A connected history of GPU programming, ASICs and hardware-supported parallelism, covering vector/SIMD processors, shared-memory and distributed programming, graphics APIs, explicit GPU kernels, directive offload, hardware description, tensor compilers and specialized accelerators. Gregorian calendar, UTC. Sources checked 3 November 2026.
Applications are dated moments
Gold application markers identify documented research demonstrations, library releases, operational deployments and benchmark results. Filter or separate by applications, linear algebra, linear programming, machine learning, molecular simulation, weather, tomography, vector search, cryptography, monte carlo, or data science.
The applications include ILLIAC IV computational fluid dynamics; BLAS and LAPACK; early GPU numerical algebra; Brook demonstrations of BLAS, FFT, segmentation and ray tracing; GPU protein simulation; CUDA N-body and Monte Carlo examples; Anton molecular-dynamics ASICs; hybrid MAGMA algorithms; AlexNet image classification; cuDNN; LAMMPS/Kokkos; TPU inference and later training; operational MeteoSwiss forecasts; TIGRE tomography; Faiss similarity search; Transformer training; RAPIDS data processing; AlphaFold structure prediction; FlashAttention; Frontier’s dense-linear-system benchmark; and GPU linear-programming solvers.
Linear algebra is not linear programming. Matrix operations and numerical factorizations are building blocks. LP solves an optimization problem with a linear objective and linear constraints. First-order methods, simplex and interior-point methods expose different parallel work. The LP track separates the original PDLP algorithm from GPU implementations and distinguishes GPU PDLP/barrier from CPU dual simplex in a concurrent solver.
Compare the programming layers
Use Separate by tags for foundations, SIMD, instruction sets, CPU parallelism, GPU languages, graphics APIs, OpenMP, OpenACC, CUDA, OpenCL, SYCL, HIP, FPGA, ASIC design, HDL, compilers, standards and application domains. Tags overlap. Blue standard markers, purple hardware-design entries and gold application entries make different kinds of contributions easier to compare.
A GPU kernel runs on an existing processor. FPGA tools configure a reconfigurable circuit. ASIC design specifies hardware for fabrication; a finished ASIC may still contain programmable engines. NKI tensor kernels and P4 packet-processing pipelines illustrate programming specialized hardware, while VHDL, SystemVerilog, Chisel and hardware compiler infrastructure address different design layers. DSPs and CPU vectors provide additional forms of specialized arithmetic and data parallelism.
Standards and interfaces at the research cutoff
| Layer | Interface or specification | Status and meaning |
|---|
| CPU instructions | MMX/SSE/AVX families, Arm SVE, RISC-V Vector | ISA features; actual support depends on the processor, compiler and OS |
| Shared memory and accelerator offload | OpenMP 6.0 | Released; TR15 is a public-comment preview of 6.1 |
| Directive accelerator offload | OpenACC 3.4 | June 2025 specification |
| Heterogeneous C++ | SYCL 2020, revision 12 | Named language/API specification; backends need not all be OpenCL |
| Cross-platform compute | OpenCL 3.0 | Released specification with optional capabilities to query |
| Explicit graphics and compute | Vulkan 1.4 | Released API baseline; device features and profiles still matter |
| Browser graphics and compute | WebGPU and WGSL | The cited September 2026 WebGPU publication is a Candidate Recommendation Draft |
| Vendor GPU kernels and runtimes | CUDA and HIP/ROCm | Vendor systems, rather than one universal cross-vendor standard; CUDA archive lists 13.4.3 in October 2026 |
| Specialized tensor kernels | AWS Neuron/NKI and accelerator-specific tools | Targeted programming interfaces; the 2024 NKI announcement is beta |
| Hardware design and modeling | IEEE 1800-2023, 1076-2019, 1666-2023 | SystemVerilog, VHDL and SystemC describe/model hardware; tool synthesis support can differ |
| Distributed communication | MPI 5.0 | Approved June 2025; complements local threading and kernels |
The dated ecosystem snapshot lists the official references. A released standard, a compiler implementation and an actually supported device feature are separate evidence. This table is a map of layers, not a ranking or a guarantee that every implementation conforms to every listed feature.
Concepts behind the milestones
Attached concept notes explain SIMD versus SIMT versus MIMD, grids/blocks/threads, data reuse, memory bandwidth and Roofline, mixed precision, training versus inference, ASIC specialization, hardware generators, simulation versus synthesis, high-level synthesis constraints, portability, intermediate representations, toolkit/driver/device versions and benchmark scope. Amdahl’s fixed-workload serial fraction and Roofline’s bandwidth bound explain why adding compute units does not automatically accelerate an entire application.
The timeline distinguishes a programmable GPU from specialized matrix units inside it, and distinguishes ASIC molecular dynamics from GPU molecular dynamics and learned protein-structure prediction. Similar application labels do not establish identical algorithms or numerical results. Source snippets and schematic equations explain concepts rather than promise portable production code.
Dates, claims and evidence
Each marker identifies its kind: paper, preprint, standard approval, standard publication, preview, hardware delivery, toolkit release, application announcement, operational migration or present-day documentation snapshot. A research paper is not automatically the first invention of its subject. A machine announcement is not proof of general availability. Hardware delivery is not the same as being declared operational.
Examples: ILLIAC IV arrives in April 1972 and is declared operational in November 1975. IEEE VHDL 1076-1987 is approved in 1987 and published in 1988. Verilog 1364-1995 is approved in 1995 and published in 1996. SystemVerilog 1800-2023 is approved in December 2023 and published in February 2024. SYCL 2020’s final specification is released in February 2021. First-generation TPU deployment in 2015 is documented by the later 2017 evaluation and is separate from a public product announcement. The Anton 2008 paper describes a scheduled system, rather than being silently treated as proof of an already completed deployment.
The OpenGL 1.0 marker dates the preserved document’s July 1994 revision, not the original API launch. CPU compiler-support entries are labeled accordingly: GCC 4.7 adds AVX2 support in 2012, before Haswell’s later launch; these compiler dates do not claim to date the first disclosure of AVX or AVX2. NVIDIA’s GeForce 256 branding is not used to claim that graphics acceleration or parallel computing began in 1999. Chrome shipping WebGPU is separate from final W3C standardization. CUDA toolkit releases are not GPU product launch dates.
Year-only and month-only dates use January 1 or the first day solely as plotting anchors. Midnight UTC does not assert a historical time of day. Coarse dates have presumptive outlines. Coverage bands organize selected entries; they do not measure the lifetime of a technology or define a support window. The map is extensive but selective rather than an inventory of every chip, ISA extension, API revision or accelerator application.
Primary evidence includes original research, standards-body catalogs, official manuals, authors’ project pages, institution histories and versioned release records. University-hosted copies preserve original papers; museums and institutional archives contextualize older machines and books. Performance claims retain the scope of their cited experiments. Vendor announcements are labeled as announcements; their peak rates are not generalized to unrelated applications. Relationship explanations and references distinguish documented use/development from conceptual comparisons and do not, by themselves, establish common project ancestry.
Media credits
Individually checked Wikimedia Commons files use public-domain, CC0 or Creative Commons licenses. The CSIRO cluster cover illustrates GPU computing; its photograph is not claimed to depict CUDA’s 2006 launch. The TPU photograph is a later TPU 3.0 device, illustrating the family rather than the exact generation announced in 2017. The Xilinx photograph depicts an XC2018, not the earlier XC2064. The historical ILLIAC image is a NASA archival photograph. Files are unchanged apart from thumbnail scaling.
timescale.info