CUDA 把 GPU 变成通用计算平台

要把矩阵乘法跑在 GPU 上，一度得先把它伪装成画图。

2006 年前后，GPU 已经擅长同时处理大量相似计算——图形渲染本来就要让许多像素和顶点并行通过相近的步骤。研究者尝试用 GPU 做通用计算（GPGPU），却常要把问题写成纹理、着色器和 OpenGL / DirectX 一类图形 API。可计算，不等于可编程。错误难查，性能模型不透明，代码也难在团队间继承。

NVIDIA 在 2006 年 11 月公布 CUDA 平台；官方工具包归档从 1.0，2007 年 6 月开始。两种日期并不冲突：前者是平台进入公众视野的起点，后者是今天仍能在版本表上找到的首个可安装节点。基础设施很少在某一天完整降临，它往往先被宣布，随后才以编译器、运行时、文档、驱动和库的形式逐项变得可用。公司时间线把它与 GPU 从图形加速器走向通用计算设备的过程写在同一条史里，而不是写成专为后来深度学习准备的预言。

CUDA 改变的是接口。开发者用接近 C 的方式编写在 GPU 上运行的 kernel，把大量工作分配给线程；编程模型把线程块（block）、网格（grid）、共享内存和设备内存变成明确对象。编译器负责把代码送往 GPU，运行时处理主机与设备之间的数据搬运，数学库逐渐封装常见线性代数与信号操作。并行硬件不再只通过图形管线露出一角，而成为一套能够学习、调试和版本化的软件平台。

它最初面向的远不止神经网络。科学计算、信号处理、分子模拟、计算金融和图像处理都能利用同一种大规模并行。神经网络后来与它相遇，是因为训练反复执行矩阵乘法与向量运算，工作负载恰好能铺到大量计算单元上。AlexNet 等 2012 年前后的工作已经示范 GPU 训练深度网络的收益；之后框架层把 CUDA 当作默认加速后端之一。2014 年前后推出的 cuDNN 等库又把卷积和其他神经网络原语进一步封装，框架作者不必为每代硬件重新从底层开始写 kernel。

稳定接口把速度变成了研究习惯。代码、教程、性能分析工具、库、人才与硬件采购相互加强：更多开发者带来更多软件，更多软件使下一块 NVIDIA GPU 更容易被采用。这种积累提高了效率，也建立了转换成本。后来所谓 NVIDIA 的护城河，包括多年兼容性、工具和开发者经验沉积出来的路径依赖，而不只有晶体管。竞争者可以做出加速器；更难复制的是一整条可继承的软件地层。

占用率、共享内存银行冲突、主机—设备拷贝开销、流（stream）与异步执行，成为论文附录与工程 checklist 里的常见项。早期 GPGPU 用图形 API 时，这些概念往往被藏在着色器技巧里；平台化之后，它们变成可教学、可文档化的对象。cuBLAS、cuFFT 等库先服务科学计算，cuDNN 再把卷积等原语接进深度学习框架——生态是分层长出来的，而不是为某一篇神经网络论文一次性定制。

深度学习的突破经常以论文与模型命名，CUDA 则位于较低的一层。公式给出扩展的可能，并行线程、内存层次、驱动版本和数学库把这种可能整理成日常工程。芯片可以迭代；接口若断裂，研究代码的继承链就会断。CUDA 赌的是接口尽量不断。

To run a matrix multiply on a GPU, you once had to disguise it as drawing.

By the mid-2000s GPUs already held large numbers of arithmetic units, because rendering needed similar operations across many pixels and vertices. Researchers tried general-purpose GPU computing—GPGPU—but often had to encode scientific work as textures, shaders, and graphics APIs such as OpenGL or DirectX. The hardware could compute; the software did not yet treat the computation as an ordinary program. Bugs were hard to chase, performance models opaque, and code difficult to hand across a team.

NVIDIA announced the CUDA platform in November 2006; the official toolkit archive begins with 1.0, June 2007. Both dates matter. One marks public intent; the other is the first surviving entry developers could install. Infrastructure rarely appears complete. It arrives as compilers, drivers, runtimes, documentation, and libraries that slowly become dependable enough to inherit. The corporate timeline places that shift alongside the GPU’s path from graphics accelerator to general computing device—not as a prophecy written for deep learning years later.

CUDA redrew the interface. Developers could write C-like kernels for execution on the GPU and distribute work across threads. Blocks, grids, shared memory, and device memory became named parts of a programming model. A compiler translated the code. A runtime coordinated host and device data movement. Mathematical libraries gradually standardized common linear-algebra and signal operations. Parallel hardware no longer appeared only through the narrow opening of a rendering pipeline.

The intended applications ranged across scientific computing, signal processing, molecular simulation, computational finance, and imaging. Neural networks met CUDA later for a practical reason: training repeatedly applies matrix and vector operations that can be spread over many computational units. Work around 2012, including AlexNet, already showed the payoff of training deep networks on GPUs; frameworks later treated CUDA as a default acceleration backend. Libraries such as cuDNN, introduced around 2014, packaged convolutions and other neural-network primitives so framework developers did not have to rewrite low-level kernels for each generation of hardware.

A stable interface turns speed into routine. Code examples produce trained developers; trained developers demand better tools; libraries attract projects; projects influence hardware purchases. Each part makes the next NVIDIA GPU easier to adopt. The same accumulation that improves productivity raises the cost of leaving. CUDA’s moat came to include silicon alongside compatibility, debugging tools, documentation, and habits built over years. Competitors can ship accelerators; reproducing a whole sedimentary software layer is harder.

Occupancy, shared-memory bank conflicts, host–device copy costs, streams, and asynchronous execution became ordinary items in paper appendices and engineering checklists. Early GPGPU work through graphics APIs often hid those ideas inside shader tricks; after platformization they became teachable, documented objects. Libraries such as cuBLAS and cuFFT first served scientific computing; cuDNN later attached convolution primitives to deep-learning frameworks. The stack grew in layers; it was not custom-built in one stroke for a single neural-network paper.

Accounts of deep learning tend to be named after papers and architectures. CUDA belongs lower in the stack, where dramatic language is less common and inheritance matters more. An equation may show that a model can scale. Threads, memory hierarchies, drivers, and versioned libraries decide whether researchers can carry that scale from one experiment to the next. Chips can iterate; if interfaces break, the inheritance chain of research code breaks with them. CUDA’s bet was to keep the interface from breaking.