ResNet 让超深网络可训练

34 层网络，训练误差比 18 层更高。

如果额外的层什么也不做，深层模型至少应该能复制浅层模型的结果。按这个推理，34 层不该比 18 层更差：多出来的层只要学会恒等映射，把输入原样传下去即可。理论表达能力随深度增加的故事，在教科书里听起来顺理成章。何恺明、张祥雨、任少卿与孙剑在 ImageNet 上的对照却给出相反数字：普通 34 层网络的训练误差和验证误差都高于 18 层版本。深度增加了，连训练集也拟合得更差。这不是“泛化差一点”的温和故事，而是优化过程本身没有把多出来的层用好——假设空间更大，搜索却更失败。

论文 Deep Residual Learning for Image Recognition 把这种现象称为退化（degradation）。作者特意把它与梯度消失区分开：实验已经使用批归一化，前向激活与反向梯度的数值尺度保持合理，单纯加长训练也消除不了 34 层对 18 层的劣势。困难不只在信号能否穿过几十层，还在优化器是否容易让新增的一叠层共同逼近一个看似简单的恒等函数。更深的网络可以表示浅网络的解，不代表随机初始化加随机梯度下降能够找到那个解。退化问题把“深度有益”从自动定理降格为需要工程保证的命题。

残差学习改变了层要回答的问题。设想一组层原本要直接拟合目标映射 (H(x))，ResNet 让它们改学残差 (F(x)=H(x)-x)，输出写成 (F(x)+x)。其中 (x) 经 shortcut 原样前传；输入输出维度相同时，恒等连接不增加参数，也几乎不增加计算。维度变化时则用 1×1 卷积投影对齐通道数与空间尺寸。若最合适的操作就是“不要改”，残差分支只需把权重推向零，而不必用一串非线性费力重建输入。论文图 3 把 VGG-19、34 层普通网络与 34 层残差网络并排画出，弧线标出跨层的恒等路径——读者几乎可以在图上直接数清“多出来的是什么”。

关键对照保持宽度与深度设置相配，唯一关键差别是残差版本加入 shortcut。结果翻转：残差 34 层不再重演退化，训练误差落到 18 层之下，增加深度终于换来准确率。团队继续堆到 50、101、152 层；为控制算力，较深模型采用瓶颈残差块——先用 1×1 降维，再 3×3 卷积，再用 1×1 升维——使 152 层网络在 ImageNet 上的计算复杂度仍低于当时的 VGG-19。六个不同深度模型组成的集成在 ImageNet 测试集得到 3.57% 的 top-5 错误率，拿下 2015 年 ILSVRC 分类任务。同一骨干迅速进入检测与定位：Faster R-CNN 一类流水线换成残差特征后，COCO 与 ImageNet 检测指标同步抬升。

作者还在 CIFAR-10 上把残差网络推到超过一百层，用以展示退化问题的解决并不依赖 ImageNet 特有的数据规模。普通网络在 CIFAR 上随深度增加出现明显的训练困难，残差版本则可以继续加深并降低错误率。ImageNet 预训练的 ResNet-50、ResNet-101 权重随后成为迁移学习的默认起点：医学影像、遥感、工业质检与检索系统大量以残差骨干微调。更耐久的贡献却不是某一个“152”整数，而是对照实验逼出来的工程原则：先保证存在一条近似恒等的解，再让优化器决定何处偏离。网络不必证明每一层都在“做事”；信息可以完整穿过，只在值得变换的地方变换。

34 层曾经输给 18 层的那张训练曲线，比任何口号都更清楚地说明为什么需要残差。ResNet 没有发明更强的单层运算，它只是允许深度在优化意义上重新变得诚实——能训，才配谈深。一百五十二层与百分之三点五七的 top-5 会过时，可训练的捷径结构却留了下来，成为后来几乎所有深层网络默认携带的记忆。

A 34-layer network had higher training error than an 18-layer one.

If extra layers did nothing, a deeper network should at least match a shallower one. By that logic a 34-layer model should not lose to an 18-layer model: the added layers need only learn the identity map. On ImageNet, Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun observed the opposite. A plain 34-layer network produced higher training error and higher validation error than its 18-layer counterpart. Depth increased, and the training set became harder to fit. That is not a mild story about slightly worse generalization; the optimizer itself failed to use the extra layers—a larger hypothesis space searched more poorly.

The paper Deep Residual Learning for Image Recognition calls this degradation. The authors carefully separate it from vanishing gradients: the experiments already used batch normalization, forward signals and backward gradients stayed in reasonable ranges, and training longer did not close the gap. The difficulty was not only whether a signal could cross dozens of layers, but whether an optimizer could easily drive those new layers toward an identity that looks simple on paper. Deeper nets can represent a shallow solution; random initialization plus SGD is not guaranteed to find it. Degradation demotes “depth helps” from an automatic theorem to a proposition that needs engineering insurance.

Residual learning reframes what a stack of layers must answer. Instead of learning a target mapping (H(x)) directly, a residual block learns a residual (F(x)=H(x)-x) and outputs (F(x)+x). The input (x) travels forward through a shortcut; when dimensions match, the identity shortcut adds neither parameters nor meaningful computation. When dimensions change, a projection shortcut—often a 1×1 convolution—aligns channels and spatial size. If the best operation is “leave it alone,” the residual branch only needs to push its weights toward zero rather than reconstruct the input through many nonlinearities. Figure 3 in the paper places VGG-19, a plain 34-layer net, and a residual 34-layer net side by side, with arcs marking the identity paths—readers can almost count on the page what was added.

The decisive control keeps everything else matched: a 34-layer plain network and a 34-layer residual network share comparable depth and width; the residual version adds shortcuts. The result flips. Degradation disappears, training error falls below the 18-layer baseline, and extra depth finally buys accuracy. The team then built 50-, 101-, and 152-layer networks; deeper models used bottleneck residual blocks—1×1 down, 3×3, 1×1 up—so even the 152-layer model had lower computational complexity than VGG-19 of the day. An ensemble of six residual models reached 3.57% top-five error on the ImageNet test set and won the 2015 ILSVRC classification task. Residual backbones also moved quickly into detection and localization pipelines such as Faster R-CNN, lifting COCO and ImageNet detection metrics when residual features replaced older trunks.

The team also pushed residual nets past one hundred layers on CIFAR-10, showing that fixing degradation did not depend on ImageNet’s particular scale. Plain nets again struggled as depth grew; residual versions could deepen and reduce error. Pretrained ResNet-50 and ResNet-101 weights became default transfer-learning starting points across medical imaging, remote sensing, industrial inspection, and retrieval. The durable export was not the integer 152 but the rule forced by the control experiment: guarantee an approximate identity solution first, then let the optimizer choose where to deviate. Layers need not prove they are “doing something” at every step; information can pass intact and change only where change is worth it.

The training curve where 34 layers once lost to 18 explains residual learning more clearly than any slogan. ResNet did not invent a stronger primitive layer. It allowed depth to become honest again in optimization terms—trainable first, deep second. The 152 layers and the 3.57% top-5 will age; the trainable shortcut structure remains, a memory almost every later deep network still carries by default.