RAG 检索增强生成论文

先查一段维基，再在查询与段落的条件下写出答案。

2020 年 5 月 22 日挂到 arXiv 上的 Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks（Facebook AI 等，后于 NeurIPS 2020 发表）把两种长期分开使用的“记东西”方式接成一句。一种是参数记忆：训练时把规律与事实压进权重，推理时只靠前向计算。一种是非参数记忆：外面放着文档库，需要时检索片段再读。前者生成流畅，更新慢、难指出来源；后者可更新、可引用，却不自动写成通顺答案。论文的动作是：先检索，再在查询与文档条件下生成。

论文里的 RAG 不是营销缩写，而是可训练流水线。查询（问题）进入稠密检索器，从维基一类文档索引里取出相关段落；预训练的序列到序列模型（参数记忆）以查询加段落为条件解码答案。检索器与生成器可以联合或分阶段训练，使“取哪段文”与“如何措辞”对着同一类知识密集型目标对齐——开放域问答是核心试验场：问题不带长上下文，答案却依赖世界知识。与只微调大生成模型相比，部分知识驻留在可替换的索引里；与只返回检索片段相比，输出是连贯文本而非链接列表。

命名后来变得比机制还响。企业知识库问答、内部规章助手、带引用的搜索摘要，都愿意自称 RAG。2020 年的论文并没有规定向量数据库品牌或提示词模板；它规定的是任务分解：非参数记忆负责可更新的事实面，参数记忆负责语言与推理面。索引可以换文档重编，而不必每次为新事实重训整网——这是工程上诱人的那一半。另一半是诚实的限制：检索错了，生成会流畅地错；段落冲突时，模型仍可能捏合；延迟与索引质量成为产品指标。RAG 降低的是“一切只靠权重”的压力，不是幻觉的物理定律。

把它放回 2020 年的邻里关系有助于防穿越。BERT 类编码器已经把稠密表示检索做热；GPT-3 把巨大参数记忆推到聚光灯下。RAG 论文站在中间：承认生成模型的条件能力，同时拒绝把百科全书只塞进参数。ChatGPT 之后的应用浪潮会把“检索增强”写成默认架构图，但账本上的公开时间点，仍是这篇把检索与生成写进同一方法名的工作。它也不同于后来产品里“先搜索网页再摘要”的黑盒编排：方法章节把检索器与生成器放进可训练目标，使“取错文档”有机会通过学习被部分纠正，而不仅靠提示词祈祷。

读实验录时，盯住接口而不是口号。输入是问题；中间是带分数的文档集合；输出是字符串答案，理想情况下可回指支撑段落。训练可以端到端到某种程度，部署却常拆成检索服务加生成服务。论文贡献是把这种拆分合法化并给出可复现的联合训练叙事，而不是发明“查资料”这一人类早会的动作。

当知识密集型 NLP 被说成一条链路，团队分工也变了：有人养索引与分块，有人养生成提示与拒答策略，有人评“是否答在文档里”。RAG 这个词能流行，是因为它给了这条分工一个短名字。短名字有时被用滥；回到 2020 年的摘要与设定，它仍然具体：检索增强的生成，服务的是需要外部文档才能站住的任务，而不是所有聊天。

First retrieve a Wikipedia passage. Then write the answer conditioned on the query and the passage.

On 22 May 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Facebook AI and coauthors; later NeurIPS 2020) joined two long-separated ways language models “know” things. Parametric memory packs regularities and facts into weights and, at inference, only runs a forward pass. Non-parametric memory keeps a document store outside and retrieves passages when needed. The first generates fluently, updates slowly, and is hard to cite. The second is updatable and citable but does not automatically write a coherent answer. The paper’s move: retrieve first, then generate conditioned on the query and the documents.

RAG in the paper is not a marketing acronym; it is a trainable pipeline. A query enters a dense retriever that pulls relevant passages from an index such as Wikipedia; a pretrained sequence-to-sequence model (parametric memory) decodes an answer conditioned on query plus passages. Retriever and generator can be trained jointly or in stages so that “which span to fetch” and “how to phrase” align to the same knowledge-intensive objectives. Open-domain question answering is the core test bed: questions arrive without long context, yet answers depend on world knowledge. Compared with only fine-tuning a large generator, some knowledge lives in a replaceable index. Compared with only returning snippets, the output is continuous text rather than a list of links.

The name later outgrew the mechanism. Enterprise knowledge-base QA, internal policy assistants, and cited search summaries all like to call themselves RAG. The 2020 paper does not mandate a vector-database brand or a prompt template. It mandates a task split: non-parametric memory for the updatable factual surface, parametric memory for language and reasoning. An index can be rebuilt with new documents without retraining the whole net for every fact—that is the half engineers find seductive. The other half is honest limit: bad retrieval yields fluent wrong answers; conflicting passages can still be mashed; latency and index quality become product metrics. RAG lowers the pressure to put everything in weights; it does not repeal the physics of hallucination.

Placing the work among 2020 neighbors prevents time travel. BERT-style encoders had already heated dense representation retrieval; GPT-3 put huge parametric memory under the spotlight. The RAG paper stands between them: it accepts generators’ conditional power and refuses to stuff the encyclopedia only into parameters. Application waves after ChatGPT would draw “retrieval-augmented” as a default architecture diagram, but the public ledger still dates the method name to this retrieve-then-generate write-up. It also differs from later product black boxes that “search the web then summarize”: the methods section puts retriever and generator into a trainable objective, so fetching the wrong document has a chance to be partly corrected by learning, not only by prompt-time prayer.

Read the lab notes for the interface, not the slogan. Input is a question; middle is a scored set of documents; output is a string answer that ideally points back to supporting passages. Training can be end-to-end to a degree; deployment often splits into a retrieval service and a generation service. The paper’s gift is to legitimize that split and give a reproducible joint-training story—not to invent the human habit of looking things up.

Once knowledge-intensive NLP is told as one pipeline, team roles change: some people own chunking and indexes, some own generation prompts and refusal policy, some score whether answers stay grounded in documents. RAG caught on because it gave that division of labor a short name. Short names get overused; return to the 2020 abstract and setup and it remains specific: retrieval-augmented generation for tasks that need external documents to stand, not for every chat.