RAG and vector search

Plain English. Retrieval-augmented generation: instead of hoping the model knows something, fetch the relevant documents first — typically via vector search over embeddings — and put them in the prompt. The standing confusion is RAG versus long context: RAG selects what the model sees; a long context window just shows it everything. Selection costs engineering; everything costs tokens.

Why it moves money. RAG-as-default-architecture is the thesis a whole enterprise database layer is priced on. The evidence is wobbling: in careful comparison, plain text search beat vector retrieval, and production teams keep reporting that simple, inspectable retrieval beats vector infrastructure. As context gets longer and cached input gets cheaper, the expensive middle of the RAG stack is the part at risk.

What to watch. Whether vector-database revenue keeps growing once long-context-plus-caching prices in — and what fraction of shipped "RAG" is actually grep.

From the signals. grep beat vector — and the harness beat both. Extency: the production-winning memory architecture is Markdown in Git, not vector databases.

Further reading. Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (2020).