BM25 in RAG: Why Exact Keywords Still Matter

An embedding model can connect “login problem” with “authentication failure.” That is useful until the answer depends on one exact string.

How do I fix ZQ7-AUTH-N9X4?

In a common retrieve-then-generate RAG pipeline, retrieval selects a small set of passages and supplies them to the generator as external evidence. If the page for ZQ7-AUTH-N9X4 never enters that set, the generator cannot cite it or ground an answer in it.

Consider an illustrative failure case. Dense retrieval returns general pages about expired sessions, while the exact troubleshooting page stays out of the candidate set. The answer can still sound plausible, but any claim that depends on the missing page cannot be grounded in the retrieved source evidence. Retrieval decides which external evidence the generator can cite or ground against.

I would not make embeddings the only retrieval path for a corpus that contains identifiers, error codes, proper nouns, niche terms, or quoted phrases. Those queries need a path that preserves literal evidence, and BM25 is one of the simplest ways to add it.

RAG fails before generation

The original RAG paper opens in a new tab combines parametric memory with retrieved non-parametric memory. RAG architectures have changed since 2020, but this dependency has not. A source passage must reach the generator before the answer can use it as retrieved evidence.

For the ZQ7-AUTH-N9X4 query, dense retrieval might return pages about invalid credentials, expired sessions, access-token failures, and general login troubleshooting. All four are related to the question, and some may help. None is a substitute for the page that documents this exact error code.

Semantic similarity tells us what a passage is about. The identifier tells us which piece of evidence the user needs. Lexical retrieval can catch the gap between those two questions.

Related passages are not substitutes for the exact source. The identifier still has to reach the context set.

RAG needs meaning and identity

Vector search Search / Jul 10, 2026 What Is Vector Search? Vector search finds related content by comparing embeddings. Learn how vectors, similarity, approximate nearest-neighbor search, and hybrid retrieval work together. helps when a query and a document use different words. Someone can search for “recover account access” and still find a page called “Reset a forgotten password.” Dense retrieval crosses that vocabulary gap.

Identifiers create the opposite problem. A query containing ZQ7-AUTH-N9X4, POST /v1/search, client_max_body_size, Acme Ultra 14, or a copied log line asks for more than a related topic. The literal text carries part of the meaning. Sometimes it carries nearly all of it.

Salient Phrase Aware Dense Retrieval opens in a new tab gives us evidence for this failure mode. Chen and colleagues report that the dense retrievers they studied lagged BM25 at reliably matching salient phrases and rare entities. Their result does not mean dense retrieval can never handle exact terms. The same paper trained a dense lexical model that recovered sparse-style matching capacity.

BM25 is a lexical ranker. It rewards passages that share important analyzed terms with the query. Our plain-English guide to BM25 Search / Jun 24, 2026 What is BM25? A Plain English Guide BM25 is a keyword search ranking function that scores documents using term frequency, term rarity, and document length. Learn how it works, where it helps, and how it differs from TF-IDF. covers its scoring mechanics, so I will keep the RAG takeaway short.

Semantic retrieval protects related meaning. BM25 can raise evidence that shares the relevant analyzed terms.

“Analyzed terms” is an important limit. Search engines tokenize text before ranking it. Depending on the tokenizer, /v1/search may remain one value or become v1 and search. Hyphens, underscores, punctuation, and case can change what the index stores.

Weaviate’s keyword-search documentation opens in a new tab shows the practical consequence. Keyword search matches indexed tokens, so fields such as URLs and email addresses need tokenization that fits the way you plan to query them.

BM25 therefore does not guarantee whole-string equality or phrase order. Use a keyword field, term lookup, or phrase query when byte-for-byte identity or ordered phrases are requirements. Those mechanisms can share the candidate path with BM25, but they are separate retrieval tools.

Hybrid retrieval widens the candidate set

Dense and lexical retrieval fail in different ways, so many RAG systems run both. A common pipeline for the error-code query works like this.

1. BM25 finds passages whose analyzed terms overlap ZQ7-AUTH-N9X4.
2. Dense retrieval finds passages about the authentication problem.
3. Fusion combines both result lists into one candidate set.
4. A reranker reorders that set. It cannot recover a passage that neither retriever found.
5. The selected passages become context for generation.

Hybrid retrieval can widen the candidate set. Reranking only works with evidence that a retrieval path already found.

NVIDIA’s RAG Blueprint opens in a new tab documents one deployable version of this design. It supports sparse-plus-dense retrieval and can combine results with reciprocal rank fusion or configured weights. The blueprint demonstrates the design. Whether it improves a different corpus remains an evaluation question.

A second retrieval path adds storage and compute. Its wall-clock cost depends on whether both searches run in parallel and where the slower path sits in the latency budget. Fusion introduces another choice because the two lists need weights or a rank-based combination. It can also send duplicates and weak candidates into the reranker.

The best balance changes with the query. The lexical result may deserve more influence for ZQ7-AUTH-N9X4, while the dense result may deserve more for “How can I recover account access?” One fixed weighting can be a poor fit for a query mix that spans both, so test the weights against each query class before considering per-query routing.

Hybrid retrieval protects against two kinds of evidence loss. It still has to earn its extra cost in evaluation.

Exact matching cannot judge source quality

Now make the example harder. Two pages contain ZQ7-AUTH-N9X4.

1. An old guide has the code in its title and body.
2. A current guide mentions the code once.

BM25 may rank the old guide first because it has stronger lexical evidence. The match is exact, yet the result is wrong for the user.

Access control must remain a hard eligibility constraint before ranking, Ranking should never be allowed to overrule access control. never a preference that a reranker can ignore. Azure AI Search documents this distinction opens in a new tab for document-level access control.

Among eligible documents, lexical overlap still says nothing about freshness, product version, or source authority. Metadata filters, version-aware indexing, authority signals, or a reranker may be needed to prefer the current guide.

Eligibility is a hard boundary. Freshness and authority still have to resolve allowed exact matches.

This is why the slogan “BM25 is precise, vectors understand meaning” falls apart under real source constraints. Lexical retrieval can be precise about the terms it indexed. It cannot decide whether a passage is current, authorized, or true.

Dense retrieval has the same boundary. A passage can be semantically close and still be stale or wrong. If an unauthorized passage reaches the candidate set, the eligibility filter has already failed.

The retriever must preserve the right evidence for this user, at this time, from the right source.

When dense retrieval is already enough

The strongest counterargument is a good one. A well-engineered dense system may already retrieve product names, codes, and technical language reliably.

Dense Passage Retrieval opens in a new tab beat its BM25 baseline on top-20 passage retrieval accuracy across the paper’s evaluated open-domain QA benchmarks. The SPAR result above also shows that dense models can learn lexical matching behavior.

In practice, a strong dense system may have a domain-tuned embedding model, chunks that keep identifiers beside their explanations, metadata filters, and enough surrounding context to make an identifier meaningful. What matters is the measured result. If top-k recall already passes for the query types you care about, adding BM25 may only add complexity.

Dense-only retrieval is most plausible when the corpus is small and consistent, users ask mostly natural-language questions, and evaluation shows that required passages already appear reliably. Lexical retrieval has its own misses. Without stemming, synonym expansion, translation support, or query rewriting, it can lose passages that use different vocabulary.

The important distinction is between model capability and system evidence. “The embedding model can retrieve this identifier” is a capability claim. “Our retriever reliably preserves this identifier class” is a measured system claim. Test the second claim against the identifiers your users actually send.

Test lexical retrieval against real failures

Do not add BM25 because hybrid retrieval sounds safer. Start with the failures in your query traffic. If exact-term misses are absent, keep the simpler system. If they are present, test whether lexical retrieval fixes them.

A small RAG evaluation set should separate these query classes.

1. Error codes and IDs
2. Proper nouns and model names
3. Niche terminology and configuration keys
4. Exact phrases copied from logs or support messages
5. Paraphrased natural-language questions
6. Stale-content traps where the strongest exact match is not the source you want

Candidate recall

• Did the required passage enter the candidate set at Recall@k?
• How did MRR or nDCG change for each query class?
• Did fusion or reranking help one class while hurting another?

Answer grounding

• Did the answer use the right passage?
• Did citation correctness improve?

System cost

• What happened to retrieval, fusion, and reranking P50/P95 latency?
• What infrastructure and tuning did the gain require?
• Did lexical candidates consume reranker or context budget without improving recall?

A polished answer can hide weak retrieval. The candidate list cannot.

I add lexical retrieval when it fixes an observed evidence gap and the gain is worth the cost. I keep dense-only retrieval when it already passes the query mix I care about.

Choose the retriever that preserves the evidence

BM25 is old and embeddings are newer. Their age tells you nothing about recall on your corpus. The useful question is whether your retriever preserves every kind of evidence the generator needs.

Use both signals when evaluation shows that one keeps losing required evidence. Keep the simpler system when it does not.