--- title: "What Is BM25? A Plain-English Guide to Keyword Search Ranking | Ceisium" description: "BM25 is a keyword search ranking function that scores documents using term frequency, term rarity, and document length. Learn how it works, where it helps, and how it differs from TF-IDF." canonical: https://ceisium.com/blog/what-is-bm25/ --- > For the complete documentation and public content index, see [llms.txt](/llms.txt). This page is the Markdown alternative to the canonical HTML page. ![BM25 scores query terms using rarity, repetition, and document length signals.](https://ceisium.com/blog/bm25-cover-ceisium-bottom-right-aligned.webp) BM25 scores query terms using rarity, repetition, and document length signals. ![BM25 scores query terms using rarity, repetition, and document length signals.](https://ceisium.com/blog/bm25-cover-ceisium-bottom-right-aligned.webp) BM25 scores query terms using rarity, repetition, and document length signals. 1. [Home](https://ceisium.com/) 2. / 3. [Blog](https://ceisium.com/blog/) 4. / 5. What is BM25? A Plain English Guide Published June 24, 2026 Updated June 24, 2026 # What is BM25? A Plain English Guide BM25 is a keyword search ranking function that scores documents using term frequency, term rarity, and document length. Learn how it works, where it helps, and how it differs from TF-IDF. [![](https://ceisium.com/blog/share-icons/x.webp)](https://twitter.com/intent/tweet?text=What%20is%20BM25%3F%20A%20Plain%20English%20Guide&url=https%3A%2F%2Fceisium.com%2Fblog%2Fwhat-is-bm25%2F)[![](https://ceisium.com/blog/share-icons/linkedin.webp) ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fceisium.com%2Fblog%2Fwhat-is-bm25%2F)[![](https://ceisium.com/blog/share-icons/hacker-news.webp)](https://news.ycombinator.com/submitlink?u=https%3A%2F%2Fceisium.com%2Fblog%2Fwhat-is-bm25%2F&t=What%20is%20BM25%3F%20A%20Plain%20English%20Guide) [Search](https://ceisium.com/blog/?tag=Search) [Learn](https://ceisium.com/blog/?tag=Learn) Search looks simple from the outside. You type a few words into a documentation search box, a product search box, or a small site search, and the system returns a list of pages. Behind that list is a ranking problem. The search engine may know about hundreds or thousands of pages, but it still has to decide which matches should come first. For example, the user may search: TEXT ``` pricing page ``` Some pages are clearly about pricing, some mention pricing once in a footer or changelog entry, and many pages have nothing to do with pricing. BM25 is one way for the search engine to sort those matches. ## What is BM25? BM25 is a ranking function for keyword search. It gives each document a score based on how well that document matches the search query. Higher score, higher chance of appearing near the top. Keyword search is also called lexical search. It works with matching words . If the query contains `pricing`, the search engine first looks for documents that contain `pricing`. BM stands for Best Matching, and the `25` is a version number from a family of ranking functions. The name sounds more mysterious than the idea behind it. BM25 looks at four main things: - Which query words appear in the document - How many times those words appear - How rare those words are across all documents - How long or short the document is ![Hand-drawn illustration of BM25 weighing term rarity, repetition, and document length instead of counting words blindly.](https://ceisium.com/blog/what-is-bm25/smarter-word-counting.webp) BM25 is still counting words, but it is not blind word counting. It weighs repetition, rarity, and length before deciding what should rank. ![Hand-drawn illustration of BM25 weighing term rarity, repetition, and document length instead of counting words blindly.](https://ceisium.com/blog/what-is-bm25/smarter-word-counting.webp) BM25 is still counting words, but it is not blind word counting. It weighs repetition, rarity, and length before deciding what should rank. BM25 is useful when exact words matter , which happens a lot in product docs, support articles, API docs, error messages, settings pages, and technical search. ## A simple matching example The user searches: TEXT ``` pricing page ``` We have two documents. Document A: TEXT ``` Pricing page for teams ``` Document B: TEXT ``` Billing and pricing ``` Document A has both words, `pricing` and `page`. Document B only has `pricing`. A very simple search system can say: TEXT ``` Document A = 2 matchesDocument B = 1 match ``` That is the first idea behind keyword search: if a document contains more words from the query, it is often a better match. But this simple method breaks quickly. ## Repeating a word should not win forever Search only for: TEXT ``` pricing ``` Document A: TEXT ``` Our pricing page explains all plans. ``` Document B: TEXT ``` Pricing pricing pricing pricing pricing pricing pricing. ``` If we only count word frequency, Document B looks much better because `pricing` appears many times. The problem is easy to see. Seven mentions of `pricing` are not seven times more useful than one mention, because after a point repetition stops giving new information. BM25 handles this using term frequency saturation The first few mentions matter most. After that, repetition starts to flatten out. : TEXT ``` 0 mentions = no evidence1 mention = useful evidence2-3 mentions = stronger evidence20 mentions = not 20x stronger ``` Repetition still helps, but it does not help forever. A page should not win only because it repeats the same word again and again. ## Rare words should count more Take another query: TEXT ``` configure sso ``` The word `configure` may appear on many documentation pages, while `sso` may appear only on a few. If a document contains `sso`, that is a stronger signal than if it only contains `configure`. BM25 uses this idea with IDF, which means inverse document frequency. In simple words: - If a word appears in many documents, it is less special. - If a word appears in fewer documents, it is more special. A user searches: TEXT ``` stripe webhook retry ``` On the website: TEXT ``` stripe appears on 40 pageswebhook appears on 15 pagesretry appears on 120 pages ``` Here, `webhook` is the rarest word, so it gives a stronger clue. `stripe` is also useful. `retry` is still useful, but it appears in more places, so it is less specific. This depends on the website. On a developer website, `api` may appear everywhere; on another website, `api` may be rare. BM25 learns term importance from the corpus itself. A word is only rare or common relative to the collection being searched. ### How does the search engine know these counts? BM25 does not read every page from scratch when a user searches. The search engine prepares this information before the query comes in. When pages are crawled or indexed, the search engine builds an inverted index. An inverted index is a map from words to documents. It looks like this: TEXT ``` stripe -> doc1, doc7, doc9, doc20, ...webhook -> doc3, doc7, doc12, ...retry -> doc2, doc3, doc7, doc8, doc21, ... ``` With this index, the search engine can know: TEXT ``` stripe appears in 40 documentswebhook appears in 15 documentsretry appears in 120 documents ``` That count is called document frequency Document frequency means: how many documents contain this word at least once. . The search engine also stores other information: - How many times each word appears in each document - How long each document is - What the average document length is At search time, the engine can use these stored numbers to calculate the BM25 score. A useful way to picture it The crawler reads pages ahead of time. [Website crawling ![](https://ceisium.com/blog/website-crawling-search-quality-cover.webp)Search / Jul 21, 2026 How Website Crawling Shapes Search Quality Learn how crawl scope, sitemaps, rendering, URL identity, and recrawl freshness shape the pages a site-search ranker can choose from.](https://ceisium.com/blog/how-website-crawling-shapes-search-quality/) determines which pages enter the index. The index remembers where words appear. BM25 uses that stored memory when a query comes in. ![Hand-drawn illustration of an inverted index as stored search memory, with query terms connected to document counts.](https://ceisium.com/blog/what-is-bm25/index-stored-memory.webp) The search engine does the slow remembering before the query arrives. At search time, BM25 uses that stored index instead of rereading the whole site. ![Hand-drawn illustration of an inverted index as stored search memory, with query terms connected to document counts.](https://ceisium.com/blog/what-is-bm25/index-stored-memory.webp) The search engine does the slow remembering before the query arrives. At search time, BM25 uses that stored index instead of rereading the whole site. ## Long documents need adjustment Search for: TEXT ``` billing ``` Document A is short: TEXT ``` BillingManage billing settings for your workspace. ``` Document B is very long: TEXT ``` ... 8,000 words ...billing appears once somewhere inside it... ``` Both documents contain `billing` once, but they should not get the same score. In Document A, `billing` is probably the main topic; in Document B, it may only be a small mention. BM25 handles this using document length normalization. A match in a short, focused document can be stronger than the same match inside a very long document. This does not mean long documents are bad. A long guide can still be the best result. BM25 is only trying to avoid giving long documents an unfair advantage because they have more words and more chances to match something. So far, BM25 has made four corrections to simple word counting. - It gives more credit when more query words match. - It lets repetition help, but only up to a point. - It gives more weight to rare words. - It adjusts the score when the same match appears inside a very long document. BM25 is still counting words, but it is counting them with useful corrections. ## BM25 in real search systems In a real website, search usually does not treat the whole page as one big block of text. A page has many fields: - Title - URL - Headings - Meta description - Body text - Anchor text - Content chunks A match in the title is usually more important than a match deep inside the body. If the user searches: TEXT ``` api keys ``` A page titled `API Keys` should usually rank higher than a long guide that mentions API keys once in the middle. Many systems solve this by scoring each field separately: TEXT ``` final keyword score = title BM25 score x title weight+ heading BM25 score x heading weight+ body BM25 score x body weight+ URL BM25 score x URL weight ``` Clear titles, headings, and URLs help both the reader and the search engine. Search systems also prepare text before ranking it. They may lowercase words, remove punctuation, handle plurals, remove stop words, or tolerate small typos: TEXT ``` API keysapi-keyapi key ``` After tokenization and normalization, these become easier to compare. Modern search systems may also split long pages into chunks, because one long page can talk about many topics. A chunk can tell the search engine, “this section is about API keys,” instead of only saying, “this whole page contains API keys somewhere.” ## The BM25 formula The formula is less scary if we connect it back to the ideas above. For the query: TEXT ``` reset api key ``` BM25 scores each query word and adds the scores: TEXT ``` score("reset") + score("api") + score("key") ``` A common BM25 formula looks like this: Same idea, written as math. Do not read this as a new idea. It is the same matching, rarity, repetition, and length logic packed into math. TEXT ``` score(D, Q) = sum over query terms:IDF(q) * ((f(q, D) * (k1 + 1)) /(f(q, D) + k1 * (1 - b + b * |D| / avgdl))) ``` The symbols mean: TEXT ``` D = the documentQ = the queryq = one query wordf(q,D) = how many times q appears in DIDF(q) = how rare q is across the collection|D| = document lengthavgdl = average document lengthk1 = how much repetition helpsb = how much document length matters ``` The two knobs to remember are `k1` and `b`. `k1` controls repetition. If `k1` is low, extra mentions stop helping quickly; if it is high, repeated words keep adding score for longer. `b` controls document length. If `b` is `0`, document length does not matter; if it is `1`, document length matters a lot. Many systems use a value somewhere in the middle. ## BM25 vs TF-IDF BM25 is related to TF-IDF. TF-IDF means: TEXT ``` term frequency x inverse document frequency ``` It uses two ideas: - Words that appear more in a document can matter more. - Words that appear in fewer documents can matter more. BM25 keeps those ideas, but adds two controls that matter in real search: - Repeated words do not keep increasing the score forever. - Long documents are adjusted so they do not win only because they contain more words. That is why BM25 is usually a better baseline than plain TF-IDF for website search, documentation search, and product search. ## Where BM25 works well BM25 works well when the user searches for words that actually appear in the content: TEXT ``` reset api keybilling invoice downloadapi_key_invalidSOC2 ``` These are good BM25 queries because the exact words matter. That makes BM25 useful for: - Product names - Feature names - API names - Error codes - SKUs - Settings pages - Technical terms - Support articles If someone searches for `api_key_invalid`, we want the result that contains `api_key_invalid`. Exact matching should have real weight here. ## Where BM25 struggles BM25 struggles when the user and the document use different words for the same idea. The user may search: TEXT ``` change password ``` while the page says: TEXT ``` Update your credentials ``` A human can understand that these are related, but BM25 may not, because `change` does not match `update`, and `password` does not match `credentials`. Another query: TEXT ``` remove teammate ``` may need this page: TEXT ``` Delete a workspace member ``` The meaning is close, but the words are different. BM25 can still do well if the page includes alternate wording, synonyms, headings, or anchor text; the core ranking function itself is lexical, so it rewards word overlap. ![Hand-drawn illustration showing exact keyword matching missing a related phrase when the words are different.](https://ceisium.com/blog/what-is-bm25/exact-match-limits.webp) This is the main limit of lexical ranking: if the useful page says the idea with different words, BM25 may need help from synonyms, rewritten headings, or another retrieval method. ![Hand-drawn illustration showing exact keyword matching missing a related phrase when the words are different.](https://ceisium.com/blog/what-is-bm25/exact-match-limits.webp) This is the main limit of lexical ranking: if the useful page says the idea with different words, BM25 may need help from synonyms, rewritten headings, or another retrieval method. ## Other ranking methods BM25 is one ranking method in the larger family of lexical and statistical search methods. TF-IDF is the closest comparison because it is simpler and helps explain why BM25 adds saturation and document length normalization. Boolean retrieval is older and more strict. It uses logic like: TEXT ``` api AND keybilling OR invoiceerror NOT deprecated ``` With Boolean retrieval, a document either matches or it does not. That is useful for filters and exact control, but it is not always enough for ranking. Language models for information retrieval ask a different question: how likely is this document to produce this query? DFR, or Divergence From Randomness, asks whether a term appears more often than we would expect by chance. BM25F is a field-aware version of BM25 for documents with fields like title, body, anchor text, and metadata. BM25+ is another variant that changes how long documents are treated in some cases. Learning to Rank goes broader. Instead of using one fixed formula, it learns how to combine many signals: BM25 score, title match, freshness, click data, phrase match, document type, and more. Even there, BM25 is often still one of the useful signals. ## FAQ ### What does BM25 stand for? BM stands for Best Matching. BM25 is one version from the Best Matching family of ranking functions. ### What is 25 in BM25? The `25` is a version number. It does not mean BM25 has 25 steps or 25 inputs. ### Is BM25 just TF-IDF? No. BM25 is related to TF-IDF, but it is not the same. Both use term frequency and inverse document frequency. BM25 also adds term frequency saturation and document length normalization. ### What is BM25 used for? BM25 is used to rank keyword search results in full-text search, documentation search, product search, support search, and technical search. ### What is BM25 in RAG? In RAG systems, BM25 can retrieve documents or chunks that contain exact query terms before the results are sent to a language model. This helps with names, IDs, error codes, and technical terms. ### Does BM25 understand meaning? Not really. BM25 mostly works with word overlap and term statistics, so it may miss two phrases that mean the same thing but use different words. ### What are k1 and b? `k1` and `b` are tuning parameters. `k1` controls how much repeated words help, and `b` controls how much document length matters. ## Summary BM25 is a practical way to rank keyword search results. It looks at: - Which query words match - How often they appear - How rare they are - How long the document is BM25 is not trying to understand language like a person. It is trying to rank exact word matches in a smarter way than simple word counting. BM25 is still useful because exact words still matter . Product names, API names, error codes, settings pages, and technical concepts often need exact matching. Once you understand BM25, many search results start to look less mysterious: the engine is asking which words matched, how strong those matches are, and whether the document looks focused enough to deserve the top spot. [![](https://ceisium.com/blog/share-icons/x.webp)](https://twitter.com/intent/tweet?text=What%20is%20BM25%3F%20A%20Plain%20English%20Guide&url=https%3A%2F%2Fceisium.com%2Fblog%2Fwhat-is-bm25%2F)[![](https://ceisium.com/blog/share-icons/linkedin.webp) ](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fceisium.com%2Fblog%2Fwhat-is-bm25%2F)[![](https://ceisium.com/blog/share-icons/hacker-news.webp)](https://news.ycombinator.com/submitlink?u=https%3A%2F%2Fceisium.com%2Fblog%2Fwhat-is-bm25%2F&t=What%20is%20BM25%3F%20A%20Plain%20English%20Guide) Copy image[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fceisium.com%2Fblog%2Fwhat-is-bm25%2F)[](https://twitter.com/intent/tweet?text=%E2%80%9CWhat%20is%20BM25%3F%20A%20Plain%20English%20Guide%E2%80%9D&url=https%3A%2F%2Fceisium.com%2Fblog%2Fwhat-is-bm25%2F) **Yuvraj** _Building Ceisium and working on growth @ SigNoz_ ceisium.com ## Next reads [ ![Five documentation cards pass through Boolean set paths and become three search candidates.](https://ceisium.com/blog/boolean-retrieval-cover.webp) Boolean Retrieval: AND, OR, and NOT in Search Learn how Boolean search uses AND, OR, and NOT to combine result sets, where exact control helps, and why website search still needs ranking. ](https://ceisium.com/blog/boolean-retrieval/)[ ![Equal query-term matches sit in short and long document lanes against one average-length reference.](https://ceisium.com/blog/document-length-normalization-cover.webp) Document Length Normalization: Why Long Pages Need Context Learn why long pages get more chances to match a search query, how BM25 adjusts for relative document length, and why shorter is not automatically better. ](https://ceisium.com/blog/document-length-normalization/)[ ![Website routes pass through crawl boundaries into a search corpus before a query selects candidates.](https://ceisium.com/blog/website-crawling-search-quality-cover.webp) How Website Crawling Shapes Search Quality Learn how crawl scope, sitemaps, rendering, URL identity, and recrawl freshness shape the pages a site-search ranker can choose from. ](https://ceisium.com/blog/how-website-crawling-shapes-search-quality/)