BM25
BM25 is the dominant sparse-retrieval algorithm and the default scoring function in Elasticsearch, Lucene, OpenSearch, and most Postgres full-text setups. It ranks documents against a query by combining term frequency and inverse document frequency with two tuning parameters: k1 controls how quickly term frequency saturates, and b controls length normalization — penalizing long documents that repeat a term mechanically. BM25 is excellent at rare words, product codes, identifiers, exact phrases, and any term the embedding model has not seen enough of during pretraining, where it routinely beats dense retrieval. Its weakness is synonymy and paraphrase — "automobile" and "car" score as unrelated — which is why modern hybrid-search pipelines pair BM25 with vector search and fuse the two result lists rather than picking one.
Example
An e-commerce search team evaluates a pure-vector retriever and sees recall on category queries go up, but recall on SKU-style queries like "SKU-48A-V2" collapses because the embedding model never learned those tokens well. They switch to BM25 + vector hybrid search with reciprocal-rank-fusion. Top-5 recall on descriptive queries holds at the embedding-model baseline, recall on SKU-style and brand-specific queries recovers to essentially 1.0, and overall search success rises from illustrative 0.82 to 0.94 without any change to the embedding model.
Put this into practice
Build polished, copy-ready prompts in under 60 seconds with SurePrompts.
Try SurePrompts