Hybrid Search with Reciprocal Rank Fusion

Why Not Just Add the Scores? The Hybrid Search Dilemma

You have two lists of results, each with its own scoring system. The cosine similarity scores are between -1 and 1, while the BM25 scores can be any positive number, often in the double digits. These scores are not on the same scale and aren't directly comparable. Simply adding them or averaging them would arbitrarily favor the method that produces larger numbers. This is a common challenge in systems engineering when fusing signals from different sources.

This is precisely why hybrid search is so powerful. It leverages the strengths of each approach to overcome their individual weaknesses.

As the article highlights, hybrid search isn't a luxury; it's a practical necessity for building a search system that is both smart and precise. The diagram below shows a high-level view of a hybrid search workflow. Our focus in this lesson is the crucial "Reciprocal Rank Fusion (RRF)" step, which merges the results.

Reciprocal Rank Fusion: The "Rank, Not Score" Method

So, if we can't use the scores, what do we use? The rank. Reciprocal Rank Fusion (RRF) is a simple and brilliant algorithm that ignores the raw scores entirely and instead looks at a document's position in each ranked list. A document that is ranked #1 by BM25 and #3 by vector search is probably more relevant than one ranked #10 by both.

RRF formalizes this intuition with the following formula:

Let's break this down:

• $d$ is a specific document.
• Rankers is the set of our search methods (in our case, {BM25, Vector Search}).
• ${\text{rank}}_r(d)$ is the rank of document $d$ in the list from ranker $r$. For example, if a document is the top result, its rank is 1. If it's the second, its rank is 2, and so on.
• $k$ is a constant, typically set to 60. We'll explore why in a moment.

For each document that appears in either result list, we calculate its RRF score by summing its reciprocal rank from each list it appears in. If a document appears in one list but not the other, its rank for the second list is effectively infinite, contributing zero to the sum.

The following video provides a fantastic numerical example that makes this process very clear.

As you saw in the video, the process is straightforward:

1. For each result list, calculate the reciprocal rank for each document (e.g., rank 1 -> 1/1, rank 2 -> 1/2, etc.).
2. For each unique document across all lists, sum its reciprocal rank scores.
3. The final RRF score determines the new, fused ranking.

The Mystery of k = 60

Why is there a k in the formula, and why is it usually 60? Why not just use 1 / rank? This is a crucial detail for a robust implementation.

Using a simple 1 / rank (which is equivalent to setting $k=0$) disproportionately penalizes documents with lower ranks.

• Rank 1 score: 1/1 = 1.0
• Rank 2 score: 1/2 = 0.5 (a 50% drop)
• Rank 3 score: 1/3 = 0.33 (another big drop)

In practice, the difference in relevance between the #1 and #2 results might be tiny. Over-penalizing the #2 result might cause you to lose a very relevant document. The k constant acts as a "smoothing" factor.

By adding k=60 to the denominator, we make the score drop-off much more gradual:

• Rank 1 score: 1 / (60 + 1) ≈ 0.0164
• Rank 2 score: 1 / (60 + 2) ≈ 0.0161 (a tiny drop)
• Rank 10 score: 1 / (60 + 10) ≈ 0.0143

This ensures that top ranks are still valued more, but not so aggressively that lower-ranked-but-still-relevant items are unfairly dismissed. It gives documents that appear high in one list and lower in another a fighting chance.

This next video explains this concept perfectly.

Implementing RRF in TypeScript

Now, let's tie this all together with a TypeScript function you can add to your Problem-Finder project. This function will take the ranked results from your BM25 and cosine similarity searches and produce a single, fused list of document IDs.

Your inputs will be two arrays of objects, where each object has an id property. The order of the arrays matters, as it represents the ranking.

interface RankedResult {
  id: string;
  // You might have other properties like score, body, etc.
  [key: string]: any;
}

/**
 * Combines multiple lists of ranked search results using Reciprocal Rank Fusion (RRF).
 *
 * @param rankedLists An array of ranked result lists. Each list is an array of objects with an 'id'.
 * @param k The RRF constant, typically 60.
 * @returns A new list of results, sorted by their combined RRF score.
 */
export function reciprocalRankFusion(
  rankedLists: RankedResult[][],
  k: number = 60
): RankedResult[] {
  const rrfScores: Record<string, number> = {};
  const allDocs: Record<string, RankedResult> = {};

  // Step 1: Iterate through each list of ranked documents
  for (const rankedList of rankedLists) {
    for (let i = 0; i < rankedList.length; i++) {
      const doc = rankedList[i];
      const rank = i + 1; // Rank is 1-based

      // Store the full document object if we haven't seen it
      if (!allDocs[doc.id]) {
        allDocs[doc.id] = doc;
      }

      // Calculate the reciprocal rank score and add it to the document's total
      const score = 1 / (k + rank);
      
      rrfScores[doc.id] = (rrfScores[doc.id] || 0) + score;
    }
  }

  // Step 2: Convert the scores map to an array of documents
  const fusedResults = Object.keys(rrfScores).map(id => {
    const doc = allDocs[id];
    // Attach the RRF score for sorting and potential debugging
    return { ...doc, rrfScore: rrfScores[id] };
  });

  // Step 3: Sort the results by the RRF score in descending order
  fusedResults.sort((a, b) => b.rrfScore - a.rrfScore);

  return fusedResults;
}

// --- Example Usage for Problem-Finder ---

// Assume you have results from your two rankers
const bm25Results: RankedResult[] = [
    { id: 'p1', body: 'My Node.js backend is slow...' }, // Rank 1
    { id: 'p3', body: 'Struggling with slow performance...' }, // Rank 2
];

const vectorResults: RankedResult[] = [
    { id: 'p3', body: 'Struggling with slow performance...' }, // Rank 1
    { id: 'p2', body: 'How to optimize PostgreSQL query...' }, // Rank 2
    { id: 'p1', body: 'My Node.js backend is slow...' }, // Rank 3
];

const hybridResults = reciprocalRankFusion([bm25Results, vectorResults]);

console.log(hybridResults.map(r => ({ id: r.id, score: r.rrfScore })));
/*
  Expected output (scores are approximate):
  [
    { id: 'p3', score: 0.0325 }, // High in both lists (Rank 2 + Rank 1)
    { id: 'p1', score: 0.0321 }, // High in one, lower in other (Rank 1 + Rank 3)
    { id: 'p2', score: 0.0161 }  // Only appears in one list (Rank 2)
  ]
*/

This implementation is generic enough to handle any number of ranked lists, making it a reusable and powerful tool in your backend developer toolkit. By adding the rrfScore to the final object, you also provide a clear, unified relevance metric for your API responses.

Conclusion

Congratulations! You have now completed the core logic for a sophisticated hybrid search engine. By building on your previous work, you've learned how to fuse the outputs of lexical and semantic search into a single, cohesive ranking that leverages the best of both worlds.

Here are the key takeaways from this lesson:

• Hybrid Search is necessary because vector search excels at semantics while keyword search excels at precision. Combining them provides a more robust and accurate user experience.
• Reciprocal Rank Fusion (RRF) is an elegant and effective algorithm for merging ranked lists. Its key advantage is that it operates on rank, not score, making it immune to differences in score scales.
• The RRF formula is simple: sum 1 / (k + rank) for each document across all result lists.
• The constant k=60 is a crucial smoothing factor that prevents the algorithm from being too harsh on lower-ranked items, leading to more balanced and relevant results.

You have now built a complete, albeit slow, in-memory search system. In our next lesson, we will begin a new module focused on a critical aspect of backend engineering: performance. We'll start by tackling the "on-the-fly" processing bottleneck in your Problem-Finder app by introducing a caching layer with Redis, a foundational step toward building a scalable and responsive application.