Cosine Similarity for Vector Search

What is Cosine Similarity?

At its heart, semantic search works by converting text (like your Reddit posts and a user's query) into numerical representations called vectors or embeddings. Once we have these vectors, we need a way to measure how "close" they are in meaning. This is where cosine similarity comes in.

Cosine similarity measures the cosine of the angle between two vectors in a multi-dimensional space. The key insight is that it evaluates their orientation, not their magnitude (or length).

Why is this important for text? Imagine two documents:

1. "The cat sat on the mat."
2. "The cat sat on the mat. It was a fluffy cat."

The second document is longer, so its vector might have a larger magnitude. However, both documents are about the same thing. Because they point in a very similar direction in "meaning space," the angle between their vectors will be very small, and their cosine similarity will be close to 1.

From your background in electrical engineering, this is analogous to comparing two AC signals, like voltage or current sinusoids. The phase difference between them tells you how "in-sync" they are, which is conceptually separate from their amplitudes (magnitudes). Cosine similarity is like measuring that phase alignment for vectors.

The formula is:

Where:

• $A\cdot B$ is the dot product of the vectors.
• $\Vert A\Vert$ and $\Vert B\Vert$ are the magnitudes (or L2-norms) of the vectors.

The resulting score ranges from -1 to 1:

• 1: The vectors point in the exact same direction (perfectly similar).
• 0: The vectors are orthogonal (90-degree angle), indicating no similarity.
• -1: The vectors point in opposite directions (perfectly dissimilar).

For a fantastic and intuitive visual explanation of this concept, the following video from StatQuest is highly recommended.

From Formula to Calculation

Now let's get comfortable with the components of the formula.

• Dot Product: For two vectors $A=[a_1,a_2,...,a_n]$ and $B=[b_1,b_2,...,b_n]$, the dot product is the sum of the products of their corresponding components: $$A\cdot B=\sum _{i=1}^n{a}_i{b}_i=a_1{b}_1+a_2{b}_2+\cdots +a_n{b}_n$$
• Magnitude: The magnitude of a vector is its length, calculated using the Euclidean norm (an extension of the Pythagorean theorem). For a vector $A$: $$\Vert A\Vert =\sqrt{a_1^2+a_2^2+\cdots +a_n^2}$$

To see how these pieces fit together, let's walk through a manual calculation. The following reading provides a clear, step-by-step example.

Implementing Cosine Similarity in TypeScript

With the theory and manual calculation covered, we can now translate this logic into a reusable TypeScript function. This function will be the workhorse for the semantic search feature in your Problem-Finder app.

The article we just looked at also provides an excellent, optimized implementation.

Here is the core function from the article for your reference:

function cosineSimilarity(vecA: number[], vecB: number[]): number {
  if (vecA.length !== vecB.length) {
    throw new Error("Vectors must have the same dimensions");
  }

  // Calculate dot product: A·B = Σ(A[i] * B[i])
  const dotProduct = vecA.reduce((sum, a, i) => sum + a * vecB[i], 0);

  // Calculate magnitudes using Math.hypot()
  const magnitudeA = Math.hypot(...vecA);
  const magnitudeB = Math.hypot(...vecB);

  // Check for zero magnitude
  if (magnitudeA === 0 || magnitudeB === 0) {
    return 0;
  }

  // Calculate cosine similarity: (A·B) / (|A|*|B|)
  return dotProduct / (magnitudeA * magnitudeB);
}

Finding the Most Similar Documents

In your Problem-Finder app, you won't be comparing just two vectors. You'll have one query vector (from the user's search term) and an array of document vectors (from the Reddit posts). Your task is to calculate the cosine similarity between the query vector and each document vector, then rank the documents by their similarity score.

The process is straightforward:

1. Loop through the array of document vectors.
2. For each document vector, calculate its cosine similarity with the query vector using the function we just created.
3. Store each document along with its calculated similarity score.
4. Sort the results in descending order of similarity score.

This pattern is very common. The Stack Overflow post below discusses this exact problem, and while it uses a pre-built library, the looping structure is what's important to see.

Let's combine these ideas into a practical function for your application. Imagine you have your Reddit posts stored as an array of objects, where each object has an id and an embedding (the vector).

// Assume cosineSimilarity function from above is available

interface Document {
  id: string;
  text: string;
  embedding: number[];
}

interface SearchResult {
  document: Document;
  similarity: number;
}

function findSimilarDocuments(queryEmbedding: number[], documents: Document[]): SearchResult[] {
  const results: SearchResult[] = documents.map(doc => {
    return {
      document: doc,
      similarity: cosineSimilarity(queryEmbedding, doc.embedding)
    };
  });

  // Sort results by similarity in descending order
  return results.sort((a, b) => b.similarity - a.similarity);
}

// --- Example Usage ---
// This is a dummy example. In your app, these would come from an embedding model.
const queryEmbedding = [1, 1, 0]; // Represents a query like "backend performance"

const documents: Document[] = [
  { id: 'doc1', text: 'How to fix slow Node.js backend...', embedding: [0.9, 1, 0.1] },
  { id: 'doc2', text: 'Best frontend frameworks...', embedding: [0, 0.2, 0.9] },
  { id: 'doc3', text: 'Optimizing Node.js server performance...', embedding: [1, 0.8, 0.2] }
];

const topResults = findSimilarDocuments(queryEmbedding, documents);

console.log(topResults);
// Expected output would show doc3 and doc1 ranked highest.

This findSimilarDocuments function is a concrete implementation of the core in-memory semantic search logic. You can see how it takes a query vector and a list of documents, and returns an ordered list of the most relevant ones.

Conclusion

In this lesson, we took our first major step towards building a functional search engine. We moved from the abstract problem of "slowness" to a concrete implementation of semantic relevance.

Here are the key takeaways:

• Cosine similarity is a powerful metric for comparing vectors because it measures orientation (direction) rather than magnitude (length), making it ideal for comparing texts of different lengths.
• The calculation relies on two fundamental vector operations: the dot product and magnitude.
• You can implement a clean, efficient cosineSimilarity function in TypeScript using built-in methods like Array.prototype.reduce and Math.hypot.
• To rank documents against a query, you simply iterate through your document embeddings, calculate the similarity for each, and sort the results.

You now have the "semantic" half of your hybrid search engine. In our next lesson, we will build the other half: a keyword-based ranking algorithm. After that, we'll be ready to combine them into a single, powerful hybrid score.