Semantic Search with Cosine Similarity

What is Semantic Search?

Before we dive into implementation, it's crucial to distinguish semantic search from traditional keyword search. Keyword search excels at finding exact textual matches. Semantic search, powered by embeddings, operates on a higher level of abstraction: meaning.

For example, a keyword search for "ways to relax outdoors" might miss a document that says "Enjoy a stroll in the park." A semantic search system, however, understands that these phrases are conceptually related and would identify the document as highly relevant.

This is achieved by representing both the query and the documents as vectors in the same high-dimensional space. The search then becomes a geometric problem: finding the document vectors that are "closest" to the query vector.

Measuring Closeness: Cosine Similarity

So, how do we mathematically define "closeness" between two embedding vectors? The most common metric in this domain is cosine similarity.

Instead of measuring the Euclidean distance between the vector endpoints (which is sensitive to vector magnitude or length), cosine similarity measures the cosine of the angle between two vectors. This makes it a measure of orientation, not magnitude.

Why is this important for text? Consider a short sentence and a long paragraph discussing the same topic. Their embedding vectors might have different lengths, but they should point in a similar direction in the vector space. Cosine similarity captures this directional alignment perfectly.

The value of cosine similarity ranges from:

• 1: The vectors point in the exact same direction (angle is 0°). This indicates identical or highly similar meaning.
• 0: The vectors are orthogonal (angle is 90°). This indicates they are unrelated.
• -1: The vectors point in opposite directions (angle is 180°). This indicates opposite meanings.

The formula for cosine similarity between two vectors, A and B, is the dot product of the vectors divided by the product of their magnitudes:
$\text{similarity}=\cos (\theta )=\frac{A\cdot B}{\Vert A\Vert \Vert B\Vert }=\frac{{\sum }_{i=1}^n{A}_i{B}_i}{\sqrt{{\sum }_{i=1}^n{A}_i^2}\sqrt{{\sum }_{i=1}^n{B}_i^2}}$

Given your background in computer science, you can see this is a straightforward and computationally efficient operation, especially when optimized in libraries.

Implementing Semantic Search from Scratch

Let's now translate this theory into a practical implementation. The core workflow for performing a semantic search is:

1. Encode the Corpus: Convert all your document chunks into embeddings (we covered this in a previous lesson).
2. Encode the Query: Take the user's incoming question and use the exact same embedding model to convert it into a query vector.
3. Compute Similarities: Calculate the cosine similarity between the query vector and every document vector in your corpus.
4. Rank and Retrieve: Sort the documents by their similarity score in descending order to find the top-k most relevant results.

This process is a fundamental building block. While vector databases like ChromaDB automate this, understanding the underlying steps is crucial for debugging, optimization, and advanced RAG strategies.

Code Walkthrough

Let's consolidate the key steps from the article into a single, runnable example.

First, ensure you have the necessary libraries installed:
pip install sentence-transformers scikit-learn numpy

Now, let's implement the search.

import numpy as np
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

# 1. Define our document corpus
corpus_of_documents = [
    "Take a leisurely walk in the park and enjoy the fresh air.",
    "Visit a local museum and discover something new.",
    "Go for a hike and admire the natural scenery.",
    "Have a picnic with friends and share some laughs.",
    "Take a yoga class and stretch your body and mind."
]

# 2. Load the pre-trained embedding model
# This must be the SAME model used to embed your documents for storage.
model = SentenceTransformer('all-MiniLM-L6-v2')

# 3. Encode the corpus into embeddings
# In a real application, these embeddings would be pre-computed and stored in your vector DB.
doc_embeddings = model.encode(corpus_of_documents)

# 4. Define a user query
query = "What are some fun activities to do outside?"

# 5. Encode the query
query_embedding = model.encode([query])

# 6. Compute cosine similarities
# The output is a 2D array, so we take the first row.
similarities = cosine_similarity(query_embedding, doc_embeddings)[0]

# 7. Rank the documents and retrieve the top-k
k = 3
# Get the indices of the top-k scores
top_k_indices = np.argsort(similarities)[::-1][:k]

print(f"Query: {query}\n")
print(f"Top {k} most relevant documents:")
for idx in top_k_indices:
    print(f"  - Score: {similarities[idx]:.4f} | Document: {corpus_of_documents[idx]}")

Expected Output:

Query: What are some fun activities to do outside?

Top 3 most relevant documents:
  - Score: 0.6397 | Document: Go for a hike and admire the natural scenery.
  - Score: 0.6279 | Document: Take a leisurely walk in the park and enjoy the fresh air.
  - Score: 0.5055 | Document: Have a picnic with friends and share some laughs.

This simple script perfectly demonstrates the mechanics of semantic search.

Connecting to Vector Databases

The "from-scratch" approach above is great for understanding, but it's a brute-force comparison that doesn't scale. This is precisely the problem that the vector databases and ANN indexes from our last lesson solve.

When you use a library like ChromaDB or FAISS, you are executing the exact same logical steps, but in a highly optimized and scalable way:

• ChromaDB: The collection.query() method abstracts this entire process.

  results = collection.query(
      query_texts=["What are some fun activities to do outside?"],
      n_results=3
  )
  # Chroma handles encoding the query, performing the ANN search (which approximates cosine similarity),
  # and returning the ranked documents and their scores.
  

• FAISS: The index.search() method is the optimized core of the process.

  # query_embedding is your numpy array from model.encode()
  distances, indices = index.search(query_embedding, k=3)
  # FAISS uses its index (e.g., HNSW) to rapidly find the nearest neighbors
  # instead of doing a full scan.
  

In essence, these tools replace our manual cosine_similarity loop and argsort with a sub-linear time complexity search, making retrieval feasible for millions or even billions of documents.

Conclusion

You have now mastered the core retrieval mechanism of a RAG system. By combining embeddings with cosine similarity, you can search for information based on conceptual meaning, a massive leap beyond simple keyword matching.

Key Takeaways:

• Semantic Search is Meaning-Based: It finds relevant information by comparing the meaning of a query to the meaning of documents, not just their words.
• Cosine Similarity Measures Orientation: It is the standard metric for comparing text embeddings because it is independent of document length and effectively captures semantic closeness.
• The Search Workflow is Universal: The process of encoding a query, comparing it to a corpus of embeddings, and ranking the results is fundamental, whether done manually or through an optimized vector database.
• Vector Databases Optimize Search: Tools like ChromaDB and FAISS implement this search workflow using highly efficient Approximate Nearest Neighbor (ANN) algorithms, making retrieval scalable to massive datasets.

What's Next?

We have now assembled the two main components of RAG: an indexed knowledge base and a method for retrieving relevant context from it. The final step is to connect these pieces.

In our next lesson, we will build a complete RAG pipeline connecting retrieval and generation using a framework like LangChain or LlamaIndex. You will learn how to take the documents retrieved from your semantic search, combine them with the user's original query into a single prompt, and pass it to an LLM to generate a final, contextually-grounded answer.