Welcome to the second module of our course! In the previous module, we established a solid foundation for our project, setting up the build system and learning how to verify memory correctness with Valgrind and, crucially, how to measure performance with perf. You now have the tools to ask "how fast is my code?" and "where are the bottlenecks?".
This lesson kicks off our journey into high-performance computing, starting with the mathematical core of our vector search engine. Before we can optimize anything, we need to understand what we're calculating and why. We will explore how complex data like text or images can be represented as high-dimensional vectors and delve into the two fundamental metrics used to measure their similarity: Euclidean distance and Cosine similarity. This foundational knowledge is essential for the practical, high-performance C implementations we will build in the upcoming lessons.
From Data to Numbers: Vector Embeddings
The central idea behind modern search and recommendation systems is to represent unstructured data—text, images, audio—as numerical lists called vector embeddings. These are typically high-dimensional arrays of floating-point numbers generated by machine learning models.
The magic of these embeddings is that they capture semantic meaning. Objects with similar meanings are mapped to points that are close to each other in a high-dimensional vector space. For example, the vector for "dog" would be closer to the vector for "puppy" than to the vector for "skyscraper". This spatial relationship allows us to perform search by measuring proximity.
Distance Metrics in Vector Search
To start, let's get a clear definition of vector embeddings and their purpose from this article by Weaviate. It provides an excellent, non-technical introduction to the concept.
Read the introduction, from the beginning, and then continue to the section Vectors in Multi-Dimensional Space. Focus on understanding how data objects are transformed into points in a vector space where distance implies semantic similarity.
With our data represented as vectors, the core task of a "similarity search" is to take a query vector and find the vectors in our database that are "closest" to it. But how do we define "closest"? This is where distance metrics come in.
Measuring Similarity: The Core Metrics
There are many ways to calculate the distance or similarity between two vectors. For our high-performance search engine, we will focus on the two most prevalent and computationally important metrics: Euclidean distance and Cosine similarity.
Euclidean Distance (L2 Distance)
Euclidean distance is the most intuitive metric. It's the straight-line "as the crow flies" distance between the two points that the vectors define in space. If you've taken any linear algebra or physics, this is simply the magnitude of the vector difference between two points.
For two vectors, and , the Euclidean distance is calculated as:
A smaller distance means the vectors are more similar. A distance of 0 means the vectors are identical.
Similarity Metrics for Vector Search
The Zilliz blog provides a concise and practical explanation of Euclidean distance.
Read the section on Euclidean Distance. Pay close attention to the formula and the note about the performance optimization: because the square root function is monotonic, for finding the nearest neighbors (ranking), we can often compare the squared Euclidean distances and skip the computationally more expensive square root operation. This is a classic performance trick we will use later.
For a verbal explanation and a simple 2D example, you can also watch this short video segment.
Euclidean Distance & Cosine Similarity | Introduction to Data Mining | Part 17
This video from Data Science Dojo provides a clear, academic-style explanation.
Watch the first part of the video covering Euclidean distance. The key takeaway is how the familiar 2D distance formula generalizes to any number of dimensions.
Euclidean distance is sensitive to both the magnitude (length) of the vectors and their orientation. This is useful when the magnitude itself carries important information.
Cosine Similarity
While Euclidean distance measures the separation between two points, Cosine similarity is concerned only with the angle between the two vectors. It asks: "Are these vectors pointing in the same direction?" This metric ignores the magnitude of the vectors entirely.
The cosine of the angle between two vectors A and B is calculated using their dot product and magnitudes:
The result of Cosine similarity ranges from -1 to 1:
- 1: The vectors point in the exact same direction (angle of 0°). Maximum similarity.
- 0: The vectors are orthogonal (angle of 90°). No correlation.
- -1: The vectors point in opposite directions (angle of 180°). Maximum dissimilarity.
Since we are typically building a search system where "distance" is a measure of dissimilarity (smaller is better), we often convert similarity to distance using the formula:
This maps the [-1, 1] similarity range to a [0, 2] distance range, where 0 represents identical vectors.
Similarity Metrics for Vector Search
Let's return to the Zilliz blog for its breakdown of Cosine similarity.
Read the section on Cosine Similarity. This part is crucial for understanding why this metric is so dominant in Natural Language Processing (NLP). It allows a long document and a short sentence about the same topic to be considered very similar, as they will "point" in the same semantic direction, even if their vector magnitudes (related to word count) are very different.
You can also watch the corresponding segment from the Data Science Dojo video for another clear explanation.
Euclidean Distance & Cosine Similarity | Introduction to Data Mining | Part 17
This segment explains how Cosine similarity is calculated and why it's well-suited for high-dimensional data like document vectors.
Watch the part of the video covering Cosine similarity.
When to Use Which? A Tale of Two Metrics
The choice between Euclidean distance and Cosine similarity is fundamental and depends entirely on the nature of your data and what you consider to be "similar".
This next video provides a fantastic, practical example that makes the distinction crystal clear.
Cosine Distance vs Euclidean Distance in Machine Learning and NLP with Word2Vec or Glove Vectors
This video by Rohan-Paul-AI uses a simple e-commerce scenario to show how the two metrics can lead to different conclusions about similarity.
Watch the segment from this timestamp. It contrasts a user who buys many items of one type with a user who buys a few items of another type, showing how Euclidean distance focuses on purchase volume while Cosine similarity focuses on purchase preference.
The key insight is this:
- Use Euclidean distance when the magnitude of the vector components is meaningful (e.g., user ratings, sensor measurements). It cares about the absolute difference in values.
- Use Cosine similarity when the orientation of the vector is what matters (e.g., text documents, where topic similarity is independent of document length). It cares about the proportional relationship between values.
A final, subtle point relates to the type of vector embedding. The advice to use Cosine for text often stems from older models based on word counts (term frequency). For modern embeddings from deep neural networks (like Word2Vec or GloVe), the argument is more nuanced.
Cosine Distance vs Euclidean Distance in Machine Learning and NLP with Word2Vec or Glove Vectors
The same video explores this nuance in the context of word embeddings.
Watch the segment from this section. It explains that since neural network-based embeddings aren't simple frequency counts, Euclidean distance can sometimes be more appropriate, as two distinct words might have vectors pointing in the same direction but be far apart, a difference Cosine similarity would miss.
In practice, the best rule of thumb is to use the distance metric that the embedding model was trained with. When this is unknown, both should be considered and tested. For our project, we will implement both to build a flexible engine.
A Note on High Dimensions
One final concept to be aware of is the "curse of dimensionality". As the number of dimensions () in our vectors grows very large (hundreds or thousands), our geometric intuitions can break down. In high-dimensional space, most points tend to be far away from each other and nearly orthogonal (at 90-degree angles).
This phenomenon can make distance metrics less discriminative. It's a key reason why simply scanning all data points (brute-force search) becomes computationally infeasible and why specialized indexing algorithms, like the HNSW we'll build, are necessary.
Cosine Distance vs Euclidean Distance in Machine Learning and NLP with Word2Vec or Glove Vectors
This segment gives a quick, intuitive explanation of the curse of dimensionality.
Watch from this point to understand why both metrics face challenges in high-dimensional spaces.
Conclusion
In this lesson, we established the conceptual framework for vector similarity search. You now understand the "what" and "why" behind the core calculations our engine will perform.
Key takeaways:
- Vector embeddings are numerical representations of complex data that capture semantic meaning.
- Similarity search is the process of finding the "nearest" vectors to a query in a high-dimensional space.
- Euclidean distance (L2) measures the straight-line distance between vector endpoints, making it sensitive to both magnitude and direction.
- Cosine similarity measures the angle between vectors, making it sensitive only to direction and ideal for cases where magnitude is irrelevant (like text document length).
- The choice of metric is critical and depends on the problem domain and the embedding model used.
We are now perfectly positioned to move from theory to practice. In the next lesson, you will take these mathematical formulas and implement them as straightforward, naive C functions. This will create the performance baseline that we will then shatter using advanced optimization techniques.