Dot Product Calculation

Welcome to your third lesson. In our previous session, we explored how to add vectors together and how to scale them using scalar multiplication. Both of these operations resulted in a new vector. Today, we'll introduce a different way to combine two vectors: the dot product.

The dot product is a fundamental operation in mathematics for AI. What makes it unique is that when you calculate the dot product of two vectors, the result is not another vector, but a single number—a scalar. This lesson is entirely focused on mastering the mechanics of this calculation. By the end, you'll be able to confidently compute the dot product for vectors of any size, a skill you will use constantly as we delve deeper into machine learning concepts.

How to Calculate the Dot Product

The most direct way to calculate the dot product is through an algebraic approach. The process involves two simple steps:

Multiply the corresponding components of the two vectors.
Sum up all of these products.

That's it. The result of this sum is the dot product. For this operation to be possible, the two vectors must have the same number of dimensions (i.e., the same number of elements).

Let's define this formally. If we have two n-dimensional vectors:

a = a_{1} a_{2} ⋮ a_{n} and b = b_{1} b_{2} ⋮ b_{n}

Their dot product, written as $a \cdot b$ , is calculated as:

a \cdot b = a_{1} b_{1} + a_{2} b_{2} + \dots + a_{n} b_{n}

The following reading provides a concise summary of this algebraic definition.

Dot Product: The Theory, Computation, and Real Uses

This article from DataCamp introduces the dot product and its algebraic definition.

Focus on the section "Algebraic definition". This section gives you the core formula for the computation we'll be practicing.

Worked Examples: From 2D to 3D

Let's solidify this with a few examples.

2D Example:
Consider two vectors, $v = [34]$ and $w = [5 - 2]$ .

To find their dot product, $v \cdot w$ :

Multiply the first components: $3 \times 5 = 15$
Multiply the second components: $4 \times (- 2) = - 8$
Sum the results: $15 + (- 8) = 7$

So, $v \cdot w = 7$ .

The video below walks through this process for 2D vectors in a very clear, step-by-step manner.

Dot Product of Two Vectors

The Organic Chemistry Tutor channel provides excellent, straightforward explanations. This video starts with the basics of calculating the dot product for 2D vectors.

Watch the first two examples presented in the video, from the beginning until the end of the second problem. This will reinforce the component-wise multiplication and summation process.

3D Example:
The process extends seamlessly to higher dimensions. Let's take two 3D vectors:
$p = 271$ and $q = 828$ .

The calculation for $p \cdot q$ is:

p \cdot q = (2 \times 8) + (7 \times 2) + (1 \times 8) = 16 + 14 + 8 = 38

This image illustrates the step-by-step calculation for the dot product of the two 3D vectors p and q, showing the element-wise multiplication and the final summation.

The same video resource from The Organic Chemistry Tutor also provides a clear example for 3D vectors.

Dot Product of Two Vectors

Let's continue with the same video to see how the calculation works in three dimensions.

Now, watch the next segment which covers a 3D vector example. Notice that the procedure is identical; we've just added one more component to the sum.

The Dot Product in AI and Code

Your goal is to learn math for AI, and the dot product is one of the most frequently used operations in machine learning libraries like NumPy, PyTorch, and TensorFlow. Whenever a model processes data, performs a calculation for a linear layer, or computes a similarity score, there's a high chance a dot product (or a series of them) is being calculated under the hood.

The following video does an excellent job of explaining the dot product from a machine learning practitioner's perspective and shows how it's done in code.

The Dot Product — Topic 14 of Machine Learning Foundations

This video from Jon Krohn's "Machine Learning Foundations" series is tailored to an AI audience. It explains the calculation and then demonstrates it in NumPy.

Watch the segments that cover the conceptual calculation and the NumPy implementation. Seeing how a simple np.dot(x, y) command performs the exact operation we've been doing by hand is a key connection between the math and the practical application.

Combining Operations

To tie this lesson together with the previous one, it's useful to see how the dot product fits into expressions involving other vector operations like addition and scalar multiplication. When you see such an expression, you simply follow the standard order of operations (parentheses first).

For example, to calculate $(3 a) \cdot (b + c)$ :

First, calculate the scalar multiplication $3 a$ .
Second, calculate the vector addition $b + c$ .
Finally, compute the dot product of the two resulting vectors.

Let's return to the video from The Organic Chemistry Tutor, which provides excellent worked examples of these combined operations.

Dot Product of Two Vectors

This final segment of the video demonstrates how to handle expressions that mix dot products with the operations you learned in the last lesson.

Watch the examples that solve for a · (b + c) and (3a) · b, from this timestamp. This is great practice for putting all the vector arithmetic you've learned so far together.

Conclusion

In this lesson, we focused entirely on the calculation of the dot product. You now have the foundational skill to compute this essential quantity.

Here are the main takeaways:

The dot product of two vectors is found by multiplying their corresponding elements and summing the results.
The two vectors must have the same number of dimensions.
The result of a dot product is always a scalar (a single number), not a vector.
This operation is a workhorse in AI, performed efficiently by functions like numpy.dot() in Python.

We've focused on the "how," but we haven't yet explored the "why." What does the number 38 or 7 that we calculated actually mean? The true power of the dot product lies in its geometric interpretation. In our next lesson, we will uncover how this simple calculation reveals the relationship between two vectors, telling us about the angle between them and how much one vector projects onto the other. This is where the dot product transitions from a mere calculation to a powerful tool for understanding data.

Recommended resources

How to Calculate the Dot Product

Worked Examples: From 2D to 3D

The Dot Product in AI and Code

Combining Operations

Conclusion