Vectors in R^n: Operations and Properties

Recommended resources

The freely available resources this lesson will direct you to.

Welcome to the first lesson of your journey into the mathematical foundations of artificial intelligence and computer graphics. Your goal is to master these fields from first principles, and there is no better place to start than with the fundamental atom of this entire domain: the vector. Everything from the position of a vertex in a 3D model, to a data point in a machine learning problem, to the forces in a physics simulation is described using vectors.

In this lesson, we will establish a solid, first-principles understanding of vectors. We will define precisely what a vector is, exploring both its geometric and numerical representations. Then, we will master the two foundational operations that allow us to manipulate them: addition and scalar multiplication. These concepts form the bedrock upon which all of linear algebra—and consequently, much of modern AI and graphics—is built.

What is a Vector?

To begin, it's essential to understand that the term "vector" can be interpreted in a few different ways, depending on the context. For our purposes in computer science, graphics, and machine learning, we will primarily focus on two complementary perspectives: vectors as arrows in space (the geometric view) and vectors as lists of numbers (the numerical or coordinate view).

The ability to fluidly translate between these two perspectives is what makes linear algebra such a powerful tool. A deep understanding of this duality will allow you to visualize the effect of your code and, conversely, to translate geometric ideas into precise, computable instructions.

The following video from the YouTube channel 3Blue1Brown provides an excellent introduction to this dual nature. Please watch the first four minutes to ground your intuition.

Vectors | Chapter 1, Essence of linear algebra

This video, "Vectors | Chapter 1, Essence of linear algebra," introduces the different perspectives on vectors and establishes the core intuition we will build upon.

Watch from the beginning until the end of the coordinate system explanation. Pay close attention to the distinction between the "physics student" view (arrows that can move freely) and the convention in linear algebra, where vectors are typically rooted at the origin.

As the video highlights, we can formally represent a vector in $n$ -dimensional space, denoted $R^{n}$ , as an ordered list (or "n-tuple") of $n$ real numbers. These numbers are the vector's components or coordinates. For example, a vector $v$ in 3D space ( $R^{3}$ ) can be written as:

v = v_{1} v_{2} v_{3}

Here, $v_{1}$ , $v_{2}$ , and $v_{3}$ are the coordinates along the x, y, and z axes, respectively. This column format is the standard convention in computer graphics and machine learning texts, as it interacts cleanly with matrices, which we will see in a future lesson.

The Fundamental Operations: Addition and Scalar Multiplication

All of the complex transformations and analyses we'll perform in this course are built from two simple operations: adding two vectors together and multiplying a vector by a number (a scalar).

Geometric Interpretation

Let's first build our intuition geometrically.

Vector Addition: To add two vectors $v$ and $w$ , we use the tip-to-tail method. We move the tail of $w$ to the tip of $v$ . The sum, $v + w$ , is the new vector drawn from the origin (the tail of $v$ ) to the tip of the repositioned $w$ . You can think of this as following one displacement, then another, with the sum being the net displacement.
Scalar Multiplication: Multiplying a vector $v$ by a scalar $c$ scales its length. If $c$ is positive, the vector's direction remains the same, but its length is multiplied by $c$ . If $c$ is negative, the vector's direction is reversed, and its length is scaled by the absolute value of $c$ .

The image below provides a clean visual summary of these two geometric operations.

This image illustrates vector addition using the tip-to-tail method for $\vec{v} + \vec{w}$ and scalar multiplication for $2\vec{v}$, which doubles the length of $\vec{v}$.

Algebraic Definition

The geometric intuition has a direct and simple numerical counterpart. These operations are performed component-wise.
Let $v = [v_{1} v_{2}]$ and $w = [w_{1} w_{2}]$ be two vectors in $R^{2}$ .

Vector Addition:
$v + w = [v_{1} v_{2}] + [w_{1} w_{2}] = [v_{1} + w_{1} v_{2} + w_{2}]$
Scalar Multiplication: Let $c$ be a scalar.
$c v = c [v_{1} v_{2}] = [c v_{1} c v_{2}]$

To see how these two views connect, let's return to the 3Blue1Brown video.

Vectors | Chapter 1, Essence of linear algebra

This segment of the video explicitly connects the geometric tip-to-tail method to the algebraic component-wise addition and also visualizes scalar multiplication.

Please watch from this section on vector operations. Notice how moving along the components of the first vector and then the second is equivalent to moving along the components of the sum vector.

A Question of Subtraction

We have defined addition and scalar multiplication. Using only these two operations, how would you define vector subtraction, $v - w$ ? What would this look like geometrically?

Think about how subtraction relates to addition and negative numbers in standard arithmetic. The same principle applies here.

Vector subtraction $v - w$ is defined as the addition of $v$ and the negative of $w$ :

v - w = v + (- 1) w

Geometrically, this means we take the vector $w$ , reverse its direction (by multiplying by -1), and then add the resulting vector to $v$ using the tip-to-tail method. Another useful visualization is that $v - w$ is the vector that points from the tip of $w$ to the tip of $v$ . This interpretation will become very useful in graphics for finding the vector between two points.

Points vs. Vectors: A Crucial Distinction

Your interest in graphics and low-level engineering demands precision. A common point of confusion is the difference between a point and a vector. A point specifies a location in space. A vector specifies a direction and magnitude (a displacement), but has no fixed location.

Point + Vector = Point: Moving a point by a displacement vector results in a new point.
Point - Point = Vector: The difference between two points is the displacement vector needed to get from one to the other.
Vector + Vector = Vector: Combining two displacements results in a net displacement.

However, an operation like Point + Point is geometrically meaningless. Adding two locations doesn't produce a logical third location. This distinction is subtle but fundamental to writing correct and robust geometric code.

The following reading from a University of Kansas tutorial on vector geometry for computer graphics provides a rigorous yet intuitive explanation of this topic.

Vector Geometry for Computer Graphics

This paper by J.R. Miller formalizes the distinction between points and vectors, explaining why certain operations are valid while others are not. This is a key "first principles" insight for anyone serious about computer graphics.

Please read the sections titled "Points and vectors" and "Spaces". Start from the beginning of the section and focus on the example in Figure 1 that demonstrates why adding points is ill-defined. Then, continue reading through the formal definitions of a vector space and an affine space, up to the sentence which concludes the argument. This will clarify which combinations of points and vectors are mathematically sound.

Conclusion

In this lesson, we established our first and most critical building block. You learned to define a vector both as a geometric arrow and as a numerical list of coordinates in $R^{n}$ . We then explored the two fundamental operations, vector addition and scalar multiplication, from both perspectives and saw how they are intrinsically linked. Finally, we drew a sharp distinction between points and vectors, a piece of formal knowledge that is essential for robust geometric programming.

These simple operations are the foundation of everything that follows. In our next lesson, we will introduce a new operation: the dot product. The dot product will unlock the geometric concepts of length, angle, and projection, allowing us to ask much more interesting questions, such as "How much is this vector pointing in the same direction as that one?"—a question at the heart of lighting calculations in graphics and similarity measures in machine learning.