Welcome to your course on the essential mathematics for AI. To understand how artificial intelligence models learn from data, we first need to understand the language they use to "see" the world: the language of mathematics. This journey begins with the most fundamental building blocks of data representation.
In this first lesson, we will explore scalars and vectors. You will learn what these terms mean and, most importantly, how they are used to translate real-world information—like a person's profile, the features of a house, or even an image—into a format that a computer can process. By the end of our session, you will be able to define scalars and vectors specifically within the context of representing data for AI.
What is a Scalar?
Let's begin with the simplest element: the scalar.
A scalar is just a single number. It's a quantity that has magnitude but no direction. Think of a person's age (e.g., 30), the temperature outside (e.g., 22.5°C), or the price of an item (e.g., $99.99). In machine learning, the individual attributes or 'features' of a data point are often scalars. For instance, if we're analyzing housing data, the square footage of a single house is a scalar value.
To get a clear definition and some practical examples, please read the following brief section from a LinkedIn article on data representation.
Data Representation in ML: Scalars, Vectors, Matrices & Sets
This section from the article "Data Representation in ML" provides a concise, practical definition of a scalar in the context of machine learning.
Please read the short section titled "1) Scalars". Notice how it connects this simple mathematical idea to a concrete component of a dataset.
While we can think of a scalar as simply "a number," the term itself comes from the world of linear algebra. Here, numbers are called scalars because one of their primary roles is to scale other objects, like vectors. Imagine you have an arrow of a certain length; multiplying it by the scalar '2' would stretch it to twice its original length. We'll explore this "scaling" operation in our next lesson.
What is a Vector?
Now, let's move on to vectors. If scalars are the individual letters of our mathematical language, vectors are the words they form. A vector is an ordered list of scalars.
There are two powerful ways to think about vectors, both of which are crucial for understanding their role in AI:
-
The Computer Science Perspective: A vector is an ordered list of numbers. Each number represents a value for a specific attribute or feature. The order matters. For example, a vector
[2100, 3, 2]could represent a house with 2100 square feet, 3 bedrooms, and 2 bathrooms. This is the most common way we use vectors to represent data. -
The Geometric Perspective: A vector is an arrow in a coordinate system, possessing both a length (magnitude) and a direction. In linear algebra, we almost always anchor the tail of this arrow at the origin (the point
[0,0]or[0,0,0]). The vector's coordinates then correspond to the position of its tip.
The following video from the "Essence of linear algebra" series provides an excellent visual introduction to these concepts. It clearly explains the different perspectives on vectors and how they relate to a coordinate system.
Vectors | Chapter 1, Essence of linear algebra
This video from the 3Blue1Brown channel is renowned for its outstanding visualizations that build deep intuition. Watching this segment will help you connect the abstract idea of a vector to both a list of numbers and a geometric arrow.
Please watch from the beginning until the end of the 3D explanation. Pay close attention to the distinction between the "computer science student perspective" (an ordered list of numbers) and the "physics student perspective" (an arrow in space), and see how they are unified in a coordinate system.
Vectors for Data Representation in AI
As the video highlighted, the computer science perspective is central to AI and machine learning. We represent a single data point or observation—be it a user, a product, or a house—as a feature vector. Each element in the vector corresponds to a specific feature of that data point.
The following readings further clarify this crucial link between vectors and data.
Data Representation in ML: Scalars, Vectors, Matrices & Sets
This reading builds on the scalar definition by explaining how vectors are constructed from them to represent a complete data record.
Please read the section titled "2) Vectors". The "real-world analogy" comparing a spreadsheet row to a vector is a particularly helpful way to think about it.
Essential Math for Data Science: Scalars and Vectors - KDnuggets
This article from KDnuggets reinforces the concept of vectors as containers for data and explicitly introduces the idea of using them with linear algebra tools.
First, read the two paragraphs under the heading "What are Vectors?". Then, skip down to the heading "Vectors in Data Science" and read that entire section.
Visualizing Data in Feature Space
The geometric view of vectors gives us a powerful way to visualize our data. A vector with two features can be drawn as a point or an arrow in a 2D plane. A vector with three features can be visualized in 3D space. Each axis in this space corresponds to one of the features. We call this the feature space.
Even though we can't visually imagine the 4-dimensional space needed for a 4-feature vector, the mathematical and conceptual logic remains the same. AI models operate in these "high-dimensional" spaces to find relationships between data points.
The following image provides a great summary of how scalars and vectors are represented, and it gives a sneak peek at matrices and tensors, which we'll cover in future lessons.
To see how this concept of high-dimensional vectors is applied in cutting-edge AI, watch this short video from Google. It demonstrates how complex data like words and images are turned into vectors that an AI can understand.
A.I. Experiments: Visualizing High-Dimensional Space
This video provides a conceptual, non-technical overview of how machine learning uses high-dimensional spaces to process complex data.
Watch from the start of the explanation to the end. Notice how very different types of data—human attributes, words, and handwritten digits—are all treated as data points (vectors) in a high-dimensional space.
Conclusion
In this lesson, we have taken our first step into the mathematics of AI. We have established the fundamental definitions of scalars and vectors and, crucially, seen how they serve as the bridge between raw information and the numerical world of machine learning.
Here are the key takeaways:
- Scalar: A single numerical value, representing a single quantity or measurement (e.g., age, price).
- Vector: An ordered list of scalars, typically representing a single data point or observation. Each element in the vector is a feature.
- Data Representation: In AI, we convert data into feature vectors. A house can become a vector of its characteristics; an image can become a vector of its pixel values.
- Feature Space: These vectors can be thought of as points in a multi-dimensional space, where each dimension corresponds to a feature. This geometric interpretation helps us reason about the relationships between data points.
In our next lesson, we will build directly on this foundation. Now that we know what vectors are, we'll learn how to perform basic mathematical operations on them, such as vector addition and scalar multiplication. These operations are the essential tools we'll need to start manipulating data and building simple models.