Matrix Multiplication and Dimension Checking

Welcome back. In the previous lesson, a dot product combined one workspace’s feature vector with one vector of weights to produce a single score. Matrix multiplication is the same operation scaled up: it applies compatible collections of weighted sums in an organized way.

This lesson focuses on two habits that prevent most linear-algebra mistakes:

Check dimensions before calculating.
Compute each output entry as a row–column dot product.

By the end, you will be able to multiply small matrices manually and recognize the matrix operation behind producing predictions for many trial workspaces at once. Plan for roughly 40 minutes, including the short video and reading.

Matrix dimensions are part of the meaning

A matrix is a rectangular array. Its shape is written:

rows \times columns

For example, a matrix with 2 rows and 3 columns has shape $2 \times 3$ :

A = [132104]

The dimensions are not merely bookkeeping. In a data-science setting, they usually represent distinct kinds of things:

rows of a feature matrix: observations, such as workspaces;
columns of a feature matrix: features, such as active days or teammates invited;
rows of a coefficient matrix: input features;
columns of a coefficient matrix: outputs, scores, or latent units.

The general matrix-multiplication pattern is:

m \times n A n \times p B = m \times p C

The two inner dimensions, $n$ and $n$ , must match. If they do, the product exists; the outer dimensions determine the output shape.

A compact memory aid is:

Inside dimensions match; outside dimensions remain.

For instance:

(3 \times 4) (4 \times 6) = 3 \times 6

This is valid because the inner dimensions are both $4$ . In contrast,

(3 \times 2) (3 \times 5)

is undefined: the first matrix has $2$ columns, but the second has $3$ rows.

The order matters. Matrix multiplication is generally not commutative: $A B$ is not assumed to equal $B A$ . Indeed, one order may be valid while the reverse order is invalid; and even if both products exist, their shapes and values may differ.

Why Matrix Multiplication Uses Rows & Columns (The Logic Explained)

Watch “Why Matrix Multiplication Uses Rows & Columns (The Logic Explained)” by Krista King. It gives a useful operational intuition: matrix multiplication chains two systems by connecting a shared intermediate category.

Watch the bakery example, focusing on how products connect to ingredients and ingredients connect to vendors. Then watch the dimension rule for the inside-match, outside-remain check. Finish with the worked calculations, noticing that every output cell comes from one row of the left matrix and one column of the right matrix.

The bakery example illustrates why the shared dimension must match. If the columns of the first matrix describe ingredients, then the rows of the second must describe prices for those same ingredients in the same order. Compatible dimensions allow the arithmetic, but semantic compatibility makes the result meaningful.

That distinction matters in production data work. Two arrays can have compatible shapes while representing mismatched feature orders. The code may run successfully and still calculate the wrong predictions.

What a product entry means: row by column

Suppose

A = [132104] and B = 205 13 - 1

Before doing arithmetic, inspect shapes:

A : 2 \times 3 B : 3 \times 2

The inner dimensions match, so $A B$ is defined. The answer must have the outer dimensions:

A B : 2 \times 2

Write an empty result matrix with that shape:

C = A B = [□ □ □ □]

Each entry of $C$ is a dot product:

take a row from $A$ ;
take a column from $B$ ;
multiply corresponding entries;
add the products.

The top-left entry uses row 1 of $A$ and column 1 of $B$ :

c_{11} = [120] 205

c_{11} = 1 (2) + 2 (0) + 0 (5) = 2

The top-right entry uses the same first row of $A$ , but column 2 of $B$ :

c_{12} = 1 (1) + 2 (3) + 0 (- 1) = 7

Now use the second row of $A$ :

c_{21} = 3 (2) + 1 (0) + 4 (5) = 26

c_{22} = 3 (1) + 1 (3) + 4 (- 1) = 2

Therefore,

A B = [22672]

In general, if $A$ has shape $m \times n$ and $B$ has shape $n \times p$ , then the entry in row $i$ , column $j$ of $C = A B$ is:

c_{ij} = k = 1 \sum n a_{ik} b_{k j}

The index $k$ runs across the shared dimension. This formula formalizes “row $i$ of $A$ dotted with column $j$ of $B$ .”

5.4 - A Matrix Formulation of the Multiple Regression Model | STAT 462

Read the “Matrix multiplication” portion of this Penn State STAT 462 page. It connects the dimension rule and row-by-column calculation directly to the regression expression X\beta, which will be central throughout this course.

In the “Matrix multiplication” section, begin with the statement that multiplication is restricted by matrix shapes. Read the dimension rule. Then continue through the numerical example immediately below, including the calculations of c_{11}, c_{12}, and c_{23}. Track two things: predict the shape before calculating, and identify precisely which row and column produce each displayed entry.

From one prediction to many predictions

The dot product from the previous lesson is already a matrix multiplication:

1 \times d [w_{1} w_{2} \dots w_{d}] d \times 1 x_{1} x_{2} ⋮ x_{d} = 1 \times 1 score

A dataset simply stacks many feature vectors together as rows.

Suppose there are $n$ trial workspaces and $d$ features. The design matrix $X$ has shape $n \times d$ :

X = — workspace 1 features — — workspace 2 features — ⋮ — workspace n features —

For a single linear model, collect the $d$ feature weights in a column vector:

β = β_{1} β_{2} ⋮ β_{d}

Its shape is $d \times 1$ . Thus:

n \times d X d \times 1 β = n \times 1 y^

The output has one row per workspace:

y^= y^_{1} y^_{2} ⋮ y^_{n}

Each row of $X$ is dotted with the same coefficient vector $β$ . So matrix multiplication means: apply the same weighted-sum model to every workspace at once.

Consider three workspaces and the three features from the previous lesson:

X = 426305213 β = 2.0 0.8 1.5

Dimension check:

(3 \times 3) (3 \times 1) = 3 \times 1

The predictions without an intercept are:

X β = 4 (2.0) + 3 (0.8) + 2 (1.5) 2 (2.0) + 0 (0.8) + 1 (1.5) 6 (2.0) + 5 (0.8) + 3 (1.5) = 13.4 5.5 20.5

If the model has intercept $b = 5$ , add it to each score:

y^= X β + 555 = 18.4 10.5 25.5

The matrix multiplication does not itself turn these scores into probabilities. Later, logistic regression will apply a nonlinear transformation to scores of this form. But the dimensional logic of $X β$ remains the same.

Implementing the shape contract in NumPy

For model code, let dimensions make the intended calculation obvious:

import numpy as np

X = np.array([
    [4.0, 3.0, 2.0],
    [2.0, 0.0, 1.0],
    [6.0, 5.0, 3.0],
])  # shape: (3, 3)

beta = np.array([
    [2.0],
    [0.8],
    [1.5],
])  # shape: (3, 1)

b = 5.0

linear_scores = X @ beta
predictions = linear_scores + b

print(X.shape)             # (3, 3)
print(beta.shape)          # (3, 1)
print(linear_scores.shape) # (3, 1)
print(predictions)

The expression X @ beta means matrix multiplication. It is not the same as X * beta, which is elementwise multiplication under NumPy’s broadcasting rules.

A useful explicit validation is:

assert X.ndim == 2
assert beta.ndim == 2
assert X.shape[1] == beta.shape[0]

The final assertion tests the formal multiplication rule: number of columns in $X$ equals number of rows in beta.

In typical scikit-learn usage, coefficients are often stored as a one-dimensional array of shape (d,), rather than a $d \times 1$ column matrix. Then this is also valid:

beta_1d = np.array([2.0, 0.8, 1.5])  # shape: (3,)

scores = X @ beta_1d
print(scores.shape)  # (3,)

Numerically, this represents the same three scores. The distinction is mainly one of representation:

(d, 1) makes the column-vector mathematics explicit and yields (n, 1);
(d,) is convenient in NumPy and yields a one-dimensional array (n,).

In a real feature pipeline, shape checks are necessary but not sufficient. You also need a feature-order contract. If beta[0] belongs to active_days, then the first column of X must also be active_days.

A safer pattern is to define the order once and enforce it:

feature_names = [
    "active_days",
    "teammates_invited",
    "key_actions",
]

X = df.loc[:, feature_names].to_numpy()

This protects against a particularly dangerous failure mode: a valid $n \times d$ matrix multiplied by a valid $d \times 1$ coefficient vector whose columns and coefficients refer to different features. Dimensions would pass; the model would not be meaningful.

Key takeaways

Matrix multiplication has a precise shape rule:

(m \times n) (n \times p) = m \times p

The inner dimensions must match. The outer dimensions give the shape of the result.

To calculate a product entry, take a row from the left matrix and a column from the right matrix, then compute their dot product:

c_{ij} = k = 1 \sum n a_{ik} b_{k j}

For predictive modeling, the central application is:

n \times d X d \times 1 β = n \times 1 y^

This applies one weighted-sum model to $n$ observations simultaneously. Before trusting such a calculation, check both the numerical shapes and the semantic alignment of feature columns with coefficient entries.

Next, you will use matrices to solve a small system of linear equations and see why singular systems correspond to redundant information in a feature matrix.

Recommended resources

Matrix dimensions are part of the meaning

What a product entry means: row by column

From one prediction to many predictions

Implementing the shape contract in NumPy

Key takeaways