Matrix Operations: Manual and NumPy

1. What is a Matrix?

Just as a vector is an ordered list of numbers (a 1D array), a matrix is a rectangular array of numbers arranged into rows and columns (a 2D array). A matrix with $m$ rows and $n$ columns is said to have dimensions or size $m\times n$.

For a quick and clear introduction to what a matrix is, how to describe its size, and how to reference its individual elements, please watch the first few minutes of the following video.

2. Manual Matrix Operations

Just like with numbers, we can perform arithmetic operations on matrices. However, there are specific rules that must be followed.

a) Addition, Subtraction, and Scalar Multiplication

These are the most straightforward operations.

• Addition and Subtraction: To add or subtract two matrices, they must have the exact same dimensions. The operation is performed element-wise, meaning you add or subtract the corresponding elements in each position.
• Scalar Multiplication: To multiply a matrix by a scalar (a single number), you simply multiply every single element in the matrix by that scalar. The dimensions of the matrix do not change.

Let's watch a video that illustrates these basic operations.

b) Matrix Multiplication

Matrix multiplication is more complex than addition. It is not an element-wise operation. There are two critical rules to remember:

1. Dimension Compatibility: To multiply matrix $A$ by matrix $B$ (i.e., $A\times B$), the number of columns in $A$ must equal the number of rows in $B$.

   • If $A$ is an $m\times n$ matrix and $B$ is an $n\times p$ matrix, you can multiply them.
   • The resulting matrix, $C=A\times B$, will have dimensions $m\times p$.

2. The Calculation Process: Each element in the resulting matrix $C$ is calculated by taking the dot product of a row from matrix $A$ and a column from matrix $B$.

   • Specifically, the element $C_{ij}$ (in row $i$, column $j$) is the dot product of the $i$-th row of $A$ and the $j$-th column of $B$.

The following image provides a fantastic visual breakdown of this process for two $2\times 2$ matrices.

Now, let's watch a detailed video explanation of this process.

A crucial property of matrix multiplication is that it is not commutative, which means in general, $A\times B\ne B\times A$. In fact, if $A\times B$ is a valid operation, $B\times A$ might not even be possible if the dimensions don't align correctly.

For a more formal mathematical definition of matrix multiplication, which you might appreciate from your engineering studies, you can refer to the resource below. It presents the operation using summation notation.

3. Matrix Operations with NumPy

Now that you understand the theory, let's put it into practice. As you saw with vectors, NumPy is the tool of choice for numerical operations in Python.

First, let's represent a matrix in NumPy. We do this by creating a 2D array, which is simply a list of lists.

import numpy as np

# A 2x3 matrix
A = np.array([
    [1, 2, 3],
    [4, 5, 6]
])

# A 3x2 matrix
B = np.array([
    [7, 8],
    [9, 10],
    [11, 12]
])

Addition, Subtraction, and Element-wise Multiplication

NumPy's operators + and - work exactly as you'd expect, performing element-wise addition and subtraction.

The * operator, however, performs element-wise multiplication, not matrix multiplication. This is an important distinction! Element-wise multiplication requires the matrices to have the same dimensions.

True Matrix Multiplication

For true matrix multiplication (the row-by-column dot product we learned), NumPy provides two main ways:

1. The np.matmul() function.
2. The @ operator (introduced in Python 3.5), which is a convenient shorthand.

Let's watch a video that demonstrates all of these operations in NumPy.

Code Example

Let's put this all together in a single code block.

import numpy as np

# Define two 2x2 matrices
A = np.array([[1, 2], 
              [3, 4]])

B = np.array([[5, 6], 
              [0, -1]])

print("Matrix A:\n", A)
print("Matrix B:\n", B)

# Addition
C_add = A + B
print("\nA + B:\n", C_add)

# Scalar Multiplication
C_scalar_mult = 2 * A
print("\n2 * A:\n", C_scalar_mult)

# Element-wise multiplication (NOT matrix multiplication)
C_elementwise = A * B
print("\nElement-wise A * B:\n", C_elementwise)

# True Matrix Multiplication
C_matmul = np.matmul(A, B)
# Or using the @ operator
C_matmul_operator = A @ B

print("\nMatrix Multiplication A @ B (using np.matmul):\n", C_matmul)
print("\nMatrix Multiplication A @ B (using @):\n", C_matmul_operator)

# Note that A @ B is not the same as B @ A
C_ba = B @ A
print("\nMatrix Multiplication B @ A:\n", C_ba)

Compare the output for A @ B with the result you calculated manually. They should match!

Conclusion

In this lesson, we demystified the fundamental operations of matrix algebra. You now have the skills to manipulate matrices both by hand and programmatically, a core competency for anyone working in data science.

Key Takeaways:

• Matrix Addition/Subtraction: Requires matrices of the same dimensions and is performed element-wise. In NumPy, this is done with the + and - operators.
• Scalar Multiplication: A scalar is multiplied with every element of the matrix.
• Matrix Multiplication: The number of columns in the first matrix must equal the number of rows in the second. The result is calculated via row-by-column dot products.
• NumPy Implementation: True matrix multiplication is done using np.matmul(A, B) or the A @ B operator. The A * B operator performs element-wise multiplication.
• Non-Commutativity: Remember, $A\times B\ne B\times A$.

In our next lesson, we will apply these new skills to a very practical problem: solving systems of linear equations using Gaussian elimination. You'll see how a system of equations like 2x + 3y = 8 can be represented and solved efficiently using matrices.