Matrix Operations with NumPy

1. The numpy.linalg Module: Your Linear Algebra Toolkit

NumPy, which you're already familiar with for creating and manipulating arrays, has a powerful submodule dedicated to linear algebra: numpy.linalg. This module contains highly optimized functions for most of the standard linear algebra operations.

To get a quick overview of what's inside, let's look at an introductory resource.

As you can see, NumPy provides a straightforward, function-based approach to performing complex matrix calculations. Let's dive into using them.

2. Determinant and Inverse

Remember that the determinant is a scalar value that tells us important properties about a square matrix, most notably whether it's invertible. A non-zero determinant means an inverse exists. The inverse of a matrix $A$, denoted $A^{-1}$, is the matrix that, when multiplied by $A$, yields the identity matrix.

NumPy makes calculating these trivial.

• Determinant: np.linalg.det(A)
• Inverse: np.linalg.inv(A)

Let's see this in action. The following resource shows clear, simple examples for both.

Now, let's combine these concepts in a single code block. We'll define a matrix, calculate its determinant, and then, if possible, find its inverse.

import numpy as np

# A non-singular (invertible) matrix
A = np.array([[4, 7], 
              [2, 6]])

# A singular (non-invertible) matrix
B = np.array([[2, 4], 
              [3, 6]])

# --- Calculate Determinant ---
det_A = np.linalg.det(A)
det_B = np.linalg.det(B)

print(f"Matrix A:\n{A}")
print(f"Determinant of A: {det_A:.2f}") # Using f-string to format output
print("-" * 20)
print(f"Matrix B:\n{B}")
print(f"Determinant of B: {det_B:.2f}")
print("-" * 20)

# --- Calculate Inverse ---
print("Calculating inverse of A...")
inv_A = np.linalg.inv(A)
print(f"Inverse of A:\n{inv_A}")

# Let's verify that A @ A_inv is the identity matrix
identity = A @ inv_A
print(f"\nVerification (A @ A_inv):\n{np.round(identity, 2)}") # Round to handle tiny floating-point errors
print("-" * 20)

# --- Attempt to invert a singular matrix ---
print("Attempting to calculate inverse of B...")
try:
    inv_B = np.linalg.inv(B)
    print(f"Inverse of B:\n{inv_B}")
except np.linalg.LinAlgError as e:
    print(f"Could not compute inverse of B. Reason: {e}")

Notice two key things in the code above:

1. The determinant of A is (4*6) - (7*2) = 24 - 14 = 10, which is non-zero. The determinant of B is (2*6) - (4*3) = 12 - 12 = 0.
2. Trying to compute the inverse of the singular matrix B results in a LinAlgError. This is NumPy's way of telling you the operation is mathematically impossible.

3. Eigenvalues and Eigenvectors

As we learned, for a square matrix $A$, an eigenvector is a special vector that, when transformed by $A$, only changes in scale, not direction. The scaling factor is the eigenvalue. This is captured by the equation $A\mathbf{v}=\lambda \mathbf{v}$.

In NumPy, we can find both the eigenvalues and the corresponding eigenvectors using a single function: np.linalg.eig(A).

This function returns a tuple of two arrays:

1. A 1D array of eigenvalues ($\lambda$).
2. A 2D array where each column is the eigenvector ($\mathbf{v}$) corresponding to the eigenvalue at the same index in the first array.

Let's review a resource that demonstrates this.

Let's write our own code to demonstrate this and, more importantly, to verify the fundamental relationship $A\mathbf{v}=\lambda \mathbf{v}$.

import numpy as np

A = np.array([[1, -3, 3],
              [3, -5, 3],
              [6, -6, 4]])

print(f"Matrix A:\n{A}\n")

# Compute eigenvalues and eigenvectors
eigenvalues, eigenvectors = np.linalg.eig(A)

print(f"Eigenvalues (lambda):\n{eigenvalues}\n")
print(f"Eigenvectors (v) [as columns]:\n{eigenvectors}\n")

# --- Verification ---
# Let's check the relationship for the first eigenvalue/eigenvector pair.
lambda_1 = eigenvalues[0]
v_1 = eigenvectors[:, 0]  # Get the first column as the first eigenvector

# Calculate A @ v
Av = A @ v_1

# Calculate lambda * v
lambda_v = lambda_1 * v_1

print("-" * 20)
print("Verifying A*v = lambda*v for the first pair:\n")
print(f"A @ v_1 = {np.round(Av, 2)}")
print(f"lambda_1 * v_1 = {np.round(lambda_v, 2)}\n")

# np.allclose() is the best way to check if two arrays are element-wise equal within a tolerance
print(f"Are they close? {np.allclose(Av, lambda_v)}")

This verification step is crucial. It confirms that the values returned by NumPy are not just abstract numbers, but that they correctly satisfy the defining equation of eigenvalues and eigenvectors.

4. Singular Value Decomposition (SVD)

In the previous lesson, we explored the theory of SVD, which decomposes any matrix $A$ (even non-square ones) into the product of three other matrices: $A=U\Sigma V^T$.

We also had a first look at the numpy.linalg.svd() function. Now, let's solidify that practical knowledge. As a refresher, the function returns three components:

• U: The matrix of left singular vectors.
• s: A 1D array of the singular values (the diagonal of $\Sigma$).
• Vt: The transposed matrix of right singular vectors ($V^T$).

To reinforce how this works in a real coding environment, let's revisit the practical implementation part of the video from our last lesson.

One of the most important practical steps with np.linalg.svd is reconstructing the full $\Sigma$ matrix from the 1D array of singular values s so you can verify the decomposition.

Here is a complete example that puts it all together:

import numpy as np

# A rectangular matrix
A = np.array([[1, 1, 0],
              [0, 1, 1]])

print(f"Original Matrix A (shape {A.shape}):\n{A}\n")

# Perform SVD
U, s, Vt = np.linalg.svd(A)

print(f"U (shape {U.shape}):\n{np.round(U, 2)}\n")
print(f"Singular values (s): {np.round(s, 2)}\n")
print(f"Vt (shape {Vt.shape}):\n{np.round(Vt, 2)}\n")

# To reconstruct A, we must build the Sigma matrix.
# It has the same shape as A, with the singular values on its diagonal.
Sigma = np.zeros(A.shape)
# The singular values go on the diagonal of a square sub-matrix
Sigma[:min(A.shape), :min(A.shape)] = np.diag(s)

print(f"Reconstructed Sigma (shape {Sigma.shape}):\n{np.round(Sigma, 2)}\n")

# Reconstruct the original matrix
A_reconstructed = U @ Sigma @ Vt

print(f"Reconstructed Matrix A:\n{np.round(A_reconstructed, 2)}\n")
print(f"Verification successful? {np.allclose(A, A_reconstructed)}")

Being able to decompose a matrix with svd and then correctly reconstruct it demonstrates a complete understanding of the practical workflow, which is a key skill for many data science applications like recommendation systems and dimensionality reduction.

Conclusion

Excellent work! You have now successfully completed the Linear Algebra module. You've journeyed from basic vector and matrix operations to advanced decomposition techniques, and most importantly, you've now translated that theory into practical coding skills with NumPy.

Key Takeaways:

• The numpy.linalg module is the primary tool for performing linear algebra operations in Python.
• Determinant: np.linalg.det() computes the determinant of a square matrix. A non-zero result indicates the matrix is invertible.
• Inverse: np.linalg.inv() computes the inverse of an invertible matrix. It will raise a LinAlgError for a singular (non-invertible) matrix.
• Eigen-decomposition: np.linalg.eig() returns the eigenvalues and corresponding eigenvectors of a square matrix.
• SVD: np.linalg.svd() decomposes any matrix into $U$, s (singular values), and $V^T$, a cornerstone for many advanced algorithms.

You are now equipped to handle the linear algebra problems you will encounter in your GATE preparation and beyond.

Preview of the Next Lesson:

We are now moving on to the next major pillar of mathematics for data science: Calculus. In our next lesson, we will begin the "Calculus for Optimization" module by learning how to compute derivatives of polynomial, exponential, logarithmic, and trigonometric functions. This will be the first step toward understanding how machine learning models "learn" by minimizing error, a process driven by calculus.