Math for Machine Learning

Perform matrix addition, scalar multiplication, and matrix multiplication, and identify their properties.

Represent a system of linear equations as a matrix equation (Ax=b) and describe the conditions for a solution's existence based on the matrix's properties.

Compute the determinant of a 2x2 or 3x3 matrix and interpret it as the scaling factor of a linear transformation's volume.

Compute the inverse of a 2x2 matrix and describe the conditions under which a matrix is invertible.

Define the rank of a matrix and its relationship to the dimension of the column space.

Define a linear transformation and represent it as a matrix.

Describe the geometric effects of standard linear transformations, such as rotation, scaling, and projection.

Define the column space and null space of a matrix as the fundamental subspaces of a linear transformation.

Find a basis for the column space and null space of a matrix.

Define eigenvalues and eigenvectors and compute them for a 2x2 matrix.

Interpret eigendecomposition geometrically as revealing the principal axes of a linear transformation.

Define matrix diagonalization and perform eigendecomposition on a diagonalizable matrix.

Define orthogonality and orthonormality of vectors and matrices.

Compute the orthogonal projection of a vector onto another vector or a subspace.

Define Singular Value Decomposition (SVD) and describe the properties of its constituent matrices (U, Σ, V^T).

Explain the relationship between SVD and the eigendecomposition of A^T*A and A*A^T.

Interpret SVD geometrically as a composition of rotation, scaling, and another rotation.

Apply SVD to compute low-rank approximations of a matrix, a core technique in interpretability.

Compute partial derivatives of multivariable functions.

Define and compute the gradient of a scalar-valued multivariable function.

Define and compute the Jacobian matrix of a vector-valued function.

Apply the multivariable chain rule to compute gradients of composed functions, forming the basis of backpropagation.

Define the Hessian matrix and relate it to the curvature of a function's surface.

Find critical points of a multivariable function and classify them using the Hessian's properties.

Describe the gradient descent algorithm, including the role of the learning rate.

Apply a single step of gradient descent to minimize a simple multivariable function.

Differentiate between discrete and continuous random variables and their distributions (PMF/PDF).

Define conditional probability and apply Bayes' theorem to update beliefs.

Compute the expectation and variance of a random variable from its distribution.

Describe the key properties of the Gaussian distribution.

Define joint, marginal, and conditional distributions for multivariate random variables.

Define covariance and correlation, and interpret the components of a covariance matrix.

Define and compute the Shannon entropy of a discrete probability distribution, interpreting it as a measure of uncertainty.

Define and compute cross-entropy between two distributions as a measure of prediction error.

Define and compute Kullback-Leibler (KL) divergence, interpreting it as a measure of the difference between two probability distributions.