Matrix Calculus and Optimization

Compute the gradient of a scalar-valued function with respect to a vector or matrix.

Compute the Jacobian of a vector-valued function.

Apply the chain rule for derivatives involving vector and matrix operations.

Use the Hessian matrix of a scalar-valued function to classify critical points.

Relate the positive definiteness of the Hessian to the convexity of a function.

Formulate and apply the gradient descent algorithm to find the minimum of a simple multivariate function.

Apply matrix calculus to derive the gradient for a linear regression cost function.