Mathematical and Optimisation Foundations

Compute the gradient and Hessian of scalar-valued functions of several variables.

Compute Jacobians for vector-valued functions and apply the multivariate chain rule.

Differentiate common matrix expressions (e.g., x^T A x, ||Ax - b||^2) with respect to a vector.

Recall the Singular Value Decomposition (SVD) and relate it to the eigendecomposition of A^T A.

Identify convex functions and explain the role of convexity in optimisation.

Formulate a constrained optimisation problem and describe the method of Lagrange multipliers.

Implement vanilla gradient descent to minimise a convex function.

Analyse the effect of the learning rate on gradient descent convergence.