Mathematical Toolkit for Deep Learning

Derive the chain rule for a multi-layered function with vector inputs, demonstrating its application to backpropagation.

Explain eigenvalues/eigenvectors and Singular Value Decomposition (SVD) and their relevance to model analysis and low-rank approximation.

Explain the probabilistic interpretation of common loss functions like Cross-Entropy and their connection to maximum likelihood estimation.

Explain the mechanics of automatic differentiation and the structure of dynamic computational graphs in modern ML frameworks.

Implement a custom PyTorch Autograd function to manually define a backward pass for a non-standard operation.

Model a simple time-series forecasting problem by defining the input/output sequence structures required for a recurrent model.

Explain the mechanics of a 2D convolution operation, detailing the roles of kernels, padding, and stride in transforming an input tensor.