AI, Graphics, and Research Mastery

Define vectors in R^n and perform vector addition and scalar multiplication.

Compute the dot product and explain its geometric interpretation in terms of projection and angle.

Compute matrix-vector and matrix-matrix multiplication, viewing matrices as transformations of vectors.

Implement matrix-vector and matrix-matrix multiplication from scratch in Python.

Determine the linear independence of a set of vectors and define span, basis, and dimension.

Compute the determinant of a matrix and interpret it as the scaling factor of area/volume.

Compute and compare L1, L2, and Frobenius norms for vectors and matrices.

Relate matrix transformations (rotation, scale, shear) to their application in 3D computer graphics.

Compute eigenvalues and eigenvectors of a matrix.

Interpret eigendecomposition geometrically as identifying the principal axes of a linear transformation.

Explain the geometric intuition of Singular Value Decomposition (SVD) as rotation, scaling, and rotation.

Derive SVD from the eigendecomposition of AᵀA.

Use SVD to perform low-rank approximation of a matrix.

Compute the orthogonal projection of a vector onto a subspace.

Reference for self-study: Identify Principal Component Analysis (PCA) as the eigendecomposition of a covariance matrix.

Compute partial derivatives of multivariable functions.

Compute the gradient of a scalar-valued function and interpret it as the direction of steepest ascent.

Apply the multivariable chain rule to compute derivatives of composite functions.

Compute the Jacobian matrix for vector-valued functions.

Compute the Hessian matrix and interpret its eigenvalues in terms of local curvature (convexity/concavity).

Construct first-order and second-order Taylor expansions to approximate multivariable functions.

Use the gradient and Hessian to classify critical points as local minima, maxima, or saddle points.

Apply axioms of probability, and rules for conditional, joint, and marginal probabilities.

Define discrete and continuous random variables, and their associated PMFs/PDFs.

Compute expectation, variance, and covariance for random variables.

Explain the Law of Large Numbers (LLN) and Central Limit Theorem (CLT) and their importance for sampling.

Apply Bayes' theorem to update beliefs given new evidence.

Derive the Maximum Likelihood Estimate (MLE) for the parameter of a Bernoulli distribution.

Define and interpret cross-entropy and Kullback-Leibler (KL) divergence as measures of difference between distributions.

Derive the closed-form Ordinary Least Squares (OLS) solution using matrix calculus.

Explain the computational trade-offs between closed-form and iterative optimization methods.

Derive the gradient descent update rule from a first-order Taylor approximation of the loss function.

Implement gradient descent for linear regression from scratch in Python.

Explain the role of the learning rate and diagnose its effect on convergence (divergence, slow convergence).

Derive Stochastic Gradient Descent (SGD) as a scalable approximation of batch gradient descent.

Derive the update rules for Momentum and RMSProp, explaining how they address challenges in SGD.

Combine Momentum and RMSProp to derive the Adam optimizer and implement it from scratch.

Describe the perceptron model and demonstrate its inability to solve non-linearly separable problems like XOR.

Define common activation functions (Sigmoid, Tanh, ReLU) and compute their derivatives.

Construct a feedforward neural network as a composition of affine transformations and non-linear activations.

Perform a forward pass for a multi-layer network to generate predictions.

Define the softmax function and its use in multi-class classification.

Derive the gradients of both Mean Squared Error (MSE) and Cross-Entropy loss functions with respect to the network's final output (logits).

Derive the full backpropagation algorithm by applying the chain rule recursively through network layers.

Explain the concepts of exploding and vanishing gradients.

Implement the forward and backward passes for a complete multi-layer network from scratch in Python.

Construct a training loop that uses computed gradients to update network parameters via an optimizer.

Diagnose overfitting and underfitting by analyzing training and validation loss/accuracy curves.

Derive L2 regularization (weight decay) and explain its effect on model parameters.

Describe the mechanisms of Dropout and Batch Normalization and their roles in regularization and training stabilization.

Explain the motivation behind weight initialization strategies like Xavier/Glorot and He initialization.

Implement a learning rate schedule (e.g., step decay) and explain its purpose.

Reference for self-study: Describe other classical regularization techniques like L1 and early stopping.

Relate the 2D convolution operation to classical image processing filters (e.g., sharpening, edge detection).

Explain the roles of filters (kernels), stride, and padding in a convolutional layer.

Describe pooling operations (max, average) and their role in creating spatial invariance.

Implement the forward and backward pass for a 2D convolution layer from scratch in Python.

Explain how a hierarchy of convolutional layers learns features from simple edges to complex objects.

Trace the architecture of a classic CNN like LeNet-5, calculating tensor shapes at each stage.

Explain the degradation problem in very deep networks.

Describe how residual connections (skip connections) in ResNet architectures mitigate degradation.

Describe the architecture of a Recurrent Neural Network (RNN) and explain its vanishing/exploding gradient problem.

Explain how LSTM and GRU gating mechanisms address the limitations of simple RNNs.

Define scaled dot-product attention and explain its function as a query-key-value lookup.

Derive self-attention as a mechanism for capturing long-range dependencies in parallel.

Describe the architecture and purpose of multi-head attention.

Trace the flow of data through a complete Transformer encoder block, identifying each sub-layer's role.

Implement scaled dot-product attention and a single self-attention head from scratch in Python.

Describe the autoencoder architecture and its use for dimensionality reduction and representation learning.

Explain the core idea of variational inference for approximating intractable posterior distributions.

Explain the reparameterization trick and why it is necessary for training VAEs with gradient descent.

Describe the Generative Adversarial Network (GAN) framework as a two-player minimax game.

Explain common GAN failure modes like mode collapse and training instability.

Describe the forward (diffusion) and reverse (denoising) processes in Denoising Diffusion Probabilistic Models (DDPMs).

Explain the simplified diffusion model training objective as a noise prediction task.

Define an implicit neural representation as a function that maps coordinates to physical properties (e.g., density, color).

Explain the concept of 'spectral bias' in MLPs and why it hinders the learning of high-frequency details.

Describe how positional encoding using Fourier features helps overcome spectral bias.

Derive the classical volumetric rendering equation for synthesizing an image from a ray's samples.

Describe the full NeRF architecture, including coarse/fine networks and hierarchical volume sampling.

Implement positional encoding for 3D coordinates and viewing directions in Python.

Implement the differentiable volume rendering function for a single ray from scratch in Python.

Decompose the original NeRF paper into its core problem, mathematical intuition, and architectural innovations.

Describe how a 3D scene is represented by a collection of explicit, anisotropic 3D Gaussian primitives.

Explain the mathematical process of projecting a 3D Gaussian onto a 2D image plane.

Compare and contrast the trade-offs of NeRF (implicit, slow render) vs. 3D Gaussian Splatting (explicit, fast render).

Decompose the 3D Gaussian Splatting paper into its problem statement, core method, and key results.

Apply a systematic framework to analyze a research paper: identify problem, distill mathematical intuition, and critique experiments.

Identify the limitations of a given research paper (e.g., NeRF, 3DGS).

Propose a concrete follow-up research question or potential improvement based on a paper's identified limitations.