Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
Math for Deep Learning
·
Module 6
Estimation and Inference
1
MLE as Loss Function Minimization
Formulate maximum likelihood estimation (MLE) and connect it to loss function minimization (e.g., MSE for Gaussian, cross-entropy for Categorical).
2
MAP Estimation and Regularization Priors
Formulate maximum a posteriori (MAP) estimation and interpret L2 and L1 regularization as Gaussian and Laplace priors, respectively.
3
Monte Carlo Estimation and Variance Reduction
Use Monte Carlo methods to approximate intractable expectations and quantify the variance of the resulting estimators.
4
Pathwise Gradient Estimator for Stochastic Operations
Derive the pathwise (reparameterization) gradient estimator to differentiate through stochastic sampling operations.
5
Understanding Entropy and Divergence
Define entropy, cross-entropy, and Kullback-Leibler (KL) divergence, and compute them for discrete and Gaussian distributions.
6
Deriving and Interpreting the Evidence Lower Bound (ELBO)
Derive the Evidence Lower Bound (ELBO) using Jensen's inequality and interpret its reconstruction and regularization terms.
7
KL Divergence in Variational Inference: Mode-Seeking vs. Mode-Covering
Contrast the behavior of forward and reverse KL divergence in variational inference, explaining mode-seeking vs. mode-covering properties.
Next module isn't ready yet,
Back to course
Previous module
Probability and Statistics for Generative Models
Next module