Foundations of Variational Inference

Derive the KL divergence between two Gaussian distributions and explain its asymmetry.

Derive the Evidence Lower Bound (ELBO) from the log-marginal likelihood using Jensen's inequality.

Explain the reparameterization trick and why it enables gradient-based optimization of the ELBO.

Derive the Variational Autoencoder (VAE) loss function as a combination of reconstruction loss and a KL regularization term.

Identify common failure modes of VAEs (e.g., posterior collapse, blurry outputs) and explain their underlying causes.

Analyze the effect of the KL divergence weight on the trade-off between reconstruction fidelity and sample quality in a trained VAE.