Training Neural Networks with Backpropagation

Manually derive the gradients using the chain rule for a simple two-layer network.

Implement the backpropagation algorithm to compute gradients for all parameters in a deep MLP.

Implement a gradient checking function to verify the correctness of your backpropagation implementation.

Integrate forward pass, backpropagation, and parameter updates into a complete training loop.

Implement the Momentum update rule to accelerate SGD.

Implement the Adam optimizer from scratch by combining momentum and adaptive learning rates.