Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
JAX for PyTorch Users
·
Module 4
Automatic Differentiation in Depth
1
Forward vs. Reverse Autodiff: Choosing the Right Mode
Distinguish forward-mode (jvp) and reverse-mode (vjp) autodiff and choose appropriately by problem shape.
2
Automatic Differentiation with JAX: Jacobians and Hessians
Compute Jacobians and Hessians using jax.jacfwd, jax.jacrev, and jax.hessian.
3
Efficient Hessian-vector Products with JVP and Grad
Implement Hessian-vector products efficiently by composing jvp and grad.
4
Implementing Custom Gradients with JAX
Define a custom forward and backward rule using jax.custom_vjp.
5
Blocking Gradients with `jax.lax.stop_gradient`
Apply jax.lax.stop_gradient to block gradient flow at chosen points.
6
Gradient Checkpointing: Compute-Memory Trade-off
Explain the mechanism of gradient checkpointing (remat) for trading compute for memory in long backward passes.
Previous module
Pytrees: Structure, Manipulation, and Custom Nodes
Next module
Building Neural Networks with Equinox