Someone wanted to learn this too, so Grasp built them a personal learning path.
You can keep exploring this one or create a path like this one shaped around you.
Recommended resources
Module 2
C++ and Reproducible Software Workflows
Module 3
Math, Numerical Computing, and Measurement
Module 4
PyTorch Modeling, Compilation, and GPU Execution
Module 5
GPU Architecture and Performance Reasoning
Module 6
CUDA Kernel Programming Fundamentals
Module 7
Profiling and Optimizing CUDA Kernels
Module 8
Triton Kernel Engineering
Module 9
Transformer Inference Computation and Memory
Module 10
LLM Inference Engines and Optimization Techniques
Module 11
Production Benchmarking and Distributed GPU Systems
Module 12