Someone wanted to learn this too, so Grasp built them a personal learning path.
You can keep exploring this one or create your own
Global Software Career Advancement
Recommended resources
The freely available resources this course will direct you to.
- GPU Warps Explained: How SIMT Really Works Under the Hood (Visual Deep Dive) | M2L3www.youtube.com
- Stanford CS149 I Parallel Computing I 2023 I Lecture 7 - GPU architecture and CUDA Programmingwww.youtube.com
- 1.2. Programming Model — CUDA Programming Guidedocs.nvidia.com
- 2.2. Writing CUDA SIMT Kernelsdocs.nvidia.com
- Understanding NVIDIA GPU Hardware as a CUDA C Programmer | Episode 2: GPU Compute Architecturewww.youtube.com
- CUDA Best Practices Guide - NVIDIA Documentation Hubdocs.nvidia.com
Module 2
CUDA Kernel Optimization by First Principles
Module 3
PTX, SASS, and Compiler Behavior
Module 4
Measurement Science and Performance Modeling
Module 5
Systematic GPU Bottleneck Diagnosis
Module 6
Designing Trustworthy GPU Profiling Infrastructure
Module 7
Multi-GPU Communication and Scaling
Module 8
Performance of Modern ML Workloads
Module 9
Virtualization, Memory Pressure, and Architectural Change
Module 10
Staff-Level Performance Architecture and Influence
Module 11