Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
High-Performance AI Computing
·
Module 12
Integrating Kernels in the AI Ecosystem
1
Custom CUDA Kernels for PyTorch Operators
Write a custom CUDA kernel for a PyTorch operator using the C++ extension API.
2
Building PyTorch Extensions with `setup.py`
Build and link the PyTorch extension using a `setup.py` file and call the custom kernel from Python.
3
Triton's Programming Model and Domain-Specific Abstractions
Explain the high-level programming model of Triton and its domain-specific abstractions.
4
Triton Basics: Implementing Element-wise Addition & Simplified GEMM
Implement a common AI kernel, such as element-wise addition or a simplified GEMM, in Triton.
5
Triton vs. CUDA: Productivity and Performance Trade-offs
Compare the Triton implementation with an equivalent raw CUDA kernel, analyzing the trade-offs between developer productivity and performance.
6
Collective Communication in Multi-GPU Training
Describe the role of collective communication libraries (e.g., NCCL) and key operations like All-Reduce in multi-GPU training.
7
Direct GPU Communication: Understanding P2P Memory Access
Explain the concept of peer-to-peer (P2P) memory access for enabling direct communication between multiple GPUs.
End of course,
Back to course
Previous module
Advanced Kernels with Tensor Cores
Next module