Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
High-Performance AI Computing
·
Module 6
Warp Execution and Synchronization
1
Warp Execution: Understanding SIMT
Explain how threads within a warp execute in a Single Instruction, Multiple Thread (SIMT) fashion.
2
Thread Divergence: Causes and Performance Impact
Describe thread divergence caused by conditional branching and analyze its performance impact.
3
Minimizing Conditional Divergence
Refactor conditional code to minimize divergence, for example by using predicated execution.
4
Atomic Operations for Thread-Safe Memory Updates
Use atomic operations (e.g., `atomicAdd`, `atomicCAS`) to safely update memory locations from multiple threads.
Memory Fences: Enforcing Memory Order
Explain the purpose of memory fences (`__threadfence`) in enforcing memory access ordering between threads.
6
Warp-Level Primitives for Data Exchange
Use warp-level primitives (e.g., `__shfl_down_sync`) for efficient intra-warp data exchange.
7
Warp-Level Collective Decisions with `__any_sync` and `__all_sync`
Use warp vote functions (`__any_sync`, `__all_sync`) for collective warp-level decisions.
Previous module
CUDA Memory Optimization
Next module
Parallel Primitives in CUDA