Skip to main content
grasp.study
About
Explore
Pricing
Log in
Try it for free
Back to course
👋
Welcome to a Grasp lesson
Create a free account
to keep studying and create your own study path.
High-Performance AI Computing
·
Module 9
GEMM Optimization: From Naive to High-Performance
1
Naive Thread-Per-Element CUDA GEMM
Implement a naive, thread-per-element matrix multiplication (GEMM) CUDA kernel.
2
Tiled Matrix Multiplication with Shared Memory
Implement tiled matrix multiplication using shared memory to reduce global memory accesses.
3
Shared Memory Tiling: Performance Benefits
Analyze the reduction in global memory traffic and performance improvement from shared memory tiling.
4
Register-Level Tiling for Data Reuse
Implement register-level tiling within a shared memory tile to maximize data reuse from registers.
5
Optimizing Memory Bandwidth with Vectorized Loads
Use vectorized memory loads (e.g., `float4`) to increase memory bandwidth utilization.
6
Handling Non-Divisible Dimensions in Matrix Tiling
Handle boundary conditions for matrices whose dimensions are not integer multiples of the tile size.
7
Benchmarking GEMM Optimizations Against cuBLAS
Benchmark each GEMM optimization stage against a library implementation (cuBLAS) and analyze remaining performance gaps.
Previous module
Performance Analysis and Asynchronous Operations
Next module
Core AI Kernels: Fusion, Normalization, and Backpropagation