Advanced Kernels with Tensor Cores

Describe the architecture of NVIDIA Tensor Cores and the `mma.sync` instruction for matrix multiply-accumulate operations.

Use the `nvcuda::wmma` C++ API to load matrices into fragments, perform warp-level MMA, and store results.

Construct a complete GEMM kernel using `wmma` intrinsics, managing the data pipeline from global memory to shared memory and finally to registers.

Explain the quadratic memory complexity of standard self-attention that FlashAttention addresses.

Describe the core concepts of FlashAttention: tiling and online softmax computation.

Implement the inner loop of a simplified FlashAttention kernel, computing a block of the output matrix.

Combine components into a simplified FlashAttention forward pass kernel and verify its output against a naive baseline.