#cuda
Every summary, chronological. Filter by category, tag, or source from the rail.
Optimizing Transformer Inference with FlashNorm
FlashNorm accelerates transformer inference by folding RMS norm gains into projection weights and parallelizing normalization and matrix multiplication via custom CUDA kernels.
AI EngineerCan LLMs Write Fast Multi-GPU Kernels?
While LLMs excel at single-GPU code, they struggle with multi-GPU kernel optimization because they lack a deep, reasoning-based understanding of interconnect topologies, data partitioning, and the complex trade-offs between copy engines and tensor memory acceleration.
AI EngineerParallelKernelBench: Frontier LLMs Struggle with Multi-GPU Kernels
While LLMs excel at single-GPU kernel generation, they currently struggle with multi-GPU tasks where communication bottlenecks and complex rank coordination dominate performance.
Building Tiled GPU Kernels with NVIDIA cuTile Python
NVIDIA cuTile allows developers to write efficient, tile-based GPU kernels directly in Python, providing a structured way to handle memory access and computation that can be benchmarked against standard PyTorch operations.
mKernel: Fusing Compute and Communication for GPU-Driven Scaling
mKernel eliminates host-driven communication bottlenecks by fusing intra-node NVLink, inter-node RDMA, and compute into persistent CUDA kernels, enabling fine-grained overlap at the tile level.
CUDA Matrix Transpose: Naive to Swizzled Optimization
Matrix transpose on GPU pits coalesced reads against writes; solve via shared memory tiling, then fix bank conflicts with padding or XOR swizzling, plus float4 vectorization for peak bandwidth.
Showing 6 of 6