GPU kernel development for LLM inference: Triton, TileLang, FlashMLA, and FlashInfer compared
How do Triton, TileLang, FlashMLA, and FlashInfer compare as GPU kernel development tools for LLM inference, and which use cases does each serve best?
The four tools fall into two families: kernel DSLs and compilers (Triton, TileLang) that generate kernels from Python-like tile programs, and attention kernel libraries and engines (FlashInfer, FlashMLA) that ship pre-built or JIT-compiled CUDA kernels for serving. The evidence shows DSLs are within reach of hand-tuned CUDA on GEMM and attention once tuned (59-105% of vendor baselines depending on workload and effort), while the libraries deliver 29-69% lower inter-token latency than compiler backends in serving benchmarks and dominate production deployment. FlashMLA is uniquely tied to one workload, DeepSeek's MLA, and has no peer-reviewed paper, so it is documented here through the DeepSeek technical reports and third-party kernel evaluations. On the training side the same DSLs power the ecosystem layer (torch.compile, Liger, MOSS's Triton FP8 GEMMs), but the training-kernel frontier — block-sparse MoE, single-kernel distributed MoE, FP8 training — is written in expert CUDA and CUTLASS, and FlashInfer and FlashMLA remain forward-only serving libraries. Confidence is moderate: almost all performance claims come from vendor-authored or single-group benchmarks with no independent head-to-head of all four tools.
Updated 13 Aug 202664 sources2018–2026Deep22 min read
GPU kernels · kernel DSLs · attention · LLM inference · LLM training · Triton · TileLang · FlashMLA · FlashInfer