Lucas Beyer, a machine learning researcher, has published a detailed technical reference for MSLK, a collection of fused GPU kernels built on PyTorch for transformer training and inference. The site, hosted at his personal domain lucasb.eyer.be, documents every public function the library exposes: attention, low precision matrix multiplication, quantization, mixture-of-experts routing, and convolution. Nothing on the page identifies an employer or company as its publisher, and the reference itself is described as generated directly from the library’s source code rather than written as a corporate announcement.
Kernel fusion is the concept doing the real work here. A GPU spends time and energy every time one operation finishes, writes its result to memory, and waits for the next operation to read that result back before starting. Combine several of those steps, a quantization pass followed immediately by a matrix multiply, for example, into a single kernel, and that round trip disappears. For most transformer workloads the limit is not how fast the chip can multiply numbers. It is how fast data can move between memory and the compute units. Fusing kernels attacks that second problem directly.
The reference organizes MSLK’s surface into three practical clusters. Fused attention runs through a function called memory_efficient_attention, which automatically selects among ten backends, including CUTLASS, Flash, Flash3, two CuTe variants tuned for Hopper and Blackwell chips, and AMD’s CK path, based on data type, head dimension, and available hardware. Quantized matrix multiplication forms a second cluster: calls such as f8f8bf16_rowwise take FP8 inputs and return BF16 output using one scale value per row, bf16i4bf16_rowwise pairs BF16 activations with packed 4-bit integer weights, and f4f4bf16 covers three low-bit floating point formats: NVFP4, MXFP4, and MXFP4-16. A third cluster handles mixture-of-experts routing as separately callable steps, index_shuffling, gather_scale_dense_tokens, and scatter_add_dense_tokens, alongside two prebuilt routing layers for teams that would rather not assemble the pieces themselves.
The compatibility constraint deserves the same attention as the kernels. The reference’s install table pairs each MSLK release to exactly one PyTorch minor version: 1.3.0 with PyTorch 2.13, 1.2.0 with 2.12, 1.1.0 with 2.11, and 1.0.0 with 2.10. The documentation states plainly that a given MSLK build is not expected to work against a PyTorch release older than the one it shipped alongside. That is an operational fact, not a footnote to skim past. A team running a pinned training stack cannot pick up newer MSLK kernels without moving PyTorch forward first, and there is no supported path for adopting a newer MSLK release and then rolling PyTorch back. Anyone planning a multi-month training run around this library needs that dependency chain mapped out before locking in a version, not discovered afterward.
The reference also flags smaller failure modes worth knowing before shipping. On ROCm, certain GEMM submodules must be imported by name before their operators register, since the top-level import does not do it automatically. Two mixture-of-experts functions, gather_scale_quant_dense_tokens and silu_mul_quant, have return signatures the page says have historically diverged from what is declared, and it recommends checking them against whatever build is currently installed, before a model is traced or exported. A variable-length attention function referenced in the library’s own tests is not currently exported at all.
This material is written for engineers already building on PyTorch who need to choose a specific fused kernel, pin an attention backend, or match a quantization scheme to the GEMM call that expects it. It is not an introduction to GPU programming or transformer architecture, and it will read as noise to anyone not already writing torch.ops calls. Teams that do not control their own PyTorch, CUDA, or ROCm versions will find the compatibility table a real constraint rather than a convenience. Anyone evaluating MSLK for a production stack should confirm the PyTorch version a target release requires before scheduling the work, since reversing that upgrade later is not something the documentation promises to support.
This reference for MSLK is self-published, undated, by machine learning researcher Lucas Beyer on his personal site, lucasb.eyer.be, and is not presented on the page as an official publication of any company.
