Ant Group, working with the vLLM community, has open-sourced Humming, a library meant to make compressed AI models run faster on graphics chips, according to a post on the vLLM blog dated 21 October 2026. The post is written by the same people who built it, so every speed figure below is the authors’ own.

A quick explanation of what this kind of library does. Teams shrink a model by storing its numbers with fewer bits, which is called quantization. A smaller model needs less memory and moves less data, but the saving only becomes speed if the low-level code doing the arithmetic is written for that exact number format and that exact chip. That code is a kernel. Writing one by hand for every format and chip is slow work, and Humming’s pitch is that it builds kernels from shared parts instead, compiling the right one when a workload arrives.

On hardware, the vLLM blog says Humming fully supports Nvidia GPUs from the Turing generation onward, plus other chips that speak Nvidia’s CUDA programming language, starting with T-Head’s Zhenwu 810E. The post names the H20, H200, B300, and the workstation-class SM120 and SM121 chips in its tests. On formats, it covers integer and floating-point weights from 1 bit up to 8 bits, paired with activations at 16, 8, or 4 bits. The shorthand is W for weights and A for activations, so W4A8 means 4-bit weights and 8-bit activations.

The headline numbers come from microbenchmarks the vLLM community ran. For 4-bit weights with 16-bit activations (W4A16), the post says Humming roughly doubles the speed of vLLM’s existing Marlin kernel on large-batch H200 work and passes three times Marlin in some B300 cases. On mixture-of-experts layers, which route each token to a few specialist sub-networks, it reports about 5 times Marlin on B300 with its grouped path. For 8-bit work, it claims about 2.6 times and 2.5 times DeepGEMM’s speed on the H20 and H200. Its 4-bit-weight, 8-bit-activation path reaches about 4.2 times and 4.5 times that same baseline on those chips.

The test closest to real use is end to end. Running DeepSeek-V4.1-Flash on eight H20 GPUs, the authors swapped Marlin for Humming in the expert layers and measured throughput gains of roughly 14 to 43 percent, depending on how many requests arrived at once. At 64 simultaneous requests, output climbed from 442.37 tokens per second to 633.94. Time to first token fell by roughly 25 to 32 percent. The authors also say compiling a kernel dropped from 23.35 seconds with CUTLASS to 1.82 seconds.

Two details cut against a pure victory lap. Marlin is vLLM’s own earlier kernel, so the project is largely beating its previous self, and the post offers no outside benchmarks. The text also notes that FlashInfer’s CUTLASS W4A8 kernel is built on Humming’s approach, which means part of the comparison is against a relative.

Adoption looks easy. Humming installs with pip, its wheel is about 500 KB, and vLLM 0.29.0 and later pulls it in automatically for CUDA setups. Two flags, one for ordinary layers and one for expert layers, switch the backend on.

The authors plan an autotuner that tries candidate settings on the actual device, and more support for Nvidia’s Rubin chips and T-Head’s Zhenwu M890. Reported by vLLM on 21 October 2026.

Anyone serving a 4-bit model on H20 or H200 hardware has a cheap experiment available: upgrade vLLM, flip the two flags, and compare throughput against your current Marlin numbers before ordering more GPUs.

Reported by the vLLM blog (Ant Group and the vLLM community) on 21 October 2026.