NVIDIA’s NeMo team has published Molt, an open-source reinforcement learning framework that lets a developer write an AI agent’s behavior directly in Python rather than describe it through a configuration file. The Python class a developer writes becomes the trainable unit itself, a structural break from how most RL pipelines for language models are currently assembled.

That distinction matters once you look at what config-driven RL frameworks demand today. Tools such as verl route training through Hydra or YAML configuration layers, with reward typically wired to a separate endpoint, reward model, or scripted rollout function sitting outside the main trainer. Molt instead lets a Python class define an environment or a chat loop, and whatever reward that class computes (a grader, a multi-turn tool call, a vision-language task, or an LLM acting as judge) feeds straight into training with no intermediate layer to configure.

The tradeoff shows up in NVIDIA’s own line-count comparison. Molt’s RL codebase runs about 9,200 lines, against roughly 7,200 for OpenRLHF, 25,000 for slime, and 62,000 for verl, by NVIDIA’s count of every file touching the training path. A team debugging a reward signal can trace it through Ray’s placement layer, vLLM rollout, and an NVIDIA AutoModel actor without leaving plain PyTorch. What that buys is a smaller surface for a research team to read end to end. What it likely costs is the built-in support for split actor, critic, and reward-model topologies that verl and OpenRLHF already ship for production RLHF work: Molt’s default topology is a single actor with an optional PPO critic bolted on.

The scaling numbers NVIDIA highlights are not aimed at a single workstation. The repository’s largest documented recipes run on Slurm-managed clusters: a Nemotron-Omni-30B model and a roughly 750 billion parameter GLM-5.2 model trained with 256-way expert parallelism, both spread across many multi-GPU nodes. Reaching trillion-parameter mixture-of-experts territory requires that scale of cluster, plus the CUDA 13 container NVIDIA ships with PyTorch 2.11, vLLM, and TransformerEngine built in for A100, H100, H200, and Blackwell-class GPUs.

A team without that hardware still gets a working entry point. Molt’s quick-start scripts run reinforcement learning on a single eight-GPU node with a 4 billion parameter Qwen3 model or a 35 billion parameter mixture-of-experts model, using the same code path that scales up later on a larger cluster. Installing the released package takes one PyPI command, without cloning the repository or building the container NVIDIA recommends for its production runs.

Molt also ships mechanisms aimed at a known failure mode in mixture-of-experts RL: the rollout engine and the training engine can pick different experts for the same tokens even at identical weights, which breaks the importance-sampling math that estimators like GRPO depend on. NVIDIA’s fix, called router replay, has the training pass reuse the exact expert selections vLLM made during rollout rather than recomputing them, and the repository documents it as the default in its mixture-of-experts recipes.

For a research team choosing between RL frameworks, Molt’s bet is that agent code can double as the specification, removing a configuration layer that normally separates a reward idea from a running experiment. Teams evaluating it this quarter should weigh that simplicity against the missing critic and reward-model scaffolding verl and OpenRLHF already provide, and should start on the single-node quick-start scripts before committing cluster time to the trillion-parameter recipes.

Details are drawn from NVIDIA’s NeMo team’s GitHub repository for the Molt reinforcement learning framework, NVIDIA-NeMo/labs-molt.