Raindrop, a startup that monitors AI agents in production, used its own blog to introduce rd-signal-2, a new model pipeline that powers what it calls Signals 2.0. The system turns raw traces of agent activity into narrow classifiers that answer a single yes-or-no question about behavior: did a tool call actually succeed, did the assistant quietly refuse, did the session end the way the team defines as a failure. That reframes work many teams currently hand to a general-purpose reasoning model acting as a judge over every interaction, a habit that gets expensive once traffic reaches production scale.
The first version of Signals, which Raindrop shipped in June 2025, judged a single input and output pair at a time, adequate for chatbot-era interactions contained in one turn. An agent failure now unfolds over many turns, invoked tools, and handoffs between subagents, a trail that can run to six figures in tokens. Catching them requires code that first filters for the relevant evidence, plus a model that then interprets it, rather than one prompt applied uniformly to everything.
Raindrop’s own illustration involves an agent that calls the same database update three times, each attempt timing out, then tells the user the record was updated successfully. No single step in that trace is wrong on its own. The failure only exists in the relationship between the repeated timeouts and the assistant’s final claim, a multi-step pattern a single-turn judge would miss and rd-signal-2 is built to catch.
Raindrop says rd-signal-2 comes close to the accuracy of GPT-5.6 Sol at its highest reasoning setting while costing 1,600 times less to run, and 260 times less than GPT-5.6 Luna at the same setting. Those figures come entirely from Raindrop’s internal testing against models Raindrop selected for comparison. The company has not published an independent benchmark alongside the release, nor disclosed the task set used to produce either multiple.
Raindrop’s own data complicates the accuracy claim further. Engineers wrote four subtly different phrasings of one single-sentence behavior policy and ran each over the same batch of 2,000 traces from production: results ranged from a 0.9 percent match rate to a 4.6 percent one, and 67 percent of flagged traces were rejected by at least one other phrasing. Raindrop frames this as proof that classification is an alignment problem between a company and its own policy, not a pure machine-learning one. It is also a reminder that an accurate classifier is only as reliable as the specification a customer wrote for it.
Raindrop says this infrastructure now evaluates more than 20 billion traces a month, with a median classification time of 100 milliseconds, inside isolated environments that hold no credentials and no internet access per customer. That volume is the real argument for the product. A team paying frontier-model prices to run a judge call on every production trace pays a cost that scales with each individual call, and at billions of traces a month that call cost becomes the entire inference budget. Training a narrow classifier once and running it cheaply afterward only pays off once a team’s classification volume is high enough and repetitive enough to amortize the upfront build. A team screening a few hundred conversations a day for one failure mode has far less to gain than one running millions.
Alongside rd-signal-2, Raindrop is releasing Signal Builder, a version of the same training and hosting pipeline built for zero data retention and aimed at regulated industries such as healthcare. It is also opening a Signals 2.0 API so outside developers can call the classifiers directly inside their own systems, with a zero-retention option available there too.
Teams currently running a frontier model as an LLM judge over production traffic should treat Raindrop’s cost multiples as a starting estimate rather than a verified number, and test rd-signal-2 against their own traces before shifting spend. Volume, not the accuracy claim, should drive the migration decision: below a few million classification calls a month, the engineering cost of building a custom Signal may exceed what a general model already charges.
Raindrop, the AI agent observability company, described Signals 2.0 and rd-signal-2 in a post on its own blog that carried no publication date.
