Salesforce AI Research Introduces xRouter: A Reinforcement Learning Router for Cost Aware LLM Orchestration
When your software can name many alternative LLMs with very completely different costs and capabilities, who ought to determine which one solutions every request? Salesforce AI analysis group introduces ‘xRouter’, a tool-calling–based mostly routing system that targets this hole with a reinforcement studying based mostly router and learns when to reply domestically and when to name exterior fashions, whereas monitoring price at token degree.
What is xRouter?
xRouter is a instrument calling based mostly orchestration system constructed on Qwen2.5-7B-Instruct because the router spine. The router is an instruction tuned mannequin with instrument calling capabilities that decides which downstream mannequin to invoke, tips on how to immediate it, and whether or not to synthesize or choose a solution. The implementation makes use of DAPO, Distributional Advantage Policy Optimization, contained in the Verl reinforcement studying framework, and exposes an OpenAI appropriate API.
The router operates over greater than 20 LLM instruments within the full system. These instruments span premium, customary, finances and specialised tiers, together with GPT-5, GPT-4.1, GPT-5-Mini, GPT-5-Nano, o3, Kimi K2, DeepSeek-R1, Qwen3-235B variants and GPT-OSS fashions. The offloading pool is a 12 mannequin subset that features GPT-5, GPT-5-Mini, GPT-5-Nano, GPT-4o, GPT-4.1, o3, o3-Pro, o4-Mini, GPT-OSS-120B, GPT-OSS-20B and two Gemini-2.5 variants.
Cost Aware Reward and Success Gating
Routing is framed as a reinforcement studying drawback. For every episode, the reward combines a binary success sign and a price penalty. The analysis group defines a reward that provides a hard and fast bonus when the ultimate reply is appropriate, then subtracts a time period proportional to the entire normalized price of all mannequin calls. If the reply is flawed, the reward is zero no matter how low-cost it was.
As per the Model weights page, reward = high quality − λ × normalized_cost, the place λ is a price penalty coefficient. Episodes with failures successfully have zero high quality. This ‘success gated, price formed’ goal forces the router to first obtain correctness, then optimize price amongst profitable methods. In apply, coaching makes use of 3 price penalty settings, which produce the xRouter-7B-1, xRouter-7B-2 and xRouter-7B-3 variants.
Training Data and Signal Design
xRouter coaching information comes from Reasoning360, which incorporates math, code and normal reasoning duties with problem estimates derived from a robust reference mannequin, Qwen3-32B. The analysis group stratify samples into straightforward, medium and onerous bands, and add easier chit chat, retrieval and factual questions to show the router when it may possibly reply instantly with out delegation. Each pattern contains descriptions and costs for fashions from completely different tiers. The system additionally refreshes the mannequin catalog and perturbs prices to keep away from overfitting to a static worth desk.
Failed trajectories, resembling flawed solutions from costly fashions or pointless calls when the router might have answered itself, nonetheless incur full price and obtain zero reward. This produces a clear studying sign, the place correctness gates reward and value shapes the routing coverage.
How the Router Behaves at Inference Time?
The router helps three execution modes. It can reply instantly from the spine with out calling instruments. It can name a number of downstream fashions, then synthesize a response utilizing its personal reasoning over their outputs. It may also name downstream fashions and use a particular select_response instrument to choose one of many replies as the ultimate reply. These modes are applied by operate calls in an OpenAI type interface, which the orchestration engine executes by LiteLLM and SGLang.
Empirically, educated xRouter situations use a mixture of direct and synthesized responses. Off the shelf routers resembling GPT-4o, GPT-4.1, GPT-5, Qwen2.5-7B and Qwen3-8B have a tendency to reply instantly more often than not, even when instructed to dump when unsure. This is a vital behavioral distinction and explains a part of the effectivity acquire.
Quantitative Results and Cost Utility
On static routing baselines throughout Minerva, MATH-500, Olympiad Bench, AIME-24, AMC-23, Codeforces, Code-Contests and Human-EvalPlus, xRouter-7B variants constantly enhance accuracy in comparison with utilizing the identical base mannequin as an untrained router. xRouter-7B-2, for instance, reaches close to GPT-5 accuracy on Olympiad Bench whereas utilizing about one eighth of the GPT-5 analysis price.
In the system degree comparability on StayCodeBenchv5, GPQADiamond, AIME25, MT-Bench, IFEval and StayBench, xRouter-7B-3 achieves the very best common accuracy on StayCodeBenchv5 amongst all examined methods, and does this with reasonable price. Across duties resembling GPQA, xRouter variants attain round 80 to 90 % of GPT-5 accuracy whereas consuming lower than one fifth of the associated fee. The analysis group summarize that their price conscious reward can scale back inference price by as much as 80 % at related completion charges. The mannequin weights HF card experiences as much as 60 % price discount for comparable high quality below different settings.
The analysis group additionally defines ‘price utility’ as accuracy divided by price. Open supply single fashions with very low API costs usually attain larger price utility, however with decrease absolute accuracy. xRouter sits within the center, buying and selling some price utility for stronger activity efficiency, which is normally what manufacturing methods care about.
Key Takeaways
- xRouter is a instrument calling router constructed on Qwen2.5 7B Instruct that learns to pick amongst 20 plus exterior LLMs with a reinforcement studying coverage that’s explicitly price conscious.
- The router makes use of successful gated reward, duties solely get constructive reward when the ultimate reply is appropriate, and inside profitable trajectories it applies a price penalty time period λ instances normalized price, which yields three xRouter 7B variants with completely different price accuracy commerce offs.
- Training on Reasoning360 with problem stratification and artificial straightforward queries teaches xRouter when to reply instantly and when to dump, whereas perturbing costs and mannequin swimming pools improves robustness to altering supplier catalogs.
- Across math, coding and reasoning benchmarks, xRouter 7B fashions obtain close to GPT 5 accuracy on onerous duties like Olympiad Bench and round 80 to 90 % of GPT 5 accuracy on GPQA, whereas reducing offloading price by as much as 60 to 80 % relying on the analysis setup.
Editorial Notes
xRouter is a sensible step towards price conscious orchestration for heterogeneous LLM fleets. It reveals {that a} mid dimension router, educated with DAPO on Reasoning360 utilizing successful gated, price formed reward, can constantly strategy GPT 5 accuracy whereas decreasing offloading price by as much as 60 to 80 %.
Check out the PAPER and Model Weight. Feel free to take a look at our GitHub Page for Tutorials, Codes and Notebooks. Also, be at liberty to comply with us on Twitter and don’t overlook to hitch our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
The publish Salesforce AI Research Introduces xRouter: A Reinforcement Learning Router for Cost Aware LLM Orchestration appeared first on MarkTechPost.
