Prime Intellect Releases Verifiers v1: Composable Tasksets, Harnesses, and Runtimes for Agentic RL Training and Evaluations

Prime Intellect launched verifiers 0.2.0. It previews a rewritten core, shipped beneath the brand new verifiers.v1 namespace. Modern evaluations now run coding brokers with instruments, compaction, and subagents. Accordingly, v1 rebuilds environments to run these agentic workloads at scale.

What is verifiers v1?

First, contemplate what verifiers is: Prime Intellect’s setting stack for agentic reinforcement studying and evaluations. Previously, an setting bundled its knowledge, agent logic, and infrastructure collectively. In distinction, v1 breaks that bundle into three composable items.

A taskset defines the work: the info, instruments, and scoring. A harness solves the duty and produces a rollout. That harness generally is a ReAct loop, a CLI agent, or your individual. The rollout then runs inside a runtime, both native or in a sandbox. Because the items decouple, any taskset runs beneath any appropriate harness.

How the Architecture Works?

With these items outlined, the subsequent query is how they convey. The central piece is the verifiers-managed interception server. It sits between the agent’s runtime and the inference server. Specifically, it proxies requests to, and responses from, inference. Meanwhile, it information the hint, units sampling parameters, and can rewrite device responses. That rewriting helps mitigate reward hacks throughout coaching.

For scale, every server multiplexes a continuing variety of rollouts, defaulting to 32. A pool then scales elastically with noticed concurrency. The server additionally owns a shopper that relays these requests. During analysis, an EvalClient acts as a blind HTTP proxy. During coaching, a TrainClient wraps renderers for trustworthy token-in RL coaching.

Because harnesses converse completely different dialects, verifiers helps three as of now. These are OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. A dialect adapter normalizes every wire format into canonical vf.sorts. Consequently, your scoring logic stays impartial of the agent examined.

Run rollout</button>
<button id=”vf-reset” class=”vf-ghost”>Reset</button>
<span class=”vf-lab”>Harness dialect:</span>
<choose id=”vf-dialect”>
<possibility worth=”Chat”>OpenAI Chat Completions</possibility>
<possibility worth=”Resp”>OpenAI Responses</possibility>
<possibility worth=”Msg”>Anthropic Messages</possibility>
</choose>
</div>

<div class=”vf-stage”>
<div class=”vf-row” type=”margin-bottom:14px”>
<div class=”vf-node vf-taskset” id=”n-taskset”>
<div class=”vf-nt”>Taskset</div>
<div class=”vf-nd”>what · knowledge · instruments · scoring</div>
</div>
</div>

<div class=”vf-runtime-wrap”>
<span class=”vf-runtime-tag”>RUNTIME · the place (subprocess · Docker · sandbox)</span>
<div class=”vf-row” id=”vf-flow”>
<div class=”vf-node vf-harness” id=”n-harness”>
<div class=”vf-nt”>Harness</div>
<div class=”vf-nd”>how · Codex · Terminus 2 · ReAct</div>
</div>
<div class=”vf-arrow”>→</div>
<div class=”vf-node vf-intercept” id=”n-intercept”>
<div class=”vf-nt”>Interception Server</div>
<div class=”vf-nd”>proxy · information hint</div>
</div>
<div class=”vf-arrow”>→</div>
<div class=”vf-node vf-infer” id=”n-infer”>
<div class=”vf-nt”>Inference Server</div>
<div class=”vf-nd”>vLLM · mannequin</div>
</div>
<div class=”vf-packet” id=”vf-packet”>req</div>
</div>
</div>

<div class=”vf-status” id=”vf-status”>Press “Run rollout” to ship a request by way of the interception server.</div>
</div>

<div class=”vf-grid”>
<div class=”vf-panel”>
<h3>Trace · message graph (v1)</h3>
<div class=”vf-hint”>Each message is a novel node. Size grows linearly in turns.</div>
<div class=”vf-graph” id=”vf-graph”>
<div class=”vf-empty”>No messages recorded but.</div>
</div>
</div>

<div class=”vf-panel”>
<h3>Trace measurement: v0 vs v1</h3>
<div class=”vf-hint”>Drag to alter turns. v0 repeats prompt-completion pairs; v1 shops distinctive nodes.</div>
<div class=”vf-chart”>
<svg viewBox=”0 0 260 150″ id=”vf-svg”>
<line x1=”30″ y1=”130″ x2=”255″ y2=”130″ stroke=”#dfe6ef” stroke-width=”1.5″/>
<line x1=”30″ y1=”10″ x2=”30″ y2=”130″ stroke=”#dfe6ef” stroke-width=”1.5″/>
<path id=”vf-v0″ fill=”none” stroke=”#d1477a” stroke-width=”2.5″/>
<path id=”vf-v1″ fill=”none” stroke=”#0b8f8f” stroke-width=”2.5″/>
<textual content x=”140″ y=”147″ font-size=”9″ fill=”#94a3b8″ text-anchor=”center”>turns →</textual content>
</svg>
</div>
<div class=”vf-legend”>
<span><i type=”background:#d1477a”></i> v0 · quadratic</span>
<span><i type=”background:#0b8f8f”></i> v1 · linear</span>
</div>
<div class=”vf-slider-row”>
<span>Turns</span>
<enter sort=”vary” id=”vf-turns” min=”4″ max=”60″ worth=”24″>
<span id=”vf-turns-val” type=”width:26px;text-align:proper”>24</span>
</div>
</div>
</div>

<div class=”vf-foot”>
Illustrative demo of the verifiers v1 structure · Built by <b>Marktechpost</b>
</div>
</div>

<script>
(operate(){
var root=doc.getElementById(“vfv1-demo”);
var packet=doc.getElementById(“vf-packet”);
var standing=doc.getElementById(“vf-status”);
var graph=doc.getElementById(“vf-graph”);
var runBtn=doc.getElementById(“vf-run”);
var resetBtn=doc.getElementById(“vf-reset”);
var dialectSel=doc.getElementById(“vf-dialect”);
var nHarness=doc.getElementById(“n-harness”);
var nIntercept=doc.getElementById(“n-intercept”);
var nInfer=doc.getElementById(“n-infer”);
var move=doc.getElementById(“vf-flow”);

var flip=0, operating=false;
var msgs=[]; // recorded nodes
var dialectLabel={Chat:”Chat”,Resp:”Resp”,Msg:”Msg”};

operate pos(el){ // heart x relative to move
var f=move.getBoundingClientRect();
var r=el.getBoundingClientRect();
return (r.left – f.left) + r.width/2 – 32;
}
operate clearActive(){ [nHarness,nIntercept,nInfer].forEach(operate(n){n.classList.take away(“vf-active”);}); }

operate movePacket(fromEl,toEl,ms,label,isResp){
return new Promise(operate(res){
packet.textContent=label;
packet.classList.toggle(“vf-resp”,!!isResp);
packet.type.transition=”none”;
packet.type.left=pos(fromEl)+”px”;
packet.type.opacity=”1″;
void packet.offsetWidth;
packet.type.transition=”left “+ms+”ms cubic-bezier(.45,.05,.35,1)”;
packet.type.left=pos(toEl)+”px”;
setTimeout(res,ms);
});
}

operate addNode(position,label,coloration){
if(msgs.size===0){ graph.innerHTML=””; }
var d=doc.createElement(“div”);
d.className=”vf-msg”;
d.innerHTML='<span class=”vf-dot” type=”background:’+coloration+'”></span><code>’+label+'</code><span class=”vf-role”>’+position+'</span>’;
graph.appendChild(d);
graph.scrollTop=graph.scrollHeight;
msgs.push(label);
}

operate sleep(ms){return new Promise(operate(r){setTimeout(r,ms);});}

async operate runTurn(){
if(operating) return;
operating=true; runBtn.disabled=true;
flip++;
var dl=dialectLabel[dialectSel.value];

// seed system + person on first flip
if(flip===1){
addNode(“system”,”S1″,”#6366f1″); await sleep(160);
addNode(“person”,”U1″,”#6366f1″);
}

clearActive();
nHarness.classList.add(“vf-active”);
standing.textContent=”Harness builds a “+dl+” request…”;
await sleep(350);

// harness -> interception
nIntercept.classList.add(“vf-active”);
standing.textContent=”Interception server proxies the request → inference.”;
await movePacket(nHarness,nInfer,850,dl+” req”);
clearActive(); nInfer.classList.add(“vf-active”);
standing.textContent=”Inference server generates the reply (vLLM).”;
await sleep(350);

// inference -> interception (information) -> harness
nIntercept.classList.add(“vf-active”);
standing.textContent=”Interception server information the hint, relays the response.”;
await movePacket(nInfer,nHarness,850,”resp”,true);
packet.type.opacity=”0″;
clearActive();

// file assistant node (+ occasional device)
addNode(“assistant”,”A”+flip,”#0b8f8f”); await sleep(150);
if(flippercent2===0){ addNode(“device”,”T”+flip,”#e0a800″); }

standing.textContent=”Turn “+flip+” recorded as a novel node within the message graph.”;
operating=false; runBtn.disabled=false;
}

operate reset(){
flip=0; msgs=[]; operating=false; runBtn.disabled=false;
clearActive(); packet.type.opacity=”0″;
graph.innerHTML='<div class=”vf-empty”>No messages recorded but.</div>’;
standing.textContent=”Press “Run rollout” to ship a request by way of the interception server.”;
}

runBtn.addEventListener(“click on”,runTurn);
resetBtn.addEventListener(“click on”,reset);

// —- v0 vs v1 development chart —-
var v0=doc.getElementById(“vf-v0”);
var v1=doc.getElementById(“vf-v1”);
var turnsR=doc.getElementById(“vf-turns”);
var turnsV=doc.getElementById(“vf-turns-val”);

operate drawChart(N){
var x0=30,x1=255,y0=130,y1=12,W=x1-x0,H=y0-y1;
var maxV0=N*N; // quadratic reference
operate ptV0(i){var x=x0+(i/N)*W;var y=y0-((i*i)/maxV0)*H;return x+”,”+y;}
operate ptV1(i){var x=x0+(i/N)*W;var y=y0-((i/N)*H);return x+”,”+y;} // linear
var p0=”M”,p1=”M”;
for(var i=0;i<=N;i++){ p0+=(i?” L”:””)+ptV0(i); p1+=(i?” L”:””)+ptV1(i); }
v0.setAttribute(“d”,p0); v1.setAttribute(“d”,p1);
}
turnsR.addEventListener(“enter”,operate(){ turnsV.textContent=turnsR.worth; drawChart(+turnsR.worth); });
drawChart(+turnsR.worth);

// —- auto-resize for PhrasePress iframe embedding —-
operate sendHeight(){
var h=doc.getElementById(“vfv1-demo”).offsetHeight+40;
if(window.mother or father){ window.mother or father.postMessage({vfv1Height:h},”*”); }
}
window.addEventListener(“load”,sendHeight);
window.addEventListener(“resize”,sendHeight);
new MutationObserver(sendHeight).observe(doc.getElementById(“vf-graph”),{childList:true});
setInterval(sendHeight,1200);
})();
</script>
</physique>
</html>
“>

v0 vs v1: A Quick Comparison

These adjustments separate v1 from v0.

Use Cases with Examples

With the structure clear, contemplate how groups use it. For instance, you possibly can run Nemotron 3 Ultra on Terminal-Bench 2 beneath Codex.

Similarly, groups can reuse Harbor datasets with out rewriting reward logic. Prime Intellect ported Terminal Bench 2 into v1 with solely a small class. In its inner testing, verifiers matched Harbor’s efficiency on the identical duties. Harbor is the primary fully-supported third-party format; NeMo Gym and OpenEnv have alpha assist.

On the coaching facet, the identical environments plug into prime-rl straight. In a length-penalty ablation, GLM-4.5-Air educated on ScaleSWE throughout six H200 nodes. That run took two days and evaluated on SWE-Bench-Verified, displaying steady agentic coaching.

A Minimal Taskset and Launch

Each run begins from a taskset that defines knowledge and scoring, impartial of any harness:

import verifiers.v1 as vf

class AdditionData(vf.TaskData):
    reply: int

class AdditionTask(vf.Task[AdditionData]):
    @vf.reward
    async def exact_match(self, hint: vf.Trace) -> float:
        return float(hint.last_reply == str(self.knowledge.reply))

class AdditionTaskset(vf.Taskset[AdditionTask, vf.TasksetConfig]):
    def load(self) -> record[AdditionTask]:
        return [
            AdditionTask(
                AdditionData(idx=i, prompt=f"What is {i} + {i}?", answer=2 * i),
                self.config.task,
            )
            for i in range(100)
        ]

__all__ = ["AdditionTaskset"]

Any taskset then runs beneath a selected harness through TOML and the CLI:

mannequin = "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B"

[taskset]
id = "primeintellect/terminal-bench-2"

[harness]
id = "codex"
model = "0.116.0"

uv run eval @ path/to/config.toml

Key Takeaways

• verifiers v1 splits an setting right into a taskset (what), a harness (how), and a runtime (the place).
• A verifiers-managed interception server proxies harness–inference requests and information traces on the fly.
• A linear message-graph hint replaces v0’s quadratic prompt-completion pairs, enabling long-horizon coaching.
• It ships with full prime-rl coaching assist; the legacy code path is now frozen.
• Harbor datasets and harnesses like Codex and Terminus 2 work out of the field.

Check out the Technical details. Also, be at liberty to comply with us on Twitter and don’t neglect to hitch our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to accomplice with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so forth.? Connect with us

The submit Prime Intellect Releases Verifiers v1: Composable Tasksets, Harnesses, and Runtimes for Agentic RL Training and Evaluations appeared first on MarkTechPost.