Decoding AI’s Open-Source Course Maps Three Ways to Run an Agent Loop and the Provider Economics Behind Each

Most groups deal with ‘which mannequin’ as the necessary choice. The harness engineering literature retains pointing some place else. In LangChain’s Terminal-Bench experiment, altering solely the harness—identical mannequin all through—moved a coding agent from roughly thirtieth place into the prime 5.

That consequence reframes the query. If the harness decides high quality, then the way you run the loop turns into an structure choice, not a deployment element. Paul Iusztin’s open-source course Building a Coding Agent From Scratch builds a Python agent referred to as Decode. Published by Decoding AI, it separates three run modes. Each mode has a unique latency profile. Each one subsequently desires a unique inference supplier.

One headless core, three shapes

The middle of the system is a headless harness with no interface of its personal. Inside it runs the agent loop each harness shares: the LLM picks an motion, a device executes, the remark feeds again. Everything reads from and writes to the context window.

The agent itself is small. In Decode it’s a ~20-line Pydantic AI definition composing a mannequin, instruments, and an output sort. In Claude Code’s leaked supply, the core loop is roughly 150 traces. Everything else—reminiscence, expertise, sandbox, permissions, LSP suggestions, compaction—is harness.

Interfaces then plug into that core. That is the place the three modes seem:

Mode 1: Interactive, on-line

A terminal UI is wired to one dwell session, in reminiscence, in the identical course of. Events stream again by async turbines as tokens arrive.

The arduous downside right here is steering. If you sort whereas a device name is in flight, injecting the message instantly corrupts the flip. Decode’s reply is a steering queue plus a precedence gate. Input is buffered on arrival and injected solely at a protected boundary. The loop exposes two: MODEL_REQUEST, earlier than the subsequent mannequin name, and WOULD_STOP, when the flip would finish.

Three enter modes map onto that. Plain Enter steers inside the flip. Alt+Enter queues a follow-up till the flip stops. Esc triggers a cooperative abort at the subsequent boundary, clearing each queues so historical past stays intact.

A human is studying each token. This mode is latency-bound, which is why it belongs on a low-latency hosted API.

Mode 2: Remote, offline

Remote mode retains the harness headless and runs it on a server by an agent runtime. Decode makes use of Kitaru, ZenML’s agent runtime, deployed to GCP, with the brokers themselves executing on Modal.

Nobody is watching. A backlog of tickets followers out to N harnesses in parallel, every producing its personal PR. Because the runtime data progress step-by-step, a sandbox that dies mid-task resumes from its final recorded step as a substitute of restarting. A run that pauses for human enter freezes and consumes no compute whereas it waits.

Tools execute inside Modal Sandboxes remotely, Docker domestically. The metric that issues is throughput per greenback, not time-to-first-token.

Mode 3: Async, on-line

The third form sits between the two. A dwell session fingers work to a job queue and returns instantly. Background workflows fan out LLM calls and publish outcomes again later.

The person is on-line however not watching every step. The queue owns the work, so the run outlives the shopper that began it. This is the sample behind Slack-triggered brokers and background PR overview, and it payments like batch, not like chat.

The interactive explainer

Why the supplier modifications with the mode

The price mannequin follows the latency requirement, and the hole is giant.

Take 1,000 paperwork at 30,000 enter tokens every, roughly 500 output tokens per doc. At frontier API charges of $3 per million enter and $15 per million output, the lesson’s arithmetic lands close to $97. Prompt caching doesn’t rescue it, as a result of each doc is a unique prefix. Batched on a serverless GPU at round 3,000 tokens per second, the identical work is underneath three hours of GPU time—roughly $13.

The reverse case is simply as sharp. Decode’s default take a look at mannequin, Qwen3.6 35B, runs on a single H200. Modal’s published pricing lists H200 SXM at $0.001261 per second, or about $4.54 per hour. Leave an interactive agent idle in a single day ready on a y affirmation, and ten idle hours add roughly $45 to the invoice.

That is the complete argument. Interactive work pays per token as a result of a human is ready. Offline and async work pays per GPU-hour as a result of throughput is the goal and idle time is the enemy.

There is a second axis: serverless versus reserved capability. Modal’s pricing analysis reduces it to one comparability. Reservations cost the peak price for the complete contract; serverless follows the demand curve. When the peak-to-average ratio exceeds the reservation low cost, serverless is cheaper. Modal studies typical reductions of two–5× towards peak-to-average ratios of 5–10× for inference, coaching, and agentic improvement. Industry surveys it cites put reservation utilization under 30%, typically underneath 10%.

Key Takeaways

Harness beats mannequin: swapping solely the harness moved an agent from ~thirtieth to prime 5 on Terminal-Bench.
Interactive mode is latency-bound and steers through a queue draining at MODEL_REQUEST and WOULD_STOP boundaries.
Remote and async modes are throughput-bound, so GPU-hour billing beats per-token billing at quantity.
1,000 paperwork price ~$97 on frontier API charges versus ~$13 of batched GPU time.
Serverless wins each time peak-to-average demand exceeds the reservation low cost, sometimes 5–10× towards 2–5×.

Sources:

The publish Decoding AI’s Open-Source Course Maps Three Ways to Run an Agent Loop and the Provider Economics Behind Each appeared first on MarkTechPost.

Decoding AI’s Open-Source Course Maps Three Ways to Run an Agent Loop and the Provider Economics Behind Each