|

Microsoft Open Sources code-testing-generator: a Polyglot Unit-Test Agent That Hits 92.1% Task Completion Versus 78.9% for Stock Copilot

ByRicardo

Microsoft has open sourced code-testing-generator, a polyglot agent that writes unit checks after which proves they work. It ships within the dotnet-test plugin contained in the MIT-licensed dotnet/skills repository.

The agent targets a hole that coding assistants normally go away open. A immediate like ‘generate unit checks’ doesn’t say which framework, file location or assertions to make use of. code-testing-generator settles these choices by studying the repository earlier than it writes something. It then plans, writes, runs and checks the checks it produces. On Microsoft’s inside 152-task benchmark, it accomplished 140 duties towards 120 for inventory GitHub Copilot. Both setups used the identical mannequin and prompts.

Is it deployable

Yes. It is an agent definition together with abilities, not a hosted service, so it runs inside your present coding agent and code stays native.

  • Company stage: viable from solo maintainers upward. Startups and mid-market groups acquire most, as a result of the agent provides repository analysis a small workforce has no time to encode. Enterprises can fork the language steering to match inside frameworks.
  • Industries: regulated or audit-heavy software program estates — monetary companies, healthcare, insurance coverage, public sector — plus platform groups paying down legacy take a look at debt.
  • Applications: backfilling checks on untested modules, producing checks for a pull-request diff, elevating protection earlier than a launch gate, and standardising conventions throughout polyglot monorepos.

What the agent really does

It coordinates work by way of a Research-Plan-Implement (RPI) pipeline. It searches the repository for code needing checks, detects the language and take a look at framework, reads present checks for conventions, and finds the true construct and take a look at instructions. That final step targets a particular failure: a take a look at undertaking that builds regionally however by no means runs in CI as a result of nothing registered it.

The agent then picks considered one of three methods. Direct writes and validates checks instantly. Single move runs one cycle. Iterative repeats it for giant scopes or protection targets. It by no means modifies manufacturing code, and avoids checks that decision exterior URLs, bind ports or rely on timing.

The verification gate

Before reporting completion, the agent runs 5 checks. It causes about small code adjustments that ought to make the checks fail, a light-weight type of mutation testing. It seems to be for weak or lacking assertions. It maps each requested situation to a take a look at. It builds the total workspace and runs the total suite. It confirms the repository’s personal take a look at command discovers the brand new checks.

Benchmark outcomes

On Microsoft’s inside benchmark of 152 duties from actual repositories, the agent accomplished 140 (92.1%) versus 120 (78.9%) for inventory GitHub Copilot on the identical mannequin and prompts (63% fewer failures).

The acquire is concentrated. On 89 obscure prompts, the agent resolved 79 (88.8%) towards 59 (66.3%), chopping failures from 30 to 10. On 63 detailed prompts, each scored 61 (96.8%). On 15 duties focusing on a particular diff, the agent handed all 15 and inventory Copilot handed none.

Notably, the agent generated 2.3% fewer checks (6,963 vs 7,129) at successfully similar line protection (72.4% vs 72.2%). Average job time was 359 seconds towards 380. Token use per accomplished job was 3.2% larger.

On 45 .NET duties, Claude Opus 4.8 reached 43/45 with the agent versus 35/45 inventory; GPT-5.5 reached 41/45 versus 36/45. On the more durable exterior SWE Atlas benchmark, completion was 16/44 versus 12/44.

Explainer: how the agent turns one immediate into verified checks


Key Takeaways

  • Open supply, MIT-licensed, polyglot unit-test agent from Microsoft’s .NET workforce.
  • Research-Plan-Implement pipeline replaces one-shot technology with repository-aware planning.
  • 92.1% vs 78.9% job completion towards inventory Copilot on the identical mannequin.
  • Gains come virtually solely from obscure prompts and diff-targeted requests.
  • Fewer checks, similar protection, 5.5% quicker — reliability, not quantity.

Check out the Technical details and Repo. Also, be happy to observe us on Twitter and don’t overlook to affix our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to accomplice with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so forth.? Connect with us

The submit Microsoft Open Sources code-testing-generator: a Polyglot Unit-Test Agent That Hits 92.1% Task Completion Versus 78.9% for Stock Copilot appeared first on MarkTechPost.

Similar Posts