Microsoft’s SkillOpt Shows Optimized Agent Skill Artifacts Transfer Across Model Scales and Between Codex and Claude Code Harnesses
SkillOpt is a text-space optimizer developed by a staff of researchers from Microsoft, Shanghai Jiao Tong University, Tongji University, and Fudan University.
SkillChoose trains a single natural-language talent doc whereas the goal mannequin stays frozen. An optimizer mannequin reads scored rollouts and proposes bounded add/delete/substitute edits. A held-out choice cut up accepts an edit solely when the rating strictly improves. The exported artifact is one file, best_skill.md.
The switch tables report three columns. Baseline is the goal’s no-skill rating. Direct is SkillChoose educated in-domain on that actual goal. Transferred applies a talent educated elsewhere, with no additional optimization.
The helpful comparability will not be transferred versus direct. It is how a lot of the in-domain acquire survives the transfer.
Cross-model switch: within-family, blended retention
Skills have been educated on GPT-5.4 and deployed on smaller variants.
| SpreadsheetBench | GPT-5.4-mini | 36.1 | 47.5 | 45.5 | +9.4 | 82% |
| SpreadsheetBench | GPT-5.4-nano | 23.5 | 42.5 | 26.5 | +3.0 | 16% |
| DwellMath | GPT-5.4-mini | 14.7 | 32.8 | 19.2 | +4.5 | 25% |
| DwellMath | GPT-5.4-nano | 23.2 | 27.2 | 28.8 | +5.6 | 140% |
Two rows deserve consideration. SpreadsheetBench on GPT-5.4-mini retains 82% of the in-domain acquire. That is near free reuse. The DwellMath row on GPT-5.4-nano is stranger: the transferred talent scores 28.8 towards an in-domain SkillChoose results of 27.2. The paper reads this as proof that some realized procedures are target-model agnostic.
The GPT-5.4-nano SpreadsheetBench row is the weak one at 16%. Retention will not be uniform, and the paper doesn’t declare it’s. Its said certain is narrower: no row falls beneath the goal’s no-skill baseline.
Note the scope. All 4 rows keep inside one GPT household. Cross-family switch, comparable to GPT to Qwen, will not be examined.
Cross-harness switch: the strongest consequence
This is the part that issues most for deployment. All rows use GPT-5.5.
| Benchmark | Source → Target | Baseline | Direct | Transferred | Gain | Share of in-domain acquire |
|---|---|---|---|---|---|---|
| SpreadsheetBench | Codex → Claude Code | 22.1 | 80.4 | 81.8 | +59.7 | 102% |
| SpreadsheetBench | Claude Code → Codex | 27.5 | 85.0 | 71.1 | +43.6 | 76% |
| DwellMath | Claude Code → Codex | 35.2 | 78.4 | 48.0 | +12.8 | 30% |
| DwellMath | Codex → Claude Code | 40.8 | 56.5 | 42.4 | +1.6 | 10% |
The first row is the headline. A talent optimized inside Codex lifted Claude Code from 22.1 to 81.8. That barely exceeds the 80.4 Claude Code reached by coaching its personal talent from scratch.
The two harnesses expose totally different device and file APIs and totally different command surfaces. A talent that survives that shift will not be encoding command recipes. The research paper attributes SpreadsheetBench’s portability to workbook-level procedures: structure-first inspection, formula-aware verification, and static-value materialization. Those maintain no matter which CLI runs the Python.
DwellMath tells the alternative story. Codex → Claude Code retains solely 10% of the in-domain acquire. The asymmetry is value sitting with. Procedural abilities — learn how to examine, confirm, and format — look like the transportable class. Reasoning-heavy abilities seem extra tied to their coaching atmosphere.
Cross-benchmark switch: actual however small
| Source → Target | Model | Baseline | Transferred | Gain |
|---|---|---|---|---|
| OlympiadBench → Omni-MATH | GPT-5.4 | 56.6 | 60.3 | +3.7 |
| OlympiadBench → Omni-MATH | GPT-5.4-mini | 34.8 | 36.6 | +1.8 |
| OlympiadBench → Omni-MATH | GPT-5.4-nano | 38.8 | 40.1 | +1.3 |
There is not any Direct column right here. No in-domain SkillChoose run on Omni-MATH is reported, so the comparability is towards no-skill solely. Gains are optimistic throughout all three mannequin scales however small. The analysis paper’s studying is that the talent retained reusable mathematical process after each the check situations and the answer-format conventions modified.
Why the artifact strikes in any respect
The mechanism is said plainly within the analysis paper. All three execution modes: direct chat, Codex, Claude Code – devour the identical best_skill.md file format. That shared contract is what makes the cross-harness experiment potential within the first place.
The Codex harness renders the present talent to a per-task SKILL.md alongside activity recordsdata, then reads again a compact execution hint. The Claude Code harness mirrors the identical workspace contract by means of the claude CLI. Neither harness will get a bespoke talent format.
The artifact’s form helps portability too. Final abilities run 379 to 1,995 tokens throughout the six benchmarks, with a median close to 920. They are assembled from 1 to 4 accepted edits. The paper’s Figure 4 samples one realized rule per benchmark, and all are procedural reasonably than instance-specific. The SpreadsheetBench rule, verbatim: examine workbook construction and formulation, then write evaluated static values throughout the complete requested goal vary as an alternative of counting on Excel recalculation.
What this means for portability
Training price is paid as soon as, offline, and measured. The analysis paper reviews 0.6M to 46.4M coaching tokens per absolute check level, relying on benchmark. SpreadsheetBench sits at 0.6M per level; DocVQA at 46.4M. The optimizer mannequin runs solely throughout coaching and provides zero inference-time calls at deployment.
If a talent educated in a single harness holds up in one other, that one-time price spreads throughout environments. The Codex → Claude Code SpreadsheetBench result’s the existence proof. It additionally implies you possibly can optimize the place tooling is least expensive and deploy the place the product lives.
The audit angle is separate and underrated. The deployed artifact is a textual content file a website practitioner can learn in minutes. Every change to it’s traceable: every step information an edit_apply_report.json with per-edit settle for and skip standing. Portability plus inspectability is a unique operational posture than transport fine-tuned weights.
Key Takeaways
- Evidence covers one GPT household and two benchmarks per axis, so portability is demonstrated, not but generalized.
- A Codex-trained SpreadsheetBench talent scored 81.8 inside Claude Code, above that harness’s personal 80.4 in-domain consequence.
- All 4 cross-model, 4 cross-harness, and 3 cross-benchmark switch rows land above the goal’s no-skill baseline.
- Transfer energy tracks activity kind: procedural spreadsheet abilities transfer effectively, math-reasoning abilities transfer weakly.
- The transportable unit is one
best_skill.mdof 379 to 1,995 tokens, constructed from 1 to 4 accepted edits.
Resources: Paper, GitHub, Project page, Docs, PyPI and Demo video
Baselines referenced: GEPA, TextGrad, EvoSkill and Trace2Skill
Benchmarks referenced: SearchQA, SpreadsheetBench, DocVQA, LiveMathematicianBench and ALFWorld
The put up Microsoft’s SkillOpt Shows Optimized Agent Skill Artifacts Transfer Across Model Scales and Between Codex and Claude Code Harnesses appeared first on MarkTechPost.
