Someone Fine-Tuned OpenBMB’s MiniCPM5-1B on Claude Fable 5 Traces to Ship a 657MB Local Thinking Model

A group developer, GnLOLot, has printed a 1B mannequin that runs absolutely on native {hardware}. The mannequin is MiniCPM5-1B-Claude-Opus-Fable5-Thinking, with GGUF builds for llama.cpp-compatible runtimes. It wants no API key and makes no cloud calls.

The Proposed Model

The mannequin is constructed on openbmb/MiniCPM5-1B. That base is a actual, documented launch from OpenBMB. It is a dense 1.08B-parameter mannequin utilizing a customary LlamaForCausalLM structure. It has 24 layers, grouped-query consideration, and a 131,072-token context size. OpenBMB studies 1B-class open-source SOTA inside its personal comparability set.

The base already ships a native considering template. Reasoning is toggled by means of enable_thinking, giving each a Think and a No Think mode. The by-product mannequin retains that template and MiniCPM5’s tool-call format.

On high of that base, the developer utilized a fine-tune. The card states the mannequin is ‘additional fine-tuned on Fable 5 information’ to enhance coding and instruction following. The GGUF card repeats this as ‘post-trained on Fable 5 information.’

How it’s truly constructed

The described technique shouldn’t be classical distillation. You don’t shrink the unique mannequin. Instead you generate many conversations with a trainer mannequin. You seize its replies and reasoning traces as textual content. You then supervised-fine-tune a smaller base mannequin on these traces.

This distinction is necessary for accuracy. Classical distillation transfers sign from a trainer’s logits or weights. No one has entry to Claude’s weights or logits. So that is supervised fine-tuning on generated outputs, not weight-level distillation. OpenBMB’s personal base mannequin, in contrast, makes use of a documented On-Policy Distillation stage between its personal trainer and pupil checkpoints.

The sensible impact is that the 1B mannequin learns to imitate response format and elegance. It doesn’t take up the trainer’s underlying functionality. A 1B parameter finances can’t maintain frontier-scale reasoning.

The specs that take a look at

The context window is 128K tokens, inherited from the bottom config.json (131,072). The GGUF repository ships 4 quantizations. Q4_K_M is roughly 657MB and is labeled the smallest footprint. Q5_K_M is roughly 751MB. Q8_0 is roughly 1.1GB and is the maintainer’s advisable default. F16 is roughly 2.1GB.

The ‘657MB footprint’ is the smallest quant, not the default construct. The mannequin masses instantly in llama.cpp, Ollama, LM Studio, jan, and KoboldCpp.

Interactive: how the construct works

The explainer beneath walks the construct pipeline, the footprint tradeoffs, and the sincere break up of what a fine-tune can and can’t carry over.