Create a Reasoning-Focused LLM: A Practical Guide to Streaming, Curating, and Fine-Tuning the SupraLabs Reasoning Corpus
In this tutorial, we construct an end-to-end workflow for working with the SupraLabs reasoning corpus. We stream a consultant subset instantly from the Hugging Face Hub, examine its supply distribution, token-length patterns, activity composition, and reasoning-to-answer ratios, and then apply a sequence of high quality filters to take away unsuitable coaching examples. We rework the retained samples into a chat-based supervised fine-tuning format with express <suppose> reasoning tags and use them to adapt SmolLM2-135M-Instruct with LoRA by means of TRL’s SFTTrainer. By combining scalable information entry, exploratory evaluation, dataset curation, parameter-efficient fine-tuning, structured inference, and Parquet export, we create a full Google Colab pipeline for turning a massive multi-model reasoning corpus into a compact reasoning-focused language mannequin.
import subprocess, sys
def pip_install(pkgs):
subprocess.check_call([sys.executable, "-m", "pip", "install", "-q", *pkgs])
subprocess.name([sys.executable, "-m", "pip", "uninstall", "-y", "-q", "torchao"])
pip_install([
"datasets>=3.0.0",
"transformers>=4.46.0",
"trl>=0.12.0",
"peft>=0.13.0",
"accelerate>=1.0.0",
"bitsandbytes",
"matplotlib",
"pandas",
])
import os, re, json, math, random, itertools, warnings
import pandas as pd
import matplotlib.pyplot as plt
import torch
from collections import Counter
from datasets import load_dataset, Dataset
warnings.filterwarnings("ignore")
random.seed(42)
torch.manual_seed(42)
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Device: {DEVICE}")
if DEVICE == "cuda":
print(f"GPU: {torch.cuda.get_device_name(0)}")
DATASET_ID = "SupraLabs/reasoning-corpus-4K-5M-v1"
SAMPLE_SIZE = 8_000
print(f"nStreaming {DATASET_ID} ...")
stream = load_dataset(DATASET_ID, cut up="prepare", streaming=True)
stream = stream.shuffle(seed=42, buffer_size=30_000)
rows = listing(itertools.islice(stream, SAMPLE_SIZE))
ds = Dataset.from_list(rows)
print(f"Materialized pattern: {len(ds):,} rows")
print(f"Columns: {ds.column_names}")
ex = ds[0]
print("n" + "=" * 70)
print("EXAMPLE ROW")
print("=" * 70)
print(f"repo_id : {ex['repo_id']}")
print(f"tok_len : {ex['tok_len']}")
print(f"consumer : {ex['user'][:300]} ...")
print(f"thought_trace : {ex['thought_trace'][:300]} ...")
print(f"assistant : {ex['assistant'][:300]} ...")
We configure the Colab atmosphere, set up the required machine studying libraries, and take away the incompatible torchao package deal. We detect the out there compute gadget, join to the SupraLabs reasoning corpus by means of Hugging Face streaming, and keep away from downloading the full dataset. We shuffle the streamed information, materialize a consultant pattern, and examine the construction and contents of an instance row.
df = ds.to_pandas()
print("nTop 15 supply repos in pattern:")
src_counts = df["repo_id"].value_counts()
print(src_counts.head(15).to_string())
fig, axes = plt.subplots(2, 2, figsize=(14, 10))
axes[0, 0].hist(df["tok_len"], bins=60, coloration="#4C72B0", edgecolor="white")
axes[0, 0].set_title("Token size distribution")
axes[0, 0].set_xlabel("tok_len"); axes[0, 0].set_ylabel("rows")
src_counts.head(12).plot(variety="barh", ax=axes[0, 1], coloration="#55A868")
axes[0, 1].invert_yaxis()
axes[0, 1].set_title("Top-12 supply repos (pattern)")
df["think_chars"] = df["thought_trace"].str.len()
df["answer_chars"] = df["assistant"].str.len()
df["reason_ratio"] = df["think_chars"] / (df["think_chars"] + df["answer_chars"] + 1)
axes[1, 0].hist(df["reason_ratio"], bins=50, coloration="#C44E52", edgecolor="white")
axes[1, 0].set_title("Reasoning ratio (suppose / (suppose + reply))")
axes[1, 0].set_xlabel("ratio")
axes[1, 1].scatter(df["tok_len"], df["reason_ratio"], s=4, alpha=0.25, coloration="#8172B2")
axes[1, 1].set_title("tok_len vs reasoning ratio")
axes[1, 1].set_xlabel("tok_len"); axes[1, 1].set_ylabel("ratio")
plt.tight_layout()
plt.present()
print("nSummary stats:")
print(df[["tok_len", "think_chars", "answer_chars", "reason_ratio"]]
.describe().spherical(2).to_string())
def tag_task(row):
u = row["user"].decrease()
a = row["assistant"]
if "```" in a or re.search(r"b(def |class |import |operate|#embrace)", a):
return "code"
if re.search(r"(show|equation|integral|theorem|frac|int|remedy for)", u):
return "math"
if re.search(r"b(affected person|prognosis|symptom|remedy|scientific)b", u):
return "medical"
if re.search(r"b(which of the following|choices?:|(a)|(b))", u):
return "mcq/logic"
return "normal"
df["task"] = df.apply(tag_task, axis=1)
print("nHeuristic activity combine:")
print(df["task"].value_counts(normalize=True).spherical(3).to_string())
We convert the sampled dataset into a pandas DataFrame and analyze the distribution of supply repositories and token lengths. We calculate reasoning and reply character counts, measure the reasoning-to-response ratio, and visualize the relationships throughout the dataset. We additionally apply light-weight heuristic guidelines to classify every file as a code, arithmetic, medical, multiple-choice, or normal activity.
def filter_length(row, min_tok=200, max_tok=3000):
"""Keep samples inside a training-friendly token price range."""
return min_tok <= row["tok_len"] <= max_tok
def filter_degenerate(row):
"""Drop empty/near-empty ideas or solutions."""
return len(row["thought_trace"]) > 100 and len(row["assistant"]) > 20
def filter_repetition(row, max_line_repeat=0.30):
"""Drop traces the place one line repeats too usually (looping fashions)."""
traces = [l.strip() for l in row["thought_trace"].cut up("n") if l.strip()]
if len(traces) < 5:
return True
most_common = Counter(traces).most_common(1)[0][1]
return (most_common / len(traces)) <= max_line_repeat
def filter_reason_ratio(row, lo=0.15, hello=0.97):
"""Keep samples that truly cause however do not ONLY cause."""
t, a = len(row["thought_trace"]), len(row["assistant"])
r = t / (t + a + 1)
return lo <= r <= hello
n0 = len(ds)
ds_f = ds.filter(filter_length)
ds_f = ds_f.filter(filter_degenerate)
ds_f = ds_f.filter(filter_repetition)
ds_f = ds_f.filter(filter_reason_ratio)
print(f"nFiltering: {n0:,} -> {len(ds_f):,} rows "
f"({100 * len(ds_f) / n0:.1f}% retained)")
MODEL_ID = "HuggingFaceTB/SmolLM2-135M-Instruct"
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
if tokenizer.pad_token is None:
tokenizer.pad_token = tokenizer.eos_token
SYSTEM_PROMPT = (
"You are a cautious reasoning assistant. Think step-by-step inside "
"<suppose>...</suppose> tags, then give your last reply."
)
def to_chat(row):
return {
"messages": [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": row["user"]},
{"position": "assistant",
"content material": f"<suppose>n{row['thought_trace']}n</suppose>nn{row['assistant']}"},
]
}
train_ds = ds_f.map(to_chat, remove_columns=ds_f.column_names)
train_ds = train_ds.shuffle(seed=42)
N_TRAIN, N_EVAL = 1_500, 100
eval_ds = train_ds.choose(vary(N_TRAIN, min(N_TRAIN + N_EVAL, len(train_ds))))
train_ds = train_ds.choose(vary(min(N_TRAIN, len(train_ds))))
print(f"nTrain: {len(train_ds):,} | Eval: {len(eval_ds):,}")
print("nRendered coaching pattern (truncated):")
print(tokenizer.apply_chat_template(train_ds[0]["messages"], tokenize=False)[:800])
We assemble a quality-filtering pipeline that removes samples with unsuitable token lengths, incomplete responses, extreme repetition, or unbalanced reasoning content material. We load the SmolLM2 tokenizer and rework every retained file into a structured dialog containing a system immediate, consumer message, and reasoning-enhanced assistant response. We then shuffle the formatted information, create coaching and analysis subsets, and examine the last chat template used for supervised fine-tuning.
from trl import SFTTrainer, SFTConfig
from peft import LoraConfig
strive:
import peft.import_utils as _piu
import peft.tuners.lora.torchao as _plt
_piu.is_torchao_available = lambda: False
_plt.is_torchao_available = lambda: False
besides Exception:
cross
mannequin = AutoModelForCausalLM.from_pretrained(
MODEL_ID,
dtype=torch.bfloat16 if DEVICE == "cuda" else torch.float32,
).to(DEVICE)
peft_config = LoraConfig(
r=16,
lora_alpha=32,
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
sft_config = SFTConfig(
output_dir="smollm2-reasoning-demo",
max_length=2048,
per_device_train_batch_size=2,
gradient_accumulation_steps=8,
num_train_epochs=1,
learning_rate=2e-4,
lr_scheduler_type="cosine",
warmup_steps=10,
logging_steps=10,
eval_strategy="steps",
eval_steps=50,
save_strategy="no",
bf16=(DEVICE == "cuda"),
gradient_checkpointing=True,
report_to="none",
)
coach = SFTTrainer(
mannequin=mannequin,
args=sft_config,
train_dataset=train_ds,
eval_dataset=eval_ds,
peft_config=peft_config,
processing_class=tokenizer,
)
print("nStarting fine-tune (≈10–20 min on a T4 with these settings)...")
coach.prepare()
print("Done. Final eval loss:", coach.consider().get("eval_loss"))
We load the SmolLM2 causal language mannequin and configure LoRA adapters for parameter-efficient coaching. We outline the optimization, batching, analysis, precision, and gradient-checkpointing settings by means of TRL’s SFTConfig. We initialize the SFTTrainer, fine-tune the mannequin on the curated reasoning conversations, and consider its last coaching efficiency.
def generate(query, max_new_tokens=512, temperature=0.7):
msgs = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": question},
]
immediate = tokenizer.apply_chat_template(
msgs, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(immediate, return_tensors="pt").to(DEVICE)
with torch.no_grad():
out = coach.mannequin.generate(
**inputs,
max_new_tokens=max_new_tokens,
temperature=temperature,
top_p=0.9,
do_sample=True,
pad_token_id=tokenizer.pad_token_id,
)
textual content = tokenizer.decode(out[0][inputs["input_ids"].form[1]:],
skip_special_tokens=True)
m = re.search(r"<suppose>(.*?)</suppose>(.*)", textual content, re.DOTALL)
if m:
print("─" * 60, "nTHINKING:n", m.group(1).strip()[:1500])
print("─" * 60, "nANSWER:n", m.group(2).strip())
else:
print(textual content)
print("nn### TEST 1: logic puzzle")
generate("If all bloops are razzies and all razzies are lazzies, "
"are all bloops undoubtedly lazzies? Explain briefly.")
train_ds.to_parquet("reasoning_subset_train.parquet")
eval_ds.to_parquet("reasoning_subset_eval.parquet")
print("nSaved: reasoning_subset_train.parquet / reasoning_subset_eval.parquet")
We create an inference operate that codecs new questions with the similar system immediate and generates responses from the fine-tuned mannequin. We separate the generated <suppose> part from the last reply and take a look at the mannequin on logic and arithmetic issues. We lastly export the processed coaching and analysis datasets as Parquet recordsdata for reuse in bigger experiments.
In conclusion, we developed a sensible pipeline that connects large-scale reasoning-data exploration with small-language-model coaching. We streamed the corpus effectively, analyzed its inside composition, filtered examples utilizing token, repetition, completeness, and reasoning-balance standards, and transformed the ensuing information into a constant conversational coaching construction. We then fine-tuned SmolLM2 with LoRA, evaluated the tailored mannequin, inspected its generated reasoning and solutions, and exported the curated datasets for future experiments. This workflow offers a reusable basis for source-aware information mixing, curriculum studying, bigger scholar fashions, longer-context coaching, and production-scale reasoning mannequin growth with out requiring the total dataset to reside in Colab reminiscence.
Check out the FULL CODES here. Also, be happy to comply with us on Twitter and don’t neglect to be a part of our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to accomplice with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so on.? Connect with us
The submit Create a Reasoning-Focused LLM: A Practical Guide to Streaming, Curating, and Fine-Tuning the SupraLabs Reasoning Corpus appeared first on MarkTechPost.
