Fine-Tuning Qwen3 with LoRA Using NVIDIA NeMo AutoModel: A Complete Single-GPU Google Colab Workflow Tutorial
In this tutorial, we construct an end-to-end NVIDIA NeMo AutoModel workflow in Google Colab and use a single GPU to discover the identical configuration-driven coaching structure that scales to distributed multi-GPU environments. We confirm the accessible CUDA {hardware} and precision assist, set up NeMo AutoModel immediately from its supply repository, load an official Qwen3-0.6B LoRA fine-tuning recipe, and programmatically adapt its precision, batch-size, checkpointing, and scheduler settings for a constrained Colab runtime. We then launch parameter-efficient fine-tuning by the automodel command-line interface, find and reload the generated LoRA checkpoint, and examine outputs from the unique and fine-tuned fashions. Finally, we use NeMoAutoModelForCausalLM by the Python API to reveal how NeMo AutoModel integrates NVIDIA-optimized execution paths whereas preserving the acquainted Hugging Face mannequin interface.
Setting Up the Colab Workspace and Shell Helper
import os, sys, glob, json, subprocess, shutil, textwrap
REPO_DIR = "/content material/Automodel"
WORK_DIR = "/content material/automodel_demo"
CKPT_DIR = os.path.be part of(WORK_DIR, "checkpoints")
os.makedirs(WORK_DIR, exist_ok=True)
def sh(cmd, examine=True):
print(f"n$ {cmd}n" + "-" * 78)
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, textual content=True, bufsize=1)
for line in p.stdout:
print(line, finish="")
p.wait()
if examine and p.returncode != 0:
elevate RuntimeError(f"Command failed ({p.returncode}): {cmd}")
return p.returncode
We import the core Python libraries required for file dealing with, course of execution, path administration, and formatted output. We outline the repository, working, and checkpoint directories used all through the workflow. We additionally create a reusable shell-command perform that streams command output and raises errors when execution fails.
Verifying the GPU and Installing NeMo AutoModel
print("=" * 78)
print("STEP 0 — Checking GPU runtime")
print("=" * 78)
import torch
assert torch.cuda.is_available(), (
"No GPU discovered! In Colab: Runtime -> Change runtime kind -> choose a GPU."
)
GPU_NAME = torch.cuda.get_device_name(0)
BF16_OK = torch.cuda.is_bf16_supported()
VRAM_GB = torch.cuda.get_device_properties(0).total_memory / 1e9
print(f"GPU: {GPU_NAME} | VRAM: {VRAM_GB:.1f} GB | bf16 supported: {BF16_OK}")
print("n" + "=" * 78)
print("STEP 1 — Installing NeMo AutoModel (takes a couple of minutes)")
print("=" * 78)
if not os.path.isdir(REPO_DIR):
sh(f"git clone --depth 1 https://github.com/NVIDIA-NeMo/Automodel.git {REPO_DIR}")
sh(f"pip -q set up -e {REPO_DIR}")
sh("pip -q set up pyyaml peft")
sh('python -c "import nemo_automodel; print('NeMo AutoModel model:', '
'getattr(nemo_automodel, '__version__', 'supply'))"')
We confirm that the Colab runtime supplies a CUDA-enabled GPU and examine its title, reminiscence capability, and bfloat16 assist. We clone the NVIDIA NeMo AutoModel repository when it’s not already accessible and set up the package deal immediately from supply. We then set up the supporting YAML and PEFT libraries and ensure that the NeMo AutoModel package deal imports accurately.
Loading and Patching the Qwen3 LoRA Recipe
print("n" + "=" * 78)
print("STEP 2 — Preparing the recipe")
print("=" * 78)
import yaml
candidates = sorted(glob.glob(
os.path.be part of(REPO_DIR, "examples", "llm_finetune", "qwen", "*0p6b*peft*.yaml")
)) or sorted(glob.glob(
os.path.be part of(REPO_DIR, "examples", "llm_finetune", "**", "*peft*.yaml"),
recursive=True,
))
assert candidates, "Could not discover a PEFT recipe within the cloned repo."
BASE_RECIPE = candidates[0]
print(f"Base recipe: {os.path.relpath(BASE_RECIPE, REPO_DIR)}")
with open(BASE_RECIPE) as f:
cfg = yaml.safe_load(f)
print("n--- Original recipe (as shipped) ---")
print(yaml.dump(cfg, sort_keys=False)[:2500])
def patch(node):
if isinstance(node, dict):
for ok, v in record(node.gadgets()):
if isinstance(v, str) and never BF16_OK and v.decrease() in (
"bf16", "bfloat16", "torch.bfloat16"):
node[k] = "float32"
elif ok in ("batch_size", "local_batch_size") and isinstance(v, int):
node[k] = min(v, 4)
elif ok == "global_batch_size" and isinstance(v, int):
node[k] = min(v, 8)
else:
patch(v)
elif isinstance(node, record):
for merchandise in node:
patch(merchandise)
patch(cfg)
cfg.setdefault("step_scheduler", {})
cfg["step_scheduler"]["max_steps"] = 40
cfg["step_scheduler"]["ckpt_every_steps"] = 40
cfg["step_scheduler"]["num_epochs"] = 1
if isinstance(cfg.get("checkpoint"), dict):
cfg["checkpoint"]["enabled"] = True
cfg["checkpoint"]["checkpoint_dir"] = CKPT_DIR
DEMO_RECIPE = os.path.be part of(WORK_DIR, "qwen3_0p6b_colab_lora.yaml")
with open(DEMO_RECIPE, "w") as f:
yaml.dump(cfg, f, sort_keys=False)
print("n--- Patched recipe (what we'll really run) ---")
print(yaml.dump(cfg, sort_keys=False)[:2500])
MODEL_ID = "Qwen/Qwen3-0.6B"
attempt:
MODEL_ID = cfg["model"]["pretrained_model_name_or_path"]
besides Exception:
cross
print(f"nBase mannequin: {MODEL_ID}")
We find an official PEFT recipe, load its YAML configuration, and examine the unique coaching settings. We recursively adapt the precision and batch measurement parameters to suit the recipe on a single Colab GPU whereas preserving its unique construction. We additionally restrict the coaching length, configure checkpoint output, save the patched recipe, and extract the Hugging Face mannequin identifier.
Running LoRA Fine-Tuning on HellaSwag
print("n" + "=" * 78)
print("STEP 3 — Training (LoRA fine-tune of Qwen3-0.6B on HellaSwag)")
print("=" * 78)
env_prefix = "HF_HUB_ENABLE_HF_TRANSFER=0 TOKENIZERS_PARALLELISM=false"
rc = sh(f"cd {WORK_DIR} && {env_prefix} automodel {DEMO_RECIPE}", examine=False)
if rc != 0:
print("nRetrying with legacy CLI syntax...")
sh(f"cd {WORK_DIR} && {env_prefix} automodel finetune llm -c {DEMO_RECIPE}")
We launch Qwen3-0.6B LoRA fine-tuning on the HellaSwag dataset by the NeMo AutoModel command-line interface. We flip off pointless Hugging Face switch and tokenizer parallelism options to maintain the Colab run extra predictable. We additionally embody a fallback command that helps older NeMo AutoModel CLI syntax when the first invocation fails.
Comparing Base and Fine-Tuned Model Outputs
print("n" + "=" * 78)
print("STEP 4 — Evaluating: base mannequin vs LoRA fine-tuned mannequin")
print("=" * 78)
from transformers import AutoModelForCausalLM, AutoTokenizer
DTYPE = torch.bfloat16 if BF16_OK else torch.float32
PROMPT = ("A man is sitting on a roof. He begins pulling up roofing shingles. "
"What occurs subsequent?")
def generate(mannequin, tok, immediate, max_new_tokens=60):
inputs = tok(immediate, return_tensors="pt").to(mannequin.machine)
with torch.no_grad():
out = mannequin.generate(**inputs, max_new_tokens=max_new_tokens,
do_sample=False, temperature=None, top_p=None,
pad_token_id=tok.eos_token_id)
return tok.decode(out[0][inputs["input_ids"].form[1]:],
skip_special_tokens=True)
tok = AutoTokenizer.from_pretrained(MODEL_ID)
base = AutoModelForCausalLM.from_pretrained(
MODEL_ID, torch_dtype=DTYPE, device_map="cuda")
print("n[BASE MODEL]")
print(textwrap.fill(generate(base, tok, PROMPT), 90))
ckpt_glob = sorted(glob.glob(os.path.be part of(CKPT_DIR, "**", "mannequin"),
recursive=True))
if not ckpt_glob:
ckpt_glob = sorted(glob.glob(os.path.be part of(WORK_DIR, "**",
"adapter_model.safetensors"),
recursive=True))
ckpt_glob = [os.path.dirname(p) for p in ckpt_glob]
if ckpt_glob:
ADAPTER_DIR = ckpt_glob[-1]
print(f"nFound checkpoint: {ADAPTER_DIR}")
attempt:
from peft import PeftModel
tuned = PeftModel.from_pretrained(base, ADAPTER_DIR)
print("n[FINE-TUNED MODEL (base + LoRA adapter)]")
print(textwrap.fill(generate(tuned, tok, PROMPT), 90))
besides Exception as e:
print(f"nCould not auto-load the adapter with peft ({e}).")
print("Inspect the checkpoint contents manually:")
for p in glob.glob(os.path.be part of(ADAPTER_DIR, "*"))[:20]:
print(" ", p)
else:
print("nNo checkpoint discovered — examine the coaching logs above.")
del base
torch.cuda.empty_cache()
We load the tokenizer and base causal language mannequin, generate a deterministic response, and set up a baseline for comparability. We search the coaching output directories for the newest LoRA checkpoint or adapter information created throughout fine-tuning. We then connect the adapter with PEFT, generate the fine-tuned response, and launch GPU reminiscence after analysis.
Using the NeMo AutoModel Python API
print("n" + "=" * 78)
print("STEP 5 — Bonus: drop-in Python API")
print("=" * 78)
attempt:
from nemo_automodel import NeMoAutoModelForCausalLM
nm = NeMoAutoModelForCausalLM.from_pretrained(
MODEL_ID, torch_dtype=DTYPE).to("cuda")
print("[NeMoAutoModelForCausalLM]")
print(textwrap.fill(generate(nm, tok, "The key concept of LoRA is"), 90))
del nm
torch.cuda.empty_cache()
besides Exception as e:
print(f"Python-API demo skipped on this model/GPU: {e}")
print("n" + "=" * 78)
print("DONE! Where to go subsequent:")
print("=" * 78)
print(f"""
1. Recipes to discover (in {REPO_DIR}/examples/):
llm_finetune/ — SFT + LoRA for Llama, Qwen, Gemma, Phi, GPT-OSS, ...
llm_pretrain/ — e.g. nanoGPT on FineWeb, DeepSearch-V3 pre-training
vlm_finetune/ — Qwen-VL, Gemma-3-VL, and different vision-language fashions
diffusion/ — FLUX / Wan / Qwen-Image LoRA fine-tuning
2. Swap the mannequin: override one discipline —
automodel recipe.yaml --model.pretrained_model_name_or_path <any-HF-LM>
3. Scale out: the SAME recipe runs on 8 GPUs with `--nproc-per-node 8`,
or multi-node by way of the shipped slurm.sub / SkyPilot / Kubernetes launchers.
4. Docs: https://docs.nvidia.com/nemo/automodel/newest/index.html
""")
We reveal the direct Python interface by loading the mannequin by NeMoAutoModelForCausalLM and operating a further technology instance. We deal with version-specific or hardware-specific failures gracefully so the pocket book can nonetheless full efficiently. We conclude by presenting the accessible recipe classes, model-override syntax, distributed scaling choices, and official documentation path.
Conclusion
In conclusion, we established a sensible NeMo AutoModel pipeline that covers surroundings validation, supply set up, recipe inspection, configuration patching, LoRA coaching, checkpoint restoration, mannequin analysis, and direct Python API inference. We noticed how NeMo AutoModel separates the distributed coaching technique from the applying code by specifying mannequin, dataset, optimizer, precision, parallelism, and checkpoint habits by reusable YAML recipes. Although we ran the workflow on a single Colab GPU, we retained the identical SPMD-oriented construction used for bigger FSDP2, tensor-parallel, context-parallel, sequence-parallel, and pipeline-parallel deployments. It provides us a technically grounded start line for adapting extra language-model, vision-language, pre-training, and diffusion recipes whereas scaling the identical workflow from experimentation to multi-node NVIDIA infrastructure.
Check out the Full Code here. Also, be happy to comply with us on Twitter and don’t overlook to hitch our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to companion with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and many others.? Connect with us
The put up Fine-Tuning Qwen3 with LoRA Using NVIDIA NeMo AutoModel: A Complete Single-GPU Google Colab Workflow Tutorial appeared first on MarkTechPost.
