|

Liquid AI Releases LFM2.5-DSpark Draft Models That Deliver Up to 3.18x Faster Decoding Without Changing Model Outputs

ByRicardo

Liquid AI has launched DSpark draft mannequin checkpoints for 3 fashions in its LFM2.5 household: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. Each drafter provides a speculative decoding path to an current goal mannequin. A roughly 300M-parameter draft proposes a block of 9 candidate tokens, and the goal mannequin verifies the entire block in a single ahead move. The commerce is a small reminiscence enhance for a big decoding speedup: up to 3.18x on an H100 and up to 2.87x on an M4 Max MacBook Pro. Output doesn’t change. Under grasping decoding, the emitted sequence is similar to the goal mannequin operating alone, so benchmark accuracy is unchanged. Both llama.cpp and SGLang have day-one help.

Is it deployable?

Yes, if you happen to self-host. The weights ship as Safetensors and GGUF, and the drafter checkpoints will not be served by any hosted inference supplier on Hugging Face right now. Running them wants an SGLang or llama.cpp construct with DSpark help for LFM2 targets.

  • Company degree: The LFM Open License v1.0 permits free business use solely whereas your entity stays below $10M in annual income. Indie builders, startups and SMBs are lined; bigger enterprises should contact Liquid AI for a business license first.
  • Industries: Developer tooling, client apps that run regionally, robotics and embedded methods, plus healthcare, finance and protection workloads that preserve knowledge on-premise or on-device.
  • Applications: Local coding assistants, on-device brokers that cause earlier than every software name, single-user chat the place batch dimension is 1, and offline copilots on laptop-class {hardware}.

What are Drafters?

Speculative decoding makes use of a small mannequin to suggest tokens {that a} bigger mannequin verifies. Each LFM2.5 drafter is roughly 300M parameters: 295.7M for the 1.2B-Instruct goal and 327.7M for the two.6B and 8B-A1B targets. The spine is 5 full-attention layers with hidden_size=2048, intermediate_size=6144, GQA at 32 heads over 8 KV heads, and a block dimension of 9. The drafter ships no vocabulary weights; embedding and LM head are tied from the goal at load time. The 2.6B drafter repository is 655 MB in BF16, which is the actual reminiscence value you’re including.

DSpark combines three components. A DFlash-style parallel spine, conditioned on the goal’s context options, produces hidden states for all draft tokens in a single ahead move. A light-weight sequential head, modeled as a Markov chain between neighboring tokens at rank 256, restores inter-token dependency and lifts acceptance at later block positions. A confidence-scheduled verifier predicts every token’s survival chance and prunes low-confidence suffixes when verification would value greater than it saves.

The Measured Results

Liquid AI studies throughput on 1xH100 in BF16 through SGLang, and on an M4 Max MacBook Pro through llama.cpp with Metal and FP16 GGUF weights. Both use block dimension 9, batch dimension 1 and temperature 0, throughout MATH500, HumanEval, MBPP, GSM8K and MT-Bench.

Target H100 imply Best H100 case M4 Max imply Best M4 Max case
LFM2.5-1.2B-Instruct 2.10x (656 → 1384 tok/s) 2.56x on MATH500 2.54x (138 → 350 tok/s) 2.87x on HumanEval (136 → 389)
LFM2.5-2.6B 2.67x (323 → 864 tok/s) 3.06x on MATH500 2.27x (61 → 139 tok/s) 2.63x on HumanEval
LFM2.5-8B-A1B 2.54x (418 → 1074 tok/s) 3.18x on MATH500 (428 → 1362) 1.18x (90 → 106 tok/s) 1.44x on GSM8K

Speedup tracks acceptance price, which tracks how predictable the output is. LFM2.5-8B-A1B accepts 8.27 of 10 tokens per step on MATH500 and solely 4.02 on GSM8K, so the identical mannequin swings from 3.18x to 1.29x on the identical GPU. On the 1.2B mannequin, MT-Bench acceptance drops to 3.90 and the H100 achieve falls to 1.66x.

The MoE end result on Apple silicon is the clearest caveat: LFM2.5-8B-A1B positive aspects just one.18x on common on the M4 Max. Liquid AI attributes this to the present MoE implementation in llama.cpp’s Metal backend, and to the truth that verifying okay tokens prompts extra consultants, and due to this fact extra weight site visitors, than a single decode step.

The Agentic Case

The achieve concentrates the place the person waits by means of reasoning earlier than each software name. Across multi-tool function-calling situations, Liquid AI studies that DSpark cuts latency by 57% on common for LFM2.5-2.6B. Test it towards your personal traces: an agent that plans, calls, and re-plans pays the decode value a number of instances per person flip.

On SGLang, launch the goal with the drafter connected:

Copy Code
python -m sglang.launch_server 
  --model-path LiquidAI/LFM2.5-2.6B 
  --speculative-algorithm DSPARK 
  --speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark 
  --speculative-draft-attention-backend flashinfer 
  --disable-radix-cache --mem-fraction-static 0.75 --port 30000

The block dimension is learn from the drafter’s config.json, and the baseline is similar command with out the three --speculative-* flags.

Key Takeaways

  • DSpark drafters add ~300M parameters and up to 3.18x sooner decoding on an H100.
  • Greedy output is similar to baseline, so benchmark accuracy is unchanged.
  • Speedup follows acceptance price and varies by workload, from 1.04x to 3.18x.
  • On-device MoE is the weak spot: LFM2.5-8B-A1B positive aspects just one.18x on M4 Max.
  • Multi-tool operate calling will get the most important sensible win: 57% decrease latency on LFM2.5-2.6B.

Check out the model card on 8B-A1B and the full technical write-up. All credit score for this analysis goes to the researchers of this venture.

Also, be happy to comply with us on Twitter and don’t neglect to be part of our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to associate with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so on.? Connect with us

The submit Liquid AI Releases LFM2.5-DSpark Draft Models That Deliver Up to 3.18x Faster Decoding Without Changing Model Outputs appeared first on MarkTechPost.

Similar Posts