|

Zhipu AI Releases ‘Glyph’: An AI Framework for Scaling the Context Length through Visual-Text Compression

ByRicardo

Can we render lengthy texts as photos and use a VLM to attain 3–4× token compression, preserving accuracy whereas scaling a 128K context towards 1M-token workloads? A staff of researchers from Zhipu AI launch Glyph, an AI framework for scaling the context size through visual-text compression. It renders lengthy textual sequences into photos and processes them utilizing imaginative and prescient–language fashions. The system renders extremely lengthy textual content into web page photos, then a imaginative and prescient language mannequin, VLM, processes these pages finish to finish. Each visible token encodes many characters, so the efficient token sequence shortens, whereas semantics are preserved. Glyph can obtain 3-4x token compression on lengthy textual content sequences with out efficiency degradation, enabling vital positive factors in reminiscence effectivity, coaching throughput, and inference pace.

https://arxiv.org/pdf/2510.17800

Why Glyph?

Conventional strategies increase positional encodings or modify consideration, compute and reminiscence nonetheless scale with token rely. Retrieval trims inputs, however dangers lacking proof and provides latency. Glyph adjustments the illustration, it converts textual content to photographs and shifts burden to a VLM that already learns OCR, structure, and reasoning. This will increase info density per token, so a hard and fast token price range covers extra unique context. Under excessive compression, the analysis staff present a 128K context VLM can deal with duties that originate from 1M token degree textual content.

https://arxiv.org/pdf/2510.17800

System design and coaching

The methodology has three phases, continuous pre coaching, LLM pushed rendering search, and submit coaching. Continual pre coaching exposes the VLM to massive corpora of rendered lengthy textual content with numerous typography and kinds. The goal aligns visible and textual representations, and transfers lengthy context abilities from textual content tokens to visible tokens. The rendering search is a genetic loop pushed by an LLM. It mutates web page measurement, dpi, font household, font measurement, line peak, alignment, indent, and spacing. It evaluates candidates on a validation set to optimize accuracy and compression collectively. Post coaching makes use of supervised tremendous tuning and reinforcement studying with Group Relative Policy Optimization, plus an auxiliary OCR alignment process. The OCR loss improves character constancy when fonts are small and spacing is tight.

https://arxiv.org/pdf/2510.17800

Results, efficiency and effectivity…

LongBench and MRCR set up accuracy and compression underneath lengthy dialogue histories and doc duties. The mannequin achieves a median efficient compression ratio about 3.3 on LongBench, with some duties close to 5, and about 3.0 on MRCR. These positive factors scale with longer inputs, since each visible token carries extra characters. Reported speedups versus the textual content spine at 128K inputs are about 4.8 occasions for prefill, about 4.4 occasions for decoding, and about 2 occasions for supervised tremendous tuning throughput. The Ruler benchmark confirms that increased dpi at inference time improves scores, since crisper glyphs assist OCR and structure parsing. The analysis staff experiences dpi 72 with common compression 4.0 and most 7.7 on particular sub duties, dpi 96 with common compression 2.2 and most 4.4, and dpi 120 with common 1.2 and most 2.8. The 7.7 most belongs to Ruler, to not MRCR.

https://arxiv.org/pdf/2510.17800

So, what? Applications

Glyph advantages multimodal doc understanding. Training on rendered pages improves efficiency on MMLongBench Doc relative to a base visible mannequin. This signifies that the rendering goal is a helpful pretext for actual doc duties that embrace figures and structure. The important failure mode is sensitivity to aggressive typography. Very small fonts and tight spacing degrade character accuracy, particularly for uncommon alphanumeric strings. The analysis staff exclude the UUID subtask on Ruler. The strategy assumes server facet rendering and a VLM with sturdy OCR and structure priors.

Key Takeaways

  • Glyph renders lengthy textual content into photos, then a imaginative and prescient language mannequin processes these pages. This reframes long-context modeling as a multimodal downside and preserves semantics whereas lowering tokens.
  • The analysis staff experiences token compression is 3 to 4 occasions with accuracy similar to sturdy 8B textual content baselines on long-context benchmarks.
  • Prefill speedup is about 4.8 occasions, decoding speedup is about 4.4 occasions, and supervised tremendous tuning throughput is about 2 occasions, measured at 128K inputs.
  • The system makes use of continuous pretraining on rendered pages, an LLM pushed genetic search over rendering parameters, then supervised tremendous tuning and reinforcement studying with GRPO, plus an OCR alignment goal.
  • Evaluations embrace LongBench, MRCR, and Ruler, with an excessive case displaying a 128K context VLM addressing 1M token degree duties. Code and mannequin card are public on GitHub and Hugging Face.

Editorial Comments

Glyph treats lengthy context scaling as visible textual content compression, it renders lengthy sequences into photos and lets a VLM course of them, lowering tokens whereas preserving semantics. The analysis staff claims 3 to 4 occasions token compression with accuracy similar to Qwen3 8B baselines, about 4 occasions sooner prefilling and decoding, and about 2 occasions sooner SFT throughput. The pipeline is disciplined, continuous pre coaching on rendered pages, an LLM genetic rendering search over typography, then submit coaching. The strategy is pragmatic for million token workloads underneath excessive compression, but it depends upon OCR and typography decisions, which stay knobs. Overall, visible textual content compression gives a concrete path to scale lengthy context whereas controlling compute and reminiscence.

Check out the Paper, Weights and Repo. Feel free to take a look at our GitHub Page for Tutorials, Codes and Notebooks. Also, be happy to comply with us on Twitter and don’t overlook to affix our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

The submit Zhipu AI Releases ‘Glyph’: An AI Framework for Scaling the Context Length through Visual-Text Compression appeared first on MarkTechPost.

Similar Posts