|

A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

ByRicardo

In most decoder-only LLMs, nothing computed on the final layer of token t feeds the primary layer of token t+1; positions talk solely via consideration over cached keys and values. A Princeton researcher’s (Yifan Zhang) technical report, Recurrent Looped Transformer (RLT), proposes closing that loop. The decoder’s last hidden state and its layerwise sliding-window consideration (SWA) cache are carried into the following token, across each immediate and response, with no reset on the boundary. The proposed analysis is a design specification. It defines the structure, execution schedules, and RL replay contract, and it explicitly stories no measured effectivity, reasoning high quality, or scaling outcomes.

How RLT is Built

Recurrent Looped Transformer (RLT) pairs a causal encoder with a recurrent decoder. The encoder processes tokens in parallel underneath a causal masks and produces representations e_t, from which key-value reminiscence M≤t is projected; reminiscence teams could be shared across decoder layers (G = 1) or saved layer-specific (G = L_D).

The decoder holds the recurrence. Its full state is Ht = (st, CtD), the place st is the ultimate decoder output and CtD holds the retained SWA keys and values at each decoder layer. For every token, a gated merge combines et with the earlier output s{t-1}, then every decoder block runs causal SWA over decoder activations, cross-attention to encoder reminiscence, and an FFN. The window W contains the present token, so at most W – 1 historic entries per layer are retained. The next-token distribution is learn from st, and initialization occurs as soon as earlier than BOS with a discovered begin state s* and an empty cache.

The reference tied configuration makes use of 48 encoder and 48 decoder layers with suitable consideration and FFN weights shared between them. Each token due to this fact executes 96 logical blocks, although decoder blocks add cross-attention, so per-block FLOPs usually are not equal. Zhang calls this parameter reuse, not activation copying.

The 3 Design Principles

  • Latent reasoning with unbounded temporal depth: After t processed tokens, the state path from s0 traverses t·LD decoder blocks, or 48t within the reference configuration. Per-token work stays fastened whereas the trail’s structural depth grows with the sequence. The analysis report warns that gates and contraction might suppress lengthy paths; structural depth shouldn’t be a reasoning assure.
  • Model-hardware co-design: Encoder options and reminiscence projections for recognized tokens use token-parallel kernels. Decoder transitions keep sequential inside a sequence, however prepared updates from unbiased sequences can share one batched kernel. The report states plainly that no precise parallel scan is assumed for the nonlinear decoder, no reduced-prefill speedup is claimed, and a typical parallel SWA decoder move shouldn’t be equal to the recurrence. Batching, kernel fusion, and checkpointing are listed as implementation targets, not accomplished kernels.
  • Model-RL algorithm co-design: Pretraining, SFT, sampling, and RL replay share one state transition. For RL, the sampler data every motion’s habits log-probability underneath its precise sampling distribution, together with temperature and truncation. The coach rebuilds encoder reminiscence, the recurrent output, and each SWA cache from the sequence begin underneath present parameters earlier than scoring every motion; outdated rollout states are by no means reused. Proposition 3.1 formalizes the payoff: transferring the prompt-response break up leaves the conditional distribution unchanged for a set token historical past.

Training and Serving

Pretraining is full-sequence next-token prediction with full backpropagation via time. SFT masks the loss to assistant targets however by no means masks state updates, so assistant losses backpropagate via person and power tokens. Appendix B exhibits why partial detaching is dangerous: the state-to-state Jacobian has cross phrases via decoder KV, so detaching solely st leaves gradient paths via the cache; any truncated-BPTT scheme should identify each indifferent tensor.

For multi-turn serving, a precise prefix snapshot contains encoder cache and reminiscence, the entire decoder state, place metadata, the window conference, and mannequin model. A fixed-weight snapshot could be reused as a result of the state is unbiased of the serving break up; weight updates invalidate outdated states, and modifying a prefix forces recomputation from an earlier checkpoint. External tokens in multi-turn RL replace the state however get no importance-ratio elements.

How It Relates to Prior Work

Encoder-derived reminiscence follows YOCO, which caches KV as soon as for a cross-decoder, and DeepSeek-V4.1-Flash, which initiatives decoder world KV from last encoder states; RLT retains the reminiscence however drops prompt-wide decoder skipping. Temporal suggestions builds on Feedback Transformer and Recurrent Transformer; RLT as a substitute feeds the earlier last decoder output into the following decoder enter and runs recurrence over the immediate too. Depth-wise reuse connects to Universal Transformers and recurrent-depth latent reasoning; the replay argument extends Zhang’s prefill-decode kernel mismatch note.

Interactive Explainer

Key Takeaways

  • RLT carries the complete decoder state (last output plus layerwise SWA cache) across each immediate and response token with no boundary reset.
  • Reference config: 48 tied encoder and decoder layers, 96 logical blocks per token, state path of 48t blocks after t tokens.
  • Hardware alternatives: encoder parallelism and batching across sequences; no parallel scan or reduced-prefill speedup is claimed.
  • RL replay rebuilds all states underneath present parameters whereas preserving recorded habits log-probabilities as ratio denominators.
  • No measured outcomes: reasoning high quality, effectivity, and RL scaling stay open validation targets.

Check out the Technical Report, GitHub repository, and Project Page. All credit score goes to the researcher of this challenge. Also, be at liberty to comply with us on Twitter and don’t overlook to affix our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to companion with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and many others.? Connect with us

The put up A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth appeared first on MarkTechPost.

Similar Posts