Transformers use attention and Mixture-of-Experts to scale computation, but they still lack a native way to perform knowledge lookup. They re-compute the same local patterns again and again, which wastes depth and FLOPs. DeepSeek’s new Engram module targets exactly this gap by adding a conditional memory axis that works alongside MoE rather than replacing it….
A team of Stanford Medicine researchers have introduced SleepFM Clinical, a multimodal sleep foundation model that learns from clinical polysomnography and predicts long term disease risk from a single night of sleep. The research work is published in Nature Medicine and the team has released the clinical code as the open source sleepfm-clinical repository on…
Tencent Hunyuan researchers have released HY-MT1.5, a multilingual machine translation family that targets both mobile devices and cloud systems with the same training recipe and metrics. HY-MT1.5 consists of 2 translation models, HY-MT1.5-1.8B and HY-MT1.5-7B, supports mutual translation across 33 languages with 5 ethnic and dialect variations, and is available on GitHub and Hugging Face…
Google DeepMind Researchers introduce Gemma Scope 2, an open suite of interpretability tools that exposes how Gemma 3 language models process and represent information across all layers, from 270M to 27B parameters. Its core goal is simple, give AI safety and alignment teams a practical way to trace model behavior back to internal features instead…
OpenAI team has released their openai/circuit-sparsity model on Hugging Face and the openai/circuit_sparsity toolkit on GitHub. The release packages the models and circuits from the paper ‘Weight-sparse transformers have interpretable circuits‘. https://arxiv.org/pdf/2511.13653 What is a weight sparse transformer? The models are GPT-2 style decoder only transformers trained on Python code. Sparsity is not added after…
OpenAI team has released their openai/circuit-sparsity model on Hugging Face and the openai/circuit_sparsity toolkit on GitHub. The release packages the models and circuits from the paper ‘Weight-sparse transformers have interpretable circuits‘. https://arxiv.org/pdf/2511.13653 What is a weight sparse transformer? The models are GPT-2 style decoder only transformers trained on Python code. Sparsity is not added after…
Can a 3B model deliver 30B class reasoning by fixing the training recipe instead of scaling parameters? Nanbeige LLM Lab at Boss Zhipin has released Nanbeige4-3B, a 3B parameter small language model family trained with an unusually heavy emphasis on data quality, curriculum scheduling, distillation, and reinforcement learning. The research team ships 2 primary checkpoints,…
Los Angeles, December 11, 2025 — Marktechpost has launched ML Global Impact Report 2025 (AIResearchTrends.com). This academic report’s evaluation contains over 5,000 articles from greater than 125 nations, all printed throughout the Nature household of journals between January 1 and September 30, 2025. The scope of this report is strictly confined to this particular physique…
How do you retain RAG methods correct and environment friendly when each question tries to stuff 1000’s of tokens into the context window and the retriever and generator are nonetheless optimized as 2 separate, disconnected methods? A group of researchers from Apple and University of Edinburgh launched CLaRa, Continuous Latent Reasoning, (CLaRa-7B-Base, CLaRa-7B-Instruct and CLaRa-7B-E2E)…
How do you retain artificial information contemporary and numerous for fashionable AI fashions with out turning a single orchestration pipeline into the bottleneck? Meta AI researchers introduce Matrix, a decentralized framework the place each management and information circulate are serialized into messages that transfer by way of distributed queues. As LLM coaching more and more…