Training large-scale transformers stably has been a longstanding challenge in deep learning, particularly as models grow in size and expressivity. MIT researchers tackle a persistent problem at its root: the unstable growth of activations and loss spikes caused by unconstrained weight and activation norms. Their solution is to enforce provable Lipschitz bounds on the transformer…
In this tutorial, we explore how to use the SHAP-IQ package to uncover and visualize feature interactions in machine learning models using Shapley Interaction Indices (SII), building on the foundation of traditional Shapley values. Shapley values are great for explaining individual feature contributions in AI models but fail to capture feature interactions. Shapley interactions go…
Deep Cogito has released Cogito v2, a new family of open-source AI models that sharpen their own reasoning skills. Released under an open-source licence, the new Cogito v2 lineup includes four hybrid reasoning AI models: two mid-sized at 70B and 109B parameters, and two large-scale versions at 405B and 671B. The largest, a 671B Mixture-of-Experts…
A leak suggests that OpenAI is about to launch a powerful new open-source AI model, potentially within hours. The evidence comes from a trail of digital breadcrumbs, eagerly pored over by developers. At the centre of it all are screenshots showing a series of model repositories with names like yofo-deepcurrent/gpt-oss-120b and yofo-wildflower/gpt-oss-20b. The repos have…
The generative AI landscape is dominated by massive language models, often designed for the vast capacities of cloud data centers. These models, while powerful, make it difficult or impossible for everyday users to deploy advanced AI privately and efficiently on local devices like laptops, smartphones, or embedded systems. Instead of compressing cloud-scale models for the…
Large language models (LLMs) specialized for coding are now integral to software development, driving productivity through code generation, bug fixing, documentation, and refactoring. The fierce competition among commercial and open-source models has led to rapid advancement as well as a proliferation of benchmarks designed to objectively measure coding performance and developer utility. Here’s a detailed,…
Estimated reading time: 5 minutes Table of contents Introduction The ThinkAct Framework Experimental Results Ablation Studies and Model Analysis Implementation Details Conclusion Introduction Embodied AI agents are increasingly being called upon to interpret complex, multimodal instructions and act robustly in dynamic environments. ThinkAct, presented by researchers from Nvidia and National Taiwan University, offers a breakthrough…
Vision Language Models (VLMs) allow both text inputs and visual understanding. However, image resolution is crucial for VLM performance for processing text and chart-rich data. Increasing image resolution creates significant challenges. First, pretrained vision encoders often struggle with high-resolution images due to inefficient pretraining requirements. Running inference on high-resolution images increases computational costs and latency…
Reinforcement Learning with Verifiable Rewards (RLVR) allows LLMs to perform complex reasoning on tasks with clear, verifiable outcomes, with strong performance in mathematics and coding. However, many real-world scenarios lack such explicit verifiable answers, posing a challenge for training models without direct reward signals. Current methods address this gap through RLHF via preference ranking, where…
Google has made its most powerful AI video creator, Veo 3, available for everyone to use on its Vertex AI platform. And for those who need to work quickly, a speedier version called Veo 3 Fast is also ready-to-go for quick creative work. Ever had a brilliant idea for a video but found yourself held…