Evaluating progress of LLMs on scientific problem-solving
General Science
Human-Computer Interaction and Visualization
The Inefficiency of Static Chain-of-Thought Reasoning in LRMs Recent LRMs achieve top performance by using detailed CoT reasoning to solve complex tasks. However, many simple tasks they handle could be solved by smaller models with fewer tokens, making such elaborate reasoning unnecessary. This echoes human thinking, where we use fast, intuitive responses for easy problems…
Navigating the dense urban canyons of cities like San Francisco or New York can be a nightmare for GPS systems. The towering skyscrapers block and reflect satellite signals, leading to location errors of tens of meters. For you and me, that might mean a missed turn. But for an autonomous vehicle or a delivery robot,…
Rethinking Audio-Based Human-Computer Interaction Machines that can respond to human speech with equally expressive and natural audio have become a major goal in intelligent interaction systems. Audio-language modeling extends this vision by combining speech recognition, natural language understanding, and audio generation. Rather than relying on text conversions, models in this space aim to understand and…
In this tutorial, we introduce TinyDev class implementation, a minimal yet powerful AI code generation tool that utilizes the Gemini API to transform simple app ideas into comprehensive, structured applications. Designed to run effortlessly in Notebook, TinyDev follows a clean three-phase workflow—Plan → Files → Code—to ensure consistency, functionality, and modular design. Whether building a…
Post-training methods for pre-trained language models (LMs) depend on human supervision through demonstrations or preference feedback to specify desired behaviors. However, this approach faces critical limitations as tasks and model behaviors become very complex. Human supervision is unreliable in these scenarios as LMs learn to mimic mistakes in demonstrations or exploit inherent flaws in feedback…
Rise of Autonomous Coding Agents in System Software Debugging The use of AI in software development has gained traction with the emergence of large language models (LLMs). These models are capable of performing coding-related tasks. This shift has led to the design of autonomous coding agents that assist or even automate tasks traditionally carried out…