Google DeepMind Ships Three Physical AI Models For Whole Body Control, Dexterity And Multi Robot Collaboration
Google DeepMind has launched Gemini Robotics 2, the intelligence layer for its subsequent technology of robots. The launch strikes the stack previous table-top manipulation into complete physique management, 5 finger dexterity and multi robotic teamwork. It ships as three separate fashions with three totally different entry tiers.
Most robots in the present day are pre-programmed or tele-operated for slender, repetitive job sequences. They don’t adapt to unpredictable environments, and expertise not often switch between robotic our bodies. Gemini Robotics 2 targets all three limits directly.
TL;DR
- Three fashions ship collectively: a VLA, an embodied reasoning VLM, and an on-device VLA.
- One checkpoint drives Apollo 2 with two totally different arms plus a Franka Duo gripper.
- Gemini Robotics ER 2 is public preview; the VLA and on-device fashions keep gated.
- Multi-finger dexterity continues to be the weak axis, starting from 32% to 92%.
- ASIMOV-Agentic, a brand new security benchmark, is on Hugging Face underneath CC-BY-4.0.
3 Models and What They Do
Gemini Robotics 2
Gemini Robotics 2 is the vision-language-action (VLA) mannequin. It converts imaginative and prescient and language enter into motor management. It can drive full humanoids from toes to fingertips, and different bi-arm robots. It additionally handles dexterous manipulation on each multi-finger arms and parallel grippers.
Gemini Robotics ER 2
Gemini Robotics ER 2 is the embodied reasoning (ER) mannequin. It is a imaginative and prescient language mannequin that acts because the excessive stage mind. It communicates with people, understands the bodily world, and plans multi-step duties lasting a number of minutes. Per its model card, ER 2 relies on Gemini 3.5 Flash. It accepts interleaved textual content, picture, video and audio with a context window of as much as 128k, and emits textual content as much as 64K tokens.
Gemini Robotics On-Device 2
Gemini Robotics On-Device 2 is the environment friendly VLA optimized to run regionally on the robotic. Its model card states it’s constructed on Gemini Robotics 1.5 know-how and Google’s on-device Gemma fashions. Inputs are textual content, photos and robotic proprioception as numerical values. Outputs are robotic actions as numerical values.
The division of labor issues for system design. ER 2 plans and tracks the duty, then arms motor execution to a VLA declared as a device. Developers register low stage management interfaces, resembling VLA fashions or navigation APIs, as callable instruments. They then stream multimodal video, audio or textual content instantly into the mannequin.
Whole physique management on Apptronik Apollo 2
Previous Gemini Robotics fashions managed solely the humanoid’s higher physique for table-top duties. Gemini Robotics 2 extends management to complete physique movement for the primary time.
The labored instance makes use of Apptronik’s Apollo 2. Given the instruction “put the watering can into the inexperienced bin within the backside shelf,” Apollo walks to the desk and picks up the watering can. It then takes a number of steps to the cabinets and locations the thing at its vacation spot.
Google DeepMind is direct in regards to the remaining hole. It states that its robots have extra to advance in motion velocity.
Dexterity throughout arms and grippers
Gemini Robotics 2 can management the five-fingered, 22 degree-of-freedom SharpaWave hand on Apollo 2. Reported actions embody tying knots and sealing a ziplock bag. The identical mannequin additionally operates customary two-fingered parallel grippers on a Franka Duo platform for duties like tight packing.
The reported success charges come from one mannequin checkpoint controlling three embodiments: Apollo 2 with SharpaWave arms, Apollo 2 with Inspire arms, and Franka Duo with a Robotiq gripper.
| Category | Embodiment | Task | Success |
|---|---|---|---|
| General complete physique manipulation | Apollo 2 + Inspire arms | Pick up from shelf | 76.3% |
| General complete physique manipulation | Apollo 2 + Inspire arms | Pick up from desk | 68.4% |
| General complete physique manipulation | Apollo 2 + Inspire arms | Pick up from flooring | 45.7% |
| Multi-finger dexterity | Apollo 2 + Sharpa arms | Unscrew bulb | 92% |
| Multi-finger dexterity | Apollo 2 + Sharpa arms | Tie trash bag | 44% |
| Multi-finger dexterity | Apollo 2 + Sharpa arms | Ziplock | 40% |
| Multi-finger dexterity | Apollo 2 + Sharpa arms | Screw bulb | 36% |
| Multi-finger dexterity | Apollo 2 + Sharpa arms | Dustpan | 32% |
| Gripper dexterity | Franka Duo | Precise insertion duties | 89.6% |
| Gripper dexterity | Franka Duo | Diverse device kitting | 78.9% |
| Gripper dexterity | Franka Duo | General decide and place | 74.2% |
ER 2: Temporal Intelligence and Tool Orchestration
The developer X post focuses on an issue that not often will get benchmarked: realizing when a job is definitely accomplished.
Progress classification:
Each body in a video feed is assigned to considered one of 5 progress ranges, from 0-20% via 80-100%. Gemini Robotics ER 2 reaches 57.4% accuracy on this job. Google DeepMind experiences this outperforms earlier technology fashions and competing frontier fashions.
Moment discovering:
This measures whether or not the mannequin can determine the precise body the place a crucial occasion happens. An instance is the second to cease pouring espresso right into a cup. ER 2 reaches 91.3% accuracy with a 0.96 second imply absolute distance. Google DeepMind experiences it competing intently with a lot bigger mannequin classes at 4x the execution velocity.
Tool orchestration:
ER 2 is evaluated throughout three management modes: actual VLA, sim VLA, and human tele-op. It outperforms Gemini Robotics ER 1.6 on all three.
ER 2 integrates with the Gemini Live API via a bidirectional streaming endpoint. The acknowledged goal is eradicating the stop-and-think pauses that break multi-step execution. It can even natively name instruments like Google Search or any user-defined perform.
Three spatial capabilities have been upgraded. Success and failure detection now runs on uncooked video feeds as a substitute of static snapshots, which catches mid-execution spills and slips. General instrument studying extends past round dials to digital shows, linear scales, rulers and liquid thermometers, examined throughout 10 instrument sorts. Spatial visible query answering improves via Gemini’s multimodal advances.
Google DeepMind constructed a demo with Spot from Boston Dynamics, utilizing ER 2 to orchestrate Spot navigation and manipulator APIs. Sample code is obtainable within the robotics-samples repository.
Multi-Robot Collaboration
Gemini Robotics 2 introduces collaboration between totally different robotic sorts. The reasoning is that no single robotic matches each job. A wheeled rover fits indoor work, whereas a humanoid handles uneven terrain higher.
Robots talk via a shared semantic understanding at hand off subtasks. The demonstrated pairing is Apptronik’s Apollo 2 with a Franka F3 Duo.
On-Device 2: Adapting to New Robot Bodies
Gemini Robotics On-Device 2 targets functions that can’t depend on community latency or connectivity. It is natively multi-embodiment and inherits the movement switch strategies from Gemini Robotics 1.5.
Google DeepMind experiences adaptation to new bi-arm embodiments in a number of hours, usually with fewer than 200 examples. This holds for embodiments with drastically totally different shapes, sensors and levels of freedom. Demonstrated platforms embody Dexmate, SO101 and Trossen.
The mannequin card publishes an information scaling comparability in opposition to On-Device 1 on platforms launched solely throughout post-training:
| Platform | Model | Start | End |
|---|---|---|---|
| SO101 | On-Device 2 | 6.7% | 53.3% |
| SO101 | On-Device 1 | 0.0% | 6.7% |
| Dexmate | On-Device 2 | 24.4% | 75.6% |
| Dexmate | On-Device 1 | 13.3% | 33.3% |
The SO101 result’s the sharper one. On-Device 1 barely leaves the ground, whereas On-Device 2 crosses 50%.
The mannequin card additionally lists limits. On-Device 2 is proscribed in generalizing to out-of-distribution duties and in controlling high-degree-of-freedom robots.
Availability
| Model | Access |
|---|---|
| Gemini Robotics ER 2 | Public preview through Gemini API and Google AI Studio; personal preview on Gemini Enterprise Agent Platform |
| Gemini Robotics 2 (VLA) | Early-access companions |
| Gemini Robotics On-Device 2 | Trusted Testers |
The AI Studio launch hyperlink makes use of the mannequin string gemini-robotics-er-2-preview. Getting-started notebooks are within the robotics-samples repository.
Interactive explainer
Sources: Gemini Robotics 2 announcement, Gemini Robotics ER 2 developer post, ER 2 model card, On-Device 2 model card, ASIMOV-Agentic dataset and Safety Technical Report.
The put up Google DeepMind Ships Three Physical AI Models For Whole Body Control, Dexterity And Multi Robot Collaboration appeared first on MarkTechPost.
