NVIDIA Releases Cosmos 3 Edge: A 4B-Parameter Open World Model That Reasons and Generates Robot Actions On-Device
NVIDIA has launched Cosmos 3 Edge, a 4-billion-parameter open world mannequin constructed to run on-device. It helps robots and imaginative and prescient AI brokers perceive environment, purpose in actual time, and generate robotic actions domestically.
The Cosmos 3 family included Cosmos 3 Nano (16B) and Cosmos 3 Super (64B) shipped on May 31, 2026 at GTC Taipei. Edge is the third and smallest tier, at roughly one-sixteenth the scale of Super.
The downside is restricted. Machines function on the edge in factories, warehouses, and hospitals. They want information middle–degree efficiency on memory-constrained methods. Cosmos 3 Edge targets that hole.
One motion illustration throughout embodiments
Physical methods describe actions in a different way. A car makes use of ego pose and motion. A digital camera makes use of digital camera movement. A robotic arm makes use of the pose of its finish effector, and a gripper provides grasp state.
Cosmos 3 maps these embodiments into a standard motion illustration. Actions are encoded as compact geometric vectors that seize translation, rotation, and manipulation state.
This connects management to the visible construction of the world. The mannequin associates pixel modifications with bodily movement and management inputs. Generated video then turns into greater than a prediction. It represents how the world ought to change in response to an motion.
Supported motion dimensions rely on the embodiment. The Cosmos GitHub repository lists digital camera movement (9D), autonomous car (9D), selfish movement (57D), single-arm robotic (10D), dual-arm robotic (20D), and humanoid robotic (29D).
Policy mode runs in each instructions
As a coverage, Cosmos 3 Edge predicts an motion along with its anticipated visible consequence. Current state goes in; an motion and its probably visible end result come out.
Action flows in each instructions. The mannequin can predict the impact of an motion, or infer the motion from its impact. This connects world modeling on to robotic coverage coaching and analysis.
NVIDIA additionally launched Cosmos 3 Edge Policy (DROID). It is a robotic manipulation coverage post-trained on the DROID dataset for pick-and-place duties, with post-training scripts included. Developers can fine-tune on a small H100 cluster or an NVIDIA DGX Station earlier than deployment.
Is it Deployable
Cosmos 3 Edge delivers memory-efficient inference throughout NVIDIA edge computer systems. Targets embrace NVIDIA RTX PRO GPUs, NVIDIA DGX, GeForce RTX GPUs, and NVIDIA Jetson, together with the newly introduced Jetson T2000 and T3000 modules.
As a post-trained world motion mannequin (WAM), the mannequin operates at robot-control decision of 640×360 observations. On NVIDIA Jetson Thor it generates 32 actions per inference, whereas reaching real-time management at 15 Hz. For era, the Edge tier helps 256p and 480p resolutions, 12–30 fps, and 50–150 frames.
Using the open Cosmos framework, builders can post-train Cosmos 3 Edge for a particular embodiment and sensor set in a couple of day. NVIDIA positions a GeForce RTX 3070 or higher as an area on-ramp for prototyping.
Key Takeaways
- Cosmos 3 Edge is a 4B open world mannequin (2B dense reasoner) that runs on-device, launched July 20 on Hugging Face.
- A Mixture-of-Transformers design pairs an autoregressive reasoner tower with a diffusion generator tower by shared multimodal consideration.
- It hits 640×360 management decision, 32 actions per inference, and 15 Hz real-time management on NVIDIA Jetson Thor.
- Actions map to a standard translation/rotation/manipulation illustration, spanning digital camera, car, single-arm, dual-arm, and humanoid embodiments.
- Benchmarks (#1 on VANTAGE-Bench at 4B) are internally claimed; the mannequin ships underneath Linux Foundation OpenMDW-1.1.
Sources: Hugging Face launch post, Cosmos3-Edge model card, Cosmos 3 collection, NVIDIA technical report (PDF), Cosmos GitHub, NVIDIA developer blog, Jetson Thor blog and NVIDIA Newsroom: Japan coalition
The put up NVIDIA Releases Cosmos 3 Edge: A 4B-Parameter Open World Model That Reasons and Generates Robot Actions On-Device appeared first on MarkTechPost.
