Unifying Perception, Reasoning, and Control
Perceptron AI advocates for moving away from the fragmented landscape of Vision-Language Models (VLMs) and Vision-Language-Action (VLA) models toward unified "embodied foundation models." The goal is to create a single architecture capable of perceiving, reasoning, and acting in the physical world. By training jointly on text, images, video, and trajectories (from desktop use to video games), these models achieve frontier-level performance in embodied reasoning at a fraction of the cost of larger proprietary models.
Solving Data Sparsity and Context Bloat
Training on video data presents two major challenges: lack of ground truth and context bloat.
- Perceptive Objectives: Rather than predicting every pixel—which treats background noise with the same weight as critical contact points—Perceptron uses a learned "perceptive objective." This allows the model to automatically identify and prioritize the percepts that matter for future actions, such as the tip of a robotic gripper.
- Data-Sparse Mixture of Experts: To handle the massive context of "always-on" cameras, the team employs a router-based architecture that decides per layer which tokens to process and which to skip. This allows the model to dynamically allocate compute based on the task (e.g., zooming into a graph or segmenting specific objects), effectively managing context bloat without hard-coded architectural priors.
New Scaling Laws for Robotics
One of the most significant findings is a new scaling law for embodied agents: video pre-training can effectively substitute for expensive teleoperation data. The team observed that 10x more video pre-training data can compensate for 10x less teleop data. This is a critical breakthrough, as teleop data costs approximately $100 per hour to collect. By leveraging this trade-off, the model can perform complex, multi-step tasks—such as reading book titles and sorting them into bins—zero-shot, by emitting control tokens directly from its reasoning process.
Agentic Detection and Robustness
By framing detection as an agentic task rather than a static classification problem, the model can actively interact with its input. It can tile images, increase contrast, and propose boxes until it successfully identifies hard-to-find objects. This joint modeling of perception and control also leads to higher robustness against environmental changes, such as shifting backgrounds or lighting conditions, compared to traditional VLA models that often fail when the visual context is slightly altered.