VIPER
Dynamic manipulation depends not only on where the robot moves, but how fast it moves.
Position-only VLAs specify where the robot moves but leave important execution dynamics implicit. For tossing, flipping, sliding, pushing, and catching, two executions can follow nearly the same spatial path and still fail or succeed depending on speed. At contact or release, velocity often decides the outcome.
VIPER adds a dedicated velocity expert to an existing VLA: the original action expert predicts joint positions, while the velocity expert predicts joint velocities. The two experts are not independent. Coupled dual-expert flow matching shares flow time, Gaussian noise, noisy-state conditioning, and inference initialization so position and velocity stay temporally aligned.
Six momentum-sensitive tasks. Simulation: Bottle Toss, Cube Slide, Bread Flip. Real world: Drop Catch, Bottle Handover, Ball Push.
Main simulation results with π0.5. VIPER is also evaluated with RTC and TEMPO; it does not improve every backbone/task pair.
| # | Method | Velocity source | Physical Robot Tasks | RoboTwin Simulation Tasks | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Ball Push | Bottle Handover | Drop Catch | Avg | Bottle Toss | Bread Flip | Cube Slide | Avg | |||
| π0.5 backbone (general-purpose VLA) | ||||||||||
| 1 | Position-Only | Controller stops at every target | 28 | 26 | 36 | 30.0 | 9 | 20 | 20 | 16.3 |
| 2 | Spline-FF | Derivative of a spline through targets | 24 | 30 | 40 | 31.3 | 40 | 30 | 12 | 27.3 |
| 3 | VIPER (ours) | Predicted from demonstrations | 52 | 44 | 46 | 47.3 | 57 | 42 | 56 | 51.7 |
| Δ Success | +24 | +18 | +10 | +17.3 | +48 | +22 | +36 | +35.3 | ||
| RTC backbone (asynchronous action-chunk execution) | ||||||||||
| 4 | Position-Only | Controller stops at every target | 32 | 44 | 50 | 42.0 | 9 | 11 | 33 | 17.7 |
| 5 | Spline-FF | Derivative of a spline through targets | 32 | 38 | 50 | 40.0 | 19 | 20 | 45 | 28.0 |
| 6 | VIPER (ours) | Predicted from demonstrations | 36 | 50 | 48 | 44.7 | 55 | 39 | 62 | 52.0 |
| Δ Success | +4 | +6 | −2 | +2.7 | +46 | +28 | +29 | +34.3 | ||
| TEMPO backbone (temporal context for dynamic tasks) | ||||||||||
| 7 | Position-Only | Controller stops at every target | 34 | 74 | 66 | 58.0 | 8 | 25 | 42 | 25.0 |
| 8 | Spline-FF | Derivative of a spline through targets | 22 | 74 | 72 | 56.0 | 28 | 35 | 30 | 31.0 |
| 9 | VIPER (ours) | Predicted from demonstrations | 32 | 86 | 86 | 68.0 | 40 | 38 | 56 | 44.7 |
| Δ Success | −2 | +12 | +20 | +10.0 | +32 | +13 | +14 | +19.7 | ||
Each pair shows the same task executed by the π0.5 backbone and the same π0.5 backbone with VIPER.
Each pair shows the same task executed by the π0.5 backbone and the same π0.5 backbone with VIPER, using same camera as the paper tasks.
Smooth motion is not necessarily the demonstrated motion.
Specifies where to move, but velocity remains implicit.
Smoother execution, but short high-speed peaks can be lost.
Learns demonstrated velocity directly, coupled to position.
Vision-Language-Action (VLA) models predict where a robot should move, but not how fast. The velocity profile is neither supervised nor predicted; the low-level controller reconstructs it from position targets. For quasi-static manipulation this is harmless. For dynamic manipulation, in which the robot tosses, flips, slides, or pushes, the velocity at contact decides the outcome, and position tracking distorts it. We show that accurate position does not imply accurate velocity and we establish the need to learn velocity from the demonstrations.
We propose VIPER, which adds a velocity expert to a pretrained VLA and denoises velocity jointly with the VLA's position expert from shared noise to ensure temporal alignment between the noise-free position-velocity actions. VIPER leaves the VLA's backbone frozen and attaches to any VLA with a flow-matching action expert.
We evaluate VIPER on three VLA backbones and six dynamic manipulation tasks in simulation and on a real robot, from tossing and flipping to catching and handover. VIPER raises mean success from 32% to 51%, and improves backbones that already reason about temporal context.