Dual-Expert Flow Matching for Dynamic Robot Manipulation

VIPER

Anonymous Authors
Under review at ICLR 2027

Dynamic manipulation depends not only on where the robot moves, but how fast it moves.

Evaluation rollout on 8 randomly selected episodes using our π0.5 backbone with VIPER policy.

Same position. Different velocity. Different outcome.

Position-only VLAs specify where the robot moves but leave important execution dynamics implicit. For tossing, flipping, sliding, pushing, and catching, two executions can follow nearly the same spatial path and still fail or succeed depending on speed. At contact or release, velocity often decides the outcome.

Motivation: same spatial trajectory, three velocity profiles, three outcomes — too slow, correct, too fast.

Predict velocity, don't reconstruct it.

VIPER adds a dedicated velocity expert to an existing VLA: the original action expert predicts joint positions, while the velocity expert predicts joint velocities. The two experts are not independent. Coupled dual-expert flow matching shares flow time, Gaussian noise, noisy-state conditioning, and inference initialization so position and velocity stay temporally aligned.

VIPER method overview: coupled dual-expert flow matching for joint position and velocity prediction.

Dynamic manipulation tasks

Six momentum-sensitive tasks. Simulation: Bottle Toss, Cube Slide, Bread Flip. Real world: Drop Catch, Bottle Handover, Ball Push.

Bottle Toss Sim
Cube Slide Sim
Bread Flip Sim
Drop Catch Real
Bottle Handover Real
Ball Push Real

Results

Main simulation results with π0.5. VIPER is also evaluated with RTC and TEMPO; it does not improve every backbone/task pair.

Table 1: VIPER improves all three backbones on nearly every task. Success rate in %. Position-Only is the backbone as published, Spline-FF adds spline velocity feedforward, and VIPER adds the velocity expert. Avg is the mean over the three tasks on each side. Bold = best, italic = second best within a block. 50 trials per physical task, 100 rollouts per simulation task.
# Method Velocity source Physical Robot Tasks RoboTwin Simulation Tasks
Ball Push Bottle Handover Drop Catch Avg Bottle Toss Bread Flip Cube Slide Avg
π0.5 backbone (general-purpose VLA)
1Position-OnlyController stops at every target 28263630.0 9202016.3
2Spline-FFDerivative of a spline through targets 24304031.3 40301227.3
3VIPER (ours)Predicted from demonstrations 524446 47.3 57425651.7
Δ Success +24+18+10+17.3 +48+22+36+35.3
RTC backbone (asynchronous action-chunk execution)
4Position-OnlyController stops at every target 32445042.0 9113317.7
5Spline-FFDerivative of a spline through targets 32385040.0 19204528.0
6VIPER (ours)Predicted from demonstrations 365048 44.7 55396252.0
Δ Success +4+6−2+2.7 +46+28+29+34.3
TEMPO backbone (temporal context for dynamic tasks)
7Position-OnlyController stops at every target 34746658.0 8254225.0
8Spline-FFDerivative of a spline through targets 22747256.0 28353031.0
9VIPER (ours)Predicted from demonstrations 328686 68.0 40385644.7
Δ Success −2+12+20+10.0 +32+13+14+19.7

Baseline vs. VIPER

Each pair shows the same task executed by the π0.5 backbone and the same π0.5 backbone with VIPER.

Bottle TossSim
Baseline
Baseline + VIPER
Cube SlideSim
Baseline
Baseline + VIPER
Bread FlipSim
Baseline
Baseline + VIPER
Drop CatchReal
Baseline
Baseline + VIPER
Bottle HandoverReal
Baseline
Baseline + VIPER
Ball PushReal
Baseline
Baseline + VIPER

Why not just differentiate position?

Smooth motion is not necessarily the demonstrated motion.

Position only

Specifies where to move, but velocity remains implicit.

Spline velocity

Smoother execution, but short high-speed peaks can be lost.

VIPER

Learns demonstrated velocity directly, coupled to position.

Velocity profile comparison: spline-differentiated velocity vs. VIPER vs. demonstration.
Abstract

Vision-Language-Action (VLA) models predict where a robot should move, but not how fast. The velocity profile is neither supervised nor predicted; the low-level controller reconstructs it from position targets. For quasi-static manipulation this is harmless. For dynamic manipulation, in which the robot tosses, flips, slides, or pushes, the velocity at contact decides the outcome, and position tracking distorts it. We show that accurate position does not imply accurate velocity and we establish the need to learn velocity from the demonstrations.

We propose VIPER, which adds a velocity expert to a pretrained VLA and denoises velocity jointly with the VLA's position expert from shared noise to ensure temporal alignment between the noise-free position-velocity actions. VIPER leaves the VLA's backbone frozen and attaches to any VLA with a flow-matching action expert.

We evaluate VIPER on three VLA backbones and six dynamic manipulation tasks in simulation and on a real robot, from tossing and flipping to catching and handover. VIPER raises mean success from 32% to 51%, and improves backbones that already reason about temporal context.