Steertuning Claims a Robot Learns Tasks From Five Human Videos in Under Five Minutes

Five human demonstrations and less than five minutes were enough to prepare a robot for a new task in a new environment, according to an arXiv preprint describing a system called Steertuning. Its developers report a 71% success rate across 12 tasks in 12 new settings, ranging from laboratories to kitchens in new homes.

The system adapts an already pretrained model rather than training a policy only on the new task. Steertuning starts with a 400-million-parameter diffusion policy pretrained on 100 hours of densely annotated, egocentric human data showing random interactions. The researchers say this pretraining helps the policy handle background differences and generalize spatially and across objects.

For each new task, the system uses five human demonstrations. The model then undergoes 2,000 training updates in two minutes. During reverse sampling, its actions are steered using the demonstration whose observation is closest to that of the deployment environment. The paper says the entire process, from collecting data for a new task to deploying the robot policy, takes less than five minutes.

Steertuning also uses a data-processing pipeline for embodiment transfer between the human demonstrations and the robot. The abstract describes the pipeline as contributing to the reported results but does not explain its operation or quantify its contribution separately.

The preprint frames the work against robot-learning methods that require large numbers of robot demonstrations to produce robust and adaptive policies. Steertuning instead combines a policy pretrained on human data with five task-specific demonstrations for adaptation in each new environment.

The 71% aggregate success rate does not show how performance varied among the 12 tasks. The supplied abstract also does not give the number of trials, per-task scores or failure modes. It does not identify the robot hardware, action speed, sensing setup or execution latency, limiting what can be concluded from the abstract about the reported hardware performance.

The work is currently presented as an arXiv preprint, and its results are claims reported by the researchers. The authors say the full dataset, model weights, codebase and robot videos will be made available on their website. If released, those materials could support closer outside examination of the method and reported experiments.