LocoManiLab: A Unified Testbed for Data, Policies, Controllers, and Deployment in Humanoid Loco-Manipulation

Dewei Wang1,2*§Jinrui Han1,3*§Jiyuan Shi1*Chenyun Zhang1Xianbo Lu1,4Jiabao Wu5Jiqin Feng1,2Jipeng Kong6Ping Luo3Chenjia Bai1,7†Xuelong Li1†
1 Institute of Artificial Intelligence (TeleAI), China Telecom 2 University of Science and Technology of China 3 The University of Hong Kong 4 Zhejiang University 5 Shanghai Jiao Tong University 6 ShanghaiTech University 7 Gamma Robotics

Equal contribution Co-project leaders Corresponding authors

Published
PaperCodeComing soonDatasetComing soonLoco-Mani Arena
234Task definitions102 atomic + 132 compositional
13Scene instancesAcross 5 household categories
12Benchmarked policiesIn Loco-Mani Arena
5Whole-body controllersOne shared G1 embodiment
14000+ episodes Episodes Teleoperated demonstrations
50+ hours Teleoperation Across five controllers

What is
LocoManiLab?

A full-stack testbed for humanoid loco-manipulation.

Bring tasks, data, policies, and whole-body control into one workflow—from simulation to real-world deployment.

TASKS & ENVIRONMENTS

Start with everyday tasks.

Open a fridge, crouch to reach a low drawer, pull out a chair, lift and carry objects, or pack them into a basket. Compose these whole-body skills into longer household tasks.

  • 102 atomic tasks
  • 132 compositional tasks
  • 13 scenes
Explore the scenes

DATA COLLECTION & GENERATION

Demonstrate. Generate. Learn.

Capture whole-body demonstrations through VR teleoperation in simulation and on real robots, and expand training data with motion synthesis. Keep vision, language, actions, and state synchronized.

  • VR teleoperation (sim & real)
  • Motion generation
  • Policy-ready data
Explore the data

POLICIES

Bring your policy.

Bring together robot state, language instructions, and images to train and compare vision-language-action and world-action policies, with a shared embodiment and policy-specific data adapters.

  • 12 integrated policies
  • VLA · WAM
  • State · Instructions · Images
Meet the policies

CONTROLLERS

Connect decisions to motion.

Pair policies with five whole-body controllers, each with its own action and state interface, on the dataset’s Unitree G1 with Dex3 dexterous hands.

  • Locomotion
  • Manipulation
  • Whole-body coordination

More humanoids to come.

Compare the controllers

DOMAIN RANDOMIZATION

One task. Many variations.

Vary appearance and lighting, add distracting objects, and change target positions and orientations. Test the same skill under different visual and spatial conditions.

  • Appearance & lighting
  • Distractors
  • Target poses
Watch appearance randomization

TASK & MOTION EVALUATION

Evaluate behavior at every level.

Compare task success and generalization alongside stability, motion quality, and efficiency—across tasks, scenes, and randomization settings. Explore demonstration scaling and open-space motion transfer in the experimental studies.

  • Task success
  • Motion quality
  • Generalization · Scaling
Explore Loco-Mani Arena

SIM2REAL

Take policies into the real world.

Bridge simulation and the physical G1 with consistent task and deployment interfaces. Explore refrigerator opening, shelf-to-table transfer, and conveyor pick-and-place across simulation and real trials.

  • Real-to-sim scenes
  • Sim-to-real transfer
  • Unitree G1
Explore sim-to-real
ONE CONNECTED TESTBED

Shared embodiment Unified data & interfaces Task & motion metrics

ENVIRONMENTS

Household scenes

13 scene instances across kitchen, living room, bedroom, bathroom, and office environments, including two real-to-sim scenes.

Loading room model…
Drag to orbit · Scroll to zoom · Right-drag to pan · Touch: pinch to zoom, two-finger drag to pan Use arrow keys to rotate, Shift and arrow keys to pan, plus and minus to zoom, and zero to reset.
100%

BENCHMARK

Tasks and demonstrations

Task examples.

Atomic tasks

102 definitions

Isolate one skill: navigate, open / close, pick / lift, place, push / pull, or pour / reorient.

Compositional tasks

132 definitions

Combine skills into two to four ordered subgoals under one instruction.

01 / WHOLE-BODY REPLAY
02 / EGOCENTRIC REPLAYSame replay states
0:00 / 0:00
Paired views rendered from the same states · seen tasks · recorded speed at 1×

SYSTEM

Policies and controllers

Native policy and controller interfaces share a common training and evaluation pipeline.

DATASET

Demonstration datasets

14000+ episodes and 50+h of teleoperation across five controllers.

TELEOPERATION CORPUS
14,160episodes
Generated corpus 20 tasks GPT augmentation · Kimodo 40 sample episodes · 2 sources Explore generated episodes

Explore GPT-augmented and Kimodo-generated trajectories across 20 tasks, with one selected episode per task and source. GPT examples include paired third-person and egocentric views; Kimodo examples currently include egocentric recordings.

VR teleoperation

Retarget VR head and hand motion; record finger targets and measured joints separately.

Synchronized episodes

Keep egocentric RGB, instructions, actions, measured state, and collection metadata.

Policy-ready data

Convert versioned episodes into each policy’s native training and deployment format.

320 × 240 egocentric RGB50 Hz nominal recording5 controller layouts
Motion synthesis & augmentation

Kimodo proposes motion; frozen SONIC executes it. Execution returns guide generator fine-tuning, comparing pretrained, full-parameter, and LoRA variants. Training and final testing are ongoing.

DOMAIN RANDOMIZATION

Visual domain randomization

Watch recorded texture and lighting changes along a continuous dataset trajectory, with third-person and ego views side by side.

EXPERIMENTAL RESULTS

From data to behavior

Explore how training data, demonstration scale, and motion adaptation shape task performance.

Explore Loco-Mani Arena

DATA SOURCES

Choosing the training data

Compare teleoperation, motion synthesis, and augmentation across two policies on the same twenty tasks.

DEMONSTRATION SCALING

Learning from more demonstrations

Follow each policy from small datasets to larger training budgets, and explore how the controller changes the curve.

SIM2REAL

Bridge sim and real

Three tasks, from simulated scenes to the physical G1. Explore each simulation alongside multiple real-world trials.

Citation