Comparison
Teleoperation data vs robot demonstration data
Robot demonstration data shows how a task should be performed. Teleoperation data is a specific kind of demonstration captured while a human controls a robot and records robot actions, states, and observations. Buyers should choose based on whether they need human behavior, robot action traces, or both.
Comparison
| Data | Usually includes | Use when |
|---|---|---|
| Human demonstration | Video, pose, task outcome | You need human behavior examples |
| Robot demonstration | Robot task episode and outcome | You need embodiment-specific examples |
| Teleoperation trace | Robot states, actions, observations | You need policy-training action data |
Why the distinction matters
Teleoperated demonstrations are not just videos of a successful robot episode. They are demonstrations where the operator's commands are recorded as robot-control data, so the resulting episode can preserve actions, observations, rewards, and metadata at each step [1]. ALOHA is a useful example because its demonstrations were collected through a custom teleoperation interface, then used for end-to-end imitation learning [2].
[2]"We present a low-cost system that performs end-to-end imitation learning directly from real demonstrations, collected with a custom teleoperation interface."
The distinction matters because embodiment-specific robot traces carry fields that a generic task demonstration may omit: teleoperation hardware, synced camera views, robot state, gripper state, and controller actions. DROID documents a robot collection rig with a Franka arm, cameras, wrist camera, and Oculus teleoperation controller [3]. RoboSet makes the source split explicit by separating kinesthetic demonstrations from teleoperated demonstrations and counting 9,500 teleoperated trajectories inside a 30,050-trajectory corpus [4]. Open X-Embodiment then shows why action representation matters for policy learning: its robot actions are represented as gripper-frame pose and gripper-opening dimensions [5].
What to ask suppliers for
When issuing a teleop sourcing request, ask suppliers to prove that the delivery contains more than demonstration footage. Require the recording format, episode metadata, action labels, and observation streams up front. LeRobot's v3 format is a practical reference point because it standardizes tabular state/action data, video shards, and metadata for robot-learning datasets [6]. For transport and replay, use a schema-preserving container such as MCAP when the delivery needs timestamped multimodal robotics logs.
[6]"LeRobotDataset v3.0 is a standardized format for robot learning data. It provides unified access to multi-modal time-series data, sensorimotor signals and multi‑camera video, as well as rich metadata for indexing, search, and visualization on the Hugging Face Hub."
The supplier checklist should include per-step joint positions, joint velocities, joint torques, gripper width, end-effector pose, image observations, action arrays, rewards, and skill or success labels where the task requires them [7]. Ask how the dataset was recorded and converted, because recording tooling and repository implementation shape what fields will survive delivery [8] [9]. Finally, ask suppliers to state whether an episode is teleoperated, kinesthetic, or another demonstration type, since sources such as LIBERO and RoboSet treat those collection modes as distinct provenance fields [10].
Teleoperation vs demonstrations: buyer matrix
Teleoperation data is the right request when the training stack needs commands and robot state, not just a camera view of a task. Human demonstrations are useful for task grammar and hand-object cues; robot demonstrations are useful for embodiment-specific behavior; teleoperation traces are useful when the policy must learn from synchronized observations, actions, and state.
| Source | Fields to require | Best use | Reject if |
|---|---|---|---|
| Human video demo | POV/external video, task labels, consent, outcome | task understanding and world-model context | sold as action-producing robot data |
| Robot demo | robot observations, state, episode outcome, embodiment | imitation examples for a target robot | no action/state schema |
| Teleoperation trace | operator controls, actions, robot state, cameras, timestamps | policy learning and recovery behavior | controller provenance omitted |
| Kinesthetic demo | human-guided robot motion, state/action, contact notes | contact-rich manipulation | collection mode not labeled |
| Eval set | held-out scenes, failures, scoring rubric | accept/reject supplier or model | duplicates training data |
Sample manifest and acceptance gates
A teleoperation sample manifest should list episode_id, task instruction, robot embodiment, operator/session ID, collection mode, observation streams, action representation, robot state fields, timestamps, controller hardware, success/failure label, split, license/provenance files, and conversion notes. The acceptance test is simple: the buyer should be able to load a sample, align frames to actions, identify episode boundaries, and audit rights without asking the supplier for hidden context; if the buyer's downstream stack expects Hugging Face tooling, align the same fields to the LeRobot format.
- Choose teleoperation when the model needs action/state traces or recovery behavior from the target embodiment.
- Choose human demonstrations when the model mainly needs task vocabulary, affordances, or first-person hand-object context.
- Choose robot demonstrations when embodiment fit matters but live teleop collection is not required for every episode.
Format choices: LeRobot, RLDS, HDF5, and MCAP
Use LeRobot when the training stack expects Hugging Face/PyTorch robot-learning datasets, RLDS when the stack is episode/step oriented, HDF5 when lab arrays and compact hierarchy matter, and MCAP when timestamped robot-native logs must preserve many channels. Format conversion can reorganize data, but it cannot recreate missing controls, state, timestamps, consent, or license files; policy teams should connect format review to VLA training data requirements before purchase.
Related pages
Use these to move from category-level context into specific task, dataset, format, and comparison detail.
External references and source context
- RLDS: Reinforcement Learning Datasets
RLDS represents episodic sequential-decision data as episodes of steps, with observations, actions, rewards, discounts, and metadata available at step level.
GitHub ↩ - Teleoperation datasets are becoming the highest-intent physical AI content category
ALOHA presents real demonstrations collected with a custom teleoperation interface and trains imitation-learning policies from those demonstrations.
tonyzhaozh.github.io ↩ - Project site
DROID uses a robot setup with a Franka Panda arm, multiple cameras, and an Oculus Quest 2 headset with controllers for teleoperation.
droid-dataset.github.io ↩ - Dataset page
RoboSet explicitly separates kinesthetic demonstrations from teleoperated demonstrations and reports 9,500 teleoperated trajectories within 30,050 total trajectories.
robopen.github.io ↩ - Project site
Open X-Embodiment describes robot actions as a seven-dimensional vector with gripper-frame pose and gripper-opening components for robot policy training.
robotics-transformer-x.github.io ↩ - LeRobot dataset documentation
LeRobotDataset v3 is a standardized robot-learning data format covering multi-modal time-series data, sensorimotor signals, multi-camera video, and metadata.
Hugging Face ↩ - Dataset documentation
FurnitureBench demonstration files include camera observations, end-effector pose, joint positions, joint velocities, joint torques, gripper width, actions, rewards, and skills.
clvrai.github.io ↩ - LeRobot documentation
LeRobot documentation is the developer-facing entry point for recording and using robot datasets in the LeRobot ecosystem.
Hugging Face ↩ - LeRobot GitHub repository
The LeRobot repository is the implementation home for recording, converting, and training with robot learning datasets.
GitHub ↩ - Dataset page
LIBERO dataset documentation distinguishes teleoperated and kinesthetic demonstrations for robot manipulation datasets.
libero-project.github.io ↩
FAQ
Is teleoperation data always robot demonstration data?
Teleoperation data is usually robot demonstration data because it records a robot performing a task under human control. It is more specific than a generic demonstration because it includes robot state and action traces.
Can human demonstrations train robots?
Human demonstrations can help with task understanding and world-model pretraining. For direct policy learning, buyers often also need robot embodiment data or a conversion pipeline.
What is robot trajectory data?
Robot trajectory data records how the robot moves through a task, often including joint states, end-effector poses, velocities, actions, timestamps, and success or failure outcomes.
Which one should I request first?
Start from the model requirement. If the model needs human interaction cues, request egocentric or human demonstrations. If it needs action-producing policy data, request teleoperation traces or robot demonstrations.
Looking for teleoperation data vs robot demonstration data?
Specify modality, task, environment, requested rights posture, and delivery format. Truelabel routes the request to candidate capture partners and helps scope consent/provenance artifacts and commercial licensing requirements for buyer review before delivery.
Request robot demonstration data