Asimov (YC W26) Alternatives: Physical AI Data Beyond Egocentric Video

Quick facts

Topic

Asimov YC W26

Audience

Procurement leads, ML ops, robotics engineers

Deliverable

Buyer-facing reference + procurement guidance

What Asimov (YC W26) Captures for Robotics Teams

Asimov joined Y Combinator's Winter 2026 batch to collect real-world human activity data for robotics training. It ships wearable hardware to collectors on several continents and returns egocentric video with layered annotations: 3D body pose, depth maps, semantic segmentation, and activity labels such as 'opening drawer' or 'pouring liquid,' plus hand trajectories and object interactions. The viewpoint matches the first-person perspective of EPIC-KITCHENS and Ego4D, the corpora most vision-language-action papers pretrain on.

That viewpoint is also the ceiling. Egocentric video carries no robot joint states, no gripper forces, and no proprioceptive feedback, so any manipulation policy trained on it still needs an inverse-kinematics or retargeting layer to turn human motion into robot commands, and that layer adds error the dataset cannot correct. Asimov fits vision-based activity recognition and high-level task planning; teams training closed-loop control need datasets that record the robot's own actions.

Asimov vs. the Alternative Categories at a Glance

The alternatives sort into four groups by what they physically record and how you get access. Match the row to the signal your policy actually consumes; that is the axis that should drive the choice.

Why Manipulation Trajectories Out-Train Human Video

A manipulation policy learns from data where every frame pairs an observation (RGB-D, proprioception) with an executable action (joint velocities, gripper commands). Human video breaks that pairing at the source.

RT-1 and RT-2 trained on 130,000+ robot trajectories carrying joint positions, gripper states, and end-effector poses sampled at 3 Hz. The Open X-Embodiment dataset aggregates 1 million trajectories across 22 embodiments, 527 skills, and 160,000 tasks in one RLDS schema^[1]; conditioning a transformer on embodiment tokens lets RT-X reach 50–70% zero-shot success on unseen robots against 10–20% for single-embodiment baselines^[2]. Egocentric video cannot supply that action distribution without added instrumentation.

Open manipulation sets cover most prototyping. DROID holds 76,000 teleoperated trajectories across 564 scenes and 84 tasks, each pairing RGB-D with 6-DOF poses and binary gripper states^[3], and BridgeData V2 adds 60,000 trajectories over 13 skills and 24 environments in RLDS^[4]. When temporal resolution matters, ALOHA records bimanual manipulation at 50 Hz; its 650-episode set trains Action Chunking Transformer policies to 80%+ success on cable routing^[5], and RoboSet contributes 2,000 WidowX episodes with per-episode failure labels. None of that exists in human video, which lacks both the action labels and the sampling rate closed-loop control needs.

Robotics-Native Formats Decide Pipeline Cost

Format choice sets how much ETL you pay before the first training step, and three containers dominate. RLDS wraps TensorFlow Datasets as (observation, action, reward) tuples in TFRecord shards, giving random-access sampling and distributed loading across GPU clusters; over 30 datasets, including BridgeData, RoboNet, and CALVIN, ship RLDS versions, so cross-dataset training needs no conversion.

MCAP is a multi-modal time-series container built as a ROS 2 bag replacement, storing synchronized RGB, depth, LiDAR, and IMU with microsecond timestamps; Foxglove plays 100 GB+ files in-browser without transcoding, and for ROS 2 capture MCAP runs roughly 3–5× smaller than rosbag2 with faster seeks. HDF5 stays the default for large manipulation sets: DROID stores its 76,000 trajectories in hierarchical groups like `/episode_000001/observations/rgb`, so you read one camera without loading depth and write in parallel during collection. LeRobot then standardizes 20+ of these datasets behind one sampling and batching API. Asimov publishes no format spec, so confirm export compatibility with your training stack before you commit budget.

When You Actually Need LiDAR, Point Clouds, and Third-Party Labels

Tabletop manipulation rarely needs LiDAR; navigation and outdoor work do. If your policy consumes LiDAR or radar, Scale AI's data engine annotates frame-synchronized RGB-D, LiDAR, radar, and IMU with 3D boxes and tracking IDs, and its Universal Robots partnership produced 50,000+ UR5e and UR10e trajectories^[6].

For point clouds specifically, Segments.ai labels 3D objects directly in point-cloud space, avoiding 2D-to-3D projection error, and open sets like the Waymo Open Dataset supply driving-grade geometry off the shelf.

If you already hold the raw sensor data and only need labels, Labelbox and Encord^[7] interpolate labels across frames, while managed crowds like Appen and CloudFactory absorb large labeling backlogs on quarter-long timelines.

The decision is simple: Asimov's egocentric pipeline records RGB and monocular depth only, so any navigation or sensor-fusion stack must add one of these paths.

How to Commission Custom Capture on a Marketplace

When no public set matches your embodiment, task distribution, or commercial-use rights, you commission capture instead of downloading it. On truelabel's physical AI data marketplace the flow is a spec-first bounty rather than a catalog purchase, which is what lets it cover tasks public repositories never recorded.

Delivered examples show the depth this reaches: Claru's kitchen set covers teleoperation episodes across common kitchen appliances, and its warehouse set captures pick-place-navigate sequences paired with LiDAR maps.

1. 01

   Post the spec

   Define task taxonomy, embodiment, sensor modalities, annotation layers, and episode targets. Buyers post the brief; matched capture partners respond.

2. 02

   Review sample packets

   Partners return a small sample with QA evidence before any scale collection, so you validate framing, calibration, and label quality against your own rubric first.

3. 03

   Scale the accepted batch

   Approve the sample, then collect at volume across around 10,000 vetted collectors in 100 countries, spanning egocentric, exocentric, and teleoperation capture.

4. 04

   Deliver rights-cleared

   Receive data in RLDS, LeRobot, MCAP, or a custom schema to S3, GCS, or Azure, each trajectory carrying consent artifacts, location releases, and per-trajectory provenance.

Choosing Between Asimov and the Alternatives

Pick Asimov when your target is egocentric human activity with rich semantic annotation and your models do activity recognition or high-level task planning; its end-to-end hardware-to-QA pipeline suits teams with no in-house capture.

Pick an alternative the moment you need robot-executable actions. For zero-cost architecture validation, start on Open X-Embodiment, DROID, and BridgeData. For enterprise multi-sensor annotation with SLAs, use Scale AI. For tasks, embodiments, or commercial rights the public sets never covered, commission capture through truelabel's marketplace. A common path prototypes on open RLDS data, then buys targeted teleoperation only for the edge cases the policy keeps failing.

Related pages

Use these to move from category-level context into specific task, dataset, format, and comparison detail.

External references and source context

1. Open X-Embodiment: Robotic Learning Datasets and RT-X Models

   OXE contains 1 million trajectories across 22 robots, 527 skills, 160,000 tasks

   arXiv ↩
2. Open X-Embodiment: Robotic Learning Datasets and RT-X Models

   RT-X achieves 50-70% zero-shot success vs 10-20% single-embodiment baselines

   arXiv ↩
3. DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset

   DROID collected 76,000 trajectories across 564 scenes and 84 tasks

   arXiv ↩
4. BridgeData V2: A Dataset for Robot Learning at Scale

   BridgeData V2 spans 13 skills and 24 environments with per-timestep actions

   arXiv ↩
5. Teleoperation datasets are becoming the highest-intent physical AI content category

   ALOHA dataset includes 650 episodes with 80%+ success on cable routing tasks

   tonyzhaozh.github.io ↩
6. scale.com scale ai universal robots physical ai

   Scale-UR partnership produced 50,000+ trajectories with force profiles

   scale.com ↩
7. Encord Series C announcement

   Encord raised $60M Series C in 2024 for video annotation infrastructure

   encord.com ↩
8. truelabel physical AI data marketplace bounty intake

   truelabel operates a network of around 10,000 vetted collectors across 100 countries

   truelabel.ai
9. RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning

   RLDS specification defines trajectory-centric schema for RL datasets

   arXiv
10. scale.com physical ai

    Scale AI physical AI data engine processes multi-sensor streams with annotations

    scale.com
11. Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World

    Domain randomization varies simulation parameters for sim-to-real transfer

    arXiv
12. Teleoperation Warehouse Dataset for Robotics AI | Claru

    Claru warehouse teleoperation bounty delivered 12,000 sequences

    claru.ai
13. iMerit model evaluation and training data

    iMerit provides managed annotation services for model training data

    imerit.net
14. sama.com computer vision

    Sama offers computer vision annotation with ethical labor practices

    sama.com

FAQ