Bender

First-person RGB-D for robot learning

Egocentric data, on demand.

Real people doing real tasks, recorded from their own point of view on the phones they already own. Every session arrives as one MCAP file: color, depth, camera pose and IMU on one clock.

Fig. 1 A modelled first-person depth frame, drawn as contour lines two centimetres apart (an illustration, not a recording). In egocentric data the hands are always the nearest thing in view, and the task is whatever they are doing.

01

One file per session

Each accepted session is delivered as a single, self-describing MCAP file. Nothing to stitch together, nothing to align after the fact.

Channel Content Timing
/rgb Color video from the phone's camera, from the contributor's point of view. up to 30 fps
/depth Depth maps from the same device, timestamped with the color frames. per frame, same clock
/pose The camera's position and orientation over time, so every frame is placed in the room. same clock
/imu Raw accelerometer and gyroscope samples. same clock

One clock. Every message in the file carries a timestamp from the same source, so depth, pose and IMU line up with the video without post-hoc alignment.

Fig. 2 Channel layout of a delivered session, drawn schematically. Ticks are timestamps; the marked column is one instant across all four streams.

02

How a session is collected

We run the collection end to end, so a lab gets validated data rather than a folder of uploads.

  1. You describe the task.

    Cooking, tidying, assembling, handling objects. We turn it into a task specification: the activity, the setting, what must stay in view, what to avoid.

  2. People do it, at home, on their own phone.

    Contributors are recruited and briefed for your task, give their consent, and record from their own point of view while doing the thing the way they normally would.

  3. We check it before you see it.

    Every session is reviewed for completeness, alignment and task compliance. Sessions that pass are delivered. Contributors are paid per accepted session.

For robotics and embodied-AI labs

Tell us the task you want demonstrated.

Use the form and we will reply by email with a draft task specification.

Contact us About Bender