How to choose a teleoperation interface for data collection

Choosing a teleoperation interface for data collection means picking how an operator drives the robot so the recorded actions match the policy you want to train. Leader-follower arms, VR, handheld controllers, and kinesthetic teaching each preserve a different mix of contact, speed, and action-space fidelity. Operant scopes that choice to your robot and task during a custom capture program, before a pilot freezes the mapping.

The interface is part of the data. It changes which corrections an operator can make, how fatigue shows up, and whether the logged commands are the ones your training code expects. Process around cameras, episode cuts, and QA is covered in teleoperation best practices. This guide is only about the control interface.

Match the interface to the action space

Start from the policy, not from the device on the shelf. A bimanual manipulation policy that needs wrist orientation and gripper width wants an interface that can express those axes continuously. A mobile base policy may only need velocity commands. A contact-rich insertion may need the operator to feel resistance, which a position-only gamepad will not provide.

During scoping for teleoperation capture, write down:

• The action vector the training pipeline will read.
• Which axes the operator must control in real time.
• Whether force or contact is part of the demonstration.
• How long a session runs before operator quality drops.
• How a new operator will be trained on the mapping.

If two interfaces can produce the same action vector, prefer the one operators can use consistently across people and shifts. Diversity of operators matters only if each person can actually drive the task.

Leader-follower arms

A leader-follower setup uses a smaller or kinematically similar arm as the input device. The operator moves the leader, and the follower robot tracks that motion. This is a strong fit for tabletop and bimanual manipulation when the leader's joints map cleanly onto the robot you will deploy.

The advantage is directness. Orientation, reach, and gripper timing are demonstrated in a motion the operator can see and correct. The risk is a quiet mismatch: different joint limits, a different gripper, or a scaling factor that was fine in the pilot and wrong in training. Log the scale, the joint order, and the gripper mapping with the calibration files.

Leader-follower capture still needs the same episode boundaries and sync plan as any other imitation learning data collection program. The interface does not replace metadata.

VR and spatial controllers

VR interfaces place the operator's hand or controller poses in a tracked space and map those poses onto the robot's end effector or hands. They are useful when the operator must see a remote scene, move a mobile base and an arm together, or demonstrate a task where a physical leader arm is impractical.

Treat tracking loss, occlusion of the headset cameras, and coordinate-frame setup as capture risks. Define the robot frame, the operator frame, and what happens to the log when tracking drops. A demonstration with a silent frame jump is worse than a discarded episode.

VR is a weaker default for delicate contact if the operator cannot feel the object. Pair it with force display or a different interface when the skill is insertion, cloth tension, or a light grasp that crushes easily. Confirm the view the operator uses is close enough to the camera views you will train on that they are solving the same perception problem.

Handheld controllers

Handheld controllers, including 6-DoF mice and gamepad-style devices, map sticks, twists, or buttons onto end-effector motion or base velocity. They are practical for navigation, coarse positioning, and tasks where the operator already knows the mapping. They are a poor fit when the demonstration depends on fluid wrist rotation or coordinated two-arm contact, unless the mapping has been proven with operators who are not the person who designed it.

Document the mode switches. A clutch that re-centers the device, a speed scale, and a separate gripper axis all belong in the action spec. If operators change scale mid-episode, the policy sees a different gain than the one you think you recorded.

Kinesthetic teaching

Kinesthetic teaching has the operator move the robot by hand, usually with gravity compensation or a freedrive mode, while joint positions are logged. It fits precise, slow contact on a robot that is safe to touch: a short insertion, a placement, or a path that must pass through a fixture.

It is a weak fit for dynamic motion, mobile bases, or any robot that is not safe to backdrive. The logged signal is often position, not the command your deployed controller will track. Say so in the data spec. If training expects velocity or torque targets, kinesthetic joint angles are a different action space and need an agreed conversion, or a different interface.

Also separate the operator's body from the scene cameras. Hands on the robot occlude the grasp. Plan a wrist view or a later camera crop so the policy input is still usable. That check belongs in the multi-sensor synchronization and camera plan, not after the batch is full.

What the pilot has to prove

Whichever interface you choose, the pilot should show that a second operator can produce accepted episodes with the same action vector, camera rig, and outcome labels. Review a handful of episodes for mapping errors, tracking drops, clutch artifacts, and contact the operator could not feel.

Keep the interface fixed between the pilot and scale unless the pilot shows it cannot express the task. Changing devices mid-program splits the dataset into two action spaces. If the task truly needs two styles of demonstration, scope them as separate episode types under robot demonstration data, with metadata that marks which interface produced each one.

To choose an interface against your robot and policy, book a discovery call.