Metrics that prove sim-to-real gap closing

Sim-to-real gap closing is proven when a policy improves on real held-out slices that match known mismatch domains, lighting, contact, wear, not when you log more simulation hours. Pair sim training with real eval cells scoped to those domains. Operant fills those cells through sim-to-real data collection so the metric is measured on the robot, in the environment, under the contacts simulation gets wrong.

More sim coverage can hide a gap. If the real eval set is still a clean cell, you have not tested the lighting, compliance, or wear that will show up after week one on a line.

Metrics that move a decision

Define the slice and the KPI before capture:

• Task success on a real held-out set that was never in the sim training mix.
• Contact or force residuals on insertion and fastening cells, such as peg-in-hole with force traces.
• Intervention or recovery rate under the lighting or clutter the sim does not render.
• Calibration and sync QA so a failed episode is a policy failure, not a clock error.

Track those numbers before and after a real capture increment. If they do not move, collect a different slice instead of more of the same.

How the program is structured

A gap-analysis workshop ranks mismatch domains by deployment risk. Each domain becomes a capture slice with sensors and acceptance criteria. Rare events use the same discipline as real-world eval benchmarks. The parent workflow is robotics data collection: scope, pilot, scale.

Book a discovery call if you need real episodes in the domains your sim still misses.