Results
Given the same simulated budget of 40 finger designs per run, does NVIDIA Nemotron 3 Super, reading the measured results of earlier designs, reach a finger that seats and centres the part in at least 37 of 48 training grasps in fewer designs than equal-budget random search? Each simulated design stands in for one print-and-test cycle.
Pre-registered confirmatory result. 32 runs per arm, seeds 1001 to 1032, run as Nebius Serverless Jobs on uk-south2. The protocol was frozen and tagged before any confirmatory run; nothing was re-run or tuned after the results were seen.
Protocol: the prereg-v1 tag (docs/PREREG.md). Full write-up with limits and deviations: docs/RESULTS.md.
PRIMARY ENDPOINT
S1: speed to a good design
A run's count is the number of its first design, counted from design 9, that seats and centres the part in at least 37 of 48 training grasps. Designs 1 to 8 are round 0 (the hand-designed entrants and seed designs), the same for both arms at each seed, so they do not count. A run that never gets there within 40 designs counts as 41. Fixed in the protocol before the prereg-v1 tag.
| Measure | Random search (A) | Nemotron with feedback (C) |
|---|---|---|
| Runs that reached the target within 40 designs | 11 of 32 | 23 of 32 |
| Median designs to the target | not reached (above 40) | 22.5 |
| Mean designs to the target (not reached counted as 41) | 34.5 | 25.6 |
Nemotron reached the target in 23 of 32 runs, random search in 11 of 32.
- Difference in means (A minus C): 8.97 designs, 95% interval 3.53 to 14.16.
- One-sided exact permutation over C(64,32) splits: p = 0.0011.
- Sensitivity, permutation log-rank: p = 0.0011.
Best training success so far
Median best training success so far. Random search (A): 74% after 40 designs; Nemotron with feedback (C): 85% after 40 designs. Threshold 37 of 48.
- Random search (A), median of 32 runs
- Nemotron with feedback (C), median of 32 runs
Thin lines are runs, heavy lines each arm's median. A run whose round 0 already reached the target still needs one of its own designs (from design 9) to count.
Designs needed to reach the target
Exploratory, lift-only scoring (the part only has to come off the table): no difference between arms (p = 1.0). Both arms find a finger that lifts the part almost at once. The advantage is in seating and centring the part, which is what the next step in a work cell depends on.
SECONDARY ENDPOINTS (HOLM FAMILY)
Held-out success and search efficiency
Each run's frozen top-1 design was scored on suites neither arm saw during search (Nebius job aijob-e05nvfme46ss0q4wp4).
H1: seated and centred, 100 unseen procedural conditions x 2 seeds
- Random (A)
- 60.5%
- Nemotron (C)
- 72.5%
- Difference
- +12.0 points
- p
- 7e-05
- Holm-adjusted p
- 0.00017
H2: seated and centred, 64 real scanned objects
- Random (A)
- 22.0%
- Nemotron (C)
- 27.5%
- Difference
- +5.5 points
- p
- 0.00077
- Holm-adjusted p
- 0.00077
S2: area under the best-so-far curve
- Random (A)
- 0.703
- Nemotron (C)
- 0.788
- Difference
- +0.085
- p
- 5.5e-05
- Holm-adjusted p
- 0.00017
All three reject the null at alpha 0.05 after Holm adjustment.
H2 by object width (planned split, reported outside the Holm family): objects 50 mm or narrower across the jaw (18 objects) 19.9% vs 24.1% (p = 0.012); wider than 50 mm (46 objects) 22.8% vs 28.8% (p = 0.001).
Real scanned objects are hard for every design in this space: the best arm seats them about a quarter of the time. The held-out real-object result says the Nemotron arm's picks generalise better, not that the fingers are ready for arbitrary objects.
Limits
- Everything is simulated, with rigid contacts. No finger was printed or tested on hardware for this result.
- One gripper geometry, six training parts, one task (seat, lift, hold).
- The effector tournament (hands, grippers, soft arm, suction, Nemotron-designed effectors) is product mode and not part of this pre-registered result.
Show the jobs behind these numbers
| Arm | Job | Where | Runs | Run time | Model calls |
|---|---|---|---|---|---|
| Random search (A) | aijob-e05yxb1awgg2d68hq1 | Nebius · cpu-d3 · uk-south2 | 32 | 28 min | none |
| Nemotron with feedback (C) | aijob-e05s3f8ka6r508p4mk | Nebius · cpu-d3 · uk-south2 | 32 | 84 min | 473 |
Image: sha256:c6162fa01d41beaab64e1431bc10e07535ce7bd59045fcef7bf0e074857b4b89
Analysis outputs: runs/analysis/s1-confirmatory/s1_v2.json, runs/analysis/secondary-confirmatory/secondary_v2.json (released with the run data).