ContactGen task-object stability benchmark (2026-07-19)

1. Question

Test whether the unmodified official ContactGen checkpoint produces stable grasp-keyframe proposals across PhysHSI task objects rather than only on the N1 cup. The benchmark contains both single-hand and two-hand task semantics and deliberately reports all samples instead of only the best-looking output.

This is a grasp-keyframe proposal benchmark. It does not claim that an isolated MANO mesh is already a dynamically executable SMPL-X grasp.

2. Fixed protocol

• Implementation: unmodified submodules/contactgen/demo.py at commit 1aea2b6ce498ec90c4af0dd4c5fbdd4a23db761a.
• Checkpoint: submodules/contactgen/checkpoint/checkpoint.pt.
• Input: canonical SAGE source mesh in Z-up; no KIMODO Y-up intermediate mesh is fed to ContactGen.
• Seed: 17.
• Samples: 4 per official invocation.
• Official weights: contact 0.1, penetration 3.0, UV 0.01; official 200 global + 1000 pose iterations.
• Contact coverage: hand vertices within 5 mm of the object surface, split into thumb/index/middle/ring/pinky/palm parts using the official MANO vertex labels.
• Task-level penetration tolerance: at most 30 mm, as allowed for a keyframe that will undergo later physics refinement.
• Strict diagnostic tolerance: at most 5 mm.
• Penetration backend: pysdf, positive inside. This is fixed because different trimesh versions gave inconsistent inside tests on the concave cup.

Single-hand tasks use the official right-hand model directly. ContactGen is not a native bimanual generator. For a two-hand diagnostic, the source and a reflected source are independently passed through the same official right-hand model; the reflected output is mapped back into a left-hand mesh. All 4 × 4 = 16 right-left pairs are then checked for:

1. valid thumb-versus-finger object contact for each hand;
2. hands on opposite sides of the object along the predeclared wide horizontal axis;
3. sufficient side separation;
4. no measured hand-hand interpenetration.

This is explicitly a mesh-level bimanual adaptation, not an official ContactGen capability and not yet a valid left-MANO/SMPL-X parameterization.

3. Cases and results

Object	Task hand	Interaction	Object size (m)	Task-tolerant result (≤30 mm)	Strict result (≤5 mm)
cup	single	pick	0.116 × 0.116 × 0.155	stable wrap: 2/4; geometry 3/4	wrap 2/4; geometry 3/4
book	single	pick	0.159 × 0.071 × 0.258	best-of-N wrap: 1/4	wrap 0/4; geometry 0/4
notebook	single	pick	0.165 × 0.053 × 0.244	pinch-only: geometry 1/4	wrap 0/4; geometry 0/4
leatherbook	single	pick	0.235 × 0.197 × 0.064	no usable candidate	no usable candidate
round cushion	two	pick	0.483 × 0.420 × 0.396	usable pairs 0/16	usable pairs 0/16
thin cushion	two	pick	0.491 × 0.174 × 0.475	usable pairs 0/16	usable pairs 0/16
basket	two	pull	0.332 × 0.286 × 0.321	usable pairs 0/16	usable pairs 0/16
instrument tray	two	pick	0.354 × 0.454 × 0.105	usable pairs 0/16	usable pairs 0/16

The 3 cm tolerance changes the thin-object interpretation but not the broad conclusion:

• cup is the only object with stable strict wraps;
• book gains one usable wrap only under the relaxed keyframe tolerance;
• notebook gains one lower-coverage pinch candidate but no wrap;
• leatherbook still lacks a thumb-opposed grasp;
• none of the four two-hand objects yields a pair in which both sides independently form a valid grasp.

The two-hand failures are not caused only by the 5 mm threshold:

• round cushion: the only right-side contact-rich sample penetrates about 51.8 mm; the task-tolerant left sample has 191 vertices within 5 mm but no thumb opposition;
• thin cushion: contacts concentrate on ring/pinky on one side and index/palm on the other;
• basket: only 14–16 near-surface vertices, primarily thumb-only; its non-watertight source also prevents a reliable object-penetration measurement;
• instrument tray: all generated hands have zero vertices within 5 mm; its source is also non-watertight.

4. Conclusion for the grasp-keyframe pipeline

ContactGen should replace the HOI-FHLI local grasp solver only as a stochastic single-hand proposal source, followed by best-of-N filtering and later SMPL-X/physics refinement. It is not stable enough to accept the first sample, and the tested mirror construction does not make it a usable general bimanual keyframe generator.

The current recommended boundary is:

single hand-sized object
    canonical Z-up source mesh
        -> official ContactGen best-of-N
        -> task/strict geometry filter
        -> left/right SMPL-X attachment
        -> whole-body collision/contact refinement

two-hand or wide object
    use task-level two-hand contact anchors / a native bimanual model
        -> joint two-hand optimization
        -> SMPL-X attachment and whole-body refinement

The 30 mm task tolerance is acceptable only at the proposal stage. Physics execution must still reduce penetration and verify force closure/contact persistence.

5. Reproducible artifacts

• Case manifest: data/hsi_generation/references/contactgen_task_object_benchmark_20260719.json.
• Runner: tools/contactgen/run_task_object_benchmark.py.
• Scoring/rendering: utils/contactgen/benchmark.py.
• Summary: output/auto_experiments/20260719_contactgen_task_object_benchmark/benchmark_summary.json.
• Batch HTML: output/auto_experiments/20260719_contactgen_task_object_benchmark/gallery.html.
• Best-sample overview: output/auto_experiments/20260719_contactgen_task_object_benchmark/best_samples_overview.png.

The HTML includes all 48 official ContactGen samples: all four single-hand samples per single-hand object, all four source-side and all four mirrored-side samples per two-hand object, and the top four of each object's 16 right-left pairings.