PhysHSI 端到端流线优化日志

本文档记录 SAGE / PhysHSI 任务生成阶段已经做过的修改，并继续记录后续从 task generation 到 HSI / HOI motion generation、hybrid stitching、rendering、 metrics 和自动优化的所有端到端流线工作。

维护规则：之后所有 task generator、visualizer、prompt、scene selection、 generated plan、adapter、motion-generator config、keyframe binding、stitching、 rendering、metric、cleanup 或 experiment 相关变更，都需要追加记录到本文档。不要记录任何私有 API key，也不要记录为了变好看而手工编辑过的生成结果。

核心约束：

未经明确批准，不修改 baseline 算法代码。
不手工编辑生成出来的 task plan 或 motion 来改善结果。
不为当前三个测试 scene 添加 scene-specific 或 object-id-specific 特判。
优先使用基于 action type、object category、geometry、reachability、 support area、clearance 和 temporal continuity 的通用规则。
允许优化 wrapper、adapter、config、keyframe binding、validation、metrics 和 orchestration scripts。
Human-like 质量评估必须发生在每个 generator 单独跑通之后：先在 HSI-only task 上优化 Kimodo，在 HOI-only task 上优化 HOIFHLI，再把优化后的 generator 集成到 hybrid flow。
临时文件在不再需要后应及时删除；保留的输出必须可复现，并记录 input task、 config、command、motion output、render output 和 metrics。

当前基线

日期：2026-05-30

基线输出根目录：

/mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center

当前 active plans：

/mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json
/mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center/hsi_2d89d0a5_tight_lie_getup_sit/task_plan.json
/mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center/hybrid_84b703fb_tight_rest_pick_place/task_plan.json

这三个 plan 均由 VLM task-generator flow 生成，并经过 task_validation.json 验证。生成的 task_plan.json 没有经过手工编辑。

旧的 preview 输出已经从以下位置删除：

/mnt/data/PhysHSI/output/e2e_task_previews
/mnt/data/PhysHSI/utils/sage_task_generator/output/e2e_task_previews

当前测试集只保留 final_v9_tight_sit_center 这棵输出目录。

已完成修改

Taxonomy 与 action 范围

公开 taxonomy 已从 locomotion / manipulation 改为 hsi / hoi。
当前公开 HSI 语义：
reach
sit
get_up
lie_down
当前公开 HOI 语义：
pick
place
carry
当前 task-generation 阶段禁用 open、close、pull 和 push。
内部 move / interact step 仍然保留为 low-level generator 结构：
普通 move 导出为 hsi:reach
持物状态下的 move 导出为 hoi:carry
interact 根据 interaction_mode 导出对应语义

Prompt 清理

已从 active flow 中移除旧版 task-generator prompt variants。
当前 active prompt 只描述现有 HSI / HOI taxonomy。
Prompt 规则强调 step 原子化：
move 只负责移动
interact 只负责交互
标准 subtask 形态是 move -> interact
内部 stand / 公开 get_up 可以是单独的 interaction

HOI 目标筛选

HOI pickup candidates 已限制为适合 carry 测试的物体。
过滤掉 pen、pencil 等过小物体。
过滤掉 decor、decorator、vase、plant、ornament 等装饰类目标。
当前生成的 HOI 示例使用 instrument tray、book 等较大的可搬运物体，而不是 pen 一类的小物体。

Pick / place 几何

pick 和 place 使用更紧的 interaction-distance 限制，约为 0.55 m。
公开 pick target metadata 导出被操作物体的 center 和 bbox。
公开 place target metadata 导出被操作物体的 target center。
Planner 内部仍可为了 feasibility 使用 surface/contact-style anchor，但 visualization 和 downstream metadata 对 pick/place 暴露 object center / target center。

Seat / sit 几何

sit 以及 seat-related stand 的最大 interaction distance 收紧到约 0.60 m。
偏好的 sit standing distance 大约为：
directional seat：0.40 m
non-directional seat：0.45 m
pouf、ottoman、stool、beanbag 和 cushion 等 non-directional seat 被视为 omnidirectional sitting target。
Non-directional seat 不再因为推断出了 front direction，就把 sit target 投影到 front edge。
Directional seat 仍然要求存在有效的 front-entry side。
Prompt candidate filtering 与 validation 现在使用对齐的 seat-entry 逻辑，因此 VLM 可见的 seat candidate 不应再因为 entry-side 不兼容而在后处理失败。
Sit standing-point search 使用 action-specific clearance，而不是总是依赖 walking clearance。

Lie scene screening

已为 lie target 增加 scene screening，用来拒绝四周被 high edge 或周围几何阻挡的 bed / lie surface。
该筛选主要用于当前测试 scene 选择；它不一定应该作为未来每个 synthetic scene 的永久 generation-time 约束。

Task constraints 与 generation modes

增加 --require-hsi-rest，用于要求 HSI rest action。
增加 --hoi-only，用于 HOI-only generation。
增加 --hsi-only，用于 HSI-only generation。
HSI-only composite task 优先生成类似 lie -> stand -> sit 的序列。
Hybrid task 在被要求时必须包含至少一个 HSI rest action，即 sit 或 lie，同时包含 HOI activity。

Visualizer 更新

Visualizer 已支持当前 hsi / hoi 公开 action schema。
Pick/place markers 优先使用 manipulated object center / target center metadata。
当前 visualizer scan root 为：

/mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center

Local config

VLM API key 只保存在本地已忽略的 config 文件中。
API key 本身不得写入本文档。
本地 config 路径：

/mnt/data/PhysHSI/.tmp/qwen3_vl_plus_dashscope_local_config.json

当前三任务测试集

HOI

Scene：889a0ed0

Plan：

/mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json

Summary：

pick instrument tray
carry to desk
place instrument tray on desk

Validation：

ok: true
pick distance：约 0.552 m
place distance：约 0.469 m

HSI

Scene：2d89d0a5

Plan：

/mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center/hsi_2d89d0a5_tight_lie_getup_sit/task_plan.json

Summary：

reach couch
lie down
get up
reach stool
sit

Validation：

ok: true
lie distance：约 0.562 m
sit distance：约 0.551 m

Hybrid

Scene：84b703fb

Plan：

/mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center/hybrid_84b703fb_tight_rest_pick_place/task_plan.json

Summary：

reach bookshelf
pick book
carry to sideboard
place book on sideboard
reach chair-like seat
sit
get up

Validation：

ok: true
pick distance：约 0.434 m
place distance：约 0.421 m
sit distance：约 0.409 m

备注：某次 hybrid generation run 中，plan 文件已经成功写出并通过验证，但 conda wrapper 在输出完成后报告 IsaacGym / Python shutdown segmentation fault。生成文件本身仍然有效。

端到端优化范围

目标流线：

task generation
-> task validation / visualization
-> task split into HSI and HOI segments
-> Kimodo for HSI segments
-> HOIFHLI for HOI segments
-> hybrid state transfer and stitching
-> full-scene render
-> metric-based and visual human-like evaluation
-> automatic parameter / adapter / binding iteration

当前 generator 职责：

Kimodo 负责 HSI actions：
reach
sit
get_up
lie_down
HOIFHLI 负责 HOI actions：
pick
carry
place
Hybrid task 必须切分成连续的 HSI / HOI segments，并在生成后重新组合。

Human-like 优化顺序：

使用 Kimodo 跑通并优化 HSI-only。
使用 HOIFHLI 跑通并优化 HOI-only。
固化每个 generator 当前最佳 config / adapter settings。
实现 hybrid splitting、dispatch、state transfer 和 stitching。
除非发现 generator-specific bug，否则 hybrid 阶段只优化 stitching / state-transfer 层。

允许的优化面：

Generator config values。
Adapter conversion logic。
Keyframe binding metadata 与 object-category binding rules。
通用 validation thresholds 与 candidate selection rules。
Stitching、resampling、root alignment、orientation alignment 和 transition smoothing。
Metrics 与 render / preview tooling。

未经明确批准不允许的优化面：

Baseline model 或核心算法源码。
手工编辑 output motions 或 task plans。
当前 scene 专用规则，例如匹配特定 scene id 或 object id。
通过删除 metrics、跳过 render check 或只改 visualization 来隐藏失败。

Human-like 评估信号：

Foot sliding during planted contact。
Body / object / scene penetration。
Pick / place 的 reach distance 与 hand-object timing。
Carry 过程中的连续 object motion。
Place 后稳定的 object pose。
Sit 和 lie 时 pelvis / torso 是否落在 support area 上。
Segment boundary 处 root-position、facing-direction、velocity 和 pose 是否连续。
Joint range 是否自然，以及是否存在突然的 pose jump。

Motion-generation 调研计划

2026-05-30 - planned flow audit

检查 Kimodo utilities、configs、keyframe-binding tools、render commands 和现有 tuned HSI examples。
检查 HOIFHLI utilities、configs、预期 task / object inputs、render commands 和 output motion format。
比较 Kimodo 与 HOIFHLI 的 coordinate frames、fps、root representation、body pose format、hand pose availability、object state format 和 output naming。
找出最干净的 wrapper layer，用于 task splitting、generator dispatch 和 hybrid stitching。

2026-05-30 - planned HSI-only optimization

将当前 active HSI plan 输入 Kimodo。
在扩展前先复现已知较好的 pouf-sit-style 配置。
通过通用 object-category rules 扩展 chair-like sit 和 sofa / bed lie 的 keyframe binding。
接受一个 config 前，需要 render full-scene preview 并计算 metrics。
每次 config change、failed run、render path、metric file 和 retained output 都记录到本文档。

2026-05-30 - planned HOI-only optimization

将当前 active HOI plan 输入 HOIFHLI。
聚焦 pick -> carry -> place，目标物体为中等或较大的 movable objects。
检查 hand-object timing、carry 期间 object attachment、place 时 object release，以及 root / hand reachability。
以通用方式调 generator / adapter 参数，不针对当前 scene id 或 object id。
每次 config change、failed run、render path、metric file 和 retained output 都记录到本文档。

2026-05-30 - planned hybrid integration

将当前 active hybrid plan 切分成 HSI 和 HOI segments。
HSI segments 交给 Kimodo，HOI segments 交给 HOIFHLI。
在 segment boundary 传递 root pose、facing direction、temporal offsets、 hand state、held-object state 和 placed-object state。
使用 resampling、root alignment、orientation alignment 和短 transition smoothing 拼接 segments。
只有在两个 single-generator flow 都稳定后，才评估 hybrid output。

临时产物策略

保留的 experiment outputs 应放在带日期或版本号的根目录下。
Failed scratch outputs 在记录 failure mode 后删除。
只保留用于比较、复现或用户查看的输出。
不要在保留的 experiment logs 中存储 API keys、credentials 或 private service responses。
每个 retained run 都需要记录：
input task path
command 或 wrapper entry point
config path
motion output path
render / preview path
metrics path
pass / fail 判断与 known caveats

Flow Optimization Entries

2026-05-30 - scope expanded to full motion pipeline

原 SAGE task-generator change log 已扩展为完整 PhysHSI generation flow 的 canonical optimization log。
当前追踪范围包括 task generation、validation、visualization、Kimodo HSI generation、HOIFHLI HOI generation、hybrid splitting / stitching、rendering、 metrics、human-like evaluation、automatic optimization 和 cleanup。
Human-like optimization 明确前置到 hybrid integration 之前：Kimodo 必须先在 HSI-only output 上调好，HOIFHLI 必须先在 HOI-only output 上调好，然后再拼接 mixed HSI / HOI tasks。
Optimization 可以调整 configs、wrappers、adapters、keyframe bindings、 validation、metrics 和 stitching logic，但未经明确批准不应修改 baseline algorithm code。
Optimization 必须保持 scene-agnostic，并基于 category / geometry；不要为当前三个 selected scenes 或 object ids 添加特判。
之后的 automatic experiments 必须在本文档中同步记录 changes、problems、 progress、retained output paths、metrics、visual judgments 和 temporary-file cleanup。

2026-05-30 - experiment: automation readiness audit

Goal：确认当前 SSH remote 上的 active plans、wrapper、adapter、测试环境和自动化实验入口是否足以支撑后续批量实验。
Input task / scene： /mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json 和 /mnt/data/PhysHSI/output/e2e_task_previews/final_v9_tight_sit_center/hsi_2d89d0a5_tight_lie_getup_sit/task_plan.json。
Generator and config：Kimodo plan-to-constraints dry conversion； HOIFHLI adapter + tools.hoifhli.legacy.generate.run_dry_run。
Command or wrapper entry point： env_protomotions_isaaclab/bin/python -m pytest utils/sage_task_generator/tests/test_hoifhli.py tests/test_kimodo_plan_to_constraints.py -q； utils.kimodo.plan_to_constraints； HOIFHLIPlanAdapter； run_dry_run。
What changed：utils/sage_task_generator/adapters/hoifhli.py now accepts the current public hsi / hoi schema, including hoi:carry steps with path_points, while preserving the older move / interact schema.
Why it changed：the active HOI plan uses pick -> carry -> place; the old adapter only found old-style pick -> locomotion move -> place clips.
Files or outputs affected： utils/sage_task_generator/adapters/hoifhli.py； utils/sage_task_generator/tests/test_hoifhli.py； temporary dry-run outputs under .tmp/codex_* were removed automatically.
Metrics and visual judgment：29 related tests passed. HSI active plan converted to 465 Kimodo frames in a temporary output. Active HOI adapter now reaches dry-run metrics construction with interaction_path_xy_acceptable_for_generation: true.
Problems found：full HOIFHLI dry-run without injected object height currently cannot import official sample.py because pytorch3d is missing from the visible env_protomotions_isaaclab environment. Kimodo generation environment also needs the correct runtime extras for text encoder / viewer workflows when running new heavy jobs.
Follow-up changes：before large automatic HOI or hybrid generation, restore or create the dedicated HOIFHLI environment expected by submodules/hoifhli_release/environment.yml, including pytorch3d, human_body_prior, TorchSDF and upstream checkpoints. Do not modify baseline algorithm code for this setup.
Temporary files removed：yes; only this log entry and adapter/test code changes were retained.

2026-05-30 - experiment: conda and remote video access audit

Goal：确认 SSH remote 上是否存在可复用 conda 环境，并确认远端生成视频可被 Codex 抽帧、传回和视觉检查。
Command or wrapper entry point： source /home/user/anaconda3/etc/profile.d/conda.sh && conda info --envs； /home/user/anaconda3/envs/hoifhli/bin/python； /home/user/anaconda3/envs/isaac-sim/bin/python； ffprobe； ffmpeg -ss 1 -i output/hoifhli_clean_sage_cushion_endpoint/scene_preview.mp4 -frames:v 1 ...。
Environment findings： hoifhli uses Python 3.8.18, torch 1.11.0, pytorch3d 0.7.4, human_body_prior, TorchSDF, smplx, trimesh and scipy; it can import official submodules/hoifhli_release/sample.py and reports get_object_static_height_floor("largebox") == 0.175。 isaac-sim uses Python 3.10.19, pytest, torch 2.4.1+cu121, pytorch3d 0.7.8, human_body_prior, TorchSDF, smplx, kimodo, viser, gradio, trimesh and scipy。 task_generator is lightweight and lacks torch; use it only for task-side utilities that do not need generation dependencies。 protomotions_isaacgym has torch and smplx but lacks pytorch3d / human_body_prior / TorchSDF。
Metrics and visual judgment：with the hoifhli conda env, active HOI HOIFHLIPlanAdapter + run_dry_run now passes pre_generation_alignment_pass: true using the official HOIFHLI object height source. The checked video is 1280x720, 9.133333 seconds, 274 frames.
Output paths retained： remote extracted frame /mnt/data/PhysHSI/.tmp/codex_video_check/scene_preview_frame.png； local Codex-visible copy /tmp/physhsi_scene_preview_frame.png。
Problems found：hoifhli does not include pytest or Kimodo; isaac-sim does not include llm2vec, so text encoder heavy jobs may still need either the previously working launch path or an explicit dependency setup check.
Follow-up changes：run HOIFHLI full generation under hoifhli; run Kimodo / rendering under isaac-sim unless a more specific script requires another env. Always extract at least one representative frame from retained videos before making visual judgments.
Temporary files removed：no; retained one small extracted frame for Codex visual-access verification.

2026-05-30 - experiment: SMPL-X Kimodo and HOIFHLI full-pipeline audit

Goal：把 final_v9 tight sit center 中的 HSI / hybrid / HOI 任务接到可复现的全流程实验入口，确认 Kimodo 生成走 SMPL-X 而不是 SOMA，并确认 HOIFHLI 任务不绕过现有 adapter。
Input task / scene： output/e2e_task_previews/final_v9_tight_sit_center/hsi_2d89d0a5_tight_lie_getup_sit； output/e2e_task_previews/final_v9_tight_sit_center/hybrid_84b703fb_tight_rest_pick_place； output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place。
Generator and config：Kimodo runs use the existing tools.kimodo.legacy.soma_keyframe_pipeline wrapper, generalized by config to pipeline_family: smplx and model nvidia/Kimodo-SMPLX-RP-v1；HOIFHLI runs use utils.sage_task_generator.adapters.hoifhli plus tools.hoifhli.legacy.generate。All Kimodo networked runs explicitly set HF_ENDPOINT=https://hf-mirror.com and keep Hugging Face cache under /mnt/data/.cache/mingxuan/huggingface。
Command or wrapper entry point： /home/user/anaconda3/envs/isaac-sim/bin/python -m tools.kimodo.legacy.soma_keyframe_pipeline --config ... --stages bind control generate skin render； /home/user/anaconda3/envs/isaac-sim/bin/python -m utils.sage_task_generator.adapters.hoifhli --run-dir ... --out-dir ... --repo-root /mnt/data/PhysHSI； /home/user/anaconda3/envs/hoifhli/bin/python -m tools.hoifhli.legacy.generate --adapter-input ... --out-dir ... --device 0。
Output paths retained： output/auto_experiments/20260530_smplx_e2e/hsi_stool_smplx_pipeline； output/auto_experiments/20260530_smplx_e2e/hybrid_seat_smplx_pipeline； output/auto_experiments/20260530_smplx_e2e/hoi_instrumenttray_hoifhli/generate_recovered。
Metrics and visual judgment：HSI and hybrid Kimodo jobs both produce SMPL-X motion/video and full-scene final frames. HOIFHLI pick/place preserves dynamic object rendering and reaches near-zero object start/end XY errors (2.13e-07m / 1.91e-07m) with object_displacement_m: 2.0851 and object_path_length_m: 4.7675。Visual review shows HOIFHLI object interaction is materially better than attempting the whole hybrid plan as Kimodo-only。
Problems found：current sit keyframe binding for stool/seat is still fallback invalid, mainly due to feet-floor and seat-penetration checks. HSI couch lie/get_up has no matching fullbody keyframe library yet, so lie/get_up remains poor. Kimodo-only hybrid does not model pick/place object interaction, so the scalable route is HOIFHLI pickup/place + Kimodo SMPL-X locomotion/sit + stitch and metrics, not a scene-specific Kimodo-only hack.
Follow-up changes：add reusable SMPL-X sit binding improvements for chair / stool / sofa / seat, then implement the hybrid composition using existing tools.hoifhli.legacy.hybrid_carry and tools.hoifhli.legacy.hybrid_sequence instead of modifying the HOIFHLI adapter. Add couch lie/get_up keyframes before judging HSI couch tasks.
Temporary files removed：no; retained videos, final frames, manifests, dry-run scenarios and metrics for later regression comparison.

2026-05-30 - experiment: current hybrid book route1 stitched baseline

Goal：在当前 final_v9 hybrid 场景上验证“HOIFHLI pickup + Kimodo SMPL-X carry + stitch”的可复现实验链路，不再把 pick/place 整段交给 Kimodo-only。
Input task / scene： output/e2e_task_previews/final_v9_tight_sit_center/hybrid_84b703fb_tight_rest_pick_place。
Generator and config：HOIFHLI adapter output in output/auto_experiments/20260530_smplx_e2e/hybrid_book_route1/adapter； HOIFHLI official generation in .../hoifhli_generate； Kimodo carry constraints in .../carry_rootonly_objectfront_ik use variant=root2d, path_control=object_com, anchor_facing_policy=object_front, cfg_weight=(0.0, 4.0), model Kimodo-SMPLX-RP-v1，and explicit HF_ENDPOINT=https://hf-mirror.com。
Command or wrapper entry point： utils.sage_task_generator.adapters.hoifhli； tools.hoifhli.legacy.generate； tools.hoifhli.legacy.hybrid_carry build； tools.kimodo.generate_with_first_heading； tools.hoifhli.legacy.hybrid_carry graft-grasp； tools.hoifhli.legacy.hybrid_sequence stitch； tools.render_motion_scene。
Output paths retained： output/auto_experiments/20260530_smplx_e2e/hybrid_book_route1/hoifhli_generate/hoifhli_scene_motion.npz； output/auto_experiments/20260530_smplx_e2e/hybrid_book_route1/carry_rootonly_objectfront_ik/kimodo_carry_motion_two_bone_ik_symmetric_front_wrist_source.npz； output/auto_experiments/20260530_smplx_e2e/hybrid_book_route1/stitched_pickup_carry_static_release/sequence.npz； rendered video output/auto_experiments/20260530_smplx_e2e/hybrid_book_route1/stitched_pickup_carry_static_release/hybrid_book_sequence_full_scene.mp4。
Metrics and visual judgment：HOIFHLI generation passed pre_generation_alignment_pass: true, gpu_generation_run: true, object_end_xy_error_m: 2.36e-07, max_frame_to_frame_object_jump_m: 0.0439。 Stitched sequence passed seam and target gates with final_target_error_m: 0.0, max_seam_object_jump_m: 0.0, max_seam_root_jump_m: 0.0, dynamic_object_rendered: true。Visual review still fails final-quality expectations: the body has obvious mid-sequence pose artifacts, and the book is rendered through the HOIFHLI largebox proxy rather than a book-shaped mesh.
Problems found：numeric stitching is now reproducible, but this is still an engineering baseline. Hand/object anchor after IK is weak for the book proxy (both_hands_object_anchor_error_mean_m: 0.315m) and the static-release tail keeps the object at the target without a convincing put-down motion. Sit/stand after placement is not included in this stitched baseline yet.
Follow-up changes：add a book-sized dynamic proxy or object-specific HOIFHLI proxy mapping, tighten hand-object anchor gates for small objects, add a real HOIFHLI or Kimodo release clip, then append the chair sit/get-up Kimodo segment after the place phase.
Temporary files removed：no; retained metrics, rendered video, final frame and contact sheet for visual regression.

2026-05-31 - experiment: couch lie HSI-only binding and visual tuning

Goal：先调通 HSI-only 视觉质量，按 sit binding 的做法迁移到 lie binding；先验证静态 binding，再验证 Kimodo keyframe，最后再调生成参数。此轮没有进入 hybrid，也没有修改 HOIFHLI adapter。
Input task / scene： output/e2e_task_previews/final_v9_tight_sit_center/hsi_2d89d0a5_tight_lie_getup_sit。
Generator and config：继续使用全流程 wrapper tools.kimodo.legacy.soma_keyframe_pipeline，但配置为 pipeline_family: smplx、model: Kimodo-SMPLX-RP-v1，并显式设置 HF_ENDPOINT=https://hf-mirror.com。参考历史 tools/kimodo/legacy/constraints_v3 可见原先效果较好的 sit 实验来自 SOMA keyframe pipeline、pre-sit alignment、seed=42、diffusion_steps=100、 cfg=(2.0,2.0) 和 4s sit duration；本轮把这些 pipeline 结构迁移到 SMPL-X lie。
Code changes：新增 utils/kimodo/lie_keyframe_binding.py，从已有 SMPL-X Kimodo motion 中读取 canonical lie pose，使用 SMPL-X skin 得到人体 mesh，寻找 couch mesh 的真实 upward support surface，而不是使用 VLM 给出的 z=0.776 交互点。脚本会做 contact patch、penetration、low-body support、 key-joint overhang 评分，并输出 native SMPL-X pose / static motion。更新 tools/kimodo/legacy/smplx_keyframe_control.py 支持 --lie-binding-dir 注入 native SMPL-X lie fullbody keyframe；更新 utils/kimodo/soma_keyframe_pipeline.py 转发该参数。之后又给 soma_keyframe_control.py / smplx_keyframe_control.py / pipeline 增加 manip_duration，让 lie_down/get_up 这类无路径 interaction duration 可配置。
Output paths retained： output/auto_experiments/20260531_hsi_visual/binding_couch_lie_smplx_v1frame134_balanced02； output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_pipeline_v2_balanced； output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_pipeline_v3_longlie。
Metrics and visual judgment：新的 couch support surface 在 z≈0.379m，比原 task plan 的 z≈0.776m 更接近实际坐垫面。balanced lie binding 的 top candidate valid，contact patch support 0.987、low body support 0.942、penetration 0。v2 生成在 frame 134 与 lie binding 精确对齐：root error 0.0m，mean joint error ~3e-6m，说明 keyframe 已被 Kimodo 保住。v3 把 lie_down 从 2.5s 拉到 4.0s 后，固定相机下 frame 100 已在 couch 上过渡，frame 140 基本成躺姿，frame 179 精确命中 binding； lie contact metrics：frame 140 contact support 0.923、low-body support 0.815、penetration 0，frame 179 contact support 0.996、low-body support 0.997、penetration 0。
Problems found：原始 lie 失败不是 constraints 没生效，而是没有 lie binding； VLM 给出的 couch lie target z 落在靠背/整体高度附近，导致旧 keyframe 漂浮。 v3 仍有早期进入 couch 的轻微穿插，frame 100 penetration count 约 873，但视觉上已经从“站在 couch 后面”改善为“上 couch 过渡”。最终 sit 仍沿用 stool top candidate；它被旧规则标为 feet_not_on_floor，但高凳脚悬空不一定是视觉错误，其他脚更低的候选反而有明显 body penetration，因此暂不写场景专用规则。
Follow-up changes：若继续精修 HSI，下一步应补一个可复用的 lie-entry intermediate binding/keyframe，或加入 short hold before get_up，降低进入 couch 和起身起始帧的穿插；另外把 stool/chair 的 feet-floor policy 区分为 low seat 与 high stool。HSI 稳定后再进入 HOI-only 物体尺寸/形状适配。
Validation：py_compile 通过；/home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q tests/test_kimodo_plan_to_constraints.py tests/test_metrics.py utils/sage_task_generator/tests/test_hoifhli.py 通过，32 passed。
Temporary files removed：no; retained videos, review frames, diagnostic frames, manifests and lie_contact_metrics.json for visual regression.

2026-05-31 - experiment: instrument tray HOI-only proxy tuning

Goal：在 HSI-only 稳定后，先单独调 HOI-only 的 pick/place 视觉质量；重点检查 HOIFHLI 支持的 proxy object 尺寸/形状与实际 instrumenttray 是否匹配。此轮没有修改 HOIFHLI adapter，只使用既有全流程入口和 CLI 参数。
Input task / scene： output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place。动态物体为 room_d624a917_instrumenttray_a06905d2，实际 mesh 为 data/selected_SAGE10k/e2e_test_scenes/889a0ed0/objects/c35084b9.ply。
Generator and config：使用既有 wrapper utils.sage_task_generator.adapters.hoifhli 生成 hoifhli_input.json，再用 tools.hoifhli.legacy.generate 执行 HOIFHLI。对比旧 baseline 的 object_name=largebox 后，本轮改用 object_name=smallbox；原因是实际 tray 高度约 0.105m，视觉 center-on-floor 为 0.053m，而 HOIFHLI largebox source center 是 0.175m，smallbox source center 是 0.080m，明显更接近 tray。
Command or wrapper entry point： utils.sage_task_generator.adapters.hoifhli --object-name smallbox --path-policy root_offset_object_com --target-spacing-m 0.7； tools.hoifhli.legacy.generate --device 0； tools.render_motion_scene --mode motion_scene。
Output paths retained： output/auto_experiments/20260531_hoi_visual/hoi_instrumenttray_smallbox/adapter； output/auto_experiments/20260531_hoi_visual/hoi_instrumenttray_smallbox/generate/hoifhli_scene_motion.npz； rendered video output/auto_experiments/20260531_hoi_visual/hoi_instrumenttray_smallbox/generate/hoifhli_full_scene.mp4； review frames/contact sheet under output/auto_experiments/20260531_hoi_visual/hoi_instrumenttray_smallbox/generate/review_frames。
Metrics and visual judgment：smallbox generation passed pre_generation_alignment_pass: true and gpu_generation_run: true。Object start/end XY errors remain near zero (2.13e-07m / 1.91e-07m), displacement is 2.085m, max frame-to-frame object jump is 0.0675m。The scene motion confirms object_name=smallbox for HOIFHLI generation while rendering the real dynamic instrumenttray mesh (dynamic_object_id=room_d624a917_instrumenttray_a06905d2, source mesh c35084b9.ply)。视觉上，pickup/carry 不再像抱一个高箱子，tray 高度比 largebox baseline 更可信，手主要位于托盘边缘/上方。
Problems found：smallbox improves z/scale matching but does not fully solve grasp quality. Pickup frame still looks like hands pressing down on the tray rather than cleanly grasping side handles; place phase is partly occluded by desk equipment and may have local interpenetration around the tabletop/device cluster. Numeric path metrics do not capture this hand/object contact defect. Compared with largebox, smallbox has a longer object path length (5.35m vs 4.77m) and slightly larger human root pickup/place offsets, so this is a visual improvement in object scale, not a final-quality HOI solution.
Follow-up changes：next HOI-only step should add contact/occlusion metrics around pickup/place frames and use smallbox as the safest available proxy for this tray unless a fully supported lower box proxy is added to HOIFHLI. A quick diagnostic tried box_c because sample.py defines its static height/orientation and its height is closer to the tray. That failed first at decide_no_force_closure_from_objects and, after a temporary local support-list test, failed again at all_object_data_dict['box_c']; this means box_c is not a complete supported generation category in the current HOIFHLI checkout. The temporary box_c support-list edit and generated mesh assets were removed.
Temporary files removed：yes for the failed box_c source/assets; retained the failed diagnostic output directory for provenance, plus smallbox metrics, full-scene video, final frame and contact sheet for visual regression.

2026-05-31 - experiment: HSI couch lie binding-first pass

Goal：按“先 binding，再 keyframe”的顺序调 HSI-only couch lie/get_up 视觉质量；不进入 hybrid，不改 HOIFHLI adapter。
Input task / scene： output/e2e_task_previews/final_v9_tight_sit_center/hsi_2d89d0a5_tight_lie_getup_sit； couch room_2b2c49cc_couch_17652d32；stool room_2b2c49cc_stool_5dce0684。
What changed： utils/kimodo/lie_keyframe_binding.py now supports multi-source-frame lie binding, emits top_candidate_fullbody_constraint.json, supports a source-frame prior analogous to sit pose_quality, and can optionally limit long-axis socket drift. tools/kimodo/legacy/smplx_keyframe_control.py now supports lie fullbody keyframe sequences, fixed sequence axis sign, and writes smplx_lie_fullbody_keyframes.json. utils/kimodo/soma_keyframe_pipeline.py passes those sequence controls through the existing full-flow wrapper.
Binding findings：the scene lie target z (0.776m) is the couch bbox/back height, not the usable cushion. The actual upward support surface inferred from mesh faces is about 0.37885m in SAGE z. A single geometric score incorrectly prefers source frame 90 because it penetrates less; adding --preferred-source-frame 134 keeps the final lie keyframe on the visually better canonical frame. Restricting long-axis drift with --max-long-delta 0.25 made the pose more centered but worse visually, with larger key-joint overhang, so that centered binding was not used for generation.
Outputs retained： output/auto_experiments/20260531_hsi_visual/binding_couch_lie_xp014_multiframe_v1； output/auto_experiments/20260531_hsi_visual/binding_couch_lie_xp014_multiframe_pref134_v1； output/auto_experiments/20260531_hsi_visual/binding_couch_lie_xp014_multiframe_pref134_centered_v1； output/auto_experiments/20260531_hsi_visual/hsi_lie_bound_v1； output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_v1； output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_v2_axisneg； output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_v3_hold134。
Generator and config：all generation experiments used tools.kimodo.legacy.soma_keyframe_pipeline with pipeline_family: smplx, model Kimodo-SMPLX-RP-v1, and HF_ENDPOINT=https://hf-mirror.com。 hsi_lie_sequence_v1 used source frames 90/105/120/134; v2_axisneg fixed all sequence candidates to axis -1; v3_hold134 held the frame-134 pose at multiple lie_down fractions.
Visual judgment：single final-frame binding (hsi_lie_bound_v1) confirms the frame 134 fullbody keyframe is obeyed exactly and puts the body on the couch, but approach/get_up transition remains weak. Multi-source sequence v1 was worse because frame 120 auto-selected the opposite axis; v2 fixed the axis flip but still left the body largely hidden/awkward around the couch arm/back. v3 hold stabilized the lie pose but did not solve the couch occlusion and entry/exit quality. Current HSI-only is improved over bbox-top binding, but not final-quality yet.
Problems found：the couch mesh has high arms/back and a narrow usable cushion; the binding score needs semantic pose priors and socket-drift constraints, not just penetration/support counts. Repeating intermediate curled source frames is not a good keyframe strategy for final lie. The next useful step is a pre-lie alignment keyframe: humanoid should stand at the front/entry point, turn/face consistently, then transition into one validated final lie keyframe rather than mixing curled lie source frames.
Validation：/home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q utils/sage_task_generator/tests/test_hoifhli.py tests/test_kimodo_plan_to_constraints.py tests/test_metrics.py utils/sage_task_generator/tests/test_lie_surface_access.py passed, 34 passed in 2.06s。
Temporary files removed：no; retained videos, review frames, configs, candidate manifests, and static binding reviews for visual comparison.

Future Entries

之后的新变更请按以下格式之一追加：

### YYYY-MM-DD - short title

- What changed.
- Why it changed.
- Files or outputs affected.
- Validation / known caveats.

### YYYY-MM-DD - experiment: short title

- Goal.
- Input task / scene.
- Generator and config.
- Command or wrapper entry point.
- Output paths retained.
- Metrics and visual judgment.
- Problems found.
- Follow-up changes.
- Temporary files removed.

2026-05-31 - experiment: migrate sit-style binding alignment to couch lie

Goal: follow the sit binding flow before tuning keyframes. The previous lie pass had a valid final fullbody pose, but it only patched final Root2D/fullbody frames; it did not derive a pre-action standing point or turn target from the bound pose the way sit does.

Changes:

Added lie_alignment_overrides support in utils/kimodo/plan_to_constraints.py. The root trajectory now treats lie_down and lie-based get_up specially: lie_down starts from a bound pre-lie standing point and ends at the bound lying root/heading; get_up starts from that lying root/heading and ends at the configured standing point.
Added automatic lie alignment export in tools/kimodo/legacy/smplx_keyframe_control.py. It reads the lie binding manifest surface axes and the selected bound SMPL-X pose, then derives: pre_lie_standing_xy_sage, pre_lie_heading, lying_root_2d_kimodo, and lying_heading. For the couch, this produced a side-entry standing point [3.7431, 4.3890], using the couch entry axis rather than hardcoding this scene.
Added pipeline config passthrough for pre_lie_distance and no_pre_lie_alignment in utils/kimodo/soma_keyframe_pipeline.py.

Experiment:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_v1_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_v1
Binding source: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_xp014_multiframe_pref134_v1
Lie fullbody frames: 182, 203, 224, all using source frame 134, axis -1.
The generated Kimodo run used Kimodo-SMPLX-RP-v1 with HF_ENDPOINT=https://hf-mirror.com.

Validation:

Control manifest shows lie_alignment.enabled = true.
Root2D path now ends the first reach at the bound pre-lie point, inserts a turn, then drives lie_down from that point to the bound lying root.
Fullbody constraint adherence is exact for the three lie keyframes: root diff 0.0, max local rotation diff about 2.03e-6, joint L2 about 2.33e-6.
Couch near-view review frames: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_v1/couch_review.
Regression tests: /home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q utils/sage_task_generator/tests/test_hoifhli.py tests/test_kimodo_plan_to_constraints.py tests/test_metrics.py utils/sage_task_generator/tests/test_lie_surface_access.py passed with 34 passed.

Visual result:

The binding-level behavior is now materially better than the prior hold-only pass. The actor stands at the couch side, turns, places a knee on the couch, and reaches a stable horizontal lying pose on the cushion.
The remaining issue is no longer the binding target itself. The generated transition still looks stiff: lie-down has an abrupt knee-on-couch phase, and get-up bends forward awkwardly before standing. Next optimization should be keyframe/generation quality: either add one semantic intermediate keyframe for a side-entry/kneeling transition, tune prompt/duration/CFG, or test a sit-then-lie bridge. The binding layer is now reusable for other couches or chairs as long as their support/entry axes are present in the scene context.

2026-05-31 - experiment: add lie get-up standing keyframe after binding

Goal: continue the binding-first debugging order. After sit-style lie binding was in place, the remaining obvious failure was that the actor reached the correct get-up root location but stayed in a kneeling pose at the final get-up frame.

Changes:

Added an optional lie-get-up standing fullbody keyframe path in tools/kimodo/legacy/smplx_keyframe_control.py.
The new keyframe is not hardcoded to this couch. It loads a SMPL-X standing template frame, rotates its root joint to the Root2D heading at the lie get-up endpoint, and writes the fullbody pose at that endpoint.
Added full-pipeline passthrough in utils/kimodo/soma_keyframe_pipeline.py: lie_getup_standing_template_motion, lie_getup_standing_template_frame, and no_lie_getup_standing_keyframe.

Experiment:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_stand_v1_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_stand_v1
Base comparison: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_single_v1
The only generation-control change from the single-final baseline is the added standing fullbody keyframe at frame 344.
Standing template: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_single_v1/kimodo_smplx_motion.npz, frame 104.

Validation:

Generated with full pipeline wrapper and HF_ENDPOINT=https://hf-mirror.com.
Kimodo used 4 constraints: Root2D, lie fullbody at frame 224, lie-get-up standing fullbody at frame 344, and final sit fullbody at frame 585.
Fullbody adherence in the generated motion: frame 224 root L2 0.0, max local rotation diff 2.26e-6; frame 344 root L2 0.0, max local rotation diff 1.73e-6; frame 585 root L2 0.0, max local rotation diff 2.74e-6.
Regression tests still pass: /home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q utils/sage_task_generator/tests/test_hoifhli.py tests/test_kimodo_plan_to_constraints.py tests/test_metrics.py utils/sage_task_generator/tests/test_lie_surface_access.py passed with 34 passed.

Visual result:

The get-up failure is substantially improved. In the previous hsi_lie_binding_first_single_v1, frame 344 was still a kneeling pose. In hsi_lie_binding_first_stand_v1, frame 300 is upright beside the couch and frame 344 is standing.
The final lie pose remains stable on the couch cushion at frame 224.
Remaining issue: the lie-down transition around frame 130 is still stiff and knee-led. Binding and get-up endpoint keyframes are now usable; the next isolated change should target a semantic intermediate lie-down keyframe or generation prompt/duration tuning for the entry phase only.

2026-05-31 - experiment: hybrid book route2 HOI/HSI visual tuning

Goal: improve the full hybrid route after validating HSI-only lie/sit quality. The user constraint was to keep the HOIFHLI adapter stable, use the existing full pipeline/wrapper modules, and make Kimodo SMPL-X based rather than SOMA.

Changes:

In utils/kimodo/object_keyframe_binding.py, chair-like sit binding now infers the real upward-facing support patch instead of using the object bbox top. For room_86161fe1_seat_fe54cd3f, support z moved from the old chair top around 0.862m to the cushion/seat surface around 0.445m.
In tools/hoifhli/legacy/hybrid_sequence.py, added append-followup so a stitched HOI pickup/carry/place sequence can be followed by a Kimodo HSI sequence while keeping the dynamic object static and checking root/object/hand seam gates.
In the same file, changed vertex baking to prefer pre-baked vertices/faces when they already exist. This is necessary for HOIFHLI pickup outputs: re-skinning their posed_joints/global_rot_mats with the Kimodo SMPL-X skin produced upside-down pickup frames in the hybrid render.

Experiments:

HOI-only book route2 used the existing adapter/generator flow with --object-name smallbox and HF_ENDPOINT=https://hf-mirror.com. Output: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_route2/hoifhli_generate. Metrics: pre_generation_alignment_pass=true, start/end object xy errors below 3e-7m, object displacement 2.354m, max frame-to-frame object jump 0.0425m.
Kimodo carry first reproduced the old root-only setup. It kept trajectory metrics valid but had weak hand-object contact. The best carry variant is: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_route2/carry_hand_book_text1. It uses variant=hand, text A person walks forward while carrying a small book with both hands., CFG (text=1.0, constraint=4.0), and trims the first 7 Kimodo carry pre-roll frames during stitching.
hand_endpoint was tested and rejected: hand metrics were good, but its fullbody endpoint constraints caused early carry frames to flip.
A later pickup anchor at frame 115 was tested and rejected as the default: it remained numerically valid, but the carry-start lunge was not visibly better and grasp error increased from 0.127m to 0.177m max.
Mesh bridging between HOIFHLI pickup vertices and Kimodo carry vertices was tested and rejected because the vertex ordering/topology is not compatible enough for direct interpolation; it produced crumpled bridge frames. The current best keeps the hard representation switch but disables pickup-carry rotation blend.
A short Kimodo transition segment was tested with build-transition/enrich-object: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_route2/transition_pickup96_to_carry7_v1. The transition's own end/root/object metrics passed, but the stitched pickup-to-transition seam failed with a left-hand/object offset jump around 1.14m. This confirms the current direct hand-carry stitch is still better than inserting that transition.
HOIFHLI proxy dimensions were rechecked before scheduling another long generation. The actual book mesh is about [0.159, 0.071, 0.258]m. smallbox remains the least-bad built-in proxy for this object; monitor, trashcan, and suitcase are substantially taller or less box-like, so no extra proxy generation was launched in this pass.

Best current full hybrid:

Output: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_route2/full_with_seat_hand_v3_trim7_norot
Video: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_route2/full_with_seat_hand_v3_trim7_norot/hybrid_book_smallbox_plus_seat_full_scene.mp4
Review sheet: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_route2/full_with_seat_hand_v3_trim7_norot/review_frames/contact_sheet.png
Clearer side-view render for review: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_route2/full_with_seat_hand_v3_trim7_norot/hybrid_book_smallbox_plus_seat_side_view.mp4
Side-view review sheet: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_route2/full_with_seat_hand_v3_trim7_norot/review_frames_side_view/contact_sheet.png
Stitch metrics pass: root/object/hand seam gates pass; final target error 0.0m; object remains static during the appended seat follow-up.
Carry grasp metrics for the selected hand variant pass: max both-hands object anchor error 0.127m, mean 0.024m.
Append metrics pass: frames=956, object_static_during_followup_max_step=0, base-to-followup root/object/hand jumps all 0.

Visual result:

The HOIFHLI-only pickup frame is now preserved in the full hybrid render; the earlier upside-down hybrid pickup frame was a stitch/render baking bug, not the HOIFHLI generator itself.
Carry now starts upright after trimming Kimodo pre-roll and disabling pickup-carry rotation blending. The book stays near the hands through the carry/release segment.
The final sit/get-up segment uses the seat support-surface binding and sits on the green chair cushion instead of the chair back.
Remaining issues: the first camera view is often occluded by the tall bookshelf/cabinet, pickup-to-carry still has a visible representation switch in mesh space, and book contact is good at wrist/hand-anchor level but not finger-precise. These are the next quality targets after this usable hybrid baseline.

Validation:

python -m py_compile tools/hoifhli/legacy/hybrid_sequence.py
/home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q tests/test_kimodo_plan_to_constraints.py tests/test_metrics.py utils/sage_task_generator/tests/test_hoifhli.py passed with 32 passed in 1.93s.

2026-05-31 - binding-first lie_down couch pass

Goal:

Revisit lie_down before doing any more keyframe or generation tuning.
Follow the existing sit binding pattern: bind to a valid object support surface, export alignment/root overrides, then insert only validated keyframes into the SMPL-X Kimodo full pipeline.

Root cause found:

Existing couch sit-entry bridge bindings were not trustworthy. Their root positions were on the wrong side/high part of the couch, e.g. SAGE x around 4.65 to 5.10, while the lie entry standing point is x about 3.74 and the bound lying root is x about 4.36.
The underlying issue was in utils/kimodo/object_keyframe_binding.py: couch was not included in the chair/sofa-like support-surface object set, so sit binding selected the couch back/top band around z 0.77m instead of the cushion support surface around z 0.38m.
The taxonomy file also did not map couch to sofa_like, so auto group selection could not naturally reuse sofa-like sit primitives for couches.

Code changes:

utils/kimodo/object_keyframe_binding.py
Treat couch as chair/sofa-like when selecting support surfaces.
Apply the same small seat-front socket offset rule to couch as to chair, armchair, sofa, and bench.
data/hsi_primitives/taxonomy/sage_sit_object_types.json
Added couch to sit_object_types.
Added couch to the sofa_like canonical group.

New binding:

Output: output/auto_experiments/20260531_hsi_visual/binding_couch_entry_cushion_v1
Command: /home/user/anaconda3/envs/isaac-sim/bin/python tools/kimodo/object_keyframe_binding.py --plan-dir output/auto_experiments/20260531_hsi_visual/hsi_couch_pseudo_plan_entry_couch --output-dir output/auto_experiments/20260531_hsi_visual/binding_couch_entry_cushion_v1 --target-id room_2b2c49cc_couch_17652d32 --groups auto --max-candidates 12 --constraint-frame 0
Result: 8 candidates, 8 valid.
Top candidate: sit_sofa_like_p00_seed0003, score 0.960963.
Corrected seat socket: [4.359707832336426, 4.531759738922119, 0.38011035323143005].
Top root position: SAGE [4.487419128417969, 4.542429447174072, 0.544672429561615], heading [0.0, 1.0].

Pipeline experiment:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_sitbridge_cushion_v1_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_sitbridge_cushion_v1
Pipeline entry point, with HF_ENDPOINT=https://hf-mirror.com: /home/user/anaconda3/envs/isaac-sim/bin/python -m tools.kimodo.legacy.soma_keyframe_pipeline --config output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_sitbridge_cushion_v1_config.json
Used the full pipeline wrapper and SMPL-X Kimodo generation, not direct low-level Kimodo commands.
Added valid lie-entry sit keyframe: lie_entry_sit_binding_dir=output/auto_experiments/20260531_hsi_visual/binding_couch_entry_cushion_v1, lie_entry_sit_candidate_rank=1, frame 147.
Kept the native couch lie binding: binding_couch_lie_xp014_multiframe_pref134_v1, lie frame 224.
Kept the get-up standing fullbody keyframe at frame 344.

Numerical checks:

Fullbody root positions match constraints exactly at frames 147, 224, 344, and 585.
Local rotation max errors:
frame 147: 2.220e-06
frame 224: 2.116e-06
frame 344: 1.669e-06
frame 585: 2.682e-06
Smooth-root drift remains from Kimodo smoothing, e.g. 0.111m at frame 224, but the constrained root positions themselves are exact.

Visual result:

Frame 130: actor stands in front of the couch without stepping onto it.
Frame 147: actor sits on the couch cushion; this fixes the earlier invalid bridge that sat on the back/edge.
Frames 182 and 203: transition into lie is usable but legs lift high over the arm/edge. This is now a keyframe/interpolation quality problem, not a binding-surface problem.
Frame 224: final lying pose is stable and supported on the couch.
Frames 300 and 344: get-up no longer ends kneeling; actor stands beside the couch.
Full scene video: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_sitbridge_cushion_v1/hsi_lie_binding_first_sitbridge_cushion_v1_full_scene.mp4
Couch review frames: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_sitbridge_cushion_v1/couch_review

Conclusion:

The binding migration from sit to lie is now on a much firmer path: couch support selection is fixed generally, entry sit binding is valid, and lie/get-up fullbody constraints are preserved.
Do not continue using binding_couch_pseudo_sofa_like or binding_couch_pseudo_reverse_couch_all for lie-entry bridge tests; they were generated before the couch support-surface fix and bind to the wrong surface.
Next quality work should tune the lie_down transition keyframes around frames 147 to 224, especially reducing the high-leg swing at frames 182/203. Candidate approaches: adjust the lie-entry sit fraction, add one intermediate side-lying binding from a real SMPL-X lie source frame, or tune generation/constraint weights after the keyframe path is inspected.

Validation:

/home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q utils/sage_task_generator/tests/test_hoifhli.py tests/test_kimodo_plan_to_constraints.py tests/test_metrics.py utils/sage_task_generator/tests/test_lie_surface_access.py passed with 34 passed in 1.94s.

2026-05-31 - lie_down keyframe timing sweep after couch binding fix

Goal:

After fixing couch binding, vary only lie_down keyframe placement to isolate visual quality issues in the transition from sit-on-cushion to lie-on-couch.
Keep the full pipeline wrapper, SMPL-X Kimodo model, couch lie binding, valid couch sit binding, and HF_ENDPOINT=https://hf-mirror.com unchanged.

Baseline retained:

output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_sitbridge_cushion_v1
lie_entry_sit_frame_fraction=0.35, sit frame 147, lie frame 224.
Best current behavior:
147: valid sit on cushion.
182/203: usable but still has raised legs during the roll-down.
224: stable final lie.
344: stable standing after get-up.

Rejected timing variants:

hsi_lie_binding_first_sitbridge_cushion_frac050_v1
Changed lie_entry_sit_frame_fraction to 0.50, sit frame 165.
Result: 165 sit frame is valid, but 182/203 become severe couch occlusion/penetration. Reject.
hsi_lie_binding_first_sitbridge_cushion_frac025_v1
Changed lie_entry_sit_frame_fraction to 0.25, sit frame 135.
Result: actor slides down/under the couch by 160, with large body/couch penetration through 182/203. Reject.
hsi_lie_binding_first_sitbridge_cushion_lieseq082_v1
Kept sit fraction 0.35.
Added repeated final lie fullbody constraints at fractions 0.82 and 1.0, frames 203 and 224.
Result: frame 203 becomes stable, but frame 182 raises legs even higher, so the full transition is not better than baseline. Reject as default.

Conclusion:

The binding and endpoint constraints are now correct; the remaining artifact is genuinely a transition/keyframe quality issue.
Keep hsi_lie_binding_first_sitbridge_cushion_v1 as the current HSI-only baseline.
The next useful keyframe work should not keep sweeping sit timing. Better options are:
add a real intermediate side-lying/sitting-to-lying source pose, not a duplicate final lying pose;
improve the lie primitive library with an entry/side-lie keyframe;
then retest generation weights or action prompt text after that keyframe exists.

2026-05-31 Lie Binding-First Follow-Up

User instruction: for lie, first make the binding follow the existing sit binding style, then inspect keyframe consumption.

Code changes:

utils/kimodo/lie_keyframe_binding.py
Added a selection_mode field to LieSurface and to output metrics.
Changed lie support-surface inference to mirror the sit binding pattern: upward-facing mesh faces are filtered into an object-type-aware support band, then the best z band is selected by area plus height prior.
Reclining objects (bed, sofa, couch, bench, daybed, chaise, seat, armchair) use reclining_support_band; generic objects still use top_support_band; missing candidates fall back to fallback_upward_faces.
Wrote selection_mode into both candidate metrics and binding_manifest.json surface metadata.
tests/test_lie_keyframe_binding.py
Added synthetic mesh tests verifying that a couch with high back/arms binds to the lower reclining support surface, while a generic table-like box keeps top-surface behavior.

New binding output:

output/auto_experiments/20260531_hsi_visual/binding_couch_lie_xp014_multiframe_pref134_v2_surfaceband
Command: /home/user/anaconda3/envs/isaac-sim/bin/python tools/kimodo/lie_keyframe_binding.py --plan-dir output/auto_experiments/20260531_hsi_visual/hsi_lie_socket_xp014_yp000 --target-id room_2b2c49cc_couch_17652d32 --source-motion output/auto_experiments/20260530_smplx_e2e/hsi_smplx_v0/kimodo_smplx_motion.npz --source-frames 75 90 105 120 134 --preferred-source-frame 134 --source-frame-prior-weight 0.2 --source-frame-prior-sigma 18.0 --surface-clearance 0.02 --output-dir output/auto_experiments/20260531_hsi_visual/binding_couch_lie_xp014_multiframe_pref134_v2_surfaceband
Top candidate remains lie_smplx_axis-1, source frame 134, valid true, score 0.902440.
Key binding metrics:
support_selection_mode=reclining_support_band
support_surface_z_sage=0.378849
contact_patch_support_ratio=1.0
contact_patch_z_gap_m=0.020000
surface_penetration_vertex_count=78
low_body_vertices_supported_ratio=0.950907

Keyframe-control check:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_surfaceband_control_v1_config.json
Wrapper command: /home/user/anaconda3/envs/isaac-sim/bin/python -m tools.kimodo.legacy.soma_keyframe_pipeline --config output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_surfaceband_control_v1_config.json
Stage: control only. No diffusion generation was run in this follow-up.
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_surfaceband_control_v1/kimodo_inputs
The generated smplx_keyframe_control_manifest.json consumes the new binding dir and carries the support_selection_mode metric through to the lie keyframe source candidate.
Numeric equivalence between smplx_lie_fullbody_keyframe.json and the bound pose bound_poses/10_lie_smplx_axis-1.npz:
keyframe frame: 224
root max abs error: 0.0
smooth-root max abs error: 0.0
axis-angle mean/max abs error: 0.0 / 0.0

Conclusion:

This is a reusable binding improvement, not a one-scene overfit. The only object-specific part is the existing run target couch; the implementation itself uses object-type classes plus mesh-derived support faces, and the new tests cover both reclining and generic support selection.
The current lie keyframe path correctly consumes the bound pose without modifying it. Remaining visual work should therefore proceed to keyframe sequencing and generation quality after this binding baseline, not by bypassing binding.

HSI-only lie keyframe sequencing tests

Experiment 1, rejected:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v1_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v1
Used the full pipeline wrapper with SMPL-X Kimodo generation: /home/user/anaconda3/envs/isaac-sim/bin/python -m tools.kimodo.legacy.soma_keyframe_pipeline --config output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v1_config.json
Added four lie_down fullbody keyframes from source frames 90,105,120,134 at default fractions, producing target frames 159,180,202,224.
Visual result:
Support/contact remained stable on the couch.
Rejected because frame 159 became horizontal too soon after the valid entry sit keyframe at frame 147; the transition looked over-constrained.
Side review: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v1/couch_review/contact_sheet.jpg

Experiment 2, current HSI-only lie baseline:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v2_delayed_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v2_delayed
Wrapper command: /home/user/anaconda3/envs/isaac-sim/bin/python -m tools.kimodo.legacy.soma_keyframe_pipeline --config output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v2_delayed_config.json
Used source frames 105,120,134, fixed axis -1, and explicit lie_down fractions 0.65,0.82,1.0, producing target frames 182,203,224.
Full scene video: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v2_delayed/hsi_lie_sequence_surfaceband_v2_delayed_full_scene.mp4
Side review video: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v2_delayed/hsi_lie_sequence_surfaceband_v2_delayed_human_right.mp4
Side review frames: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v2_delayed/couch_review/contact_sheet.jpg

Visual result:

Frame 147: valid sit on the couch cushion.
Frame 159: still sitting/leaning forward, no longer over-constrained into an immediate horizontal pose.
Frames 180/182/203/224: lying pose stays supported on the couch surface.
Frame 260: get-up midpoint is still visually awkward, with the body bent over the couch and limbs extending outside the support. This is the next HSI-only keyframe target; it is separate from lie binding.
Frames 300/344: standing recovery is acceptable and no longer kneeling.

Numeric constraint check for v2:

Root positions match all fullbody constraints exactly at frames 147,182,203,224,344,585.
Local axis-angle max errors:
frame 147: 2.146e-06
frame 182: 1.602e-04
frame 203: 3.195e-05
frame 224: 2.265e-06
frame 344: 1.729e-06
frame 585: 2.742e-06
Smooth-root drift remains due to Kimodo smoothing, max observed here 0.081m at frame 224.

Conclusion:

The delayed sequence is better than both the original single-lie-keyframe run and v1's too-early four-keyframe run. Keep hsi_lie_sequence_surfaceband_v2_delayed as the current HSI-only lie_down baseline if get_up is ignored.
Next HSI-only work should target the get-up midpoint around frame 260, likely by adding a bound intermediate get-up/sit/side-support keyframe or adjusting the get_up segment timing, before moving on to HOI-only tuning.

HSI-only get_up sit bridge

Code changes:

tools/kimodo/legacy/smplx_keyframe_control.py
Added optional --lie-getup-sit-binding-dir, --lie-getup-sit-frame-fraction, --lie-getup-sit-candidate-rank, and --allow-invalid-lie-getup-sit-binding-candidate.
The new option reuses an existing SOMA sit binding as a SMPL-X fullbody constraint inside the lie get_up segment, mirroring the already-working lie_entry_sit_binding_dir path.
This is a reusable get-up keyframe control hook, not a direct edit to one generated motion.
utils/kimodo/soma_keyframe_pipeline.py
Passes the new get-up sit options from JSON/YAML pipeline configs to smplx_keyframe_control.

Experiment 3, current HSI-only baseline:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v3_getupsit_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v3_getupsit
Wrapper command: /home/user/anaconda3/envs/isaac-sim/bin/python -m tools.kimodo.legacy.soma_keyframe_pipeline --config output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v3_getupsit_config.json
Lie_down keyframes: same delayed sequence as v2, frames 182,203,224.
New get_up sit keyframe:
binding dir: output/auto_experiments/20260531_hsi_visual/binding_couch_entry_cushion_v1
candidate rank: 1
target fraction: 0.35
target frame: 267
Full scene video: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v3_getupsit/hsi_lie_sequence_surfaceband_v3_getupsit_full_scene.mp4
Side review video: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v3_getupsit/hsi_lie_sequence_surfaceband_v3_getupsit_human_right.mp4
Side review frames: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v3_getupsit/couch_review/contact_sheet.jpg

Visual result:

Frames 147/159: sit and lean-forward transition are preserved.
Frames 180/182/203/224: lie_down remains supported on the couch.
Frames 250/260/267/280: get_up now passes through a seated posture on the cushion instead of hanging over the couch edge.
Frame 300: standing-up transition is acceptable.
Frame 344: upright beside the couch.

Numeric constraint check for v3:

Root positions match all fullbody constraints exactly at frames 147,182,203,224,267,344,585.
Local axis-angle max errors:
frame 147: 2.146e-06
frame 182: 1.602e-04
frame 203: 3.195e-05
frame 224: 2.265e-06
frame 267: 2.146e-06
frame 344: 1.729e-06
frame 585: 2.742e-06
Smooth-root drift remains due to Kimodo smoothing, max observed here 0.067m at frame 224.

Conclusion:

hsi_lie_sequence_surfaceband_v3_getupsit supersedes v2 as the current HSI-only baseline for the couch lie/get_up sequence.
Remaining HSI-only issues are now smaller visual polish problems, not binding failures: foot/leg placement during final lie could still be improved, but the sequence now follows a coherent sit -> lie -> sit -> stand keyframe path.

Lie source-frame 75 probe

I also checked whether the old Kimodo source frame 75 could serve as a sit-to-lie intermediate keyframe instead of jumping directly from sit to a nearly horizontal lie pose.

Experiment, rejected:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_sitbridge_cushion_lieseq75_v1_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_binding_first_sitbridge_cushion_lieseq75_v1
Parameters:
allow_invalid_lie_binding_candidate=true
lie_sequence_source_frames=[75,134]
lie_sequence_frame_fractions=[0.65,1.0]
lie_sequence_axis_sign=-1
Static source-frame review:
frame 75 visually resembles an all-fours/kneeling entry pose on the couch, which is directionally useful for lie_down.
the bound candidate is invalid because of body_mesh_penetrates_lie_surface, so it should not be promoted to the default binding library.
Visual result:
frames 160/182 look more plausible than an immediate horizontal lie: the actor leans forward and transitions through a supported kneeling pose.
frame 203 then collapses into a bad lie pose with the legs lifted nearly vertical, so the sequence is rejected.

Conclusion:

Frame 75 is useful evidence that lie_down needs a real intermediate entry/side-support pose, but this exact candidate is not clean enough.
Do not keep sweeping timing around this invalid candidate. The next binding-first improvement should create or mine a valid intermediate side-lie / supported-entry keyframe and bind it with the same support-band logic before another generation sweep.

HSI-only lie prompt and keyframe-density sweep

I ran three additional HSI-only checks after the binding-first baseline. All used the full SMPL-X pipeline wrapper and kept the couch lie binding plus couch entry/get-up sit binding unchanged.

Experiment 4, rejected:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v4_twolie_getupsit_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v4_twolie_getupsit
Change from v3: removed the frame-182 lie source keyframe and kept only source frames 120,134 at fractions 0.82,1.0.
Close side review: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v4_twolie_getupsit/couch_review_close/contact_sheet.jpg
Result: not better than v3. The actor still becomes nearly horizontal by frames 180/182, and the leg/foot lift artifact remains visible.

Experiment 5, rejected:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v5_singlelie_getupsit_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v5_singlelie_getupsit
Change from v3: kept only the final source frame 134 at fraction 1.0.
Close side review: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v5_singlelie_getupsit/couch_review_close/contact_sheet.jpg
Result: worse than v3. Frame 159 already shows the body occluded by the couch side/back, and frames 180/182/203/224 still do not fix the leg lift.

Code change before experiment 6:

utils/kimodo/plan_to_constraints.py
Added _contextual_english_prompt(...).
When lie_info exists, lie_down now uses A person sits on the support surface and carefully lies down.
When lie_info exists, lie get_up now uses A person sits up from lying down and stands up calmly.
Generic non-lie prompts are unchanged. This is binding-contextual, not a couch-specific prompt override.

Experiment 6, neutral:

Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v6_prompt_getupsit_config.json
Output: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v6_prompt_getupsit
Change from v3: same keyframe schedule as v3, but with the contextual lie prompts above.
Close side review: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v6_prompt_getupsit/couch_review_close/contact_sheet.jpg
Result: prompt text is correctly passed into Kimodo and does not regress the sequence, but it also does not remove the main leg/foot artifact during lie_down.

Current HSI-only conclusion:

Keep hsi_lie_sequence_surfaceband_v3_getupsit as the stable baseline; v6 is acceptable but not a clear visual upgrade.
The remaining HSI-only fix should not keep sweeping prompt text or keyframe density. The next meaningful change is to add/mine a valid bound intermediate side-support / entry-lie pose and then rerun the same v3-style pipeline.
Tests after the contextual prompt change: /home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q tests/test_kimodo_plan_to_constraints.py tests/test_lie_keyframe_binding.py utils/sage_task_generator/tests/test_lie_surface_access.py -> 28 passed.

2026-05-31 - HSI/HOI quality pass after binding-first lie debugging

Goal:

Continue in the requested order: finish the lie binding migration first, verify keyframe consumption, then tune HSI-only visual quality, then HOI-only visual quality, and only then rebuild hybrid variants.
Keep the HOIFHLI adapter implementation unchanged. All HOI changes in this pass are experiment-input/config changes.

HSI-only status:

The sit-style lie support-band binding is now the baseline binding path: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_xp014_multiframe_pref134_v2_surfaceband.
The binding selects support_selection_mode=reclining_support_band, support_surface_z_sage=0.378849, and keeps the preferred SMPL-X source frame 134 with top candidate lie_smplx_axis-1.
A control-only wrapper run verified that smplx_lie_fullbody_keyframe.json exactly matches bound_poses/10_lie_smplx_axis-1.npz: root error 0, smooth-root error 0, axis-angle mean/max error 0/0.
Additional root-aligned sit bridge trials were rendered: hsi_lie_sequence_surfaceband_v7_rootaligned_getupsit and hsi_lie_sequence_surfaceband_v8_mid90_rootaligned. They improved the front-facing entry sit pose but worsened the lie-down leg swing/occlusion.
Current HSI-only visual baseline therefore remains output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v3_getupsit. It has the coherent path sit -> lie -> sit -> stand; the remaining defect is transition polish around the lie-down leg/foot motion, not binding failure.

HOI-only book status:

The old book/smallbox HOIFHLI route was visually wrong because generation constraints were not aligned with the original scene support height: floor_with_final_support made HOIFHLI reach a floor-height pickup while the rendered real book mesh starts on the bookshelf.
I kept the adapter code stable and copied the adapter input to output/auto_experiments/20260531_hoi_visual/adapter_book_smallbox_linear_z/hoifhli_input.json, changing only interaction_z_policy to linear_pickup_to_place.
HOIFHLI generation used the hoifhli conda env and HF_ENDPOINT=https://hf-mirror.com: output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a.
Metrics improved versus the default/floor-z run:
object_path_length_m: 3.1396 -> 2.8164
max_frame_to_frame_object_jump_m: 0.0444 -> 0.0252
human_root_pickup_xy_error_m: 0.2001 -> 0.1741
nearest wrist/object distance over the generated motion: mean about 0.564m -> 0.172m
Review video: output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a/hoifhli_book_smallbox_linear_z_side_view.mp4.
Side contact sheet: output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a/review_frames_side/contact_sheet.png.
Visual judgment: the book is now carried near the hands/forearms rather than near the head/shoulder. Finger contact is still not precise, but the main HOI-only z/scale failure is fixed.

Hybrid rebuilds:

First rebuild used the existing full route: hybrid_carry build -> Kimodo SMPL-X carry generation -> graft-grasp -> hybrid_sequence stitch -> append-followup, with the new linear_z HOIFHLI pickup scene as input.
Output: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_with_seat_hand_v1_trim7_norot.
Numeric gates passed: root/object/hand seam jumps are 0, final target error is 0, object static during follow-up max step is 0, and max object step is 0.0210m.
Visual judgment: this is numerically clean, but it degrades the HOI-only visual result because the reconstructed Kimodo carry introduces a hunched pickup-to-carry posture. It is not the preferred visual baseline.

Preferred hybrid baseline:

I then preserved the complete optimized HOIFHLI pick/carry/place segment and appended the Kimodo seat/get-up follow-up directly.
Best output: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_seat_v3_nobridge_norot.
Rendered videos: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_seat_v3_nobridge_norot/hybrid_hoifhli_linear_z_plus_seat_full_scene.mp4 and output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_seat_v3_nobridge_norot/hybrid_hoifhli_linear_z_plus_seat_side_view.mp4.
Review sheets: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_seat_v3_nobridge_norot/review_frames/contact_sheet.png and output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_seat_v3_nobridge_norot/review_frames_side_view/contact_sheet.png.
Metrics:
all_gates_pass=true
max_frame_object_step_m=0.02516
max_seam_root_jump_m=0
max_seam_object_jump_m=0
max_seam_hand_offset_jump_m=0
object_static_during_followup_max_step_m=0
mesh_bridge_applied=false
rotation_blend=false
I tried two rejected append variants first:
full_hoifhli_plus_seat_v1: mesh bridge caused a folded gray body at the generator seam because HOIFHLI and Kimodo baked vertices/topology are not bridge-compatible.
full_hoifhli_plus_seat_v2_nobridge: disabling mesh bridge removed the fold, but rotation blend still made the first follow-up frame look like a sideways/fallen body.
v3_nobridge_norot fixes both by disabling mesh bridge and rotation blend while keeping root/object/hand seam metrics exact.
I also tried inserting a short Kimodo release/step-back segment after the HOIFHLI place frame: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/release_from_hoifhli_v1. This is rejected for now because the generated release did not preserve the HOIFHLI final fullbody start pose: start_joint_error_max_m=1.370m, start_gate_pass=false, even though root/object gates passed. It would introduce a larger pose discontinuity than the direct v3 append.

Current conclusion:

HSI-only is usable with known lie-down transition polish still open.
HOI-only book visual quality is materially better with linear_pickup_to_place.
The preferred hybrid should preserve the optimized HOIFHLI HOI segment rather than regenerate carry with Kimodo. The next quality target is a smoother semantic transition from HOIFHLI place/end pose into the Kimodo follow-up, but the current v3 hybrid is the cleanest baseline produced in this pass.

HSI lie binding-first pass: sit-to-lie alignment and transition probes

Scope and constraint reminder:

This pass followed the binding-first instruction for couch lie: first compare the existing sit binding path with the lie binding, then inspect keyframes.
HOIFHLI adapter code was not touched.
All generation runs used the full wrapper /home/user/anaconda3/envs/isaac-sim/bin/python -m tools.kimodo.legacy.soma_keyframe_pipeline --config ... with HF_ENDPOINT=https://hf-mirror.com and Kimodo-SMPLX-RP-v1.

Reusable binding change:

utils/kimodo/object_keyframe_binding.py
Added seat_contact_target_xy_sage and CLI option --seat-contact-target-xy X Y.
This lets a sit binding target an explicit support-surface socket instead of always using the default couch center. It is a reusable binding override for sit-on-support actions, not a one-off edit to a generated motion.
Manifest output now records seat_contact_target_xy_override_sage under object_sockets when the override is active.
Added focused tests in tests/test_object_keyframe_binding.py for both default couch socket behavior and explicit xy override behavior.
Test command: /home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q tests/test_object_keyframe_binding.py tests/test_lie_keyframe_binding.py -> 4 passed.

Binding outputs:

Rejected direct socket alignment: output/auto_experiments/20260531_hsi_visual/binding_couch_entry_lie_aligned_v1. It aligned the sit contact socket to the lie socket, but worsened the sit-root to lie-root 2D gap to about 0.598m.
Preferred root-aligned sit binding: output/auto_experiments/20260531_hsi_visual/binding_couch_entry_lie_root_aligned_v1. It preserves the sit front/back contact while aligning the long-axis root to the final lie root. Top candidate root-to-lie 2D gap is about 0.123m, versus about 0.197m for the old couch-center sit binding.
Static preview: output/auto_experiments/20260531_hsi_visual/binding_couch_entry_lie_root_aligned_v1/top_candidate_static_preview_smplx22.png.

Generation/keyframe probes after binding:

hsi_lie_sequence_surfaceband_v7_rootaligned_getupsit: root-aligned sit binding, original lie frames [105,120,134]. Seated keyframes are better aligned, but the lie-down middle still has the leg/foot vertical artifact around frame 170.
hsi_lie_sequence_surfaceband_v8_mid90_rootaligned: adds valid source frame 90 before 105/120/134. This improves the frame-170 vertical-leg artifact versus v7 and remains one of the better HSI-only candidates, but frame 159 still starts a rough forward-fold/drop.
hsi_lie_sequence_surfaceband_v9_latersit_mid90: rejected. Delaying sit and source-90 too far makes frame 170 a large side-floating transition.
hsi_lie_sequence_surfaceband_v10_sit159_mid90_170: sitting lasts longer, but the transition from sit to lie is still a sudden fold/bridge around frames 165-170; not a clear upgrade over v8.
hsi_lie_sequence_surfaceband_v11_frame75_midsequence: probe only. It explicitly allowed invalid source frame 75 plus [90,105,120,134] to test whether an all-fours support pose helps. Rejected: source-75 has body_mesh_penetrates_lie_surface, and generation shows a crawl-like entry plus a leg flip around frame 170.
Dense transition binding: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_xp014_transition_dense_v1.
Evaluated source frames 70,75,80,85,90,95,100,105,110,115,120,127,134.
Valid useful frames include 85/90/95/100/105/110/115/120/127/134.
75 and 80 remain invalid due to body_mesh_penetrates_lie_surface.
70 is valid but its root/socket is too far along the couch long axis for the current sit-to-lie sequence.
hsi_lie_sequence_surfaceband_v12_valid85_dense: uses only legal dense binding frames [85,95,105,120,134], no invalid lie candidates. It is geometrically cleaner than v11, but enters a lying pose too early; frame 159 is already largely horizontal, so it does not solve semantic sit-then-lie timing.
First v12 retry failed from GPU OOM; the completed run succeeded after the stale allocation disappeared. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is not usable with Kimodo here because the text encoder shares CUDA tensors across multiprocessing workers.

Review artifacts:

v10 front/right contact sheets: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v10_sit159_mid90_170/couch_review/contact_sheet.jpg and output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v10_sit159_mid90_170/couch_review_right/contact_sheet.jpg.
v11 front/right contact sheets: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v11_frame75_midsequence/couch_review/contact_sheet.jpg and output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v11_frame75_midsequence/couch_review_right/contact_sheet.jpg.
v12 front/right contact sheets: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v12_valid85_dense/couch_review/contact_sheet.jpg and output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v12_valid85_dense/couch_review_right/contact_sheet.jpg.

Current HSI conclusion:

The root alignment part of sit binding is now fixed in a reusable way.
The remaining visual issue is keyframe semantics, not ordinary socket binding: we need a valid intermediate side-support / swing-legs / sit-to-prone pose that is close to the root-aligned sit socket and does not force an immediate horizontal lie.
Do not promote source-75 directly; it is useful evidence but invalid. Do not promote source-85 alone either; it is legal but already too close to final lie.
Best current HSI-only baseline remains in the v8/v10 family depending on whether longer visible sitting or earlier smoother lying is preferred. Neither should be considered final for the couch lie action.

HSI lie binding migration into the full pipeline

What changed:

utils/kimodo/soma_keyframe_pipeline.py
Added optional lie_bind stage so lie binding can be run through the same full wrapper instead of manually calling tools/kimodo/lie_keyframe_binding.py.
The normal bind stage now passes binding.seat_contact_target_xy through to object_keyframe_binding --seat-contact-target-xy.
When a config has a lie_binding section, control defaults --lie-binding-dir to that stage's output.
utils/kimodo/object_keyframe_binding.py
Added a generic interaction-target fallback for sit binding on objects that only expose lie targets. This was needed for the couch in hsi_2d89d0a5_tight_lie_getup_sit; it has explicit_interaction_targets_by_mode.lie but no sit.
seat_front_direction_xy now falls back to lie_entry_direction_xy for this case.
utils/kimodo/lie_keyframe_binding.py
Added --lie-contact-target-xy X Y, analogous to sit's explicit seat socket override.
Manifest now records lie_contact_target_xy_override_sage.
Tests:
/home/user/anaconda3/envs/isaac-sim/bin/python -m pytest -q tests/test_object_keyframe_binding.py tests/test_lie_keyframe_binding.py -> 6 passed.

Binding findings:

Full-wrapper default scene lie target:
Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v14_binding_pipeline_config.json
Sit binding output: output/auto_experiments/20260531_hsi_visual/binding_couch_entry_lie_root_aligned_pipeline_v1
Lie binding output: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_transition_dense_pipeline_v1
This ran through soma_keyframe_pipeline --stages bind lie_bind.
It was valid, but root alignment was worse: entry-sit root to selected axis-1 lie roots was about 0.17-0.26m.
Explicit lie socket override:
Config: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v15_lie_xyoverride_binding_config.json
Lie binding output: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_transition_dense_pipeline_v2_xoverride
lie_contact_target_xy=[4.499875545501709, 4.45].
This restored the good geometry: entry-sit root to axis-1 lie source frames 85/95/105/120/134 is about 0.05-0.12m.
Static source review: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_transition_dense_pipeline_v2_xoverride/static_review_right/source_static_sheet.jpg. Source 85/95/105/120/134 are legal and visually plausible; source85 is already a prone/side-prone pose, not a true sit-to-side-support pose.

Generation runs after the binding fix:

hsi_lie_sequence_surfaceband_v15_lie_xyoverride_binding
Full wrapper stages: control generate skin render.
Contact sheets:
- output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v15_lie_xyoverride_binding/couch_review/contact_sheet.jpg
- output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v15_lie_xyoverride_binding/couch_review_right/contact_sheet.jpg
Current best HSI-only candidate from this pass. Binding/root placement is good, the person sits first, then transitions into a valid couch lie. Remaining issue: frame 165-180 still has an abrupt sit-to-prone interpolation and a raised-leg/forward-fold artifact.
hsi_lie_sequence_surfaceband_v16_lateblend25
Tried later sit/blend/source timing: sit frame 155, blend frame 172, first source85 frame 191, blend alpha 0.25.
Rejected. It keeps sitting longer, but makes the model rush into the first lie keyframe; frame 180 becomes a more obvious suspended plank / raised-leg pose.
hsi_lie_sequence_surfaceband_v17_blend25_v15timing
Same timing as v15 but lower blend alpha 0.25.
Rejected as not clearly better; frame 172 has heavier fold/head placement than v15.

Current HSI recommendation:

Use v15 as the current HSI-only baseline.
The binding is now a reusable, pipeline-backed solution, not a scene-only hack:
sit socket override handles couch entry sit;
lie socket override handles couch/sofa lateral placement;
lie_bind lets future chairs/sofas run through the same wrapper.
The remaining visual limitation is a missing semantic intermediate pose between sit and prone. Simple timing/alpha changes did not solve it. The next real upgrade should add or derive a valid side-support/swing-legs source keyframe rather than just moving source85 earlier/later.

Hybrid follow-up diagnostics: release/reskin rejected, stable HSI lie source preferred

Scope:

HOIFHLI adapter implementation was not changed.
The optimized HOIFHLI book segment is still output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a/hoifhli_scene_motion.npz.
All hybrid assembly used the wrapper /home/user/anaconda3/envs/isaac-sim/bin/python -m tools.hoifhli.legacy.hybrid_sequence ....

Release diagnostic:

Added --pose-constraint-space {local,world} to hybrid_sequence build-release and made the local mode use the same Kimodo-native local_joints_rot + root_positions + smooth_root_2d fullbody format as the HSI sit/lie keyframes.
Probe output: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/release_from_hoifhli_v3_local.
Result: rejected. Local fullbody constraints did not improve the release start pose: start_joint_error_max_m=1.361m, start_gate_pass=false, while root/object gates passed. This confirms the release issue is not only the old world-vs-local constraint format.

Append/reskin diagnostic:

Added optional --force-reskin-followup to hybrid_sequence append-followup so cached follow-up vertices can be ignored after joint/rotation blending.
full_hoifhli_plus_seat_v5_start398_reskin_rotblend: rejected. Re-skinning with rotation blend exposed cross-generator rotation mismatch; the body becomes sideways/folded after the seam and later jumps.
full_hoifhli_plus_seat_v6_start398_reskin_norot: rejected. No-rotation reskin avoids the immediate side-fall but still inherits the same bad seat follow-up height excursion.

Seat follow-up source audit:

The previous seat-only source output/auto_experiments/20260531_hybrid_visual/hybrid_seat_support_pipeline_v1/kimodo_smplx_motion.npz is not a good HSI source: root_y_max=3.620m, max_root_step=2.140m.
The older seat-only hybrid v4 therefore improves the seam relative to v3 but still inherits an unstable HSI follow-up source: output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_seat_v4_start398_nobridge_norot.
Stable HSI lie source: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v3_getupsit/kimodo_smplx_motion.npz, with root_y_max=1.054m and max_root_step=0.038m.

Current hybrid candidate:

output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_hsi_lie_v7_hsiv3_start0_norot.
Metrics:
all_gates_pass=true
max_frame_root_step_m=0.0676
max_frame_object_step_m=0.0252
max_seam_root_jump_m=0
max_seam_object_jump_m=0
max_seam_hand_offset_jump_m=0
object_static_during_followup_max_step_m=0
rotation_blend=false
mesh_bridge_applied=false
Rendered review videos:
.../full_hoifhli_plus_hsi_lie_v7_hsiv3_start0_norot/hybrid_hoifhli_linear_z_plus_hsi_lie_side_view.mp4
.../full_hoifhli_plus_hsi_lie_v7_hsiv3_start0_norot/hybrid_hoifhli_linear_z_plus_hsi_lie_full_scene.mp4
Review sheets:
.../full_hoifhli_plus_hsi_lie_v7_hsiv3_start0_norot/review_compare_side/contact_sheet.png
.../full_hoifhli_plus_hsi_lie_v7_hsiv3_start0_norot/review_compare_front/contact_sheet.png
full_hoifhli_plus_hsi_lie_v8_hsiv3_start33_norot was also tested. It has similar metrics, but does not clearly improve the seam and shortens the HSI follow-up, so v7 is preferred for now.

Current conclusion:

HOI-only best remains the linear-z HOIFHLI book run.
HSI-only lie v3/v8/v10 family remains the source of usable HSI behavior; the old hybrid seat support source should not be reused until regenerated.
Hybrid v7 is the current best assembled candidate because it replaces the unstable seat-only follow-up with a stable HSI lie sequence. The remaining visible issue is still the semantic transition from HOIFHLI placing the book to the HSI follow-up start; release generation and rotation/vertex blending are not reliable fixes yet.

Hybrid follow-up correction: same-scene seat source instead of cross-scene couch lie

Correction after auditing scene ids:

The stable HSI lie source used by v7 is bound to couch id room_2b2c49cc_couch_17652d32, while the book HOI scene is layout_84b703fb with dynamic object id room_86161fe1_book_17ef5d4c.
layout_84b703fb does not contain a couch/sofa/bed; it contains several chair_* and seat_* objects, including room_86161fe1_seat_fe54cd3f.
Therefore v7 is useful as a stable-motion diagnostic, but it is not the best scene-consistent book hybrid.

Same-scene seat source audit:

hybrid_seat_support_pipeline_v1 is bound to room_86161fe1_seat_fe54cd3f, which is scene-consistent for the book task.
Its original full clip should not be appended from frame 398: the source has large root jumps in the HOI-like pick/carry segment around frames 203-206 and a vertical excursion around frames 759-766.
The actual reusable seat follow-up portion starts after that bad reach phase, around frame 766, just before the final turn/sit/get-up stage.

Current preferred book hybrid:

output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_seat_v11_start766_norot
Inputs:
base: output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a/hoifhli_scene_motion.npz
follow-up: output/auto_experiments/20260531_hybrid_visual/hybrid_seat_support_pipeline_v1/kimodo_smplx_motion.npz
follow-up slice: 766..1054
rotation blend: disabled
mesh bridge: disabled
Metrics:
all_gates_pass=true
max_frame_root_step_m=0.0564
max_frame_object_step_m=0.0252
max_seam_root_jump_m=0
max_seam_object_jump_m=0
max_seam_hand_offset_jump_m=0
object_static_during_followup_max_step_m=0
Review videos:
.../full_hoifhli_plus_seat_v11_start766_norot/hybrid_hoifhli_linear_z_plus_seat_side_view.mp4
.../full_hoifhli_plus_seat_v11_start766_norot/hybrid_hoifhli_linear_z_plus_seat_full_scene.mp4
Review sheets:
.../full_hoifhli_plus_seat_v11_start766_norot/review_compare_side/contact_sheet.png
.../full_hoifhli_plus_seat_v11_start766_norot/review_compare_front/contact_sheet.png

Other same-pass probes:

full_hoifhli_plus_hsi_lie_v9_hsiv8_start0_norot: assembled from the HSI v8 couch-lie source. Metrics pass, but it remains cross-scene for the book task and does not visually justify replacing v11.
full_hoifhli_plus_hsi_lie_v10_hsiv12_start0_norot: assembled from the HSI v12 couch-lie source. Metrics pass, but it has the same cross-scene limitation and no clear visual advantage.
full_hoifhli_plus_seat_v12_start787_norot: same-scene seat source starting at frame 787, selected by pose similarity to the HOIFHLI final frame while still preserving turn/sit/get-up. Metrics pass (max_frame_root_step_m=0.0608), but the seam is not visually better than v11 and it removes some useful turn-before-sit context. Keep v11.
full_hoifhli_plus_seat_v13_start766_bridge12_norot: same as v11 but with a 12-frame mesh bridge. Metrics pass and the bridge is technically applied, but review frames 275-278 show an inflated/stretched blue body blob at the seam. Reject; mesh bridge remains worse than the direct no-rotation append.

Updated conclusion:

For the book scene, v11 supersedes v4 and v7 as the current preferred hybrid because it preserves the optimized HOIFHLI segment and uses the same-scene seat_fe54cd follow-up without the unstable source frames.
Remaining issue: there is still a semantic seam from HOIFHLI's final place pose into the start of the seat follow-up. Release generation, mesh bridge, rotation blend, and force-reskin attempts were all worse than the direct no-rotation append.

Latest status note: HSI lie binding/control pass

Latest HSI work is the section HSI lie binding migration into the full pipeline above.
Current recommended HSI-only baseline from this pass: output/auto_experiments/20260531_hsi_visual/hsi_lie_sequence_surfaceband_v15_lie_xyoverride_binding.
Rejected later keyframe variants: hsi_lie_sequence_surfaceband_v16_lateblend25 and hsi_lie_sequence_surfaceband_v17_blend25_v15timing.
The useful reusable changes are now in the pipeline: bind supports seat_contact_target_xy, lie_bind supports lie_contact_target_xy, and control can consume the generated lie binding without manually passing a separate lower-level command.

Latest status note: HOI-only visual audit after HSI binding

Current HOI-only baseline remains:

output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a

The run is intentionally still using the HOIFHLI wrapper:

Adapter input: output/auto_experiments/20260531_hoi_visual/adapter_book_smallbox_linear_z/hoifhli_input.json
Generator wrapper: python -m tools.hoifhli.legacy.generate
HOIFHLI object proxy: smallbox
Rendered SAGE dynamic object: room_86161fe1_book_17ef5d4c
Actual rendered mesh: /mnt/data/PhysHSI/data/selected_SAGE10k/e2e_test_scenes/84b703fb/objects/982a932e.ply
Z policy: linear_pickup_to_place

Why smallbox is still the safer proxy:

HOIFHLI's available grasp-cache proxy set is effectively limited to smallbox and largebox for this kind of carry. largebox is too coarse for the current book-sized object.
The rendered book mesh local bbox is approximately [0.159, 0.258, 0.071] meters, while the HOIFHLI internal source center height is 0.08m. The exported motion records the true visual center height as 0.1297971m, so the render/conversion layer compensates the final support height without changing the adapter.

Current metrics:

object_path_length_m = 2.8164
max_frame_to_frame_object_jump_m = 0.0252
human_root_pickup_xy_error_m = 0.1741
human_root_place_xy_error_m = 0.4078
Object floor height reaches the target support height at the end: final object floor z is 0.7569, matching the planned support z.
Wrist-to-object-center distance after pickup stays roughly 0.16-0.18m.

New review artifacts:

Right-side review video: output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a/hoifhli_book_smallbox_linear_z_right_review.mp4
Right-side contact sheet: output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a/review_frames_right_clear/contact_sheet.png

Visual conclusion:

Path, object continuity, and support placement are acceptable for the HOI-only baseline.
The remaining visual issue is hand/finger fidelity: the hands carry the book-sized mesh near the palms but do not tightly wrap the true book geometry. This is consistent with using HOIFHLI's smallbox proxy and should not be fixed by changing the existing adapter implementation.
No new HOIFHLI generation was started in this pass because the current linear-z run is already the best validated HOI-only candidate. If a later pass specifically targets hand fidelity, the next controlled experiment should use the existing wrapper with --grasp-iter or --no-finger-motion variants, then compare against this baseline.

Latest status note: hybrid baseline after HSI/HOI audit

Hybrid baseline remains:

output/auto_experiments/20260531_hybrid_visual/hybrid_book_smallbox_linear_z_route2/full_hoifhli_plus_seat_v11_start766_norot

Inputs:

HOI base: output/auto_experiments/20260531_hoi_visual/hoifhli_book_smallbox_linear_z_default_a/hoifhli_scene_motion.npz
Same-scene HSI follow-up: output/auto_experiments/20260531_hybrid_visual/hybrid_seat_support_pipeline_v1/kimodo_smplx_motion.npz
Follow-up slice: 766..1054

Metrics:

all_gates_pass = true
max_frame_root_step_m = 0.0564
max_frame_object_step_m = 0.0252
max_seam_root_jump_m = 0
max_seam_object_jump_m = 0
max_seam_hand_offset_jump_m = 0
object_static_during_followup_max_step_m = 0

Why the new couch-lie HSI v15 is not used for this book hybrid:

HSI v15 is the current best couch-lie baseline, but it is bound to the separate couch scene/object. The book HOI scene does not contain that couch.
Directly appending v15 would recreate the earlier cross-scene diagnostic problem, so the current book hybrid keeps the same-scene seat follow-up.

Additional hybrid check:

full_hoifhli_plus_seat_v13_start766_bridge12_norot was rechecked. It passes the same gates and applies a 12-frame mesh bridge, but the bridge only smooths the body mesh around the seam. It does not solve the semantic transition from the HOIFHLI final place pose into the seat follow-up, so it does not supersede v11.

Current hybrid conclusion:

v11 remains the preferred book-scene hybrid because it is scene-consistent and numerically stable.
The remaining issue is semantic, not a simple root/object continuity bug: after placing the book, the human still needs a better generated transition into the same-scene seat follow-up. Prior release generation, mesh bridge, rotation blend, and force-reskin variants did not produce a better result than the direct no-rotation append.

Latest status note: couch lie armrest binding

Goal addressed in this pass:

Allow a couch lie binding where the head, upper body, and feet can be supported by the two couch armrests instead of forcing every key joint onto a single flat cushion plane.

Code changes:

utils/kimodo/lie_keyframe_binding.py
Added explicit armrest-region extraction from the target object mesh.
Split core lower-body support from upper-body support, so spine3, neck, shoulders, and head may be supported by an armrest while pelvis/hips/knees remain on the cushion.
Kept foot-on-armrest valid when the foot is inside the armrest region and not laterally outside the couch.
Added metrics such as head_armrest_support_ratio, upper_armrest_support_ratio, foot_armrest_support_ratio, and endpoint_bad_max_overhang_m.
tools/kimodo/action_keyframe_factory.py
Added explicit prompt-index selection so new canonical keyframes can be appended through the full pipeline without calling Kimodo's lower-level generate command directly.
Added couch prompts for head/shoulder/upper-back-on-armrest lies.
tools/kimodo/generate_lie_keyframe_library.local.sh
Added environment overrides for prompt indices, prompt limit, seeds, target count, CUDA device, and cache paths while preserving the full pipeline entry.
data/hsi_primitives/taxonomy/sage_lie_object_types.json
Added couch-like armrest lie prompts.

Generated keyframes:

Full pipeline command used: PROMPT_INDICES="4 7 8" SEEDS="0:2" TARGET_PER_GROUP="-1" PROMPT_LIMIT="9" CUDA_VISIBLE_DEVICES="1" TEXT_ENCODER_MODE="api" TEXT_ENCODER_URL="http://127.0.0.1:9550/" bash tools/kimodo/generate_lie_keyframe_library.local.sh
Accepted:
lie_couch_like_p04_seed0000
lie_couch_like_p04_seed0001
lie_couch_like_p07_seed0000
lie_couch_like_p07_seed0001
lie_couch_like_p08_seed0000
Rejected:
lie_couch_like_p08_seed0001, because its arms were too wide for the reclining support filter.

Binding results:

Best armrest-aware baseline: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v11_armrest_surface
Same library plus newly generated armrest prompts: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v12_armrest_prompts
Slightly lifted clearance variant: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v13_armrest_clearance03

Current best candidate:

Source keyframe: lie_couch_like_p01_seed0000
Prompt: A person lies on a couch with arms resting by their sides.
It remains better than the new explicit armrest prompts because the new prompt generations are often too long for this particular couch and push feet beyond the opposite armrest.

Key v11/v12 metrics:

head_armrest_support_ratio = 1.0
upper_armrest_support_ratio = 0.8
foot_armrest_support_ratio = 1.0
foot_bad_max_overhang_m = 0
surface_penetration_vertex_count = 0
hand_backrest_intrusion_m = 0

Review videos:

v11 full multi-view: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v11_armrest_surface/static_rank1_human-front.mp4 output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v11_armrest_surface/static_rank1_human-back.mp4 output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v11_armrest_surface/static_rank1_human-left.mp4 output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v11_armrest_surface/static_rank1_human-right.mp4 output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v11_armrest_surface/static_rank1_fixed_oblique.mp4
v13 clearance comparison: output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v13_armrest_clearance03/static_rank1_human-front.mp4 output/auto_experiments/20260531_hsi_visual/binding_couch_lie_library_v13_armrest_clearance03/static_rank1_fixed_oblique.mp4

Visual conclusion:

v11/v12 now model the couch as a cushion plus two armrest support regions and match the intended pattern: head/upper body can use one armrest while feet can use the opposite armrest.
v13 removes a little visual risk by lifting the body contact patch from 0.01m to 0.03m, but the score drops because the contact patch is less tightly seated on the cushion. Prefer v11 if contact should look grounded; use v13 if the next review still flags slight cushion penetration.

2026-05-31 2cdc81db short-bed HSI lie prompt and binding check

Context:

Scene: data/selected_SAGE10k/e2e_test_scenes/2cdc81db/layout_2cdc81db.json
Task output: output/e2e_task_previews/20260531_2cdc81db_hsi_only_lie_bed_v1
Target bed: room_e8121c03_bed_c9809501
Bed support metrics from binding:
usable support length: 1.2718246637821198m
physical long bounds length: 1.6305444407463074m

Prompts previously used for bed_like Kimodo lie keyframes:

A person lies on a bed.
A person lies on a bed with arms resting by their sides.
A person lies on their back on a bed.
A person lies on their side on a bed with knees slightly bent.
A person lies on a bed with knees gently bent and arms relaxed.
A person lies on their side on a short bed with legs slightly curled.

Finding:

The short-bed / knees-bent prompts above still generated mostly straight lying clips. The shortest selected source spans were around 1.70m, too long for the modified bed's safe support length.

Change:

Added three stronger but still reusable bed_like prompts to:
data/hsi_primitives/taxonomy/sage_lie_object_types.json
tools/kimodo/action_keyframe_factory.py
New prompts:
A person lies in a curled side-lying pose on a short bed with knees bent and arms relaxed.
A person lies on their side on a short bed with knees bent enough to fit on the bed.
A person rests on a short bed in a curled side-lie pose with knees bent and feet kept on the bed.

Generation command:

export HF_ENDPOINT=https://hf-mirror.com
CANONICAL_GROUPS=bed_like PROMPT_INDICES="6 7 8" SEEDS=0:4 TARGET_PER_GROUP=-1 PROMPT_LIMIT=9 CUDA_VISIBLE_DEVICES=1 TEXT_ENCODER_MODE=api TEXT_ENCODER_URL=http://127.0.0.1:9550/ bash tools/kimodo/generate_lie_keyframe_library.local.sh

Generation result:

Accepted all 12 generated candidates for prompt indices 6, 7, and 8.
Best new short-bed source by geometry:
item: lie_bed_like_p06_seed0002
prompt: A person lies in a curled side-lying pose on a short bed with knees bent and arms relaxed.
selected frame: 3
pose score: 0.9984628290892674
source body long span: about 1.588m

Binding command:

export HF_ENDPOINT=https://hf-mirror.com
/home/user/anaconda3/envs/isaac-sim/bin/python -m tools.kimodo.legacy.soma_keyframe_pipeline --config output/auto_experiments/20260531_hsi_visual/2cdc81db_hsi_bed_lie_v4_curled_prompt_config.json --stages lie_bind

Binding result:

Binding output: output/auto_experiments/20260531_hsi_visual/2cdc81db_bed_lie_binding_v4_curled_prompt_library
Top candidate: lie_bed_like_p06_seed0002_axis+1
Source prompt: A person lies in a curled side-lying pose on a short bed with knees bent and arms relaxed.
Binding score: 0.8643030956719664
Valid: true
Invalid reasons: none
Key metrics:
pose_body_long_span_m = 1.5875614881515503
support_usable_length_m = 1.2718246637821198
support_physical_length_m = 1.6305444407463074
foot_bad_max_overhang_m = 0
head_bad_max_overhang_m = 0
hand_backrest_intrusion_m = 0
surface_penetration_vertex_count = 0

Review images:

Full scene front: output/auto_experiments/20260531_hsi_visual/2cdc81db_bed_lie_binding_v4_curled_prompt_library/static_rank1_human_front.png
Full scene right: output/auto_experiments/20260531_hsi_visual/2cdc81db_bed_lie_binding_v4_curled_prompt_library/static_rank1_human_right.png
Bed-only front: output/auto_experiments/20260531_hsi_visual/2cdc81db_bed_lie_binding_v4_curled_prompt_library/static_rank1_bedonly_human_front.png
Bed-only right: output/auto_experiments/20260531_hsi_visual/2cdc81db_bed_lie_binding_v4_curled_prompt_library/static_rank1_bedonly_human_right.png

Visual conclusion:

The stronger curled side-lying prompt produced one useful short-bed pose. It is materially better than the earlier straight-body candidates and should be the current static lie-binding baseline for this scene.
The pose is not fully solved. It still needs visual tuning of hand/foot placement near the bed frame and possibly an explicit support-length-aware scorer so future short beds prefer these curled poses automatically rather than relying on a lucky seed.

2026-05-31 short-support lie filter update

Change:

tools/kimodo/action_keyframe_factory.py
Added stronger short-bed side-lie prompts.
Added prompt-aware short_side_lie scoring. Short-bed curled/knee-bent prompts now prefer a shorter head-to-feet footprint and bent legs instead of rewarding fully stretched lying poses.
utils/kimodo/lie_keyframe_binding.py
Added short-support length filtering. For short reclining supports, poses whose body long-axis span exceeds the physical support length minus a configurable margin are marked invalid.
Increased the score weight of support-length fit for short beds, so less stretched candidates rank above long straight candidates.
utils/kimodo/soma_keyframe_pipeline.py
Exposes the new lie-binding length-filter parameters through pipeline config.
Config added: output/auto_experiments/20260531_hsi_visual/2cdc81db_hsi_bed_lie_v5_short_filter_config.json

Suggested next run:

Generate new short-bed prompts with indices 9 10 11 and more seeds.
Run only lie_bind first. If there are no valid candidates, loosen short_support_physical_margin from 0.06 to 0.04; otherwise inspect the top candidate before running full HSI sequence generation.

2026-05-31 HOIFHLI official demo path replay

Goal:

Replay the existing HOI-only instrument tray / smallbox path inside the official HOIFHLI no-scene demo environment, to separate scene-collision problems from HOIFHLI's own navigation/interaction behavior.

Command:

cd /mnt/data/PhysHSI
source /home/user/anaconda3/etc/profile.d/conda.sh
conda activate hoifhli
export HF_ENDPOINT=https://hf-mirror.com
export CUDA_VISIBLE_DEVICES=1
python -m tools.hoifhli.legacy.generate \
  --adapter-input output/auto_experiments/20260531_hoi_visual/hoi_instrumenttray_smallbox/adapter/hoifhli_input.json \
  --out-dir output/auto_experiments/20260531_hoi_visual/hoifhli_demo_path_replay_generate \
  --device 0

Outputs:

Demo visualizer video: submodules/hoifhli_release/visualizer_results/20260531_hoi_visual_hoifhli_demo_path_replay_generate/0_1/output.mp4
Navigation-only video: submodules/hoifhli_release/visualizer_results/20260531_hoi_visual_hoifhli_demo_path_replay_generate/0_1/output_0.mp4
Motion package: output/auto_experiments/20260531_hoi_visual/hoifhli_demo_path_replay_generate/hoifhli_scene_motion.npz
Metrics: output/auto_experiments/20260531_hoi_visual/hoifhli_demo_path_replay_generate/metrics.json

Result:

pre_generation_alignment_pass = true
navigation_path_xy_matches_stage1_step1 = true
interaction_path_xy_matches_source_object_com_path = true
Object start/end XY errors are near zero.
Human root pickup/place XY errors remain large:
pickup: about 0.718m
place: about 0.733m

Conclusion:

The replay confirms that the supplied path is being passed through correctly, but in interaction/carry the effective constraint is still the object COM path, not a hard human-root path constraint.
Because this run uses the official no-scene visualizer, chair/desk collision cannot occur here. Any collision in the full scene is a scene-integration issue on top of HOIFHLI's loose human-root tracking.
The crouched/hunched posture is still visible during pick/place in the demo replay, so that part is not caused only by the SAGE scene adapter.

2026-05-31 HOIFHLI pick/place + Kimodo carry hybrid v1

Goal:

Test a hybrid HOI-only sequence where HOIFHLI handles pick and place, while Kimodo generates the carry segment.
During Kimodo carry, bind the moved object to the predicted hand trajectory so the object visibly travels with the human instead of being teleported only at the final target.
Keep baseline HOIFHLI/Kimodo source untouched; add only an orchestration script around existing utilities.

Script added:

tools/hoifhli/legacy/run_pick_kimodo_carry_place.py

Command:

cd /mnt/data/PhysHSI
export HF_ENDPOINT=https://hf-mirror.com
/home/user/anaconda3/envs/isaac-sim/bin/python -m tools.hoifhli.legacy.run_pick_kimodo_carry_place

Inputs:

Adapter input: output/auto_experiments/20260531_hoi_visual/hoi_instrumenttray_smallbox/adapter/hoifhli_input.json
Source HOIFHLI motion: output/auto_experiments/20260531_hoi_visual/hoi_instrumenttray_smallbox/generate/hoifhli_scene_motion.npz

Outputs:

Hybrid sequence: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v1/hybrid_sequence.npz
Front video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v1/hybrid_pick_kimodo_carry_place_front.mp4
Right video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v1/hybrid_pick_kimodo_carry_place_right.mp4
Overview video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v1/hybrid_pick_kimodo_carry_place_overview.mp4
Metrics: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v1/hybrid_sequence_metrics.json

Result:

The pipeline completed end to end.
Stitch metrics pass after final target alignment:
max_seam_object_jump_m = 0.0
max_seam_root_jump_m = 0.0
max_seam_hand_offset_jump_m = 0.0
final_target_error_m = 0.0
all_gates_pass = true
Kimodo carry root follows the carry plan well:
root_error_mean_m ~= 0.027
root_error_max_m ~= 0.040
Carry object plan tracking is acceptable for this first pass:
object_plan_error_mean_m ~= 0.140
object_plan_error_max_m ~= 0.246

Known issues:

Hand-object attachment during Kimodo carry is still weak:
both_hands_object_anchor_error_mean_m ~= 0.413
both_hands_object_anchor_error_max_m ~= 0.608
both_hands_gate_pass = false
The front/right human-following camera views can be temporarily occluded by scene geometry; the overview video is easier for checking global trajectory.
This hybrid v1 is useful for validating the architecture, but visual quality still needs a better grasp target policy or a carry-specific hand constraint.

2026-06-01 HOIFHLI/Kimodo hybrid carry front-sign fix v2

Goal:

Fix the Kimodo carry stage where the hands appeared behind the body.
Keep baseline HOIFHLI/Kimodo code untouched; update only the added hybrid orchestration script.

Script updated:

tools/hoifhli/legacy/run_pick_kimodo_carry_place.py

Change:

Replaced the fixed symmetric_front graft call with an in-script candidate selector.
The script now tries front-sign +1, front-sign -1, and source hand-offset policies, with planned and both_hands_endpoint object-follow candidates.
Candidates are scored by:
hand-object anchor error,
object path error,
whether the hand center stays in front of the body.
The chosen carry graft for this run was:
grasp_target_policy = front_sign_pos
object_follow_mode = planned
hand_front_fraction = 1.0

Command:

cd /mnt/data/PhysHSI
export HF_ENDPOINT=https://hf-mirror.com
/home/user/anaconda3/envs/isaac-sim/bin/python -m tools.hoifhli.legacy.run_pick_kimodo_carry_place \
  --out-dir output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v2_frontfix

Outputs:

Front video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v2_frontfix/hybrid_pick_kimodo_carry_place_front.mp4
Right video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v2_frontfix/hybrid_pick_kimodo_carry_place_right.mp4
Overview video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v2_frontfix/hybrid_pick_kimodo_carry_place_overview.mp4
Hybrid sequence: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v2_frontfix/hybrid_sequence.npz
Sequence metrics: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v2_frontfix/hybrid_sequence_metrics.json
Candidate graft metrics: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v2_frontfix/kimodo_carry/graft_candidate_metrics_bound_to_hands.json

Result:

Sequence/stitch gates pass:
max_seam_object_jump_m ~= 0
max_seam_root_jump_m = 0
max_seam_hand_offset_jump_m ~= 0
final_target_error_m = 0
all_gates_pass = true
Carry graft improved compared with v1:
v1 both_hands_object_anchor_error_mean_m ~= 0.413
v2 both_hands_object_anchor_error_mean_m ~= 0.309
v1 both_hands_object_anchor_error_max_m ~= 0.608
v2 both_hands_object_anchor_error_max_m ~= 0.420
The selected v2 candidate keeps hands in front:
hand_front_fraction = 1.0

Remaining issue:

Hand-object contact is still not tight enough:
both_hands_gate_pass = false
ik_target_clamp_mean_m ~= 0.297
This suggests the next improvement should be carry-specific hand/keypoint constraints in Kimodo generation itself, not only post-generation IK grafting.

2026-06-01 Hybrid carry transition stabilization v7

Goal:

Fix the pickup-to-Kimodo transition without hard-coding a scene-specific frame.
Avoid the earlier visual failure where the body briefly twisted or went horizontal around the pickup/carry boundary.

Script updated:

tools/hoifhli/legacy/run_pick_kimodo_carry_place.py

Generic changes:

Auto pickup anchor selection now scores candidate frames by:
object lift,
stable/standing body posture,
hand-object proximity,
not being too far into the carry trajectory.
Added a repeated stable hold transition after pickup:
pickup_settle_transition.npz
default pickup_settle_frames = 12
Added Kimodo --start-fullbody at carry frame 0 so the generated carry starts from the stable pickup pose rather than an arbitrary walking pose.
Added automatic Kimodo carry pre-roll trimming:
scan the generated carry start for a consecutive stable window,
skip frames where head/root height indicates collapsed or horizontal body,
also require reasonable hand-object distance.
Kept mesh morph bridging disabled to avoid hiding bad pose changes with vertex interpolation.

Command:

cd /mnt/data/PhysHSI
export HF_ENDPOINT=https://hf-mirror.com
/home/user/anaconda3/envs/isaac-sim/bin/python -m tools.hoifhli.legacy.run_pick_kimodo_carry_place \
  --out-dir output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v7_auto_settle_trim

Outputs:

Front video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v7_auto_settle_trim/hybrid_pick_kimodo_carry_place_front.mp4
Right video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v7_auto_settle_trim/hybrid_pick_kimodo_carry_place_right.mp4
Overview video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v7_auto_settle_trim/hybrid_pick_kimodo_carry_place_overview.mp4
Hybrid sequence: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v7_auto_settle_trim/hybrid_sequence.npz
Metrics: output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v7_auto_settle_trim/hybrid_sequence_metrics.json

Result:

Auto-selected pickup_anchor_frame = 97.
Inserted pickup_settle_frames = 12.
Auto-trimmed Kimodo pre-roll:
carry_source_start_frame = 18
Kimodo carry grasp quality improved substantially:
both_hands_object_anchor_error_mean_m ~= 0.091
both_hands_gate_pass = true
object_plan_gate_pass = true
all_gates_pass = true for graft metrics.
Stitched sequence gates pass:
final_target_error_m = 0
seam_object_gate_pass = true
seam_root_gate_pass = true
seam_hand_offset_gate_pass = true
all_gates_pass = true
Manual frame check around the pickup/carry boundary no longer shows the earlier horizontal-body twist.

Remaining issue:

Vertex jump metrics at model-family boundaries remain high because mesh morph bridging is intentionally disabled:
max_seam_vertex_mean_jump_m ~= 0.599
max_seam_vertex_max_jump_m ~= 1.773
This is preferable to the previous morph artifact, but a cleaner future solution is to generate a true Kimodo transition motion conditioned on both the pickup hold pose and the carry pose.

2026-06-01 HOI keyframed Kimodo carry/place alignment pass

Goal:

Test the revised HOI-only plan: use HOIFHLI as the source of pick/place keyframes and object trajectory, then use Kimodo for the carry/place motion with keyframe constraints.
Keep baseline HOIFHLI/Kimodo code untouched; add only orchestration logic.
Explicitly verify coordinate axes and representation compatibility.

Script updated:

tools/hoifhli/legacy/run_pick_keyframed_kimodo_place.py

Generic changes:

Added a separate keyframed runner that:
builds the carry Root2D/object plan through the existing hybrid_carry path,
patches Kimodo constraints with HOIFHLI pickup/place fullbody keyframes,
aligns planned root/object paths to the chosen HOIFHLI boundary frames,
renders source HOIFHLI frames from their cached vertices and Kimodo frames from Kimodo SMPL-X skinning, instead of mixing HOIFHLI rotations into Kimodo skinning.
Added GRADIO_TEMP_DIR under the project cache so the Kimodo text encoder client no longer writes to the server-global /tmp/gradio directory owned by another user.
Exposed --anchor-facing-policy; default is now source for this runner so the HOIFHLI heading is not silently flipped.
Added --kimodo-trim-start-frames to support a generic handoff where unstable Kimodo pre-roll frames are skipped and replaced with the matching HOIFHLI source frames.

Important representation finding:

HOIFHLI root_positions and posed_joints[:, 0] are not the same point: the observed offset is about [0.001, -0.014, 0.380] meters.
HOIFHLI cached vertices/faces_seq render correctly, but directly skinning HOIFHLI posed_joints/global_rot_mats with Kimodo SMPL-X can produce inverted/collapsed bodies.
Therefore, source HOIFHLI visual frames must use cached vertices. Kimodo frames must use Kimodo-generated rotations/skin. Copying HOIFHLI global_rot_mats into Kimodo frames is invalid.

Commands tried:

cd /mnt/data/PhysHSI
export HF_ENDPOINT=https://hf-mirror.com
/home/user/anaconda3/envs/isaac-sim/bin/python -m tools.hoifhli.legacy.run_pick_keyframed_kimodo_place \
  --out-dir output/auto_experiments/20260531_hoi_visual/hybrid_pick_keyframed_kimodo_place_v14_anchor97_text_trim13_stride16_cw4 \
  --pickup-anchor-frame 97 \
  --carry-variant hand \
  --anchor-facing-policy source \
  --hand-stride 16 \
  --cfg-text-weight 1.2 \
  --cfg-constraint-weight 4.0 \
  --graft-object-follow-mode planned \
  --boundary-hold-frames 4 \
  --boundary-blend-frames 18 \
  --kimodo-trim-start-frames 13 \
  --render-frame-stride 2 \
  --render-width 960 \
  --render-height 544

Current best output:

Front video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_keyframed_kimodo_place_v14_anchor97_text_trim13_stride16_cw4/hybrid_keyframed_front.mp4
Right video: output/auto_experiments/20260531_hoi_visual/hybrid_pick_keyframed_kimodo_place_v14_anchor97_text_trim13_stride16_cw4/hybrid_keyframed_right.mp4
Sequence: output/auto_experiments/20260531_hoi_visual/hybrid_pick_keyframed_kimodo_place_v14_anchor97_text_trim13_stride16_cw4/hybrid_keyframed_sequence.npz
Metrics: output/auto_experiments/20260531_hoi_visual/hybrid_pick_keyframed_kimodo_place_v14_anchor97_text_trim13_stride16_cw4/hybrid_keyframed_metrics.json

Current result:

Text encoder path now works after setting project-local GRADIO_TEMP_DIR.
pickup_anchor_frame = 97, place_keyframe = 355.
kimodo_trim_start_frames = 13, so HOIFHLI frames through 110 are used for the handoff before switching to Kimodo.
Graft/object metrics pass:
root_gate_pass = true
both_hands_gate_pass = true
object_plan_gate_pass = true
all_gates_pass = true
The start horizontal-body artifact is removed compared with v11/v12.

Remaining issue:

Handoff is still not strictly aligned:
sequence_handoff_to_kimodo_root_jump_m ~= 0.127
sequence_handoff_to_kimodo_joint_mean_jump_m ~= 0.588
Visual quality is better than v11/v12 but still not fully human-like; the carry phase can show a crouched/overreaching posture near the target.
Root/object/hand coordinate alignment is now understood; the remaining hard problem is making Kimodo respect a HOIFHLI-derived upright fullbody boundary without copying incompatible HOIFHLI rotations.

2026-06-01 HOIFHLI frame to Kimodo fullbody keyframe converter

Goal:

Add a small, non-baseline converter for turning one HOIFHLI frame into a Kimodo-native SMPLX22 fullbody keyframe constraint.
Avoid the previous failure mode where HOIFHLI root_positions and global_rot_mats were copied into Kimodo constraints even though their semantics differ from Kimodo's root/joint0 and native local rotations.

Script added:

tools/hoifhli/legacy/hoifhli_to_kimodo_keyframe.py

Behavior:

Uses hoifhli posed_joints[frame, 0] as the Kimodo root and smooth_root_2d, not HOIFHLI root_positions.
Fits one Kimodo-native SMPLX22 local-rotation pose to the HOIFHLI joint positions with a short single-frame IK optimization.
Writes Kimodo's native JSON shape:

type: fullbody
frame_indices: [...]
local_joints_rot: axis-angle SMPLX22 local rotations
root_positions: Kimodo pelvis/root position from HOIFHLI joint0
smooth_root_2d: root x/z from HOIFHLI joint0

Does not copy HOIFHLI global_rot_mats into the constraint.
Emits metrics for root offset, joint fitting error, hand/head errors, and a load-validation check through Kimodo's load_constraints_lst.

2026-06-01 planned experiment: HOIFHLI pick-turnstand to Kimodo carry

Added orchestration entry point tools/hoifhli/legacy/run_pick_turnstand_kimodo_carry.py.
The experiment regenerates a short HOIFHLI segment from the HOI-only instrument-tray adapter by replacing the interaction path with a local pickup/turnstand endpoint. The original full task endpoint is still used as the Kimodo carry destination.
The handoff frame is selected from the regenerated HOIFHLI motion using lift, hand-object proximity, local stability, heading alignment to the original path, and upright body height.
That handoff frame is converted through hoifhli_to_kimodo_keyframe.py into a Kimodo-native fullbody frame-0 constraint. The Kimodo carry plan is patched so root trajectories use HOIFHLI joint0/pelvis rather than HOIFHLI root_positions.
This wrapper is an experiment layer only; it does not modify baseline HOIFHLI, Kimodo, or task-generator code.

2026-06-02 controlled transition bridge ablation

Goal:

Reset the HOIFHLI-to-Kimodo bridge to a controlled setting and add only one variable per run.
Keep Kimodo and HOIFHLI baseline code unchanged.
Use task-generator carry.path_points_xy as the path source. Do not infer Kimodo waypoints from the HOIFHLI object trajectory.

Scripts added or updated:

utils/transition_bridge/build_kimodo_bridge_example.py
utils/transition_bridge/run_stable_kimodo_bridge.py
utils/transition_bridge/run_bridge_ablation.py
utils/transition_bridge/motion_metrics.py

Main finding:

HOIFHLI root_positions and HOIFHLI posed_joints[:, 0] differ by about 0.38 m in this source motion. Kimodo exports use root_positions == posed_joints[:, 0].
Therefore fullbody handoff constraints must not mix HOIFHLI root_positions with HOIFHLI joint0. Earlier "root fixed but body warped" behavior came from this adapter-level semantic mismatch.
Direct local/world fullbody constraints from HOIFHLI keep the handoff frame numerically close, but they inherit HOIFHLI's crouched or unstable pose and make Kimodo walk in a bent posture.
Hands-only constraints were worse in this scene: the first frame collapsed toward the ground and right-hand target error was about 0.79 m.

Controlled ablations:

v02_start_local_clipped had the best old numeric score but bad visual posture because it hard-locked the HOIFHLI body pose.
Removing holding a box from the prompt did not fix the start collapse, so prompt text was not the primary cause.
Removing the off-path handoff root helped only partially, so the connector point alone was not the primary cause.
Sparse root2d controls every 15 frames improved the start posture compared with per-frame root2d controls.
Reducing Kimodo constraint guidance from 3.0 to 1.0 improved smoothness.
Adding a 30-frame Kimodo pre-roll and trimming it after generation produced the first visually stable result while preserving the task-generator path.

Current stable default:

python -m utils.transition_bridge.run_stable_kimodo_bridge \
  --task-plan output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json \
  --hoifhli-motion output/auto_experiments/20260602_guided_navigation_before_pick/D_guided_navigation_aligned_same_prompt/generate/hoifhli_scene_motion.npz \
  --out-dir output/auto_experiments/20260602_transition_bridge_debug/stable_runner_v1_D_frame205 \
  --handoff-frame 205 \
  --render

Non-standard adapter settings used by the stable route:

fullbody_mode = none
prepend_handoff_root = false
path_clip_mode = nearest_handoff_root
preroll_frames = 30
trim_frames = 30
root2d_stride = 15
cfg_text_weight = 1.0
cfg_constraint_weight = 1.0
prompt: A person walks forward holding a box

Verified output:

Motion: output/auto_experiments/20260602_transition_bridge_debug/stable_runner_v1_D_frame205/kimodo_motion.npz
Native Kimodo render: output/auto_experiments/20260602_transition_bridge_debug/stable_runner_v1_D_frame205/kimodo_motion_root.mp4
Summary: output/auto_experiments/20260602_transition_bridge_debug/stable_runner_v1_D_frame205/summary.json

Metrics after using Kimodo foot_contacts for foot-skate evaluation:

quality_gate_pass = true
quality_gate_pass_count = 5 / 5
root_step_p95_m = 0.0308
joint_mean_step_p95_m = 0.0302
foot_horizontal_sep_min_m = 0.2674
foot_sliding_p95_mps = 0.3071
carry_root_path_error_m_mean = 0.0342

Remaining K-to-H note:

The current HOIFHLI adapter exposes scenario-level navigation and interaction paths, object name, text, and object poses. It does not expose a supported external SMPL-X initial pose or prefix-motion condition.
For Kimodo-to-HOIFHLI transitions, the controlled route is therefore: generate HOIFHLI with its native inputs, choose a stable start frame, then apply adapter-level stitching, root/object translation ramps, and optional rotation/mesh blending. Treat this as post-generation alignment, not as HOIFHLI being conditioned on Kimodo pose.

2026-06-02 selected Kimodo bridge runner

Goal:

Make the HOIFHLI-to-Kimodo carry bridge less dependent on one lucky Kimodo sampling seed.
Keep Kimodo and HOIFHLI baseline code unchanged.
Preserve the task-generator carry.path_points_xy as the path source and continue to avoid HOIFHLI object-derived carry waypoints.

Script added:

utils/transition_bridge/run_selected_kimodo_bridge.py

Behavior:

Runs a small adapter-level candidate sweep.
Default profiles:
fast20: avg_speed=0.75, diffusion_steps=20
slow20: avg_speed=0.65, diffusion_steps=20
slow50: avg_speed=0.65, diffusion_steps=50
Default seeds: 41 42 43.
Each candidate still uses the stable bridge settings: fullbody_mode=none, path_clip_mode=nearest_handoff_root, prepend_handoff_root=false, root2d_stride=15, preroll_frames=30, cfg_text_weight=1.0, cfg_constraint_weight=1.0.
The selector prefers candidates that pass every quality gate. If no candidate passes, it falls back to fewer gate failures, lower foot/path/root violations, then lower transition score.
The selected candidate is copied into best/, and only the best candidate is rendered when --render-best is provided.

Verified command:

cd /mnt/data/PhysHSI
/home/user/anaconda3/envs/isaac-sim/bin/python -m utils.transition_bridge.run_selected_kimodo_bridge \
  --task-plan output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json \
  --hoifhli-motion output/auto_experiments/20260602_guided_navigation_before_pick/D_guided_navigation_aligned_same_prompt/generate/hoifhli_scene_motion.npz \
  --out-dir output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D \
  --handoff-frame 205 \
  --render-best

Selected result:

Selected candidate: slow50, seed 43.
Best motion: output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/best/kimodo_motion.npz
Native Kimodo render: output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/best/kimodo_motion_root.mp4
Contact sheet: output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/visual_check/best_contact_sheet.jpg
Selection summary: output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/selection_summary.json

Selected metrics:

quality_gate_pass = true
quality_gate_pass_count = 5 / 5
score = 1.33858
root_step_p95_m = 0.02629
joint_mean_step_p95_m = 0.02810
foot_sep_min_m = 0.31777
foot_sliding_p95_mps = 0.28875
carry_root_path_error_m_mean = 0.03516

Candidate sweep result:

slow50 seed=43 pass=true  gates=5/5 score=1.338580 foot=0.288750 path=0.035159
fast20 seed=41 pass=true  gates=5/5 score=1.414916 foot=0.307095 path=0.034206
slow20 seed=42 pass=true  gates=5/5 score=1.565130 foot=0.347133 path=0.033899
slow50 seed=42 pass=true  gates=5/5 score=1.591366 foot=0.334216 path=0.034757
slow50 seed=41 pass=false gates=4/5 score=1.634989 foot=0.467840 path=0.035106
fast20 seed=42 pass=false gates=4/5 score=1.638835 foot=0.524055 path=0.032974
slow20 seed=41 pass=false gates=4/5 score=1.613699 foot=0.532359 path=0.034576
fast20 seed=43 pass=false gates=4/5 score=1.584927 foot=0.573204 path=0.038587
slow20 seed=43 pass=false gates=4/5 score=1.603742 foot=0.636266 path=0.033316

Conclusion:

A single fixed Kimodo seed/profile is not robust: different seeds fail the foot-sliding gate under different speed/step settings.
The bridge conversion itself is stable enough under the root2d-only, task-path, pre-roll/trim setting. The remaining variance is mostly Kimodo sampling quality, especially foot contact and sliding.
The currently preferred H-to-K production path is therefore the selected runner, not the single-candidate runner.
Native-render contact sheet for the selected result shows an upright walking motion without the earlier start collapse, crossed-leg start, or prone body failure.

Remaining limitation:

This selected runner currently validates only the H-to-K carry stage on the instrument-tray HOI scene. The code is scene-agnostic as long as the task plan has a usable carry.path_points_xy and the HOIFHLI motion has posed_joints.
K-to-H is still not solved as true conditional HOIFHLI generation because the available HOIFHLI adapter does not expose a supported prefix-motion or external SMPL-X initial-pose input.

2026-06-02 Kimodo-to-HOIFHLI follow-up wrapper

Goal:

Add a controlled adapter-level route for the opposite transition direction: append a HOIFHLI follow-up segment after a Kimodo sequence.
Keep the result honest: this is post-generation seam alignment, not HOIFHLI conditioned on the Kimodo final body state.

Script added:

utils/transition_bridge/run_kimodo_to_hoifhli_followup.py

Behavior:

Input: a Kimodo motion and a HOIFHLI follow-up motion.
If the Kimodo motion has no object track, the wrapper creates a prepared base motion with a static object copied from the first HOIFHLI follow-up frame after root alignment. This is only a seam aid.
Calls existing tools.hoifhli.legacy.hybrid_sequence append-followup.
Defaults to position/root blend plus rotation slerp blend over the first follow-up frames.
Computes both:
body_transition_gate_pass: root/joint/rotation seam continuity.
full_motion_quality_gate_pass: global motion quality including foot sliding and hand-object distance.

Verified command:

cd /mnt/data/PhysHSI
/home/user/anaconda3/envs/isaac-sim/bin/python -m utils.transition_bridge.run_kimodo_to_hoifhli_followup \
  --kimodo-motion output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/best/kimodo_motion.npz \
  --hoifhli-followup output/auto_experiments/20260531_hoi_visual/hybrid_pick_hoifhli_kimodo_carry_place_v9_vertex_bridge/hoifhli_place_clip.npz \
  --out-dir output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/k_to_h_followup_runner_v1 \
  --task-plan output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json

Output:

Sequence: output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/k_to_h_followup_runner_v1/kimodo_to_hoifhli_followup.npz
Summary: output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/k_to_h_followup_runner_v1/summary.json
Motion metrics: output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/k_to_h_followup_runner_v1/motion_metrics.json

Result:

body_transition_gate_pass = true
append_all_gates_pass = true
root_jump_m = 0.0
joint_mean_jump_m = 0.0
rot_geodesic_mean_rad = 0.000111
rot_geodesic_max_rad = 0.000488
full_motion_quality_gate_pass = false
foot_sliding_p95_mps = 2.73471
hand_center_object_mean_m = 1.61768

Controlled comparison:

With --no-rotation-blend, root/joint/object seam still looked perfect numerically, but rotation seam was bad: rot_geodesic_mean_rad = 2.81635, rot_geodesic_max_rad = 3.01947.
Therefore K-to-H post-stitch should use rotation blend by default.

Conclusion:

K-to-H body seam continuity can be made numerically smooth by post-generation alignment and rotation blending.
This does not solve full HOI semantics. The tested follow-up uses a historical HOIFHLI place clip and a static-object enrichment because the selected Kimodo carry output is body-only.
A true K-to-H solution still needs either:
a HOIFHLI initial-state/prefix-motion conditioning entry point, or
a generated HOIFHLI follow-up clip whose object/hand state is consistent with the selected Kimodo terminal object state.

2026-06-02 full HOIFHLI navigation+pick to selected Kimodo carry

Goal:

Produce the requested visual sequence: HOIFHLI navigation + pick / lift / stand through the stable handoff frame, then selected Kimodo carry.
Reuse the current selected H-to-K carry result. Do not regenerate Kimodo and do not edit HOIFHLI/Kimodo baseline code.

Script added:

utils/transition_bridge/run_full_h_to_k_sequence.py

Verified command:

cd /mnt/data/PhysHSI
/home/user/anaconda3/envs/isaac-sim/bin/python -m utils.transition_bridge.run_full_h_to_k_sequence \
  --hoifhli-motion output/auto_experiments/20260602_guided_navigation_before_pick/D_guided_navigation_aligned_same_prompt/generate/hoifhli_scene_motion.npz \
  --kimodo-motion output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/best/kimodo_motion.npz \
  --out-dir output/auto_experiments/20260602_transition_bridge_debug/h_nav_pick_to_selected_k_v2_scripted \
  --task-plan output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json \
  --handoff-frame 205 \
  --render

Important implementation detail:

The selected Kimodo carry is body-only. For visualization, the script creates an adapter-only object track:
copy the HOIFHLI dynamic object mesh and rotation at the handoff frame;
place the first Kimodo object frame exactly at the HOIFHLI handoff object;
move the object with the Kimodo hand center plus the HOIFHLI hand-object offset.
This lets the video show a carried object, but the object motion is not Kimodo-generated object motion.

Renderer:

Correct renderer: tools.render_motion_scene
This renderer reads baked vertices/faces_seq and draws the dynamic object.
Do not use render_kimodo_motion_offline_fast for this full sequence. That renderer re-skins the HOIFHLI frames from rotations and can show an inverted body in the HOIFHLI part.

Outputs:

Sequence: output/auto_experiments/20260602_transition_bridge_debug/h_nav_pick_to_selected_k_v2_scripted/h_nav_pick_to_k_sequence.npz
Root-follow video: output/auto_experiments/20260602_transition_bridge_debug/h_nav_pick_to_selected_k_v2_scripted/sequence_viewer_root.mp4
Whole-motion video: output/auto_experiments/20260602_transition_bridge_debug/h_nav_pick_to_selected_k_v2_scripted/sequence_viewer_motion.mp4
Root-follow contact sheet: output/auto_experiments/20260602_transition_bridge_debug/h_nav_pick_to_selected_k_v2_scripted/visual_check/sequence_viewer_root_sheet.jpg
Whole-motion contact sheet: output/auto_experiments/20260602_transition_bridge_debug/h_nav_pick_to_selected_k_v2_scripted/visual_check/sequence_viewer_motion_sheet.jpg
Summary: output/auto_experiments/20260602_transition_bridge_debug/h_nav_pick_to_selected_k_v2_scripted/summary.json

Metrics:

stitch_all_gates_pass = true
stitch_max_seam_root_jump_m = 0.0
stitch_max_seam_object_jump_m = 0.0
stitch_max_seam_hand_offset_jump_m = 0.0
seam rot_geodesic_mean_rad = 0.000506
seam rot_geodesic_max_rad = 0.000846
root_step_p95_m = 0.02963
joint_mean_step_p95_m = 0.03633
hand_center_object_mean_m = 0.20257
both_hands_object_mean_m = 0.39944
motion_quality_gate_pass = false
foot_sliding_p95_mps = 2.46277

Visual check:

The correct viewer-rendered contact sheets show HOIFHLI standing, bending, picking up, standing with the object, and transitioning into Kimodo carry.
No fast-render body inversion is present in the viewer-rendered videos.

Conclusion:

This output is suitable for visually inspecting the requested H-to-K transition with the HOIFHLI front segment added.
The seam itself is numerically smooth after blending and mesh bridging.
The full-sequence physical-quality metrics are not clean. In particular, foot sliding remains high over the full sequence, and the carry object track is adapter-synthesized for visualization.

2026-06-02 follow-up failure analysis for full H-to-K render

User visual review found two visible failures in h_nav_pick_to_selected_k_v2_scripted/sequence_viewer_root.mp4:

Around 3 seconds, the HOIFHLI pickup pose looks like the head and torso are facing inconsistently.
Around the H-to-K handoff, the body temporarily collapses into a spiky blob and then appears unstable.

Controlled checks:

The stitched sequence frames 0..205 are exactly equal to the source HOIFHLI motion for vertices, faces_seq, root_positions, and posed_joints: max_abs_diff_front_0_205 = 0.0.
The HOIFHLI native video output/auto_experiments/20260602_guided_navigation_before_pick/D_guided_navigation_aligned_same_prompt/generate/hoifhli_native_output_0.mp4 already shows the same bent pickup posture around 3 seconds. This is a HOIFHLI front-stage generation-quality issue, not a H-to-K stitch artifact.
The spike is localized to frames 206..208, immediately after the handoff. In the bad render, mesh edge lengths jump from normal SMPL-scale values to invalid long triangles:

bad sequence edge_max:
frame 204: 0.0610
frame 205: 0.0607
frame 206: 1.6942
frame 207: 1.2547
frame 208: 0.4430
frame 209: 0.0600

Root cause:

tools.hoifhli.legacy.hybrid_sequence._apply_start_mesh_bridge linearly interpolates vertices across the H-to-K seam but does not synchronize faces_seq.
HOIFHLI frames use pre-baked vertices/faces_seq; Kimodo frames are skinned with the Kimodo SMPL-X topology. Even when both have 10475 vertices and 20908 faces, the vertex ordering/topology cannot be assumed identical.
Therefore a bridge frame can contain HOIFHLI-like vertices but Kimodo faces, causing long invalid triangles and the visible spiky blob.
Separately, hybrid_sequence._blend_start_from_reference was slerping global_rot_mats from HOIFHLI to Kimodo over the first 12 carry frames. This is unsafe because the two motions expose compatible-shaped arrays but not necessarily compatible global rotation frames. The selected Kimodo carry starts upright by itself, but the stitched render became unstable after this cross-system rotation blend.

Ablation:

cd /mnt/data/PhysHSI
/home/user/anaconda3/envs/isaac-sim/bin/python -m utils.transition_bridge.run_full_h_to_k_sequence \
  --hoifhli-motion output/auto_experiments/20260602_guided_navigation_before_pick/D_guided_navigation_aligned_same_prompt/generate/hoifhli_scene_motion.npz \
  --kimodo-motion output/auto_experiments/20260602_transition_bridge_debug/selected_runner_v1_D/best/kimodo_motion.npz \
  --task-plan output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json \
  --out-dir output/auto_experiments/20260602_transition_bridge_debug/h_nav_pick_to_selected_k_v2_debug_no_rot_no_mesh_bridge \
  --handoff-frame 205 \
  --pickup-carry-blend-frames 12 \
  --release-frames 1 \
  --bake-vertices \
  --pickup-carry-mesh-bridge-frames 0 \
  --no-pickup-carry-rotation-blend \
  --render \
  --render-camera-modes root \
  --render-frame-stride 1 \
  --render-width 960 \
  --render-height 544

The ablation removes the invalid long triangles:

ablation edge_max:
frame 204: 0.0610
frame 205: 0.0607
frame 206: 0.0615
frame 207: 0.0622
frame 208: 0.0628
frame 209: 0.0630

Updated conclusion:

The earlier v2 scripted result should not be treated as a stable high-quality output.
Do not use vertex interpolation as a H-to-K mesh transition unless the vertex correspondence and faces are proven identical frame-by-frame.
Do not slerp HOIFHLI and Kimodo global_rot_mats directly as a default bridge operation. A safer adapter should preserve each generator's native rotations and only align root/object/joint positions in a shared world frame, or use a true retargeted SMPL-X parameter representation before blending rotations.

2026-06-04: Kimodo-only HOI pipeline orchestrator

Goal:

Consolidate the current Kimodo-only HOI flow into a single adapter-level script before integrating it with HSI. The script must keep Kimodo/HOIFHLI baseline code unchanged and should make pick/place keyframes, object binding, hand baking, metrics, rendering, and provenance explicit.

New script:

/mnt/data/PhysHSI/tools/kimodo/run_kimodo_hoi_pipeline.py

Pipeline stages:

Resolve task_plan.json and layout_json, then compute generic task geometry: pick/carry/place step IDs, manipulated object, support object, scene_origin_xy, initial object center, target object center, pick root, and carry endpoint in Kimodo coordinates.
Call build_smplx_multiprompt_pickfree_carrypath_place.py to write meta.json, info.json, and baseline constraints.json.
Snapshot baseline constraints as constraints_base.json.
Add the table-facing place reach keyframe with build_kimodo_place_reach_keyframe_constraint.py.
Add the pickup reach keyframe with build_kimodo_pick_reach_keyframe_constraint.py.
Generate body motion through tools.kimodo.generate_with_first_heading.
Attach the dynamic object with attach_task_object_to_kimodo_motion.py.
Bake hand/SMPL-X vertices with bake_kimodo_vertices_with_hands.py. Supported hand modes are prior, npz, schedule, and optimize.
Run motion/object and hand-object contact metrics.
Render with render_motion_viewer_offline.py, explicitly passing the resolved --scene-origin-xy.

Important implementation choices:

The orchestrator computes the pick target object center directly from task_plan + layout mesh, rather than depending on a previous motion_augmented.npz from an older experiment.
Vector command-line values that may start with negative numbers are passed as --arg=value to avoid argparse interpreting them as new options.
The script writes pipeline_manifest.json incrementally, including every delegated command, resolved geometry, output path, return code, and elapsed time.
The default turn duration is 1.6666666666666667s to match the current E01 590-frame setting. --turn-s auto remains available for generic automatic heading-based timing.
The default place reach keyframe fraction is 0.76, which resolves to frame 560 for the current 120-frame place segment.

Verification:

Constraints-only smoke:

source /home/user/anaconda3/etc/profile.d/conda.sh
conda activate isaac-sim
cd /mnt/data/PhysHSI
python tools/kimodo/run_kimodo_hoi_pipeline.py \
  --out-dir output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E00_constraints_smoke \
  --task-plan output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json \
  --layout-json data/selected_SAGE10k/e2e_test_scenes/889a0ed0/layout_889a0ed0.json \
  --source-npz output/kimodo_smplx_segment_stitch_20260602/pick_only_firstframe_free/motion.npz \
  --place-end-keyframe-source output/kimodo_smplx_segment_stitch_20260602/pick_only_firstframe_free/motion.npz \
  --stop-after constraints

Smoke output:

constraints: 7
frames: pick 150, turn 50, carry 270, place 120, total 590
fullbody constraints: frame 0, frame 589, frame 560 place reach, frame 67 pick reach
stages: build_inputs, snapshot_constraints_base, place_reach_keyframe, pick_reach_keyframe

Full run using the existing optimized hand schedule:

LOCAL_CACHE=True HF_HOME=/mnt/data/PhysHSI/.cache/huggingface \
python tools/kimodo/run_kimodo_hoi_pipeline.py \
  --out-dir output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E01_full_schedule \
  --task-plan output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json \
  --layout-json data/selected_SAGE10k/e2e_test_scenes/889a0ed0/layout_889a0ed0.json \
  --source-npz output/kimodo_smplx_segment_stitch_20260602/pick_only_firstframe_free/motion.npz \
  --place-end-keyframe-source output/kimodo_smplx_segment_stitch_20260602/pick_only_firstframe_free/motion.npz \
  --hand-mode schedule \
  --hand-pose-schedule-json output/auto_experiments/20260604_kimodo_hand_refine/E12_e10_e11_phase_schedule/hand_pose_schedule.json

Full-run outputs:

output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E01_full_schedule/motion.npz
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E01_full_schedule/motion_augmented.npz
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E01_full_schedule/motion_augmented_hands.npz
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E01_full_schedule/motion_scene_dynamic_object_hands.mp4
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E01_full_schedule/pipeline_manifest.json

Full-run stage status:

build_inputs: 0
place_reach_keyframe: 0
pick_reach_keyframe: 0
generate_motion: 0
attach_object: 0
bake_hands: 0
motion_object_metrics: 0
hand_object_contact_metrics: 0
render: 0

Notes:

The first full-generation attempt failed because the Kimodo loader tried to access the gated HuggingFace repo without authentication. Setting LOCAL_CACHE=True and HF_HOME=/mnt/data/PhysHSI/.cache/huggingface made the loader use the existing local snapshot.
The full-run metric gate reported quality_gate_pass = false with 9/10 gates passing. This means the script is operational, but the generated motion is not being marked as globally "solved" by the current metric suite.

2026-06-04 Unified Kimodo HOI autogating and pick hand optimization

Goal: keep pick in the unified Kimodo-only HOI pipeline, but make it quality controlled in the same spirit as the place reach keyframe. Baseline Kimodo and HOIFHLI source files were not modified; all changes are adapter-side.

Adapter changes:

tools/kimodo/build_kimodo_pick_reach_keyframe_constraint.py now supports a reference-motion pool and hard quality gates for automatic pick keyframe selection. It rejects candidates with bad root placement, foot penetration, implausible head-root height, hand separation, wrist height, or missing post-pick recovery. If no candidate passes, it fails instead of silently inserting a bad full-body keyframe.
tools/kimodo/build_kimodo_place_reach_keyframe_constraint.py now uses the same reference-pool and gate pattern for place reach keyframes. This prevents the old place_only_table frame 31 failure mode where the bound pose had feet below the ground.
tools/kimodo/run_kimodo_hoi_pipeline.py now passes default pick/place reference pools, records the selected candidates in metadata, and can automatically optimize multiple hand poses.
The automatic hand-pose frame selector now includes a pick_contact frame near the attach frame. For this frame, selection uses a balanced wrist-to- object-mesh-surface score instead of only wrist-center-to-object-center distance, because fixed-wrist grasp optimization cannot repair a frame where one wrist is far from the object surface.
Contact metrics now include pick_after_attach and pick_contact_window subsegments. The full pick segment includes approach and crouch frames before grasping, so whole-segment contact ratio is not a valid pick-grasp quality metric by itself.

Controlled experiments:

E02_autogated_constraints:
  stop-after constraints.
  pick selected v10 frame 47, expected root XZ error 0.043m,
  head-root y 0.421m, bound min foot y 0.013m.
  place selected E03 canonical low-table frame 41, expected root XZ error
  0.094m, head-root y 0.447m, bound min foot y 0.005m.

E03_autogated_full_prior:
  full body/object/render with hand prior.
  video:
  output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E03_autogated_full_prior/motion_scene_dynamic_object_hands.mp4

E04_autogated_handopt_smoke:
  60-iter hand optimization, old auto frames only:
  [267, 450, 475].

E05_autogated_handopt_pickframe_smoke:
  60-iter hand optimization with added pick_contact, selected frame 112.

E06_autogated_handopt_pick_surface_smoke:
  60-iter hand optimization with pick_contact selected by balanced
  wrist-to-surface distance, selected frame 108.

E07_autogated_handopt_pick_surface_full:
  600-iter hand optimization and full render.
  selected hand frames: [108, 267, 450, 475].
  video:
  output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E07_autogated_handopt_pick_surface_full/motion_scene_dynamic_object_hands.mp4

E07 numeric checks:

carry root path error mean/max: 0.0248m / 0.0509m
carry wrist-center-object distance mean/max: 0.0685m / 0.0736m
place wrist-center-object distance mean/max: 0.1689m / 0.4192m
final object center error: 0.0000m
foot_y_p01: 0.0033m
head_root_y_p05: 0.3423m

hand contact, closest50 mean / contact ratio <5cm:
pick full segment:        0.1996m / 0.2534
pick_after_attach:        0.0073m / 0.4226
pick_contact_window:      0.0095m / 0.3829
carry:                    0.0238m / 0.2148
place:                    0.0300m / 0.2125

Interpretation:

The keyframe autogates fixed the earlier unified-pipeline failure where bad reference frames made the person crouch through the floor or place the object on the ground.
Pick must be evaluated after attach. The whole pick segment is expected to have low contact before the person reaches the object; pick_after_attach is the meaningful grasp-holding metric.
The current remaining weakness is asymmetric hand quality, especially the right hand in carry/place. The optimizer is run with fixed wrists; when the generated body motion places a wrist too far from the object surface, the grasp energy can curl fingers but cannot move the wrist to a physically good contact pose.
quality_gate_pass is still false under the old global metric because head_root_y_p05 > 0.45m is too strict for a sequence with a deliberate pick crouch. This gate should be segment-aware rather than global.

2026-06-04 E08 unfixed-wrist HOIFHLI grasp ablation

Question: should the HOIFHLI grasp optimizer be allowed to change wrist pose instead of freezing the Kimodo wrist?

Controlled setting:

Body/object motion was reused from E07_autogated_handopt_pick_surface_full/motion_augmented.npz.
Hand optimization frames stayed [108, 267, 450, 475].
HOIFHLI batch_size=16, n_iter=600, w_prior=0.5, same seeds as E07.
The only intended optimization change was omitting --fixed-wrist when calling optimize_hoifhli_grasp_for_kimodo_object.py.
The bake step still applies only the optimized finger axis-angles to the original Kimodo wrists, because the current bake adapter does not rewrite the body skeleton wrist joints.

Outputs:

output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E08_unfixed_wrist_ablation/motion_scene_dynamic_object_hands.mp4
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E08_unfixed_wrist_ablation/wrist_deviation_summary.json
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E08_unfixed_wrist_ablation/hand_object_contact_metrics.json

Result compared with E07 fixed-wrist:

pick_after_attach contact, closest50 / ratio<5cm:
  E07 fixed wrist:    0.0073m / 0.4226
  E08 unfixed wrist:  0.0057m / 0.3629

carry contact, closest50 / ratio<5cm:
  E07 fixed wrist:    0.0238m / 0.2148
  E08 unfixed wrist:  0.0282m / 0.1906

place contact, closest50 / ratio<5cm:
  E07 fixed wrist:    0.0300m / 0.2125
  E08 unfixed wrist:  0.0412m / 0.1507

Returned HOIFHLI wrist deviation in E08:

frame 108: left 0.039m / 14.1deg, right 0.084m / 26.8deg
frame 267: left 0.093m / 6.4deg,  right 0.137m / 19.6deg
frame 450: left 0.062m / 4.9deg,  right 0.174m / 9.7deg
frame 475: left 0.080m / 6.0deg,  right 0.159m / 8.8deg

Interpretation:

Unfixed HOIFHLI wrist substantially reduced the internal grasp energy, but it did so by moving the wrist away from the Kimodo body motion, especially the right wrist.
Because the current hand bake keeps Kimodo body wrists fixed, applying only the unfixed-wrist fingers back to Kimodo did not improve the rendered contact metrics; it made carry/place worse.
Therefore fixed_wrist=True remains the correct default for the current adapter. To benefit from unfixed-wrist optimization, a later stage would need to propagate the returned wrist correction into arm IK/body correction before baking, rather than discarding it.

2026-06-04 E09/E10 propagated wrist-correction IK ablation

Question: if HOIFHLI's unfixed-wrist grasp result moves the wrist, can that returned wrist correction be propagated back into Kimodo's arm motion instead of being discarded?

New adapter:

tools/kimodo/apply_hoifhli_wrist_correction_to_kimodo_motion.py

What it does:

Reads Kimodo motion_augmented.npz, info.json, and a HOIFHLI hand-pose schedule.
Converts each HOIFHLI returned wrist pose from object-local coordinates back to Kimodo world coordinates.
Uses CCD inverse kinematics on SMPL-X arm chains: left shoulder/elbow/wrist = 16/18/20, right shoulder/elbow/wrist = 17/19/21.
Recomputes local_rot_mats, global_rot_mats, and posed_joints.
Does not change root, legs, object trajectory, Kimodo baseline code, or HOIFHLI baseline code.

Sanity check:

FK validation against saved Kimodo joints:
  max sample joint error ~= 1e-7m

E09 setting:

source body/object: E07 motion_augmented.npz
hand poses: E08 unfixed-wrist HOIFHLI hand schedule
correction_scale: 1.0
max_wrist_correction_m: 0.20
wrist_rotation_scale: 1.0
output:
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E09_unfixed_wrist_arm_ik/motion_scene_dynamic_object_hands.mp4

E09 result:

pick_after_attach contact ratio <5cm: 0.6637  (E07: 0.4226)
carry contact ratio <5cm:             0.8623  (E07: 0.2148)
place contact ratio <5cm:             0.5935  (E07: 0.2125)

upper-body velocity max:
  E07: 2.81 m/s
  E09: 5.06 m/s

Interpretation: full propagated wrist correction greatly improves hand-object contact, but introduces a local upper-body/wrist velocity spike. It is useful as evidence that the route works, but is too aggressive as a default.

E10 setting:

same as E09, except correction_scale = 0.5
output:
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E10_unfixed_wrist_arm_ik_halfscale/motion_scene_dynamic_object_hands.mp4

E10 result:

pick_after_attach contact ratio <5cm: 0.4869  (E07: 0.4226)
carry contact ratio <5cm:             0.4738  (E07: 0.2148)
place contact ratio <5cm:             0.3986  (E07: 0.2125)

upper-body velocity max:
  E07: 2.81 m/s
  E10: 2.79 m/s

Interpretation: half-scale propagated wrist correction is currently the better candidate default. It keeps velocity peaks near the E07 baseline while improving contact substantially, especially during carry/place. The next controlled improvement should replace this global scalar with a segment-aware or velocity-constrained correction schedule.

2026-06-04 E11 weighted arm-IK correction ramp

Observed issue: in E10, around 2s the hands stayed near the middle, then around 3s appeared to jump toward the grasp pose.

Root cause:

The hand bake already uses a smooth active_alpha starting at attach_frame = 67.
The new arm-IK adapter, however, used weight_sum > 1e-8 only as a binary active mask. Therefore frame 67 had no correction, but frame 68 received the full correction_scale=0.5 correction even though the smooth alpha was only 0.00044.
Quantitatively, E10's wrist delta relative to E07 jumped at frame 68 by about 5cm:

E10 frame 67: left/right wrist delta ~= 0m
E10 frame 68: left 0.055m, right 0.047m

Fix:

apply_hoifhli_wrist_correction_to_kimodo_motion.py now computes correction_weight = clip(weight_sum, 0, 1) and applies:

frame_scale = correction_scale * correction_weight[frame]
scaled_target = current + frame_scale * delta

E11 setting:

same as E10, except arm-IK wrist correction is weighted by the continuous
schedule/active-alpha weight.
output:
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E11_weighted_halfscale_arm_ik/motion_scene_dynamic_object_hands.mp4

E11 result:

Early wrist delta relative to E07:
  frame 67: left/right ~= 0m
  frame 68: left/right ~= 0m
  frame 70: left 0.00021m, right 0.00019m
  frame 90: left 0.00293m, right 0.00507m

contact ratio <5cm:
  pick_after_attach: 0.4323  (E07: 0.4226, E10: 0.4869)
  carry:             0.4738  (E07: 0.2148, E10: 0.4738)
  place:             0.3508  (E07: 0.2125, E10: 0.3986)

velocity:
  upper-body velocity max: 2.78 m/s  (E07: 2.81, E10: 2.79)
  wrist velocity max:      2.56 m/s  (E07: 2.68, E10: 2.79)

Interpretation: E11 fixes the pickup-time discontinuity while preserving most of the carry/place contact improvement from E10. It is a safer candidate than E10 for continuing the unified Kimodo+HOIFHLI-grasp adapter work.

2026-06-04 E12 hand-width gated pick IK ramp

Motivation: E11 removed the one-frame discontinuity, but the pick phase could still visually close the hands first and open them later. The better trigger is not a fixed attach frame; it should depend on when the generated body has already reached a plausible grasp width for the object.

Adapter change:

apply_hoifhli_wrist_correction_to_kimodo_motion.py now supports --pick-width-ramp.
The target width is read from the first HOIFHLI returned left/right wrist anchors. For diagnostics, the object mesh width projected onto the same anchor axis is also recorded.
The adapter searches the pick segment after attach_frame for the first frame where Kimodo's left/right wrist distance is within a tolerance of the target width. From that detected frame to the HOIFHLI grasp frame, it ramps up arm-IK correction smoothly.

E12 setting:

same as E11, plus:
  --pick-width-ramp
  --pick-width-tolerance-m 0.05
output:
output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E12_width_gated_halfscale_arm_ik/motion_scene_dynamic_object_hands.mp4

Detected pick ramp:

target HOIFHLI anchor width:        0.4650m
object width on anchor axis:        0.4827m
search range:                       frames 67..108
detected ramp start:                frame 77
ramp end:                           frame 108
hand width at start/end:            0.4153m / 0.4744m

E12 result:

pick_after_attach contact ratio <5cm: 0.4706  (E07: 0.4226, E11: 0.4323)
pick_contact_window contact ratio:     0.4052  (E07: 0.3829, E11: 0.3785)
carry contact ratio <5cm:             0.4738  (same as E11)
place contact ratio <5cm:             0.3508  (same as E11)

velocity:
  upper-body velocity max: 2.78 m/s
  wrist velocity max:      2.56 m/s

Interpretation: E12 is the current best candidate. It keeps E11's smooth activation and carry/place gains, while improving the pick segment by starting the arm-IK ramp when the generated hand width becomes object-compatible rather than waiting only on the attach-frame alpha.

2026-06-04 E13/E14 Kimodo HOI hand-anchor controlled probes

Scope:

Kept Kimodo and HOIFHLI source untouched.
Added adapter-only script tools/kimodo/add_kimodo_hand_anchor_constraints.py.
Task remained the HOI-only instrument-tray task: output/e2e_task_previews/final_v9_tight_sit_center/hoi_889a0ed0_tight_pick_place/task_plan.json.

E13 changed exactly one variable relative to the prior generated-input baseline:

Added Kimodo-native left-hand and right-hand world-target constraints before generation.
Frames: 84,96,108.
Targets came from HOIFHLI returned object-local wrist anchors at pick grasp frame 108, transformed through the E07 dynamic object trajectory.
Wrist rotations were deliberately disabled in the hand constraints (rot_indices=[]) to avoid forcing HOIFHLI wrist orientation into Kimodo generation.
Output: output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E13_pregen_hand_anchor_constraints/.

E13 result:

Loader verification: 9 constraints total; the added constraints loaded as LeftHandConstraintSet index 20 and RightHandConstraintSet index 21.
Motion/object quality gates: 9/10. The failing gate remained head_root_y_p05_gt_0p45m; pick still uses a very low squat.
No visible mesh explosion or neck flip in sampled frames 84/96/108/120.
Hand contact regressed relative to E12 because E13 removed the width-gated wrist correction:
pick_after_attach both<5cm = 0.376.
pick_contact_window both<5cm = 0.299.
carry both<5cm = 0.191.
place both<5cm = 0.243.

E14 changed exactly one variable relative to E13:

Re-applied the existing width-gated HOIFHLI wrist correction on top of the E13 generated body motion.
Parameters matched E12: correction_scale=0.5, max_wrist_correction_m=0.20, ik_iterations=8, wrist_rotation_scale=1.0, pick_width_ramp=true, pick_width_tolerance_m=0.05.
Detected pick ramp: frame 74 -> 108. This starts earlier than E12 frame 77 because E13's generation-time hand constraints make the Kimodo hand width reach the object-compatible width sooner.
Output: output/auto_experiments/20260604_unified_kimodo_hoi_pipeline/E14_pregen_anchor_plus_width_gated_ik/.

E14 result:

Motion/object quality gates: 9/10. The same low-squat head/root gate failed (head_root_y_p05_m = 0.303).
Contact improved over both E13 and E12:
pick_after_attach both<5cm = 0.527 (E12: 0.471, E13: 0.376).
pick_contact_window both<5cm = 0.450 (E12: 0.405, E13: 0.299).
carry both<5cm = 0.473 (E12: 0.474, E13: 0.191).
place both<5cm = 0.369 (E12: 0.351, E13: 0.243).
No visible mesh explosion or neck flip in sampled frames 84/96/108/120.
Residual issue: final render still reads visually as "standing while holding the tray" more than an explicit release/place-on-desk action, even though object metrics report final center error 0.0m and object bottom height error about 0.001m. This indicates the remaining place problem is semantic/visual release and support visibility, not target object coordinate placement.

Current interpretation:

The user's hand-width-trigger intuition is supported. Generation-time hand anchors alone are not enough, but they make the width-gated wrist correction start earlier and improve pick contact without introducing the previous severe twitch.
The next controlled variable should target place release semantics, not more pick grasp correction. Candidate: add a late-place hand-open/retract keyframe or object-release hand fade that explicitly separates hands from the object after the object reaches the support.

2026-06-05: Kimodo-only HOI full pipeline stabilization

Detailed log:

/mnt/data/PhysHSI/output/auto_experiments/20260605_kimodo_hoi_final_experiment_log.md

Adapter-only changes, with Kimodo and HOIFHLI baseline source left untouched:

Added adaptive pick root gate in tools/kimodo/run_kimodo_hoi_pipeline.py.
Added local root despike postprocess in tools/kimodo/despike_kimodo_root.py.
Added minimum pickup hand attach ramp in tools/kimodo/bake_kimodo_vertices_with_hands.py.
Made object initial pose prefer task manipulated_object_center_xyz in tools/kimodo/attach_task_object_to_kimodo_motion.py.
Added batch/generalization runner and quality_grid preset in tools/kimodo/run_kimodo_hoi_batch.py.

Final validated runs:

Instrument tray -> desk, cw2/default: /mnt/data/PhysHSI/output/auto_experiments/20260605_kimodo_hoi_final_2task_cw2/hoi_889a0ed0_tight_pick_place__default/motion_scene_dynamic_object_hands.mp4
Book -> table, cw2/default: /mnt/data/PhysHSI/output/auto_experiments/20260605_kimodo_hoi_final_2task_cw2/hoifhli_hoi_97183b84_pick_place_local__default/motion_scene_dynamic_object_hands.mp4
Decor -> sideboard, cw3: /mnt/data/PhysHSI/output/auto_experiments/20260605_kimodo_hoi_hybrid_local_root_despike_cw3/hybrid_84b703fb_sit_pick_place__constraint_weight_3p0/motion_scene_dynamic_object_hands.mp4

Summary:

All three final recommended runs pass the adapter batch gates for object final placement, path following, hand/finger continuity, wrist continuity, foot penetration, and contact-window metrics.
Book and hybrid pass motion/object quality 10/10. Instrument tray remains 9/10 only because head_root_y_p05_gt_0p45m fails during the low pickup bend; root step, joint step, foot sliding, contact, path, and object placement gates pass.
A single fixed constraint weight is not robust across tasks. The recommended entrypoint is run_kimodo_hoi_batch.py --preset quality_grid, which tries cw2 and cw3 and selects by gate/score.

2026-06-07: Task-generator root target and VLM id robustness fixes

Scope:

Changed task-generator code only; Kimodo and HOIFHLI baseline source remain untouched.
Main file: utils/sage_task_generator/vlm_task_generator.py.
Added regression test: utils/sage_task_generator/tests/test_vlm_task_generator_locomotion_targets.py.

Problem found in E27 task outputs:

In task_plan.json, HOI action steps exported root_target_point_xy as the object interaction/contact point rather than the humanoid safe standing point.
On the 10-scene E27 output, every HOI/sit action with a root target had this mismatch. HOI mismatches were typically 0.39m to 0.57m, exactly the offset between "stand near object" and "touch object".
Consequence: downstream Kimodo task adapters could pull the humanoid root into the object/support surface even though validation had already computed a safe standing_point_xy. This explains task-level penetration and "not really moving to the usable interaction stance".

Fix 1: root/interaction semantic split:

_locomotion_target_point_xyz(...) now treats pick, place, push, pull, kick, and synthesized carry as humanoid root targets.
For those HOI modes it returns [standing_x, standing_y, room_floor_z].
The actual hand/object target remains in interaction_point_xyz and manipulated-object metadata. This preserves object/contact information while preventing body-root collision.
sit/lie were intentionally not folded into this change because their root target can legitimately differ from the pre-action standing point.

Fix 2: unique-prefix target-id recovery:

_validate_step_target(...) now expands a VLM-produced truncated target_id only when it is a unique prefix of exactly one known entity id.
Ambiguous or unknown ids still fail validation.
This addresses real Qwen3-VL outputs such as room_..._table_bdab90 when the full entity id is room_..._table_bdab90d5, reducing avoidable retry attempts without guessing across objects.

Verification:

py_compile passed for utils/sage_task_generator/vlm_task_generator.py.
Unit tests passed: python -m unittest utils.sage_task_generator.tests.test_vlm_task_generator_locomotion_targets.
E27 regression re-translation: output/auto_experiments/kimodo_vlm_pipeline_20260607/taskgen/regression_root_target_after_locomotion_fix
58 total steps.
HOI root/standing mismatches after fix: 0.
Previously observed HOI mismatch count in E27: 27.
Fresh true-VLM smoke: output/auto_experiments/kimodo_vlm_pipeline_20260607/taskgen/E28_true_vlm_smoke3_root_target_fix
layout_27720065: validation ok, covers pick -> carry -> place plus pull; all HOI root/standing distances are 0.0m.
layout_616f1245: validation ok, covers kick -> sit -> get_up; HOI root/standing distance is 0.0m.
The attempted layout_84b703fb path was unavailable under data/selected_SAGE10k/scene_bank/; it belongs to a different e2e/cache scene source, not a validation failure.
Real failed VLM attempt replay:
Replayed fresh layout_27720065 attempt 1, which previously failed because Qwen truncated a table id.
With unique-prefix recovery, the same parsed VLM response validates ok and resolves all target ids to full entity ids.

Remaining risks:

Fresh smoke was 2 valid scene-bank scenes, not yet the full random-10 run.
sit root targets still differ from standing points by design; later motion validation should separately check sit-specific penetration and seat entry.
The task generator still depends on VLM semantic selection diversity; larger batch experiments should report per-action coverage and target diversity after these geometry fixes.