Migrate Franka tasks to the shared asset and add minimal presets - #8170
kellyguo11 merged 32 commits into
Conversation
|
run-ci |
|
There was a problem hiding this comment.
Isaac Lab Review Bot
The flat-asset migration and additive differential-IK action offset are generally coherent, but backend-specific Physics variants are not selected in the controller tutorials or several contributed stack/lift configurations. The Reach task also changes success termination and reward behavior without a clear changelog entry. Two localized style inconsistencies should be cleaned up.
- Design and architecture: Backend selection is correctly kept at configuration boundaries for Reach, Drawer, cabinet IK, Pour, and the render benchmark, and cabinet IK preserves the parent variant mapping across robot replacement. However, the dual-backend controller tutorials and contributed stack/lift configurations adopt the variant-bearing flat asset without applying the same backend-keyed
Physicsselection, leaving those paths architecturally inconsistent with the rest of the migration. - API:
DifferentialInverseKinematicsActionCfg.offsetis an additive API with a compatibility-preserving zero default. It is materialized as an owned batched tensor, applied asraw * scale + offsetbefore clipping, exposed through the IO descriptor, documented in the action-term contract, and covered by a changelog fragment. The reviewed task-facing surfaces also include backend and collider variants, migrated prim-path expressions, actuator keys, and the newarm_collisionspreset. - Compatibility and deprecation: Breaking changes:
Isaac-Reach-FrankaandIsaac-Reach-Franka-OSCno longer terminate on success and no longer provide the previous weight-10 success reward. This changes episode behavior and return scale, while the major changelog fragment currently discusses the asset, collision defaults, checkpoint requalification, and action/controller contracts without naming the success-term removal. The flat-asset dynamics and gripper-only collision default are documented, and the existingdiffik_absalias retains a targeted deprecation warning and functional replacement path. - Implementation: The affine action path through
process_actionsandset_command, OSC actuator-property preservation, Pour actuator migration, cabinet variant preservation, and render-benchmark variant resolution are internally consistent. The remaining implementation gap is thatrun_diff_ik.py,run_osc.py, and contributed stack/lift configurations use the flat asset without selectingmujocoversusphysxaccording to the active backend, despite supporting more than one backend. - Style consistency: Import grouping,
preset()usage, comments, and changelog placement generally follow adjacent patterns. Rename the cabinet IK localphysics_variantsto reflect that it preserves the complete robot variant mapping, includingColliders, and keep the shorttarget_specstuple entries consistently on one line rather than forcing unnecessary multi-line formatting. - Test quality: The new differential-IK action test owns the affine transform contract at the action-term boundary. The Reach preset tests cover asset identity, backend payloads, collision presets, normalized absolute actions, success tracking, and OSC actuator preservation; cabinet IK and render-benchmark tests cover backend-specific variants. Existing integration tests were also migrated deliberately between the flat and legacy plants. Coverage does not currently guard the missing backend variant selection in the controller tutorials or affected contributed stack/lift configurations.
Minor fixes needed. Posted 5 actionable findings inline.
Automated review; human maintainers own approval decisions.
AntoineRichard
left a comment
There was a problem hiding this comment.
Couple of NITs but looks good otherwise I think.
|
run-ci |
|
run-ci |
|
run-ci |
|
run-ci |
## Description Migrate maintained Franka tasks, contributed configs, benchmarks, and controller examples to the shared `FRANKA_PANDA_CFG`. Standard tasks use full primitive arm/gripper collisions with explicit `Physics=physx` or `Physics=mujoco` payloads. Add `FRANKA_MINIMAL_CFG` and expose its gripper-only collision scope through `preset()` in the existing Franka Reach and rigid Lift configs. Select it with `presets=minimal`, including Reach OSC, while retaining task-local controller and actuator tuning. Backend selection stays independent of collision scope. No additional task ID or environment-page section is added. Preserve task-local actuator tuning, correct absolute Reach action mapping and OSC actuator handling, and support differential-IK action offsets in the zero agent. Reset clearance ignores disabled colliders, and cuRobo excludes robot geometry from world obstacles even with custom ignore lists. Legacy compatibility configs remain available; maintained tasks no longer use them. Lift reset/curriculum and continuous Reach tracking improvements are isolated in dependent #8171. No USD packages, checkpoints, or new dependencies are added to this source PR. **Breaking:** migrated tasks change robot dynamics and collision scope. Existing checkpoints require requalification; use explicit legacy configs where the old contract is needed. ## Validation At `25ad1ecfc`, based on current `develop` (`ae5d4e397`): - 72 focused tests passed on HORDE; formatting, changelog checks, and warning-free documentation build passed. - 28 rollouts used fresh, SHA-256-verified public-S3 checkpoints, with no checkpoint lookup cache and zero non-finite actions. - Reach/OSC, Drawer, and non-camera Soft/Cloth/Cable exceeded 90% task-level success. - Rigid Lift is not fully qualified by this asset-only change: Newton averaged 55.2% across seeds 40–42. The obsolete Newton IK-Rel artifact also remains unqualified. CI fixes at `90960d451`: excluded the robot from cuRobo world obstacles, and prevented randomized RL resets during static cable golden capture. The frontend checks, all six cuRobo tests, Isaac Sim RTX and active legacy kitless cable goldens passed; existing goldens and thresholds were unchanged. Formatting, changelog checks, and the warning-free docs build passed. Full CI is being rerun. The controlled speed comparison remains pending; no performance multiplier or complete checkpoint qualification is claimed. The separate ovstage-only cable renderer check also fails on unmodified `develop`; it is post-merge-only and is not part of the PR CI gate. ## Release backport - [x] <!-- backport-active-release --> Backport this pull request to the active release branch after it merges into `develop` Merged latest upstream `develop` (`7c05e96ca`). Audited every test added or changed by this PR and removed 14 redundant cases: broad asset inventories, repeated preset combinations, comparisons of copied defaults, and a synthetic USD-variant test. Kept regression coverage for action normalization and offsets, zero-agent holds, disabled collider filtering, OSC solver limits, backend payloads surviving IK overrides, and deterministic cable golden capture. Latest local validation: 128 focused tests passed across the six affected suites. Nine temporary production mutations failed on the intended retained assertions; production files were restored. Formatting and changelog checks passed. Earlier validation included minimal Reach, Reach OSC, and Lift random-agent rollouts and a clean warning-free documentation build. The final test pruning does not affect rendered docs. --------- Co-authored-by: Mustafa Haiderbhai <mhaiderbhai@nvidia.com> (cherry picked from commit b7c0225)
|
Backported to |
## Description Improve Franka rigid Lift resets and motion regularization: - Associate each of the eight cloned object geometries with its matching finger opening and hand-frame orientation, independent of prototype order. - Apply aligned pre-grasps after the generic finger-width reset, cache static reset data, and bound reset-bank prefill. - Scale Franka motion penalties with success-driven ADR, avoid terminating nominal arm-velocity excursions, and terminate non-finite rigid-object states. Restore continuous Franka Reach tracking without an early-success termination or terminal success reward. Kuka reward/curriculum configs remain unchanged. Asset, backend, actuator, Drawer, and deformable migrations belong to #8170. Updated against `develop` at `b7c022547` after #8170 merged. Minimal Reach, Reach-OSC, and Lift use `presets=minimal`; there is no separate minimal task config. The diff against `develop` contains only the 11 task behavior, test, and release-note files. **Behavior change:** training and play use a mixed table/pregrasp reset bank. Aligned pre-grasps have a 0.75 proposal probability before rejection and bank sampling; these scores are not table-only pickup success. For a separate table-only bank, set `env.events.conditional_reset.params.terms.reset_object_to_target.params.probability=0` before initialization. ## Validation At `00387a373`, 60 focused Reach/Lift/preset tests and formatting/changelog checks passed against current `develop`. Minimal Reach (2 environments) and Lift (8 environments, reset bank reduced to two states per group) each completed 10 random-agent steps with Newton MJWarp; the Lift reset bank filled successfully. These are smoke checks, not checkpoint requalification. At `325bdd34e`: 79 focused tests, formatting/changelog checks, and a warning-free documentation build passed. All 28 fresh-public-S3 checkpoint rollouts completed with zero non-finite actions. Rigid Lift mean success: Isaac Sim PhysX 94.7%, Newton 83.5% (seeds 40–42), and OvPhysX 91.5% (seed 42). The same checkpoints are used in #8170 and #8171; different reset distributions make this an MDP comparison, not evidence of new policy training. At `4f0c7ce8d`, the verified CI fixes from #8170 are included; 67 focused Lift/Reach/action tests and formatting/changelog checks passed. Active Isaac Sim and legacy kitless cable golden tests now pass without modifying goldens or thresholds. Full CI is being rerun. **Remaining gates:** Newton's thin capsule scored 40.6%; OvPhysX's thin plate scored 52.9%. The old Newton IK-Rel artifact remains unqualified. Controlled speed measurements remain pending. The separate ovstage-only cable renderer check also fails on unmodified `develop`; it is post-merge-only and is not part of the PR CI gate. ## Release backport - [x] <!-- backport-active-release --> Backport this pull request to the active release branch after it merges into `develop` --------- Co-authored-by: Mustafa Haiderbhai <mhaiderbhai@nvidia.com>
## Description Improve Franka rigid Lift resets and motion regularization: - Associate each of the eight cloned object geometries with its matching finger opening and hand-frame orientation, independent of prototype order. - Apply aligned pre-grasps after the generic finger-width reset, cache static reset data, and bound reset-bank prefill. - Scale Franka motion penalties with success-driven ADR, avoid terminating nominal arm-velocity excursions, and terminate non-finite rigid-object states. Restore continuous Franka Reach tracking without an early-success termination or terminal success reward. Kuka reward/curriculum configs remain unchanged. Asset, backend, actuator, Drawer, and deformable migrations belong to #8170. Updated against `develop` at `b7c022547` after #8170 merged. Minimal Reach, Reach-OSC, and Lift use `presets=minimal`; there is no separate minimal task config. The diff against `develop` contains only the 11 task behavior, test, and release-note files. **Behavior change:** training and play use a mixed table/pregrasp reset bank. Aligned pre-grasps have a 0.75 proposal probability before rejection and bank sampling; these scores are not table-only pickup success. For a separate table-only bank, set `env.events.conditional_reset.params.terms.reset_object_to_target.params.probability=0` before initialization. ## Validation At `00387a373`, 60 focused Reach/Lift/preset tests and formatting/changelog checks passed against current `develop`. Minimal Reach (2 environments) and Lift (8 environments, reset bank reduced to two states per group) each completed 10 random-agent steps with Newton MJWarp; the Lift reset bank filled successfully. These are smoke checks, not checkpoint requalification. At `325bdd34e`: 79 focused tests, formatting/changelog checks, and a warning-free documentation build passed. All 28 fresh-public-S3 checkpoint rollouts completed with zero non-finite actions. Rigid Lift mean success: Isaac Sim PhysX 94.7%, Newton 83.5% (seeds 40–42), and OvPhysX 91.5% (seed 42). The same checkpoints are used in #8170 and #8171; different reset distributions make this an MDP comparison, not evidence of new policy training. At `4f0c7ce8d`, the verified CI fixes from #8170 are included; 67 focused Lift/Reach/action tests and formatting/changelog checks passed. Active Isaac Sim and legacy kitless cable golden tests now pass without modifying goldens or thresholds. Full CI is being rerun. **Remaining gates:** Newton's thin capsule scored 40.6%; OvPhysX's thin plate scored 52.9%. The old Newton IK-Rel artifact remains unqualified. Controlled speed measurements remain pending. The separate ovstage-only cable renderer check also fails on unmodified `develop`; it is post-merge-only and is not part of the PR CI gate. ## Release backport - [x] <!-- backport-active-release --> Backport this pull request to the active release branch after it merges into `develop` --------- Co-authored-by: Mustafa Haiderbhai <mhaiderbhai@nvidia.com> (cherry picked from commit 34040cd)
Description
Migrate maintained Franka tasks, contributed configs, benchmarks, and controller examples to the shared
FRANKA_PANDA_CFG. Standard tasks use full primitive arm/gripper collisions with explicitPhysics=physxorPhysics=mujocopayloads.Add
FRANKA_MINIMAL_CFGand expose its gripper-only collision scope throughpreset()in the existing Franka Reach and rigid Lift configs. Select it withpresets=minimal, including Reach OSC, while retaining task-local controller and actuator tuning. Backend selection stays independent of collision scope. No additional task ID or environment-page section is added.Preserve task-local actuator tuning, correct absolute Reach action mapping and OSC actuator handling, and support differential-IK action offsets in the zero agent. Reset clearance ignores disabled colliders, and cuRobo excludes robot geometry from world obstacles even with custom ignore lists. Legacy compatibility configs remain available; maintained tasks no longer use them.
Lift reset/curriculum and continuous Reach tracking improvements are isolated in dependent #8171. No USD packages, checkpoints, or new dependencies are added to this source PR.
Breaking: migrated tasks change robot dynamics and collision scope. Existing checkpoints require requalification; use explicit legacy configs where the old contract is needed.
Validation
At
25ad1ecfc, based on currentdevelop(ae5d4e397):CI fixes at
90960d451: excluded the robot from cuRobo world obstacles, and prevented randomized RL resets during static cable golden capture. The frontend checks, all six cuRobo tests, Isaac Sim RTX and active legacy kitless cable goldens passed; existing goldens and thresholds were unchanged. Formatting, changelog checks, and the warning-free docs build passed.Full CI is being rerun. The controlled speed comparison remains pending; no performance multiplier or complete checkpoint qualification is claimed.
The separate ovstage-only cable renderer check also fails on unmodified
develop; it is post-merge-only and is not part of the PR CI gate.Release backport
developMerged latest upstream
develop(7c05e96ca). Audited every test added or changed by this PR and removed 14 redundant cases: broad asset inventories, repeated preset combinations, comparisons of copied defaults, and a synthetic USD-variant test. Kept regression coverage for action normalization and offsets, zero-agent holds, disabled collider filtering, OSC solver limits, backend payloads surviving IK overrides, and deterministic cable golden capture.Latest local validation: 128 focused tests passed across the six affected suites. Nine temporary production mutations failed on the intended retained assertions; production files were restored. Formatting and changelog checks passed. Earlier validation included minimal Reach, Reach OSC, and Lift random-agent rollouts and a clean warning-free documentation build. The final test pruning does not affect rendered docs.