Skip to content

Use scene dome lighting in Newton RTX - #7796

Open
maxkra15 wants to merge 6 commits into
isaac-sim:developfrom
maxkra15:add-visualizer-sky-backgrounds
Open

maxkra15 wants to merge 6 commits into
isaac-sim:developfrom
maxkra15:add-visualizer-sky-backgrounds

Conversation

@maxkra15

@maxkra15 maxkra15 commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Description

Makes the scene-authored dome light the single source of truth for the visible sky and environment lighting across Kit and Newton RTX.

  • Newton RTX's rtx_environment="default" mirrors authored scene DomeLight properties and the composed transform into ViewerRTX's private stage.
  • Kit and Newton RTX show the native scene background by default; no renderer-specific HDR path, intensity, rotation, or background-mode selector is required.
  • An explicit VisualizerCfg.background_color changes only the visible background and preserves dome lighting and reflections.
  • Newton GL continues to use its procedural gradient because it cannot display HDR environment textures.
  • If a scene has no dome, Newton RTX falls back to Newton's existing default light rig. The explicit "studio" and "none" environments remain unchanged.

This keeps task authoring in the existing DomeLightCfg, for example:

env_cfg.scene.sky_light.spawn.texture_file = "/path/to/evening.hdr"
env_cfg.scene.sky_light.spawn.intensity = 1000.0

The implementation copies authored dome attributes generically, resolves remote texture assets for the private OVRTX stage, and preserves the dome's composed transform and visibility. It deliberately does not add new sky assets or change task lighting rigs.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Documentation update

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Validation

  • Regression-first visualizer tests: the previous implementation failed 6 new assertions covering duplicated configuration and missing scene-dome propagation
  • Focused visualizer suite: 134 passed
  • uv run --frozen isaaclab -f: passed
  • uv run --frozen python tools/changelog/cli.py check develop: passed
  • uv run --isolated --extra dev -- make -C docs current-docs: passed without warnings
  • Live headless ANYmal-D Newton RTX rollout and video capture: passed using only the scene-authored Kloofendal HDR dome

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with uv run isaaclab -f
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • My name already exists in CONTRIBUTORS.md

@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team labels Sep 14, 2026
@maxkra15

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 16, 2026
@maxkra15
maxkra15 marked this pull request as ready for review September 16, 2026 12:21
@maxkra15
maxkra15 requested a review from a team September 16, 2026 12:21
@greptile-apps

greptile-apps Bot commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

Fix the mutable-configuration compatibility regression before merging.

Findings

  1. P1 Mutable background shortcut now crashes

Summary

This PR adds a shared solid/sky background selector and configurable Newton RTX HDR dome lighting.

  • Preserves the solid sky-blue default and exposes native backgrounds across Kit, Newton GL, and Newton RTX.
  • Replaces Newton RTX's default dome-plus-distant-light setup with a configurable dome.
  • Adds documentation and focused configuration/backend tests.
  • Post-construction assignment of the documented None shortcut can now prevent Kit and Newton GL initialization.

Reviews (1) · Last reviewed commit: "Merge remote-tracking branch 'origin/dev..."

Comment thread source/isaaclab_visualizers/isaaclab_visualizers/newton/newton_visualizer.py Outdated

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The shared background_mode API is wired consistently across Kit, Newton GL, and Newton RTX, but compatibility with post-construction background_color=None assignments is broken because mode normalization occurs only during initialization.

  • Design and architecture: Centralizing solid-versus-sky selection in VisualizerCfg and translating it at each backend boundary fits the existing visualizer layering. The Newton RTX dome configuration and single-light rig are coherent with that design.
  • API: The additive Literal["solid", "sky"] field is documented and validated. However, the stated backward-compatible background_color=None shortcut is incomplete: assigning None after config construction no longer selects the native sky, unlike the previous consumer-side behavior.
  • Implementation: Kit, Newton GL, and Newton RTX consistently branch on background_mode, but their solid-background paths assume background_color remains non-null after __post_init__. A later assignment to None causes Newton GL's assertion to fail and Kit to unpack None; the effective mode must account for background_color is None at consumption time or maintain the invariant on assignment.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

Comment thread source/isaaclab_visualizers/isaaclab_visualizers/newton/newton_visualizer.py Outdated
@maxkra15

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 16, 2026

@ooctipus ooctipus left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on sky support across the visualizers. I think we can simplify the configuration by building on the environment that tasks already define.

My main concern is the experience for a task author: once a scene has an HDR dome configured, I would expect both Kit and Newton RTX to use it automatically for lighting and the visible background, without needing another mode or a repeated HDR path.

For example, the shared velocity scene already defines /World/skyLight, including kloofendal_43d_clear_puresky_4k.hdr and intensity 750.0. If a user changes it:

env_cfg.scene.sky_light.spawn.texture_file = "evening.hdr"
env_cfg.scene.sky_light.spawn.intensity = 1000.0

I would expect either KitVisualizerCfg() or NewtonRTXVisualizerCfg() to render that environment. With the current implementation, Newton RTX creates a separate dome from its visualizer settings, defaulting to blue_sky.hdr and intensity 500.0, so users would need to configure the same environment in two places.

Could we adjust the approach along these lines?

  • Have Newton RTX consume the scene dome's authored texture, intensity, and transform. Recreating the dome in the viewer's separate stage is fine; sourcing those values from the scene would keep configuration in one place.
  • Keep background_color as the optional solid-color override and drop the new background_mode field. This would avoid having two settings that select the background and need to stay consistent.
  • Apply no background override by default, so the configured scene HDR is visible automatically. An explicitly supplied RGB color would change only the visible background while preserving scene lighting. This also means revisiting the existing default solid RGB and documenting that behavior change.
  • Express any convenient sky/solid presets through the existing settings, while allowing a task's configured HDR to work without selecting a preset or repeating its dome configuration.

It would be helpful to cover this with regression tests: an HDR configured only on the scene should be consumed by both Kit and Newton RTX with default visualizer configs, and an explicit solid-color override should preserve the lighting. We should also guard against reintroducing the duplicate selector and independently configured default dome. Newton GL's procedural sky limitation can be documented separately.

I'd suggest focusing this change on connecting scene lighting to Newton RTX, with new HDR assets and default lighting-rig changes considered separately. If there are ViewerRTX constraints that make reusing the scene dome difficult, it would be helpful to discuss those here.

@ooctipus
ooctipus self-requested a review September 19, 2026 09:44
@maxkra15 maxkra15 changed the title Add selectable visualizer sky backgrounds Use scene dome lighting in Newton RTX Sep 19, 2026
@maxkra15

Copy link
Copy Markdown
Contributor Author

Thanks, this is now reworked around the scene configuration as suggested.

  • background_mode and the Newton-specific dome texture/intensity/rotation fields are removed.
  • background_color=None is the default, so Kit and Newton RTX show the scene-authored dome without another selector.
  • Newton RTX now mirrors every authored scene DomeLight into ViewerRTX's private stage, including authored light attributes, the resolved HDR asset, composed transform, and visibility.
  • An explicit solid background_color remains a visible-background override only; the scene dome continues to provide lighting and reflections.
  • Newton's existing default rig remains the no-dome fallback, while studio and none remain explicit alternatives.
  • Newton GL's procedural-sky limitation is documented.

The regression coverage exercises both native-HDR and solid-background cases and verifies that no second distant light is added. I also validated this with a live headless ANYmal-D Newton RTX rollout using only the velocity scene's existing Kloofendal dome.

@maxkra15

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants