Skip to content

Add a physics-plus-render mode to the render benchmark - #7797

Open
mataylor-nvidia wants to merge 6 commits into
isaac-sim:developfrom
mataylor-nvidia:mataylor/render-benchmark-physics-mode
Open

mataylor-nvidia wants to merge 6 commits into
isaac-sim:developfrom
mataylor-nvidia:mataylor/render-benchmark-physics-mode

Conversation

@mataylor-nvidia

@mataylor-nvidia mataylor-nvidia commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Description

Follow-up to #7702. That PR added the Isaac-RenderBenchmark-Franka-Cabinet task, whose scene is always driven through the physics solver. This adds a mode switch so the same scene can be benchmarked with or without the solver doing that actuation work, plus a physics timer so what physics costs is visible either way.

  • source/isaaclab/isaaclab/sim/simulation_context.py: adds PHYSICS_PROFILE_SCOPE around the physics step in SimulationContext.step, gated by the ISAACLAB_PHYSICS_PROFILE environment variable. This mirrors the existing ISAACLAB_RENDER_PROFILE timer added by Added Rendering Benchmark #7702 — same scope-name-per-backend approach, same wp.ScopedTimer print format — and is off by default, so an ordinary run pays neither the print nor the device synchronization.
  • source/isaaclab_tasks/.../render_benchmark_env_cfg.py: new benchmark_mode field ("render" | "physics_render"), read from the BENCHMARK_MODE environment variable and rejecting unknown values at import, so a typo fails at launch instead of quietly benchmarking the default for a whole sweep.
  • source/isaaclab_tasks/.../render_benchmark_env.py: _animate_joints picks its drive path from the mode. "physics_render" requests the sinusoid as actuator position targets, which is what the task did before. "render" instead writes the joint positions and zero velocities straight into the simulation.
  • scripts/benchmarks/benchmark_renderer.py: turns the physics timer on for every run, the same way Added Rendering Benchmark #7702 turns on the render timer, and declares the scope name and its log pattern. The script's parsing and reporting are untouched — physics timings land in the run log next to the render ones.
uv run --no-sync python scripts/benchmarks/benchmark_renderer.py newton_*
BENCHMARK_MODE=physics_render uv run --no-sync python scripts/benchmarks/benchmark_renderer.py newton_*

Note that the physics backend still integrates in "render" mode, because an Isaac Lab environment has no way to skip its own physics step; the mode changes how the joints are driven, not whether the step happens.

Known issue

Greptile and the Isaac Lab review bot both flagged the same defect in "render" mode, and they are right: _animate_joints runs from _pre_physics_step, so the direct joint write is followed by decimation physics steps before the camera is read. The implicit actuator drives are still active with their previous targets, and gravity and joint limits also apply, so the rendered frame is the post-step state rather than the analytic sinusoid the docstrings currently claim. The fix is to either apply the pose after physics and before the render, or soften the documented contract. Not yet addressed — do not merge as-is.

Type of change

  • New feature (non-breaking change which adds functionality)

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

The render benchmark could only ever report renderer time. Add a
benchmark_mode switch so the same scene can be measured either way.

In "render" mode (the default, and what the task did before) the
articulations are posed by writing joint positions straight into the
simulation, so the frame a renderer is timed on is the analytic sinusoid
and nothing has to be actuated to produce it. In "physics_render" mode
the same poses are requested as actuator targets, so the solver does an
ordinary task's tracking work.

Measuring the second mode needs a physics timer, so add
PHYSICS_PROFILE_SCOPE around the physics step in SimulationContext.step,
gated by ISAACLAB_PHYSICS_PROFILE and mirroring the existing
ISAACLAB_RENDER_PROFILE timer. benchmark_renderer.py selects the mode
with --mode, sums the decimated physics steps that precede each rendered
frame, and grows PHYSICS and TOTAL columns when a sweep has them.
@greptile-apps

greptile-apps Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; the current changes address both previously reported render-pose and stale-transform defects without introducing a new actionable issue.

Summary

This PR adds separate render and physics-plus-render benchmark modes, introduces opt-in synchronized physics-step profiling, and extends benchmark reporting with per-frame physics and combined timing statistics.

  • Direct render-mode poses are now applied after physics and explicitly propagated before the camera read.
  • Renderer scene-state cadence is invalidated so the direct pose reaches the next lazy camera render.
  • Physics timings are accumulated across decimated steps and reported alongside render and total timings.
  • Configuration, lifecycle, parsing, and mode ordering receive focused tests.

Diagram

sequenceDiagram
    participant E as Benchmark environment
    participant P as Physics backend
    participant S as Simulation context
    participant R as Render context
    participant C as Camera

    E->>P: Run decimated physics steps
    S->>S: Record physics-step timings
    alt render mode
        E->>P: Write analytic joint pose and zero velocity
        E->>S: forward()
        E->>R: Reset scene-state cadence
    else physics_render mode
        E->>P: Set actuator targets before physics
    end
    E->>C: Read lazy camera output
    C->>R: Synchronize scene and render
    R->>R: Record render timing
Loading

Reviews (4) · Last reviewed commit: "Parse every timed scope out of a run log..."

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The physics-plus-render benchmark plumbing is coherent, but render mode writes the analytic joint pose before the environment’s decimated physics steps. The camera therefore observes a subsequently integrated state rather than necessarily the exact pose promised by the new documentation.

  • Design and architecture: The mode switch and profiling scope are consistently connected across task configuration, environment behavior, subprocess environment variables, timing parsing, and reporting. However, the render-mode design does not fully isolate the rendered pose from physics because the direct state write occurs in _pre_physics_step, before physics integration.
  • API: The new mode values, profiling scope, CLI option, and additive JSON fields are kept consistent and covered by tests. The documented contract for benchmark_mode="render" overstates the behavior by claiming the rendered scene is exactly the analytic sinusoid even though physics steps occur after the direct write.
  • Implementation: Physics timings are accumulated across decimated steps and paired with each render timing appropriately. The render-mode state path needs adjustment: apply the direct pose immediately before rendering if exact analytic poses are required, or revise the documentation and benchmark claims to describe the post-integration pose accurately.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

Follows how isaac-sim#7702 handled ISAACLAB_RENDER_PROFILE: the timer is simply
always on rather than gated behind a flag. Doing the same for
ISAACLAB_PHYSICS_PROFILE removes the reason for everything --mode was
carrying. Every log now has physics timings, so the parser no longer
tracks whether a frame has them, the table no longer chooses its columns
per run, and the test log writer no longer makes physics optional.

The scene switch stays on BENCHMARK_MODE, read directly the way the
profile env vars are. Both modes report physics, render, and total; the
mode decides what the physics number means, not whether there is one.
Keep only the physics scope name, its pattern, and the env var that turns
the timer on. Physics timings land in the run log next to the render
ones; how they get summarized is a separate question from adding the
timer, and reworking parse_log, build_record, and the results table to
answer it here was scope this change did not need.
Greptile and the review bot both caught that render mode wrote the joint
pose from _pre_physics_step, so decimation physics steps ran before the
camera was read. The implicit drives were never retargeted and stayed
active, and gravity and joint limits applied too, so the solver pulled
the joints off the written pose and the renderer was timed on its output
rather than the analytic one.

Split the drive paths: _pre_physics_step now only advances the clock and,
in physics_render mode, requests actuator targets. Render mode writes the
pose in _get_observations instead, after the last physics step and just
before the lazy camera read, with sim.forward() to propagate the joint
write to the body transforms the renderer reads.

The regression test asserts the call order across the step, so moving the
write back ahead of physics fails it.
@mataylor-nvidia

Copy link
Copy Markdown
Contributor Author

@greptile review

Greptile caught that the post-physics joint write can miss the render.
RenderContext.update_scene_state syncs at most once per physics step and
sim.forward() does not advance that count, so with lazy_sensor_update off
InteractiveScene.update has already stamped the current step and the lazy
camera read reuses the transforms captured before the write.

The task's default config leaves lazy_sensor_update on, so nothing had
synced yet and the default path was already correct, but that is one
config flag away from silently rendering the solver pose.
reset_scene_state_cadence() is the documented hook for this and costs an
extra sync only in the case that needs one.
@mataylor-nvidia

Copy link
Copy Markdown
Contributor Author

@greptile review

@mataylor-nvidia

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 14, 2026
"""

RENDER_SCOPE_PATTERN = re.compile(rf"{re.escape(RENDER_SCOPE)} took ([\d.]+) ms")
PHYSICS_SCOPE_PATTERN = re.compile(rf"{re.escape(PHYSICS_SCOPE)} took ([\d.]+) ms")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PHYSICS_SCOPE and PHYSICS_SCOPE_PATTERN seems to not be used anywhere, is that right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated parse_log -> parse_frames to ingest these

parse_log took one hardcoded regex, so PHYSICS_SCOPE_PATTERN was compiled
and never read: the timer ran and its numbers sat in the log unsummarized.
Scopes are now a name-to-pattern mapping, and a caller can pass its own to
summarize a timer this script does not know about.

Scopes cannot be collected independently and zipped. Decimation puts
several physics steps in front of one render, so a non-frame scope
accumulates until the frame scope closes it out; counting the steps as
frames instead would report the cost of a single step and shift the
samples the warm-up padding skips past. parse_frames carries that, and a
scope missing from a log reads as zero, so a run captured without
ISAACLAB_PHYSICS_PROFILE still parses.

Render stays flat in the results, so build_record and the existing tests
did not move. Physics and total ride along as sub-dicts, and the record
picks up sub-dicts by shape rather than by name, so another scope reaches
the JSON without further edits. The table gained PHYSICS and TOTAL, which
meant rebuilding it from column pairs: its widths were baked into noqa'd
string literals that adding a column would have rewritten anyway.
@mataylor-nvidia

Copy link
Copy Markdown
Contributor Author

@greptile review

@mataylor-nvidia

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants