fix: preserve integer precision in cumulative sums - #4706
Open
shoumikhin wants to merge 1 commit into
Open
Conversation
github-actions
Bot
added
component: tests
## Problem The `cumsum` converter builds a loop with a float32 accumulator. Integer totals can lose precision: a row starting at 16777216 followed by five ones should end at 16777221, but the converter loses those increments. ## Change Use TensorRT's cumulative-sum layer when available. Accumulate integer and boolean inputs in int64 by default. Forward an explicit `dtype` argument and keep unsupported requested types in PyTorch. Float64 is allowed only when `truncate_double=True` permits float32 arithmetic. Accumulate float16 and bfloat16 inputs in float32, then convert the output back, preserving the existing reduced-precision behavior. Keep the old loop on TensorRT versions before 10.8. ## Tests Passed 28/28 cumulative-sum tests covering fixed and changing shapes, integer and boolean outputs, and reduced-precision floating-point inputs. The two exact integer tests fail without this change. A separate dynamic-batch int32 check also matched PyTorch exactly for the row described above. Without this change, it loses the unit increments. An existing limit remains: explicitly requesting float16 or bfloat16 for a wider input does not always match PyTorch's input-rounding behavior. The same mismatches were measured with and without this change. The passing small `dtype` test does not prove that broader behavior is fixed. Tested on Linux x86_64 with Python 3.12, TensorRT 11.2, and the native runtime. Windows, aarch64, TensorRT-RTX, the older-version loop path, and updates to engine weights after compilation were not tested.
shoumikhin
force-pushed
the
upstream/cumsum-native-cumulative-layer
branch
from
September 12, 2026 04:58
2f82de5 to
2355556
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
cumsumconverter builds a loop with a float32 accumulator. Integer totals can lose precision: a row starting at 16777216 followed by five ones should end at 16777221, but the converter loses those increments.Change
Use TensorRT's cumulative-sum layer when available. Accumulate integer and boolean inputs in int64 by default. Forward an explicit
dtypeargument and keep unsupported requested types in PyTorch. Float64 is allowed only whentruncate_double=Truepermits float32 arithmetic.Accumulate float16 and bfloat16 inputs in float32, then convert the output back, preserving the existing reduced-precision behavior. Keep the old loop on TensorRT versions before 10.8.
Tests
Passed 28/28 cumulative-sum tests covering fixed and changing shapes, integer and boolean outputs, and reduced-precision floating-point inputs. Both exact integer tests fail on the base branch, losing unit increments at large int32 and int64 values.
An earlier separate dynamic-batch int32 check also matched PyTorch exactly for the row described above and lost its unit increments without this change.
An existing limit remains: explicitly requesting float16 or bfloat16 for a wider input does not always match PyTorch's input-rounding behavior. Earlier comparisons found the same mismatches with and without this change. The passing small
dtypetest does not prove that broader behavior is fixed.The focused rerun used Linux x86_64, Python 3.12, PyTorch 2.15 nightly, and TensorRT 11.3 with the Python runtime. Earlier native-runtime coverage used TensorRT 11.2; a native TensorRT 11.3 build was not tested.
Windows, aarch64, TensorRT-RTX, the older-version loop path, and updates to engine weights after compilation were not tested.