fix: preserve results when evaluating constant tensor operations - #4708
Open
shoumikhin wants to merge 1 commit into
Open
fix: preserve results when evaluating constant tensor operations#4708shoumikhin wants to merge 1 commit into
shoumikhin wants to merge 1 commit into
Conversation
github-actions
Bot
added
component: tests
## Problem The converter can evaluate an operation during compilation when both inputs are constants. This is called constant folding. Python's `max`, `min`, and logical operators do not work on tensors element by element, so folding can fail or return a whole input tensor. Simply converting every constant through the engine's input helper would create another problem. Python numbers would become one-element tensors with fixed data types. That can overflow integers, change float16 arithmetic, and turn boolean results into integers. ## Change Keep Python arithmetic and comparison operators for scalar values (single numbers). These operators also preserve PyTorch's choice of data type when a tensor is combined with a scalar. Use PyTorch's element-by-element max, min, and logical operations for tensor inputs. Do not change how inputs are prepared for operations that run inside TensorRT. Use non-trainable parameters for the logical tests' constants. This keeps their operations in the traced graph so the converter, not tracing, evaluates them. ## Tests Passed 119/119 tests in the binary-operator module. Direct folding tests check exact values, data types, dimensions, both input orders, and mixed scalar/tensor inputs. Engine tests check float16 overflow and boolean addition, plus max/min/logical operations over multiple elements. The scalar regression tests fail without the scalar fix, including both engine-output checks. The five max/min/logical engine cases fail without the element-by-element fix. Tested on Linux x86_64 with Python 3.12, TensorRT 11.2, and the native runtime. Windows, aarch64, TensorRT-RTX, TensorRT 10.x, and the Python-only runtime were not tested.
shoumikhin
force-pushed
the
upstream/elementwise-python-fold
branch
from
September 12, 2026 04:58
f9bc85a to
47e572d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The converter can evaluate an operation during compilation when both inputs are constants. This is called constant folding. Python's
max,min, and logical operators do not work on tensors element by element, so folding can fail or return a whole input tensor.Simply converting every constant through the engine's input helper would create another problem. Python numbers would become one-element tensors with fixed data types. That can overflow integers, change float16 arithmetic, and turn boolean results into integers.
Change
Keep Python arithmetic and comparison operators for scalar values (single numbers). These operators also preserve PyTorch's choice of data type when a tensor is combined with a scalar.
Use PyTorch's element-by-element max, min, and logical operations for tensor inputs. Do not change how inputs are prepared for operations that run inside TensorRT.
Use non-trainable parameters for the logical tests' constants. This keeps their operations in the traced graph so the converter, not tracing, evaluates them.
Tests
Passed 119/119 tests in the binary-operator module. Direct folding tests check exact values, data types, dimensions, both input orders, and mixed scalar/tensor inputs. Engine tests check float16 overflow and boolean addition, plus max/min/logical operations over multiple elements.
The five max/min/logical engine cases fail on the base branch because Python tries to interpret a multi-element tensor as one boolean. They pass with this change.
Earlier tests that removed only the scalar-preservation logic failed the scalar regressions, including both engine-output checks. Scalar arithmetic itself also works on the base branch; those checks protect existing behavior.
The focused rerun used Linux x86_64, Python 3.12, PyTorch 2.15 nightly, and TensorRT 11.3 with the Python runtime. Earlier native-runtime coverage used TensorRT 11.2; a native TensorRT 11.3 build was not tested.
Windows, aarch64, TensorRT-RTX, and TensorRT 10.x were not tested.