HIVE-29819: Fix Iceberg COW UPDATE for null identity partition column filters - #6763
Open
kokila-19 wants to merge 1 commit into
Open
HIVE-29819: Fix Iceberg COW UPDATE for null identity partition column filters#6763kokila-19 wants to merge 1 commit into
kokila-19 wants to merge 1 commit into
Conversation
kokila-19
force-pushed
the
fix_update
branch
from
September 9, 2026 04:07
041239c to
32df432
Compare
asf-ci-hive
added
tests pending
tests passed
and removed
tests unstable
tests pending
labels
kokila-19
force-pushed
the
fix_update
branch
from
September 9, 2026 10:33
32df432 to
a9b8b5f
Compare
asf-ci-hive
added
tests pending
tests passed
and removed
tests passed
tests pending
labels
kokila-19
changed the title
Contributor
Author
|
Before fix: After fix: Tablescan now has str_col partition column in them , so it won't throw error. |
kokila-19
marked this pull request as ready for review
September 10, 2026 04:57
Contributor
Author
|
@kasakrisz @chinnaraolalam |
kasakrisz
reviewed
Sep 10, 2026
Comment on lines
+529
to
+530
| for (Operator<? extends OperatorDesc> child : scanOp.getChildOperators()) { | ||
| if (child instanceof FilterOperator filterOperator) { |
Contributor
There was a problem hiding this comment.
Could you please verify whether FilterOperator is guaranteed to be a direct child of TS?
Contributor
Author
There was a problem hiding this comment.
Thanks for telling me this!
Not necessarily, Hive’s PPD code already handles cases where a Select sits between them, so plans like TableScan → Select → Filter are normal. I’ve replaced the direct-child check with a passthrough pipeline walk plus filterExpr handling.
| @@ -0,0 +1,33 @@ | |||
| -- SORT_QUERY_RESULTS | |||
|
|
|||
Contributor
There was a problem hiding this comment.
Please set
set hive.explain.user=false;
That type of plan has more details.
… filters Column pruning dropped partition columns from the TableScan while filters above the scan still referenced them. Keep filter-referenced partition columns in the Iceberg scan projection.
kokila-19
force-pushed
the
fix_update
branch
from
September 11, 2026 17:19
a9b8b5f to
80f0268
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Column pruning now keeps Iceberg identity partition columns in the TableScan when a filter uses them.
Before, UPDATE ... WHERE str_col IS NULL could leave str_col in the Filter but remove it from the TableScan.
Now, if a filter references a partition column, that column stays in the scan.
Why are the changes needed?
Iceberg COW UPDATE builds a plan with a Filter above the TableScan. Column pruning was dropping partition columns the Filter still needed causing errors like cannot find field str_col.
The fix keeps the Filter and TableScan aligned on which columns are read.
Does this PR introduce any user-facing change?
User will get error while executing update query with is null predicate in partition column.
It executes successfully after this fix.
How was this patch tested?
qtest: update_iceberg_cow_null_identity_partition.q