HIVE-30020: Fix TIMESTAMP WITH LOCAL TIME ZONE data loss and read failure on ORC tables - #6761
HIVE-30020: Fix TIMESTAMP WITH LOCAL TIME ZONE data loss and read failure on ORC tables#6761vikramahuja1001 wants to merge 1 commit into
Conversation
|
Can you help in reviewing this PR as per your availability? |
| vector.time[rowId] = timestampTZ.toEpochMilli(); | ||
| vector.nanos[rowId] = timestampTZ.getNanos(); | ||
| break; | ||
| } |
There was a problem hiding this comment.
default case is not handled for primitive category due to that the data is getting written in this case. Handle default case and throw exception so that the error will come while writing the data itself for unsupported/unhandled primitive type.
There was a problem hiding this comment.
done, thanks for pointing it out
| .getPrimitiveJavaObject(obj).toSqlTimestamp()); | ||
| break; | ||
| } | ||
| case TIMESTAMPLOCALTZ: { |
There was a problem hiding this comment.
It looks indentation problem for this block, please check and fix.
There was a problem hiding this comment.
I am not able to understand why indentation is being flagged here. Indentation level is same as other case statements above and below. Can you take a look once?
|
|
||
| insert into timestampltz_orc_format select * from timestampltz_formats; | ||
|
|
||
| SELECT * FROM timestampltz_orc_format; No newline at end of file |
There was a problem hiding this comment.
Adding ORDER BY clause to the select query is resulting an exception with the patch, please check.
There was a problem hiding this comment.
Yes rightly pointed, ORDER BY clause is giving an exception. To fix that will require a lot of changes, which are separate from the scope of this PR. I have raised a follow up JIRA and will work on that as well, hope that's fine.
JIRA ID: https://issues.apache.org/jira/browse/HIVE-30042
There was a problem hiding this comment.
I am ok with handling this scenario in a separate JIRA mentioned above. But I request you to add proper details with exception in the description section of the JIRA. Thanks.
There was a problem hiding this comment.
Sure, will do.
…lure on ORC tables
23a937d to
cf89476
Compare
|
Thanks for reviewing this PR @mdayakar . I have raised a follow up JIRA to fix the order by query clause as well and will raise a PR for that as well. |
|
|
@mdayakar , could you please merge this PR as well? |
3 participants
What changes were proposed in this pull request?
Fix TIMESTAMP WITH LOCAL TIME ZONE data loss and read failure on ORC tables
Why are the changes needed?
Check https://issues.apache.org/jira/browse/HIVE-30020 for more details
Does this PR introduce any user-facing change?
No
How was this patch tested?
Added a new test case(.q file)