Repository navigation
fix(common): restore the overflow check in Timestamp::getEpochNanoSeconds - #1584
Conversation
…h helpers Address review feedback on PR LadybugDB#1584: - Make Multiply checked integral operations accept inputs by value/const so constexpr/const operands (e.g. Interval scale constants) select the checked implementation instead of the unchecked primary template. - Simplify Timestamp::getEpochNanoSeconds to rely on the safe Multiply API. - Route Date::getEpochNanoSeconds through checked Multiply operations to avoid raw wrapping multiplication. - Make string-to-timestamp tryCast paths return false on overflow instead of propagating exceptions, preserving try-cast failure semantics. Note: build/tests not executed in this environment (cmake/ninja unavailable).
…h helpers Address review feedback on PR LadybugDB#1584: - Make Multiply checked integral operations accept inputs by value/const so constexpr/const operands (e.g. Interval scale constants) select the checked implementation instead of the unchecked primary template. - Simplify Timestamp::getEpochNanoSeconds to rely on the safe Multiply API. - Route Date::getEpochNanoSeconds through checked Multiply operations to avoid raw wrapping multiplication. - Make string-to-timestamp tryCast paths return false on overflow instead of propagating exceptions, preserving try-cast failure semantics. Note: build/tests not executed in this environment (cmake/ninja unavailable).
c3a1f1c to
a38e3f6
Compare
|
@pratik-saptarshi Could you look into the test failure? Also, you probably want to --reset-author on the commits to yourself instead of the bot so attribution is clear. |
I am training an expert system that is adhering to my rules. I have 9 bots, with self-learning loops. |
…onds
Casting an out-of-range timestamp to TIMESTAMP_NS silently wrapped:
CAST('9999-01-01 00:00:00' AS TIMESTAMP_NS) produced an instant in 1815
instead of failing. getEpochNanoSeconds multiplies the microsecond value
by 1000 through function::Multiply::operation, intending its
overflow-checked int64_t specialization (which throws OverflowException),
but the parameter is const-qualified, so template argument deduction
selected the primary unchecked template and the product wrapped.
Copy the operands to non-const locals so the checked specialization
applies, matching how fromEpochMilliSeconds/fromEpochSeconds already call
it. All three conversion paths route through this helper (string,
timestamp, and date casts to TIMESTAMP_NS), so all three now throw
'Overflow exception: Value ... * 1000 is not within INT64 range.' for
instants outside the nanosecond-representable range (roughly
1677-09-21 to 2262-04-11); in-range casts are unchanged.
Adds timestamp_ns boundary cases to cast_bounds.test covering the three
cast paths in both directions plus in-range controls.
…h helpers Address review feedback on PR LadybugDB#1584: - Make Multiply checked integral operations accept inputs by value/const so constexpr/const operands (e.g. Interval scale constants) select the checked implementation instead of the unchecked primary template. - Simplify Timestamp::getEpochNanoSeconds to rely on the safe Multiply API. - Route Date::getEpochNanoSeconds through checked Multiply operations to avoid raw wrapping multiplication. - Make string-to-timestamp tryCast paths return false on overflow instead of propagating exceptions, preserving try-cast failure semantics. Note: build/tests not executed in this environment (cmake/ninja unavailable).
Keep tryCast returning false on overflow for try-cast paths, but let strict CAST propagate overflow exceptions instead of converting them to parse errors.
…h helpers Address review feedback on PR LadybugDB#1584: - Make Multiply checked integral operations accept inputs by value/const so constexpr/const operands (e.g. Interval scale constants) select the checked implementation instead of the unchecked primary template. - Simplify Timestamp::getEpochNanoSeconds to rely on the safe Multiply API. - Route Date::getEpochNanoSeconds through checked Multiply operations to avoid raw wrapping multiplication. - Make string-to-timestamp tryCast paths return false on overflow instead of propagating exceptions, preserving try-cast failure semantics. Note: build/tests not executed in this environment (cmake/ninja unavailable).
d6027ca to
27dcb1c
Compare
27dcb1c to
4149bdc
Compare
|
Authorship: all commits on this branch have been reset to the submitting account ( What changed in response to the review:
Verification, run locally on the final tree ( — Musebot-Astro |
| } | ||
|
|
||
| template<> | ||
| void TryCastStringToTimestamp::cast<timestamp_ns_t>(const char* input, uint64_t len, |
There was a problem hiding this comment.
With IGNORE_ERRORS=true, an out-of-range row now aborts the whole COPY, and the error gives no file or line:
// ts.csv: id,t / 1,2020-01-01 00:00:00 / 2,9999-01-01 00:00:00
COPY T FROM 'ts.csv' (HEADER=true, IGNORE_ERRORS=true);
-- Error: Overflow exception: Value 253370764800000000 * 1000 is not within INT64 range.The CSV reader only skips rows on ConversionException (csv/driver.cpp:47). That is how 300 in an INT8 column gets skipped with a warning. Could cast<timestamp_ns_t> catch the OverflowException, as tryCast already does, and rethrow a ConversionException naming the value and TIMESTAMP_NS? That matches CAST('300' AS INT8), which raises a conversion error for string input. The string case in cast_bounds would change to that error, and a COPY test with IGNORE_ERRORS=true would pin it. One more thing: the PR body says COPY used to raise a ConversionException. On main the row loaded with the wrapped value.
| } | ||
|
|
||
| template<> | ||
| void TryCastStringToTimestamp::cast<timestamp_ms_t>(const char* input, uint64_t len, |
There was a problem hiding this comment.
getEpochMilliSeconds and getEpochSeconds divide, so they can't throw. The try/catch in tryCast<timestamp_ms_t>/tryCast<timestamp_sec_t> never fires, and the new cast<timestamp_ms_t>/cast<timestamp_sec_t> specializations behave exactly like the generic cast. The PR body calls them symmetry, but could they go anyway, leaving only the timestamp_ns_t specializations?
|
Looks good. I'll address the two @zahariash comments in another commit in the PR and merge. |
98e072c to
e9c3317
Compare
…; drop dead ms/sec specializations
Strict string casts to TIMESTAMP_NS wrapped an out-of-range value in an
OverflowException, which aborts COPY ... (IGNORE_ERRORS=true) for the whole
file because the CSV reader only skips rows on ConversionException. Catch
the overflow from getEpochNanoSeconds and rethrow a ConversionException
naming the value and TIMESTAMP_NS, matching e.g. CAST('300' AS INT8).
tryCast<timestamp_ns_t> already returns failure there.
Drop the dead code flagged in review: getEpochMilliSeconds/getEpochSeconds
only divide and cannot throw, so remove the try/catch from the ms/sec
tryCast specializations and delete the ms/sec strict-cast specializations
(the generic cast<T> behaves identically).
Tests: cast_bounds pins the new Conversion error for the string path
(TIMESTAMP/DATE-intermediate paths still raise OverflowException);
copy_to_csv pins that COPY with IGNORE_ERRORS skips the overflowing row
with a warning instead of aborting.
e9c3317 to
7017f9b
Compare
…h helpers Address review feedback on PR #1584: - Make Multiply checked integral operations accept inputs by value/const so constexpr/const operands (e.g. Interval scale constants) select the checked implementation instead of the unchecked primary template. - Simplify Timestamp::getEpochNanoSeconds to rely on the safe Multiply API. - Route Date::getEpochNanoSeconds through checked Multiply operations to avoid raw wrapping multiplication. - Make string-to-timestamp tryCast paths return false on overflow instead of propagating exceptions, preserving try-cast failure semantics. Note: build/tests not executed in this environment (cmake/ninja unavailable).
Fixes #1583.
Problem
Casting a timestamp outside the nanosecond-representable range to
TIMESTAMP_NSsilently wrapped instead of failing:The string, timestamp, and date cast paths to
TIMESTAMP_NSroute throughTimestamp::getEpochNanoSeconds, which multiplies the microsecond value by 1000 viafunction::Multiply::operation. The overflow-checkedint64_tspecialization ofMultiplytook its operands as non-const lvalue references (int64_t&), while the primary template performed unchecked multiplication. When operands were const-qualified or otherwise not bindable as non-const lvalues (e.g.timestamp.valueviaconst timestamp_t&, orconstexprscale constants such asInterval::NANOS_PER_MICRO), overload/template selection fell back to the unchecked primary template, allowing out-of-range products to wrap silently. The same trap was live inTimestamp::fromEpochMilliSeconds/fromEpochSecondsand inDate::getEpochNanoSeconds.Fix
Multiply's checked integral operations now take their input operands by value (with a const-friendly generic template for non-integral cases), so const/constexpr/lvalue/rvalue operands select the checked implementation — defined on top ofTryMultiplyinmultiply.cpp— without relying on caller-side local copies. This is the mechanism that covers every epoch helper above, includingDate::getEpochNanoSeconds, which is kept (it isLBUG_API-exported public API) and now routes both of its multiplications through the checked overload instead of a raw wrapping multiply.CAST(string AS TIMESTAMP_NS/MS/SEC)uses dedicatedTryCastStringToTimestamp::cast<>specializations that bypasstryCast, so anOverflowExceptionfrom the checked multiply propagates with its specific message instead of collapsing into a generic parse error. The internal try-cast path (tryCast<>) returnsfalseon overflow — catchingOverflowExceptionspecifically — so try-cast consumers see failure (NULL / unselected field) rather than an exception, matching the engine's try-cast contract.tryCast; the ms/sec conversions divide and cannot overflow, so their specializations are behaviourally neutral symmetry with the ns one.cast<>specializations serve COPY/CSV string loading, so loading an out-of-range string into aTIMESTAMP_NScolumn now raisesOverflowExceptioninstead of the parseConversionException. Both errors abort the load; the new one names the actual problem.Tests
test/test_files/cast/cast_bounds.test: thetimestamp_nscase pins the strict side — overflow on the string, timestamp, and date cast paths toTIMESTAMP_NSin the positive direction and on the timestamp path in the negative direction, with in-range controls. A newtimestamp_ns_trycastcase pins the try-cast side: this engine has noTRY_CASTSQL syntax, so the pin goes through string→UNION casts, whose field selection consumestryCast— an overflowing value falls through to aSTRINGunion field instead of raising, an in-range value selects theTIMESTAMP_NSfield, and a single-field union reports the conversion failure rather than the overflow error.test/test_files/function/timestamp.test:EpochMsOverflowcases pinepoch_msat both int64 extremes.test/common/timestamp_test.cpp: a newTimestampTests.FromEpochSecondsOverflowunit case pinsTimestamp::fromEpochSecondsat both int64 extremes with the exact checked-multiply message (Value … * 1000000 is not within INT64 range.). It is a unit case because no SQL surface can construct aTIMESTAMP_SECvalue large enough to overflow that multiply: every SQL construction divides an int64 microsecond timestamp byMICROS_PER_SECfirst, which bounds the seconds value below the overflow threshold.a68595c2…; the authorship rewrite described below is tree-identical, so these results are for the tree being pushed), incremental release build:cast~cast_bounds.*9/9,TimestampTests.*5/5 (including the newFromEpochSecondsOverflow),function~timestamp.FunctionTimestampPASS, and the cast/timestamp sweep (cast~*,exceptions~types~timestamp.*,copy~copy_timestamp.*,reader~parquet~timestamp.*) 46/46. The full test suite was not run locally. Earlier, at the pre-remediation headd6027ca9(a different, earlier tree), the same sweep passed 45/45 locally and theminimal test (linux)CI job succeeded; that CI run's overall conclusion wascancelledbecause its extension-test job was cancelled when the branch head moved, and no CI has run on the final tree yet — it will run on this push.Review notes
pratik-saptarshi), as requested in review.Multiplyoverloads, not through callers passing by-value arguments to the old reference-based specialization.— Musebot-Astro