Repository navigation
[BUG] roll back state when set_params fails validation - #589
OfficialAbhinavSingh wants to merge 3 commits into
Conversation
set_params writes parameter values to self via setattr before calling reset(), which is what re-runs __init__ to validate them. If __init__ rejects a value, the raise happens after the write, so the rejected value survives on self -- a state __init__ could never have produced. get_params, clone, and later set_params calls then operate on that invalid state. Snapshot self.__dict__ before the write loop in set_params and restore it if the loop or the following reset() call raises. Adds a regression test with a fixture that actually validates in __init__, since the existing test_get_params_after_set_params never exercised the raise path (its fixture class never raises). Fixes sktime/sktime#10695
fkiraly
left a comment
There was a problem hiding this comment.
Thanks for the contribution. I think this is potentially something to add, but the current implementation has major issues.
- storing the current state can multiply memory requirements. Imagine expensive model weights, which have to exist in two copies.
- I think "silent failure" is the wrong handling of this case. I think there should be an explicit exception at the end.
Further:
- I to not think the entire code needs to be in the try/except loop - only
reset. - a clear exception message is missing, the user will not know what has happened
|
Thanks for the review, taking the points in order. Only Memory — the snapshot is "Silent failure" / missing message — the previous version did re-raise (bare chained One point for your call: Rationale for rollback vs message-only is in sktime#10695. |
|
Went with keeping the original exception type, in a1200bb, so Worth noting the skbase suite did not catch this either way, since the fixture classes do not validate in 1618 passed locally. |
|
closing this, after the discussion in sktime#10695 I don't think the rollback is worth the extra complexity in |
Reference Issues/PRs
Fixes sktime/sktime#10695. See also sktime/sktime#10821, an earlier attempt at this issue that patched
set_paramson thesktimeside and was closed unmerged, since the bug is inBaseObject.set_paramshere inskbase, not insktime.What does this implement/fix? Explain your changes.
BaseObject.set_paramswrites new parameter values toselfviasetattrbefore callingself.reset(), which is what actually re-runs__init__to validate them. If__init__rejects a value, the raise happens after the write, so the rejected value survives onself-- a state__init__could never have produced.get_params,clone, and any laterset_paramscall then operate on that invalid state.Fix snapshots
self.__dict__before the parameter-writing loop inset_params, and restores it if anything in that loop or the followingself.reset()call raises, then re-raises the original exception. Confirmed against the estimator that surfaced this insktime(DilationMappingTransformer):set_params(dilation=0)now leavesdilationat its prior valid value instead of0, andclone()succeeds afterward.Does your contribution introduce a new dependency? If yes, which one?
No.
What should a reviewer concentrate their feedback on?
self.__dict__is the right mechanism, versus validating before writing toselfin the first place -- the issue write-up considered both directions and this seemed the smaller, more general change, since it doesn't require touching every__init__.test_get_params_after_set_paramstest intest_base.pyalready asserts rollback semantics under fuzz values, but the fixture class it uses (Parent) never raises in__init__, so that assertion was never actually exercised. Added a new test with a fixture that does validate,test_set_params_rolls_back_state_on_invalid_value.Any other comments?
An LLM was used as a search and navigation tool to help trace the bug to its root cause in
set_params/reset()and to draft this description. Every claim above -- the repro, the root cause, the before/after behavior, and the test counts -- was produced by running the code locally on both sides of the fix, including a run of the new test against the pre-fix source to confirm it fails without the change.PR checklist
For all contributions
the PR topic is related to enhancement, CI/CD, maintenance, documentation, or a bug.
For code contributions