Repository navigation
fix(stats): greeks, r_squared and treynor_ratio break on any gap - #553
WatchTree-19 wants to merge 1 commit into
Conversation
Since missing returns stay NaN (ranaroussi#546), a single gap made np.cov and linregress return NaN for the whole series: greeks() reported beta and alpha as 0 through its fillna(0), r_squared() returned NaN, and treynor_ratio() fell back to 0 with a "beta is zero" warning. Estimate over the dates on which both the strategy and the benchmark have an observation.
…, #553) Estimate greeks(), r_squared()/r2() and treynor_ratio() over the dates on which both the strategy and the benchmark were observed. 0.0.83 stopped filling gaps with 0, and np.cov/linregress propagate NaN through the whole estimate, so a single missing day reduced beta and alpha to 0 and R-squared to NaN. greeks() reported that zero without complaint, since its trailing fillna(0) could not tell "undefined" from "no sensitivity". Reported with a fix by @WatchTree-19 in #553. Revert kelly_criterion() to the classic fixed-odds fraction p - q/b. 0.0.82 divided it by the average-loss magnitude, on the argument that a fraction which does not move when the series is rescaled must be wrong. That quantity is the growth-optimal leverage for a per-period P&L, not the Kelly fraction: on daily returns it reaches double and triple digits, and reports showed figures like 10792% where the fraction reads 21% (#552). The fixed-odds fraction depends only on the odds by construction, so its scale-invariance is a property rather than a defect. Bumps version to 0.0.84.
|
Thank you — this was a real regression, and your diagnosis was exactly right. Shipped in v0.0.84. One correction to the report: this was in a release. I introduced it in 0.0.83 when I stopped filling missing observations with Reproducing your snippet on 0.0.83: And on 0.0.84: I implemented your approach rather than merging the branch directly, because I was also reverting the Kelly change in the same release and wanted one coherent diff; the logic is yours. It is factored into a I also swept every benchmark-taking function in One subtlety worth recording, which your approach gets right: estimating over the shared observations is deliberately not the same as calling Closing as shipped. This is the second report of yours that held up under scrutiny (#545 was the other) — genuinely useful, please keep them coming. |
Thanks for picking up #546 and #547 so quickly. Running the new gap handling on main (5a751dd) against the benchmark metrics, one thing breaks: since a missing return now stays
NaN, a single gap makesnp.covandlinregressreturnNaNfor the whole series.greeks()reports beta and alpha as0, because theNaNgoes through its.fillna(0)r_squared()/r2()returnNaNtreynor_ratio()falls back to0with the "beta is zero" warningThis is only on main, not in any release.
reports.metrics()is not affected, because it fills the frame with 0 before calling these.The fix estimates over the dates on which both the strategy and the benchmark have an observation, which is the same "skip the gap" rule #546 applies to the single-series statistics. It is a small helper,
_paired_observations(), called fromgreeks()andr_squared();treynor_ratio()andr2()go through those. Series without gaps give exactly the same numbers as before.Tests:
TestBenchmarkMetricsSkipGapsintests/test_regressions.py, following the pattern ofTestMissingObservationsAreNotZeroReturns. Each metric on a gapped series must match the same metric on the observed dates, andtreynor_ratio()must not warn or return 0. All three fail on main and pass with the fix. Full suite: 207 passed.ruff checkandruff format --checkare clean. I added a line to the 0.0.83 section of the changelog.