Skip to content

Clarify average odds cancellation in metric explanations - #573

Open
Hanabi9248 wants to merge 4 commits into
Trusted-AI:mainfrom
Hanabi9248:codex/average-odds-explanation
Open

Hanabi9248 wants to merge 4 commits into
Trusted-AI:mainfrom
Hanabi9248:codex/average-odds-explanation

Conversation

@Hanabi9248

@Hanabi9248 Hanabi9248 commented Sep 11, 2026 •

Copy link
Copy Markdown

Fixes #528.

Opposite-signed FPR and TPR differences can cancel: a four-row example returns average odds difference 0 while both average absolute odds difference and equalized odds difference are 1. Correct the two API docstrings and the text/JSON explanations so zero is no longer presented as sufficient for equality of odds. Metric formulas are unchanged.

The new explanation regression fails before the change. Local validation on Python 3.11:

  • 50 tests pass across the explanation regression, classification metrics, structured datasets, sample distortion, regression metrics, and sklearn metrics (using the Adult dataset).
  • Flake8 critical-error checks pass on the five changed files.
  • A focused Sphinx build of the affected APIs passes with warnings treated as errors. The full documentation site also builds; its 27 warnings match a build of the unmodified base.

The full optional-dependency Python/R suite has not been run locally. The GitHub Actions workflow is awaiting maintainer approval.

Signed-off-by: Hanabi <317387557+Hanabi9248@users.noreply.github.com>
Signed-off-by: Hanabi <317387557+Hanabi9248@users.noreply.github.com>
Signed-off-by: Hanabi <317387557+Hanabi9248@users.noreply.github.com>
Signed-off-by: Hanabi <317387557+Hanabi9248@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

average_odds_difference metric is wrongly represented as an equalized odds relaxation

1 participant