Skip to content

Publish Nemotron 3.5 and Ornith 1.5 evals and add family-only filtering - #159

Merged
antoinezambelli merged 3 commits into
mainfrom
az/lfm25-24b-evals
Oct 23, 2026
Merged

antoinezambelli merged 3 commits into
mainfrom
az/lfm25-24b-evals

Conversation

@antoinezambelli

@antoinezambelli antoinezambelli commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Adds 13,000 gen-3 evaluation runs for Nemotron 3.5 Bolt, Nemotron 3.5 Super VL, and Ornith 1.5 122B, with updated publication pins and regenerated reports.

Includes the corresponding batch configurations, serving recipes, sampling defaults, and dashboard model labels.

Adds an only button beside each Family checkbox: click to isolate that family, then click again to restore all families. Other filters remain unchanged.

Temporarily cap the Anthropic SDK below 2 in both extras; old/new SDK compatibility is tracked separately in #160. No auth implementation changes.

Validation:

  • Publication audit passed.
  • Full Python 3.13 unit suite passed against Anthropic SDK 0.86.0 and 1.17.0: 1,564 passed, 1 skipped on each.
  • Deterministic Proxy contract smoke against SDK 1.17.0: 19/19 passed.
  • SDK 2.2.0 locally reproduced the same three existing auth failures observed in CI.
  • Dashboard tests, lint, and build passed.
  • Dashboard embedded eval data unchanged by the UI update.
  • UI plan pressure test and correctness-only review clean; dashboard visually approved.

No version bump, PyPI release, or standalone Proxy release.

🤖 Generated with Codex, human reviewed.

@antoinezambelli
antoinezambelli merged commit 743e0f7 into main Oct 23, 2026
3 checks passed
@antoinezambelli
antoinezambelli deleted the az/lfm25-24b-evals branch October 23, 2026 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant