Repository navigation
tflite: ship sm-music + sm-sfx as w8a8 rung ladders (default w8a8) - #118
Merged
Merged
Conversation
Apply the from-repo cache-safe rung technique (medium PR #112/#114) to both small DiTs. 8-rung ladder capped at the 2-min max (valid_T_lat(120)=1292): 8,48,96,192,416,704,1056,1292. GCE_OUT is derived from the config (6*embed_dim = 6144 for the small DiTs vs 9216 medium); rung_dit.py reads the gc width from the model, so one runtime handles both. Defaults flip to w8a8 for every family (CJ's call 2026-10-10, ear-approved): - DEFAULT_DIT_PRECISION[sm-*] fp32 -> w8a8; DIT_PRECISIONS_BY_FAMILY[sm-*] -> ("fp32","w8a8"), retiring the varlen w16a32/w8a32/w8a8-dyn -> legacy/ - install bundle for small downloads dit_w8a8 + SAME-S (~0.7 GB, was ~2.0 GB) - --lora still auto-falls-back to fp32 (the int8 w8a8 rung can't be merged) Verified: cache-safe 999 dB (gce-outside holds for small), per-forward vs fp32 sm-music 23.3 dB/cos 0.9977, sm-sfx 25.3 dB/cos 0.9985; peak RAM flat ~0.8 GB vs the old varlen's 3.8 GB at 2 min (4.5x less). Ear-approved A/B. New build driver build/build_dit_small.sh (leaves medium build_dit.sh untouched). The PRECISIONS union is now {fp32, w8a8} across all families; docs/gradio/CLI help updated accordingly. Pairs with the HF swap. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applies the cache-safe rung technique (medium #112/#114) to both small DiTs, and flips their default to w8a8 (you've ear-approved — board at the swap). Pairs with the HF swap moving the old small varlen tiers to
legacy/.What changes
fp32/w16a32/w8a32/w8a8-dynw8a8(default) /fp32dit_fp32(varlen) + SAME-S, ~2.0 GBdit_w8a8(rung) + SAME-S, ~0.7 GBThe old varlen tiers retire to
tflite/sa3-sm-*/legacy/on HF.fp32(rung) stays for reference/LoRA;--loraauto-selects it.Implementation
valid_T_lat(120)=1292):8,48,96,192,416,704,1056,1292.export_dit.pyis now family-aware (SA3_DIT_FAMILY) and derivesGCE_OUTfrom the config (6·embed_dim = 6144 small / 9216 medium);rung_dit.pyreads the gc width from the model, so one runtime handles both. Medium's build is untouched (default family).build/build_dit_small.sh(the mediumbuild_dit.shis unchanged).PRECISIONSunion is now{fp32, w8a8}across all families — docs, gradio, and CLI help updated; the retiredw16a32/w8a32/w8a8-dynare no longer offered.Verified
tflite==torch(max|d| ~1e-3), gcond ~1e-6.🤖 Generated with Claude Code