Skip to content

ci: stop flaky unit-test failures (Robolectric 429s, wall-clock budgets) - #17

Merged
R44VC0RP merged 2 commits into
mainfrom
ci-flakes
Nov 9, 2026
Merged

R44VC0RP merged 2 commits into
mainfrom
ci-flakes

Conversation

@R44VC0RP

@R44VC0RP R44VC0RP commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

CI failed randomly today and blocked merges and release cuts (CI-01). Two causes:

  • Robolectric downloads were rate-limited. Robolectric fetched its android-all-instrumented jars from Maven Central at test time; the JUnit XML from main run 42653023505 and PR fix(build): install on Android 14; microphone and location hardware optional #16 run 42663258902 shows HTTP 429 from repo1.maven.org for the API 35 jar. Gradle now resolves the two SDK jars the tests use (API 34 14-robolectric-10818077-i7, API 35 15-robolectric-13954326-i7, pinned in the version catalog to match Robolectric's DefaultSdkProvider) into a build dir, so the existing Gradle cache covers them, and Robolectric runs with robolectric.offline=true from that dir. A test asking for another SDK now fails fast instead of downloading.
  • Wall-clock budget asserts (e.g. elapsedMs < 50 in AgentFileTests) failed on busy runners. On CI (CI set) the timing limits aren't enforced; test names, correctness asserts, simulated-clock checks and polling timeouts are unchanged, and local runs keep the original budgets. CI is a declared test input so results aren't reused across the two.

Verified on the QA VM: full scripts/check.sh passes (6m 29s); the previously failing classes (ComposerAutocompleteTests, AppWideWorkTests, LabVoiceKeysTests, LiveHapticsFrameworkTests, plus an API 34 test) pass with Gradle --offline, an empty Maven repo and an unreachable download URL.

Replace downloads from inside Robolectric workers with a shared directory of the exact API 34 and 35 instrumented jars. Resolve each version separately so Gradle caches both; leave modules without Robolectric alone.

Verified on the Linux QA VM: Gradle --offline force-ran ComposerAutocompleteTests, AppWideWorkTests, LabVoiceKeysTests and SharedImageEncoderTests (19 passing) with an empty Maven repository and an unreachable Robolectric download URL.
Keep the iOS test names, correctness checks and original local limits. Skip only the elapsed-time assertions on shared CI runners, where unrelated load can exceed a budget. Track CI as a Gradle test input so local runs cannot reuse CI results.

Verified on the Linux QA VM: vm.sh check passed for all modules with local budgets enabled; AgentFileLoadingTests also ran all 10 cases with CI=true.
@R44VC0RP
R44VC0RP merged commit ba842ad into main Nov 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant