Skip to content

Report kept env.py failures and skip empty autogenerated revisions - #7809

Merged
FarhanAliRaza merged 5 commits into
mainfrom
fix/kept-env-py-errors
Nov 11, 2026
Merged

FarhanAliRaza merged 5 commits into
mainfrom
fix/kept-env-py-errors

Conversation

@masenf

@masenf masenf commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

All Submissions:

  • Have you followed the guidelines stated in CONTRIBUTING.md file?
  • Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • This change requires a documentation update

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

#7797, which this PR stacked on, is merged; the PR is rebased onto main.

Fixes FINDING-005, FINDING-006 and FINDING-017 from the reflex 0.11.0a1 pre-release testing (artifacts on branch claude/vibrant-pasteur-3921g3, prerelease-testing/runs/2026-11-11-v0.11.0a1/). FINDING-012 is fixed separately in #7815, which is also stacked on #7797.

Defect

  • FINDING-005 (MEDIUM, regression): Say a 0.10.x user edited alembic/env.py by hand (target_metadata = SQLModel.metadata) and left the driver:// placeholder URL in alembic.ini. 0.11 keeps that file, because legacy detection only looks for the literal target_metadata = None, and then runs it. The result: reflex run dies at startup with NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:driver, reflex db status/migrate/makemigrations exit 1 with a raw traceback, and reflex db init only says "already initialized". 0.10.1 never ran env.py and worked. The stock env.py also calls fileConfig(alembic.ini), which disables every reflex.* logger, so Reflex's own messages vanished for the rest of the process.
  • FINDING-006 (MEDIUM, regression): With such a kept env.py, every reflex db makemigrations wrote an empty pass revision. rx.model.migrate(autogenerate=True) at import chained several of them on each dev start, because only the template env.py skipped empty revisions.
  • FINDING-017 (LOW): A second legacy env.py replacement silently overwrote alembic/env.old.

Fix

  • migrations.env_kind() sorts env.py into one of three kinds:

    • template: it contains the docstring marker Generated by reflex db init . Every template since 0.11.0a1 has it, so 0.11.0a1 templates count as the template.
    • legacy: it contains target_metadata = None.
    • custom: anything else.

    check_env() logs a warning once per process (via the dedupe log filter) when env.py is custom: "alembic/env.py is not the Reflex template, so what the template adds may be missing: ... Move it aside and run reflex db init to adopt the template; existing revisions are kept."

  • Any exception from an alembic run, not only CommandError, is now reported by migrations.report_failure(). The message is Alembic failed with <Type>: <message>. When the failure comes from a custom env.py itself (the innermost traceback frame in the alembic script directory is env.py), it adds the hint "alembic/env.py is not the Reflex template: move it aside and run reflex db init to write the current template." and the traceback goes to the debug log. Any other failure, such as a failing revision script under either env.py, is logged with its traceback and without the hint. Each caller handles it differently:

    • check_schema_up_to_date (startup and after db migrate) logs it, and the app still starts. This restores the 0.10.1 behaviour of starting.
    • _alembic_errors (reflex db ...) exits 1 with the message.
    • migrate() logs it and returns False when env.py is custom. Under the template it still raises: there the error is real, and a script calling migrate() must see it.
  • migrations.run() restores the root handlers and level after each run, and re-enables the loggers that an env.py's fileConfig disabled.

  • migrations.autogenerate() runs alembic check first and writes a revision only on AutogenerateDiffsDetected. This is what 0.10.1 and the deprecated alembic_autogenerate already did. reflex db makemigrations, migrate(autogenerate=True) and alembic_autogenerate now all use it, so the result no longer depends on env.py. When nothing changed, makemigrations logs "No model changes detected; no revision was written."

  • update_legacy_env() saves the backup under the first free name: env.old, then env.old.1, and so on.

  • Docs: docs/database/overview.md (upgrade note and the empty-revision sentence).

Known behaviour

These were confirmed by the release owner.

  • With a custom env.py, migrate() logs and continues. An import-time rx.model.migrate() logs any non-CommandError failure of a custom env.py and returns False instead of raising, so the app still starts. That includes a failing revision script, which is logged with its traceback. Explicit callers who need the error can check the False return. Under the template env.py, migrate() raises as before.
  • No hint for a failing template env.py. The message names the exception only. Moving the template aside and re-running reflex db init would just write the same file again.

Test plan

New tests in tests/units/utils/test_migrations.py use real alembic and sqlite. They are parametrized over sqlmodel and sqlalchemy where the fixture supports it. All of them failed before the fix:

FAILED test_custom_env_failure_is_reported_with_the_fix        (NoSuchModuleError raised out of get_compiled_app)
FAILED test_template_env_failure_is_reported_without_the_hint  (RuntimeError raised out of reflex db migrate)
FAILED test_custom_env_warns_once_that_it_is_not_the_template[sqlmodel|sqlalchemy]
FAILED test_custom_env_writes_no_empty_revision[sqlmodel|sqlalchemy]   (2nd makemigrations: empty revision 1 not applied -> exit 1; no log captured, fileConfig had removed the handlers)
FAILED test_custom_env_import_time_autogenerate_writes_no_empty_revision   (assert 4 == 1 revisions)
FAILED test_second_legacy_env_keeps_the_first_backup             (env.old held "# third")

All of them pass after the fix.

test_revision_script_failure_keeps_its_traceback[template|custom] covers a revision script whose upgrade() raises: the error is logged with its traceback and without the hint. Both cases failed before the third commit.

CI fix (second commit): the first push failed test_custom_env_writes_no_empty_revision on the Windows legs and on ubuntu 3.14, with near "ALTER": syntax error. Under the stock env.py, which has no render_as_batch, that test's model change produced a plain ALTER TABLE thing ALTER COLUMN t1 DROP NOT NULL. Only recent SQLite builds accept that statement: the SQLite 3.53 in uv's Pythons does, while the system Pythons on those runners have an older build. I reproduced it locally with SQLite 3.45.1 (/usr/bin/python3.12). The test now adds a nullable column and asserts the revision has no alter_column. It passes on SQLite 3.45 and 3.53. Product code is unaffected: the Reflex template uses batch mode.

  • uv run pytest tests/units --cov (without tests/units/reflex_bench/drivers/test_browser.py, since there is no Chromium in this sandbox): 15652 passed, 66 skipped, coverage 81.45%. After the rebase onto 93beac9, the migration and model tests pass again: 50/50 locally, and test_migrations 37/37 on SQLite 3.45.
  • uv run ruff check . and uv run ruff format --check . are clean. uv run pyright reflex tests gives 0 errors.
  • End to end: I adapted the verifier's verify_dbu/scripts/run_all.sh to an editable install of this branch, using form-designer seeded on 0.10.1:
    • Stock env.py: reflex run comes up, env.old matches 0.10.1, and no revision is written.
    • Edited env.py with the placeholder URL, with or without import-time migrate(): reflex run comes UP (8–10 s) with no traceback, and logs the warning plus Alembic failed with NoSuchModuleError ... move it aside and run reflex db init. db status/migrate/makemigrations exit 1 with the same message and no traceback. db init shows the warning before "already initialized". The remedy (move it aside, then db init) works.
    • Edited env.py with a working URL: two makemigrations add 0 revisions (before: 2). migrate(autogenerate=True) at import plus reflex run adds 0 (before: 3). The railway-style env.py adds 0 (before: 2).
    • Second legacy replacement: env.old keeps the custom edit and env.old.1 holds the stock file.

Notes

  • makemigrations now runs env.py twice when there are changes (check, then revision), as 0.10.1 did.
  • The "not the template" warning is once per process, not persisted. reflex run shows it in the main process and again in the backend worker.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

@masenf
masenf requested review from a team and Alek99 as code owners November 11, 2026 09:04
@masenf masenf added this to the v0.11.0 milestone Nov 11, 2026
@masenf masenf added the on deck PRs lined up to review / merge next label Nov 11, 2026
@greptile-apps

greptile-apps Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

This PR reports failures from kept env.py files, skips empty autogenerated revisions, restores logging after Alembic runs, and preserves earlier backups.

  • Apps keep starting when a custom migration file fails.
  • Migration commands write revisions only when models changed.
  • Repeated upgrades keep each stock migration file.

Reviews (7) · Last reviewed commit: "Build the expected revision path with Pa..." · Reviewed by Greptile

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 18 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/database/tables.md Resolved
Comment thread reflex/model.py Resolved
Comment thread reflex/utils/prerequisites.py Resolved
Comment thread reflex/.templates/alembic/env.py Resolved
Comment thread reflex/reflex.py Resolved
@codspeed

codspeed Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 4.64%

1 warning

⚠️ 18 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 1 improved benchmark
✅ 150 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ test_process_event_burst_independent_tokens[1] 2.9 ms 2.7 ms +4.64%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix/kept-env-py-errors (9e53d00) with main (3184cad)

Open in CodSpeed

masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf
masenf force-pushed the fix/kept-env-py-errors branch from 71cefac to e5930a6 Compare November 11, 2026 09:29
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
Comment thread docs/database/overview.md Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="news/+detect-declarative-bases.breaking.md">

<violation number="1" location="news/+detect-declarative-bases.breaking.md:1">
P3: This breaking note overstates auto-detection: Reflex does not migrate every loaded `DeclarativeBase`; it only detects eligible module-level bases and excludes function-local and legacy `declarative_base()` bases. Say “every eligible module-level `DeclarativeBase`” so users are not promised migration for bases they still need to register.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

View guided diff | Turn on auto-fix | Re-trigger cubic

@masenf
masenf force-pushed the fix/kept-env-py-errors branch 2 times, most recently from 8464b7c to 75b8f29 Compare November 11, 2026 09:34
A hand-edited 0.10.x alembic/env.py, kept by 0.11 because it no longer
has the stock `target_metadata = None` line, is now executed by alembic.
With the placeholder URL of alembic.ini it raised NoSuchModuleError,
which crashed `reflex run` and gave `reflex db` a raw traceback, and
without the template's empty-revision skip every makemigrations and
`migrate(autogenerate=True)` wrote an empty revision.

- Classify env.py as template (docstring marker present since 0.11.0a1),
  legacy (stock alembic) or custom, and warn once per process about a
  custom one.
- Report any exception an alembic run raises: the startup schema check
  logs it and lets the app start, `reflex db` exits 1 with it, and
  `migrate()` logs it and returns False when env.py is custom (it still
  raises under the template). The custom-env hint says to move env.py
  aside and run `reflex db init`; the traceback is logged at debug.
- Restore the root handlers and re-enable the loggers a custom env.py
  disables through fileConfig, so Reflex's own messages still show.
- makemigrations and migrate(autogenerate=True) run `alembic check`
  first and write a revision only when it reports changes.
- A second legacy replacement keeps the earlier env.old, backing up to
  the first free name of env.old, env.old.1, ...

Fixes FINDING-005, FINDING-006 and FINDING-017 of the 0.11.0a1
pre-release testing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
test_custom_env_writes_no_empty_revision made t1 nullable and added a
NOT NULL t2 under a stock alembic env.py, which has no render_as_batch.
The revision then holds a plain `ALTER TABLE ... ALTER COLUMN ... DROP
NOT NULL`, which only recent SQLite builds accept: it passed with the
SQLite 3.53 of uv's Pythons and failed on the system Pythons of the
Windows runners and ubuntu 3.14 (SQLite 3.45 reproduces it). The step
now adds a nullable column, and the test asserts the revision has no
alter_column.

`reflex db makemigrations` also says when it writes no revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf
masenf force-pushed the fix/kept-env-py-errors branch from 75b8f29 to 38150cb Compare November 11, 2026 17:40
masenf pushed a commit that referenced this pull request Nov 11, 2026
…onto main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf

masenf commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

CodSpeed on 38150cb flags test_state_update_wire_serialization[model_1000] (−5.4%) and test_process_event_burst_independent_tokens[1] (−4.2%). Neither comes from this PR:

  • The reflex/model.py changes are only in the migration helpers. One function now calls migrations.autogenerate(...), and migrate() reports kept-env.py failures. Serialization and event processing never call either one.
  • The model_1000 payload is a pydantic BaseModel (tests/benchmarks/support/events.py, PayloadModel), not an ORM model.
  • test_process_event_burst_independent_tokens[1] shows the same −4.2% on Refuse to autogenerate a NOT NULL column filled only by a dataclass default_factory #7815, whose only source change is the alembic env.py template. CodSpeed also reports three benchmarks in this run measured on different hardware.

Generated by Claude Code

A failing revision script was reported in one line, without its file or
line, even under the template env.py; only REFLEX_LOGLEVEL=debug showed the
traceback. With a hand-edited env.py that works, the same failure also got
the "move env.py aside" hint, which points at the wrong file.

report_failure now hints and keeps the traceback at debug level only when
the innermost frame in the alembic script directory is a custom env.py.
Any other failure is logged with its traceback.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread reflex/utils/migrations.py Resolved
Comment thread reflex/utils/migrations.py Resolved
A SyntaxError in a hand-edited env.py is raised while alembic compiles
it, so no traceback frame is in the script directory and the "move
env.py aside" hint was lost. Its filename is now taken as the innermost
location. The frame is compared with the script directory's env.py
itself, so a revision script named env.py under versions/ keeps its
traceback and gets no hint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread tests/units/utils/test_migrations.py Resolved
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@FarhanAliRaza
FarhanAliRaza merged commit 1e20c11 into main Nov 11, 2026
168 checks passed
@FarhanAliRaza
FarhanAliRaza deleted the fix/kept-env-py-errors branch November 11, 2026 19:03
masenf added a commit that referenced this pull request Nov 11, 2026
…7809)

* Report kept env.py failures and skip empty revisions

A hand-edited 0.10.x alembic/env.py, kept by 0.11 because it no longer
has the stock `target_metadata = None` line, is now executed by alembic.
With the placeholder URL of alembic.ini it raised NoSuchModuleError,
which crashed `reflex run` and gave `reflex db` a raw traceback, and
without the template's empty-revision skip every makemigrations and
`migrate(autogenerate=True)` wrote an empty revision.

- Classify env.py as template (docstring marker present since 0.11.0a1),
  legacy (stock alembic) or custom, and warn once per process about a
  custom one.
- Report any exception an alembic run raises: the startup schema check
  logs it and lets the app start, `reflex db` exits 1 with it, and
  `migrate()` logs it and returns False when env.py is custom (it still
  raises under the template). The custom-env hint says to move env.py
  aside and run `reflex db init`; the traceback is logged at debug.
- Restore the root handlers and re-enable the loggers a custom env.py
  disables through fileConfig, so Reflex's own messages still show.
- makemigrations and migrate(autogenerate=True) run `alembic check`
  first and write a revision only when it reports changes.
- A second legacy replacement keeps the earlier env.old, backing up to
  the first free name of env.old, env.old.1, ...

Fixes FINDING-005, FINDING-006 and FINDING-017 of the 0.11.0a1
pre-release testing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

* Keep the custom env.py test within what every SQLite runs

test_custom_env_writes_no_empty_revision made t1 nullable and added a
NOT NULL t2 under a stock alembic env.py, which has no render_as_batch.
The revision then holds a plain `ALTER TABLE ... ALTER COLUMN ... DROP
NOT NULL`, which only recent SQLite builds accept: it passed with the
SQLite 3.53 of uv's Pythons and failed on the system Pythons of the
Windows runners and ubuntu 3.14 (SQLite 3.45 reproduces it). The step
now adds a nullable column, and the test asserts the revision has no
alter_column.

`reflex db makemigrations` also says when it writes no revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

* Keep revision script tracebacks and hint only at a failing env.py

A failing revision script was reported in one line, without its file or
line, even under the template env.py; only REFLEX_LOGLEVEL=debug showed the
traceback. With a hand-edited env.py that works, the same failure also got
the "move env.py aside" hint, which points at the wrong file.

report_failure now hints and keeps the traceback at debug level only when
the innermost frame in the alembic script directory is a custom env.py.
Any other failure is logged with its traceback.

* Hint at a custom env.py that fails to compile; match env.py by its path

A SyntaxError in a hand-edited env.py is raised while alembic compiles
it, so no traceback frame is in the script directory and the "move
env.py aside" hint was lost. Its filename is now taken as the innermost
location. The frame is compared with the script directory's env.py
itself, so a revision script named env.py under versions/ keeps its
traceback and gets no hint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

* Build the expected revision path with Path in the traceback test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Farhan <www.mfarvirus@gmail.com>
(cherry picked from commit 1e20c11)
masenf pushed a commit that referenced this pull request Nov 11, 2026
…last open

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf masenf added the release-train Issues found during agentic /prerelease-testing fanout, not necessarily validated bugs. label Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

on deck PRs lined up to review / merge next release-train Issues found during agentic /prerelease-testing fanout, not necessarily validated bugs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants