Skip to content

Refuse to autogenerate a NOT NULL column filled only by a dataclass default_factory - #7815

Open
masenf wants to merge 7 commits into
mainfrom
fix/default-factory-not-null
Open

masenf wants to merge 7 commits into
mainfrom
fix/default-factory-not-null

Conversation

@masenf

@masenf masenf commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

All Submissions:

  • Have you followed the guidelines stated in CONTRIBUTING.md file?
  • Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • This change requires a documentation update

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

Stacks on #7797 (fix/autogenerate-never-drops-tables, rebased onto its final head 93beac9). The template reflex/.templates/alembic/env.py and the alembic tests change in both PRs. Until #7797 merges, GitHub also shows its commits here. Only the top two commits belong to this PR: "Refuse a NOT NULL column filled only by a dataclass default_factory" and "Look up the added column by key and its property directly". It does not depend on #7809, which handles FINDING-005/006/017. The new test sits next to the callable-default test, so the two PRs do not conflict.

Fixes FINDING-012 from the reflex 0.11.0a1 pre-release testing (artifacts on branch claude/vibrant-pasteur-3921g3, prerelease-testing/runs/2026-11-11-v0.11.0a1/).

Defect

Take a non-empty table and add score: Mapped[int] = mapped_column(default_factory=lambda: 7) on a MappedAsDataclass model. Autogenerate wrote add_column(... nullable=False) with no server_default, so reflex db migrate failed with NOT NULL constraint failed. When the revision rebuilt the table in sqlite batch mode, alembic also left an _alembic_tmp_* table behind that broke the next migrate.

The cause is the env.py template's add-column hook, which only reads Column.default. A dataclass default_factory is stored on the dataclass field only, so Column.default is None.

Fix

The hook now handles a NOT NULL added column that has no server_default and no Column.default like this:

  • It looks the column up in the mapped dataclasses of the registered or detected model bases. It uses table.c[op.column.key] (keyed by Column.key, which can differ from the name), then mapper.get_property_by_column(column).key, then __dataclass_fields__. A mapper that doesn't map the column, as with a single-table-inheritance parent, is skipped.

  • If the column's attribute has a default_factory, the hook raises a CommandError rather than guess a value:

    Cannot add the NOT NULL column 'item.score': its default_factory gives the rows the table already holds no value. Give the column a server_default=..., or a literal default= or insert_default=, or make it nullable.

I put the refusal at autogenerate time because it fails before a broken revision is written, so no _alembic_tmp_* leftover can come from this path. It also fires in alembic check, so the startup schema check reports it.

Docs: docs/database/tables.md gets a sentence about default_factory and migrations.

Known behaviour

This was confirmed by the release owner.

  • A callable Column.default is still evaluated once, not refused. Examples are sqlmodel Field(default_factory=...) and mapped_column(insert_default=callable). The value is taken when the revision is generated and becomes the server default for existing rows, as in 0.10.1. test_callable_default_is_evaluated_for_existing_rows covers it. Only a dataclass default_factory that leaves Column.default empty is refused.
  • A non-callable Column.default still becomes the server default, and a nullable column is never refused.

Test plan

  • New test test_default_factory_column_is_refused in tests/units/utils/test_migrations.py (real alembic and sqlite, MappedAsDataclass app). It failed before the fix: makemigrations exited 0 and wrote ..._factory.py. After the fix it checks:

    • makemigrations exits 1 with the message naming 'thing.score', and no revision is written.
    • Bare alembic check fails with the same message.
    • A column with a custom key, mapped_column('points', key='points_key', default_factory=...), is refused too, naming 'thing.points'. This case was added for the review thread and failed before the key fix.
    • A literal default=7 gets server_default=sa.text('7') and migrates the existing row.
    • A nullable default_factory column is accepted.
  • uv run pytest tests/units --cov (without tests/units/reflex_bench/drivers/test_browser.py, since there is no Chromium in this sandbox): 15645 passed, 66 skipped, coverage 81.44%. After the rebase onto 93beac9, the migration and model tests pass again: 43/43 locally, and test_migrations 30/30 on SQLite 3.45.

  • uv run ruff check . and uv run ruff format --check . are clean. uv run pyright reflex tests gives 0 errors.

  • End to end with the verifier's verify_dbm/v3_default_factory.sh, run on an editable install:

    Variant Result
    df makemigrations rc 1 with the message, no revision; migrate rc 0 and its schema check logs the refusal; row untouched; no _alembic_tmp_* table
    dfbatch same as df
    dflt server_default=sa.text('7'), rows [(1, 'a', 7)]
    ins server_default=sa.text('7'), rows [(1, 'a', 7)]
    rxdf server_default=sa.text('7'), rows [(1, 'a', 7)]

Notes

  • The fix lives in the env.py template. Apps upgrading from 0.10.x get it when their stock env.py is replaced. An env.py that 0.11.0a1's reflex db init wrote keeps the old hook until it is moved aside and re-initialized.
  • I did not add cleanup of an _alembic_tmp_* table left by an already-written broken revision.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi

An app whose models use a SQLAlchemy DeclarativeBase, the style the docs
recommend, had to call rx.ModelRegistry.register(Base). Without it the
alembic target metadata held none of its tables, so `reflex db
makemigrations` wrote a revision dropping every one of them and `reflex db
migrate` deleted their data, while `alembic check` and the startup hint
pointed at makemigrations.

ModelRegistry.get_models now adds every module-level DeclarativeBase
subclass with its own metadata until a base other than SQLModel's is
registered, so registering bases still restricts migrations to them, e.g.
for an app with a second database. A base is detected only while its module
holds it under its name, so one replaced by a reload of its module is not.
The walk runs when the metadata is built, which is cached as before.

The env.py template also warns when an autogenerated revision drops tables,
naming them, so that the remaining ways to drop a table by accident (a base
that is not detected, such as a legacy declarative_base(), or a base left
unregistered beside a registered one) are visible before `reflex db migrate`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
Say that detection applies only until a base is registered, that it covers
module-level bases the app or its pages import, a library's included, and
not one created inside a function, and update the key-takeaways bullet that
still required registration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf
masenf requested review from a team and Alek99 as code owners November 11, 2026 09:13
@masenf masenf added this to the v0.11.0 milestone Nov 11, 2026
@masenf masenf added the on deck PRs lined up to review / merge next label Nov 11, 2026
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@greptile-apps

greptile-apps Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge.

Summary

The PR refuses unsafe column additions when a dataclass default_factory supplies no database default.

  • Reflex finds imported model bases even when apps skip registration.
  • Migrations stop on a required column filled only by a factory.
  • Generated revisions warn when they remove tables.

Reviews (5) · Last reviewed commit: "Look up the added column by key and its ..." · Reviewed by Greptile

Comment thread reflex/.templates/alembic/env.py Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 14 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread docs/database/overview.md Resolved
Comment thread reflex/model.py Resolved
Comment thread reflex/.templates/alembic/env.py Resolved
@masenf

masenf commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

app-harness-playwright (redis, 3.11, 1) is red because of an ERROR at teardown of tests/integration/tests_playwright/test_client_error.py::test_state_mismatch_prompts_reload[prod-chromium]. The test itself passed on all three reruns. The raise_console_error fixture caught the [Reflex Frontend Exception] Skipped state update for unknown substate(s) log that this test provokes on purpose, so the log landed in the wrong capture window. The test exercises a frontend/backend substate mismatch and touches none of alembic, env.py, DeclarativeBase or reflex db, which is all this PR (and #7797 under it) changes. The same job passed on #7797 and #7809. The workflow is still running other jobs, so the failed job can't be re-run yet. It will be re-run once when the run finishes, or by the next push to this branch.


Generated by Claude Code

@codspeed

codspeed Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 5.05%

2 warnings

⚠️ 17 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 2 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

❌ 2 regressed benchmarks
✅ 149 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ test_route_matching[10] 50.4 µs 53.5 µs -5.85%
❌ test_process_event_burst_independent_tokens[1] 2.7 ms 2.8 ms -4.25%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing fix/default-factory-not-null (3e9f0a8) with main (2d71015)

Open in CodSpeed

@masenf

masenf commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator Author

CodSpeed is red here (test_route_matching[1000] -5.8%, test_process_event_burst_independent_tokens[1] -4.0%), but this PR doesn't cause it. CodSpeed flags both as measured on different hardware, and both show the same 4–6% drop on every PR in this batch. This PR (and #7797 under it) changes only migration generation: the alembic env.py template, reflex/model.py base discovery and reflex/utils/migrations.py. None of those runs in route matching or event processing. CodSpeed is not a required check; leaving it for acknowledgement on CodSpeed.


Generated by Claude Code

@masenf
masenf force-pushed the fix/default-factory-not-null branch from 3faf080 to 342353a Compare November 11, 2026 09:29
masenf pushed a commit that referenced this pull request Nov 11, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
…e the base is made

A factory-made base bound under its own class name is detected; only a
base its module holds under a different name, or one made with
declarative_base(), is not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf
masenf force-pushed the fix/default-factory-not-null branch from 342353a to fcfb9ff Compare November 11, 2026 09:32
`mapped_column(default_factory=...)` on a MappedAsDataclass model is the
dataclass's default alone: the Column has no default, so the env.py
add-column hook gave the revision no server_default and adding the
column to a table with rows failed with "NOT NULL constraint failed"
(on sqlite batch mode leaving an `_alembic_tmp_*` table behind).

The hook now looks the column up in the mapped dataclasses and, for a
NOT NULL column whose only default is a default_factory, raises a
CommandError naming the table and column before any revision is
written (and in `alembic check`). Column defaults, literal or callable,
still become the server default as before; a nullable column needs none.

Fixes FINDING-012 of the 0.11.0a1 pre-release testing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
`Table.c` is keyed by `Column.key`, so a column with an explicit key
other than its name was missed and its default_factory not refused.
The property is now found with `mapper.get_property_by_column` and the
dataclass field with `__dataclass_fields__`, instead of scanning
`mapper.columns` and `dataclasses.fields`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dxjadq3qkhKnqZR2b9WBFi
@masenf
masenf force-pushed the fix/default-factory-not-null branch from fcfb9ff to 3e9f0a8 Compare November 11, 2026 09:34

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

on deck PRs lined up to review / merge next

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants