Repository navigation
Conversation
Backend unit coveragePASS · minimum combined coverage 98.00%. Tests: 11971 passed, 0 failed, 0 skipped. Source files: 400.
Unit coverage includes every backend workspace. PostgreSQL/Redis integration tests run separately. Commit: e68cfb4 · HTML, JSON, XML reports and job logs · attempt 1 |
1 of 2 tasks
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



What changes and why?
PostgreSQL assessment: this is not approved as a zero-overhead optimization under the user's constraint. The new provenance boolean adds approximately one WAL byte per newly inserted outbox event, and migration 0041 requests a transient exclusive DDL lock with a five-second wait deadline. Normal-workload SQL reductions do not erase those costs. Keep this PR in draft and
DEVFEED_SEARCH_CACHE_ISOLATION_ENABLED=false; merging or enabling is blocked pending review of that tradeoff and a representative canary.Every completed search-index batch currently invalidates the same public generation used by article, feed, and catalog responses. Split index publication into its own epoch, and make Typesense search-result cache entries depend atomically on both the public-data and search-index epochs.
The outbox now records conservative provenance: existing, trigger-created, manual and unknown events default to
public_change=true. Those batches retain public-data invalidation, including the existing fallback for raw SQL writers. Only indexer-generated projection/fanout pages explicitly carry false. Repair backfills invalidate public data once after enqueue commit and mark their pages projection-only. This preserves source withdrawal and unknown-maintenance safety while removing repeated unrelated cache eviction from index-only work.The default-off rollout gate preserves old public API replicas during a mixed-version upgrade. Updated search routes always understand both epochs; enable isolation only after every public API and indexer replica is compatible.
Before/after PostgreSQL measurements
Local x86-64 PostgreSQL 18, session-mode PgBouncer and Redis; three samples per
isolation mode, alternating order. Each sample includes the producer insertion,
indexer processing/acknowledgement and public API read for 20 projection-only pages.
The baseline is isolation off, which retains the old broad invalidation behavior.
A separate observer role captures PostgreSQL utility/nested statements, driver
configuration, transaction commands and pooler resets. Warm-up is excluded. All
samples use the same hydrated response and preserve the raw-writer/publication fences.
These gate-off/on results isolate runtime invalidation behavior on the new schema.
They do not hide the additive schema cost below, establish an across-the-board
PostgreSQL CPU reduction under failure, or predict whole-production container savings.
The degraded cases retain equal SQL work; they are not an outage capacity guarantee.
Migration and ongoing outbox storage
A fresh disposable database at revision 0040 held 103,000 events before the actual
Alembic upgrade. Before/after checks matched every ID/xmin checksum, relation filenode
and the 7,888,896-byte heap. All old rows read
public_change=true. No heap rewriteor row-by-row backfill occurred. A deliberately held reader made the requested
AccessExclusiveLock wait and fail at the migration's five-second lock deadline;
a subsequent uncontended upgrade succeeded.
The boolean provenance has a concrete storage/WAL cost, even while the fixed runtime
workload avoids API SQL. Keep the PR in draft and isolation disabled until migration
contention, native ARM64 and a representative production canary have been reviewed.
Do not weaken raw-SQL or source-withdrawal fences to remove this byte. Rollout still
requires every public API and indexer to support both epochs before enabling isolation.
Validation
Rollout notes
Requires the single additive migration
0041_search_cache_publication.py. KeepDEVFEED_SEARCH_CACHE_ISOLATION_ENABLED=falseuntil all public APIs and indexers are upgraded; then enable and compare cache hit rate, API SQL/CPU and search freshness. Source business events remain conservative, so this does not remove every indexer-originated public invalidation.Disable the gate before an older API rollback. Keep the additive column until compatible image rollback is complete. No TTL, replicas, polling interval or production deployment changes are included. The cache epoch guide documents provenance, raw-writer fallback and rollout boundaries.
Ready for review