Repository navigation
Added email analytics ingestion lag to Debug - #30685
Conversation
ref https://linear.app/ghost/issue/BER-3921/ Event timestamps alone cannot distinguish ingestion backlog from quiet periods. Track completed fetch progress so Debug can show how far ingestion is behind. - Expose fetchedThrough, lagSeconds and measuredAt for each analytics pipeline. - Advance empty completed windows and use the safe cursor for capped fetches. - Show separate delivery, opens and recovery lag with UTC progress timestamps.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: QUIET Plan: Essentials Run ID: 📒 Files selected for processing (4)
Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (12)
🧰 Additional context used📓 Path-based instructions (7)Review new or changed service boundaries for explicit dependency ownership, deterministic/idempotent initialisation, boot ordering, transaction and event semantics, cache coherence, and restart/multi-instance safety.⚙️ CodeRabbit configuration file Files:
Review whether tests prove changed behaviour, meaningful error/edge paths, and externally observable contracts without coupling to implementation details.⚙️ CodeRabbit configuration file Files:
New source files must be TypeScript: flag new JS files as a required change unless exempt (DB migrations, apps/ember-admin/, tool/config files, scripts/, docker/, generated code).⚙️ CodeRabbit configuration file Files:
Review lens: "where does this data become trusted?" Boundary data (HTTP input, external API/SDK responses, env/config, DB/filesystem reads, queue/webhook/event payloads) is `unknown` until validated — Zod by default.⚙️ CodeRabbit configuration file Files:
Prioritise concrete correctness, security, data-integrity, compatibility, and regression risks.⚙️ CodeRabbit configuration file Files:
Type-safe boundaries: Fail only if the PR: consumes boundary data (HTTP input, external API/SDK responses, env/config, DB/filesystem reads, queue/webhook/event payloads) without validating it first — Zod by default, another format only wher...📄 CodeRabbit inference engine (Custom checks) Files:
New files are TypeScript: Fail if the PR adds a new .js/.jsx/.cjs/.mjs source file, unless it is: a DB migration (ghost/core/core/server/data/migrations/), under apps/ember-admin/, a tool/config file, under scripts/ or docker/, or generated...📄 CodeRabbit inference engine (Custom checks) Files:
WalkthroughThe email analytics service records successful fetch boundaries and exposes Suggested reviewers: Merge Risk: ⚪ Minimal · up to This adds per-pipeline email analytics progress and ingestion-lag details to the Debug overview. No merge-blocking risk is evident. 🚥 Pre-merge checks | ✅ 5 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (5 passed)
Full details: Type-Safe BoundariesExplanation The PR adds unvalidated consumption of an HTTP response in Resolution Add a Zod schema at the
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
|
| Command | Status | Duration | Result |
|---|---|---|---|
nx run @tryghost/admin:test:acceptance |
✅ Succeeded | 11m 58s | View ↗ |
nx run ghost:test:ci:integration |
✅ Succeeded | 4m 40s | View ↗ |
nx run ghost:test:integration |
✅ Succeeded | 3m 57s | View ↗ |
nx run-many -t test:unit -p @tryghost/admin,ghost |
✅ Succeeded | 5m 19s | View ↗ |
nx run ghost:test:ci:e2e |
✅ Succeeded | 4m 16s | View ↗ |
nx run ghost:test:legacy |
✅ Succeeded | 3m 34s | View ↗ |
nx run ghost:test:e2e |
✅ Succeeded | 2m 59s | View ↗ |
nx run ghost-admin:test |
✅ Succeeded | 2m 42s | View ↗ |
Additional runs (6) |
✅ Succeeded | ... | View ↗ |
💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗
☁️ Nx Cloud last updated this comment at 2026-09-10 16:47:30 UTC
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 69f017a1ee
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| <tr> | ||
| <td>Fetched through:</td> | ||
| <td>{{ this.analyticsStatus.latest.fetchedThrough }}</td> |
There was a problem hiding this comment.
This adds a new Admin feature directly to the legacy Ember debug screen, while repository guidance requires new Admin UI to be built in apps/admin using admin-x-framework and Shade. Move the ingestion-lag presentation and API consumption into the React Admin surface and include the required older-backend acceptance case rather than extending the Ember implementation.
AGENTS.md reference: AGENTS.md:L40-L44
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
There's a follow-up in #30688 that migrates the debug screen
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #30685 +/- ##
==========================================
- Coverage 67.79% 67.72% -0.08%
==========================================
Files 1670 1670
Lines 60326 60347 +21
Branches 10435 10441 +6
==========================================
- Hits 40899 40867 -32
- Misses 17123 17170 +47
- Partials 2304 2310 +6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ref https://linear.app/ghost/issue/BER-3921/ Keep ingestion lag focused on ongoing pipelines and remove unused metadata and misleading captions. - Exclude scheduled refetches from wall-clock lag and remove measuredAt. - Remove the redundant cancellation guard and hard-coded buffer captions. - Omit zero duration units, retaining 0s for zero lag.
jonatansberg
left a comment
jonatansberg
left a comment
There was a problem hiding this comment.
Nice! Two small, non-blocking things for your consideration:
- The "Ingestion lag" row under Analytics Missing will always read ~30m on a healthy site, since that job's window ends at
now - TRUST_THRESHOLD_MS. Might be worth relabelling or dropping the lag row there so it doesn't look like the job is behind? scheduled.fetchedThroughis set on the backend and formatted indebug.js, but the scheduled block in the template doesn't render it yet. Show or drop?
ref https://linear.app/ghost/issue/BER-3911 #30685 added fetchedThrough and lagSeconds for the debug screen, which measure the same thing as this branch's caughtUpTo. Keeping both would let the lag logs, the metric and the debug screen disagree, so lag reporting now derives from lagSeconds and the separate status shape is gone. fetchedThrough also avoids two edge cases caughtUpTo had: a fetch skipped because Mailgun isn't configured no longer counts as caught up, and a failed first fetch after a restart no longer reports lag from a days-old restored cursor.
ref https://linear.app/ghost/issue/BER-3911 The opened-events job warned when its cursor was older than emailAnalytics:openedJobLagWarningMinutes. That cursor only moves when events are processed, so quiet sites warned for days and an outage was hard to tell apart from inactivity. The job.completed log for the opened and delivery pipelines now carries lag_seconds, read after each fetch from the ingestion lag getStatus() already reports since #30685: the time since the end of the last successful fetch window, or its safe cursor when a run was capped. It is left out until a fetch has succeeded in the current process, so a restart or an unconfigured Mailgun is reported neither as behind nor as caught up. The threshold and its warning are removed; failed fetches already log their own error. The missing-events sweep gets no lag entry because its window trails the delivery pipeline by design. The email debug screen keeps polling after a failed status request, and GET /emails/:id/analytics has an e2e-api test. Co-authored-by: Ricardo Pinto <hessian_loom_0u@icloud.com>
2 participants
ref https://linear.app/ghost/issue/BER-3921/
Added server-calculated ingestion lag and UTC fetched-through timestamps to the analytics Debug overview, with separate delivery/failure, opens and recovery sections.
Progress advances through empty completed windows and uses the safe cursor when fetching hits an event limit. Failed fetches preserve previous progress. Unknown progress displays N/A, including on older backends. Scheduled refetches expose progress without wall-clock lag.