Skip to content

feat(cli): add inspect db toast-sizes command - #6528

Open
bruno222 wants to merge 6 commits into
developfrom
feat/inspect-db-toast-sizes
Open

feat(cli): add inspect db toast-sizes command#6528
bruno222 wants to merge 6 commits into
developfrom
feat/inspect-db-toast-sizes

Conversation

@bruno222

@bruno222 bruno222 commented Sep 8, 2026

Copy link
Copy Markdown

Summary

Adds supabase inspect db toast-sizes — a new inspect db subcommand that lists user tables with TOAST storage, ordered by TOAST size (largest first).

What changed

  • apps/cli/src/commands/inspect/db/toast-sizes/ — 3-file subcommand (query, handler, command)
  • apps/cli/docs/supabase/inspect/db-toast-sizes.md — docs overlay with description and sample output
  • apps/cli/src/commands/inspect/db/db.command.ts — registers the new subcommand
  • apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md — updates active count (13 → 14) and query table
  • apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts — adds a test case for toastSizesSpec

Why it matters

TOAST (The Oversized-Attribute Storage Technique) is PostgreSQL's overflow storage for large values in TEXT, JSONB, and bytea columns. Autovacuum runs on TOAST tables independently from the main heap, so dead TOAST chunks accumulate silently. A high TOAST dead-tuple ratio increases I/O and inflates effective database size — but it doesn't appear in the existing bloat or vacuum-stats commands, which only cover the main heap.

This command surfaces:

  • Heap vs. TOAST size split per table
  • Live/dead TOAST chunk counts and dead-chunk percentage
  • Last autovacuum/vacuum timestamps on the TOAST table

It's a common source of invisible bloat, especially in tables with large JSONB columns.

Testing

  • New integration test case in inspect-specs.integration.test.ts (count updated 13 → 14)
  • Query filters out internal Supabase schemas via the shared $1 LIKE-escape parameter (consistent with bloat, vacuum-stats, index-stats, table-stats, db-stats)
  • Docs overlay at apps/cli/docs/supabase/inspect/db-toast-sizes.md follows the same format as existing overlays

Docs

A follow-up PR to supabase/supabase will add this command to the Inspect guide under the Disk Storage Analysis section after this merges.

Adds `supabase inspect db toast-sizes`, which lists user tables that
have TOAST storage, ordered by TOAST size (largest first).

TOAST (The Oversized-Attribute Storage Technique) is PostgreSQL's
overflow storage for large values in TEXT, JSONB, and bytea columns.
Autovacuum runs on TOAST tables independently from the main heap, so
dead TOAST chunks accumulate silently. High TOAST dead-tuple ratios
increase I/O and inflate effective database size without appearing in
the standard `bloat` or `vacuum-stats` commands.

The query joins pg_class with pg_stat_all_tables on the TOAST relid,
filters out tables with no TOAST (reltoastrelid = 0) and internal
Supabase schemas, and returns table name, heap/TOAST sizes, live/dead
chunk counts, dead-chunk percentage, and last autovacuum/vacuum times.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@bruno222
bruno222 requested a review from a team as a code owner September 8, 2026 20:10
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Supabase CLI preview

npx --yes https://pkg.pr.new/supabase/cli/supabase@f7f6e49b657aefb54ea18c9b441416b30be12a02

Preview package for commit f7f6e49.

bruno222 and others added 3 commits September 8, 2026 22:24
Adds toast_sizes to REPORT_QUERIES so `supabase inspect report --db-url`
includes TOAST bloat data in its CSV output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

Only Claude's independent review was available; Codex did not complete. Nine findings are confirmed, mostly documentation and consistency issues. The substantive concerns are an incorrect published --linked default and understated TOAST storage. The claim about typical output being dominated by empty tables remains uncertain.

Findings

Severity Location Category Sources Claim
🟡 MINOR apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.command.ts:6 documentation claude The new command lacks a docs default override, so its published --linked default will incorrectly appear as false even though omitting target flags selects the linked project.
🟡 MINOR apps/cli/src/commands/inspect/db/inspect-specs.integration.test.ts:253 test-coverage claude toast-sizes is omitted from the inspect cli-e2e subcommand matrix and has no PostgreSQL fixture, unlike every other active inspect-db subcommand.
🟡 MINOR apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md:67 documentation claude The new SIDE_EFFECTS row identifies a nonexistent toast_sizes.sql file even though the query is defined inline in TypeScript.
🟡 MINOR apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.query.ts:15 correctness claude toast_size excludes the TOAST index while total_size includes it and parent indexes, making the reported TOAST footprint incomplete and the documentation's exact additivity misleading.
⚪ NIT apps/cli/src/commands/inspect/db/db.command.ts:46 documentation claude Adding toast-sizes raises the inspect-db leaf count to 26, leaving four comments that still say 25.
⚪ NIT apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.query.ts:27 usability claude The query includes tables whose TOAST relation has no allocated data and uses no deterministic tie-breaker, potentially producing output dominated by arbitrarily ordered zero-size rows.
⚪ NIT apps/cli/docs/supabase/inspect/db-toast-sizes.md:3 documentation claude The documentation incorrectly describes the approximately 2 kB TOAST threshold as applying to an individual column value rather than the row tuple.
⚪ NIT apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.command.ts:7 consistency claude The command's five-sentence help description is substantially longer than every sibling inspect-db description.
⚪ NIT apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.query.ts:32 consistency claude toastSizesSpec is the only active inspect-db query spec without the explanatory JSDoc used by all sibling specs.
⚪ NIT apps/cli/src/commands/inspect/db/db.command.ts:26 style claude The new imports do not follow the established ordering in either the command registry or sibling command files.

Stats

Claude findings: 10 · Codex findings: 0 · Confirmed: 9 · Refuted: 0 · Uncertain: 1


Models: claude-opus-5 + gpt-5.6-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

import { inspectDbRuntimeLayer } from "../db.layers.ts";
import { inspectDbToastSizes } from "./toast-sizes.handler.ts";

export const inspectDbToastSizesCommand = Command.make("toast-sizes", INSPECT_DB_FLAGS).pipe(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MINOR · documentation · source: claude

The new command lacks a docs default override, so its published --linked default will incorrectly appear as false even though omitting target flags selects the linked project.

Evidence: apps/cli/src/commands/inspect/db/inspect-db-command.ts:21-24 declares the primitive default as false, while apps/cli/src/commands/inspect/db/inspect-query.ts:193-196 defaults execution to linked. apps/cli/src/docs/docs-spec.ts:254 falls back to the primitive default, and apps/cli/src/docs/docs-spec.tables.ts:167-179 overrides every other active inspect-db command but not toast-sizes.

Suggested fix: Add "supabase-inspect-db-toast-sizes linked": "true" to DOCS_DEFAULT_OVERRIDES.

Comment on lines +253 to +267
spec: toastSizesSpec,
params: "schemas1",
row: {
name: "public.events",
total_size: "120 kB",
heap_size: "80 kB",
toast_size: "40 kB",
toast_live_chunks: 1200,
toast_dead_chunks: 80,
toast_dead_pct: "6.3",
last_autovacuum: "2025-01-15 03:00",
last_vacuum: "",
},
expect: ["public.events", "120 kB", "40 kB", "1200", "80", "6.3", "2025-01-15 03:00"],
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MINOR · test-coverage · source: claude

toast-sizes is omitted from the inspect cli-e2e subcommand matrix and has no PostgreSQL fixture, unlike every other active inspect-db subcommand.

Evidence: apps/cli-e2e/src/tests/inspect.e2e.test.ts:31-45 lists 13 active subcommands and drives success and connection-failure subprocess tests at lines 66-90. apps/cli-e2e/fixtures/pg contains matching fixtures for those 13 commands but none for toast-sizes. The added integration case exercises runInspectQuery directly, not command registration or subprocess wiring.

Suggested fix: Add a toast-sizes fixture and entry to the cli-e2e SUBCOMMANDS table.

inspectDbVacuumStatsCommand,
inspectDbTableStatsCommand,
inspectDbTrafficProfileCommand,
inspectDbToastSizesCommand,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · documentation · source: claude

Adding toast-sizes raises the inspect-db leaf count to 26, leaving four comments that still say 25.

Evidence: apps/cli/src/commands/inspect/db/db.command.ts:32-59 registers 26 leaves. Stale counts remain at inspect-db-command.ts:10 and :40, db.layers.ts:13, and db.layers.unit.test.ts:9.

Suggested fix: Change the four counts from 25 to 26.

| long-running-queries | long_running_queries.sql | no |
| role-stats | role_stats.sql | no |
| traffic-profile | traffic_profile.sql | no |
| toast-sizes | toast_sizes.sql | yes (`$1`) |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MINOR · documentation · source: claude

The new SIDE_EFFECTS row identifies a nonexistent toast_sizes.sql file even though the query is defined inline in TypeScript.

Evidence: apps/cli/src/commands/inspect/db/SIDE_EFFECTS.md:67 names toast_sizes.sql, while apps/cli/src/commands/inspect/db/toast-sizes/toast-sizes.query.ts:10-30 contains the actual SQL and the repository has no toast_sizes.sql. Trusted apps/cli/CLAUDE.md:30-43 says TypeScript is authoritative and new Go-parity framing must not be added.

Suggested fix: Reference toast-sizes.query.ts and update the table's obsolete Go SQL-file framing.

FORMAT('%I.%I', n.nspname, main.relname) AS name,
pg_size_pretty(pg_total_relation_size(main.oid)) AS total_size,
pg_size_pretty(pg_relation_size(main.oid)) AS heap_size,
pg_size_pretty(pg_relation_size(main.reltoastrelid)) AS toast_size,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MINOR · correctness · source: claude

toast_size excludes the TOAST index while total_size includes it and parent indexes, making the reported TOAST footprint incomplete and the documentation's exact additivity misleading.

Evidence: toast-sizes.query.ts:13 uses pg_total_relation_size(main.oid), while lines 14-15 use pg_relation_size for only the parent and TOAST main forks. docs/supabase/inspect/db-toast-sizes.md:10-12 shows Total Size exactly equal to Heap Size plus TOAST Size in every sample row.

Suggested fix: Use pg_total_relation_size(main.reltoastrelid) for the full TOAST footprint and revise the sample or documentation to explain that parent Total Size also includes parent indexes and auxiliary forks.

Comment on lines +27 to +30
WHERE main.relkind = 'r'
AND main.reltoastrelid <> 0
AND NOT n.nspname LIKE ANY($1)
ORDER BY pg_relation_size(main.reltoastrelid) DESC`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · usability · source: claude

The query includes tables whose TOAST relation has no allocated data and uses no deterministic tie-breaker, potentially producing output dominated by arbitrarily ordered zero-size rows.

Evidence: toast-sizes.query.ts:27-29 filters only by relation kind, existence of a TOAST relation, and schema. Line 30 orders solely by TOAST relation size.

Suggested fix: If empty relations are not useful, filter them out; independently, add a table-name secondary ordering key.

Adjudication (uncertain): The code confirms the lack of a size filter and tie-breaker, but static inspection cannot establish that typical databases are dominated by empty rows. The command description also explicitly promises every user table with a TOAST relation.

@@ -0,0 +1,13 @@
# db-toast-sizes

This command displays TOAST table sizes and dead chunk counts for every user table that has a TOAST relation. When a column value exceeds ~2 kB (TEXT, JSONB, bytea), Postgres stores it out-of-line in a companion TOAST table. Autovacuum runs on the TOAST table independently from the main heap, so it can accumulate dead chunks even when the parent table looks healthy by its own dead-tuple count.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · documentation · source: claude

The documentation incorrectly describes the approximately 2 kB TOAST threshold as applying to an individual column value rather than the row tuple.

Evidence: docs/supabase/inspect/db-toast-sizes.md:3 says Postgres moves a value out-of-line when that value exceeds approximately 2 kB. PostgreSQL instead applies the threshold to the tuple and compresses or moves eligible attributes until it fits.

Suggested fix: Explain that when a row exceeds the threshold, PostgreSQL compresses and/or moves eligible variable-length attributes out of line.

Comment on lines +7 to +13
Command.withDescription(
"Displays TOAST table sizes and dead chunk counts for every user table that has a TOAST table. " +
"Tables with TEXT, JSONB, or bytea columns store overflow values in a separate TOAST relation. " +
"Autovacuum runs on the TOAST table independently, so it can accumulate dead chunks even when " +
"the main heap looks healthy. A table that appears fine by dead-tuple count alone can still " +
"have significant TOAST bloat that wastes disk and slows queries.",
),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · consistency · source: claude

The command's five-sentence help description is substantially longer than every sibling inspect-db description.

Evidence: toast-sizes.command.ts:7-13 contains five sentences. The other 25 command files each use a single-sentence description; examples include vacuum-stats.command.ts:7 and traffic-profile.command.ts:10-12.

Suggested fix: Use a concise one-sentence CLI description and retain the detailed explanation in the docs overlay.

AND NOT n.nspname LIKE ANY($1)
ORDER BY pg_relation_size(main.reltoastrelid) DESC`;

export const toastSizesSpec: InspectQuerySpec = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · consistency · source: claude

toastSizesSpec is the only active inspect-db query spec without the explanatory JSDoc used by all sibling specs.

Evidence: toast-sizes.query.ts:32 declares the spec without JSDoc. Each of the other 13 active query files has a JSDoc immediately before its exported InspectQuerySpec, such as table-stats.query.ts:32-37 and vacuum-stats.query.ts:68-73.

Suggested fix: Add a short JSDoc describing the report and its notable size and timestamp semantics.

import { inspectDbTotalTableSizesCommand } from "./total-table-sizes/total-table-sizes.command.ts";
import { inspectDbTrafficProfileCommand } from "./traffic-profile/traffic-profile.command.ts";
import { inspectDbUnusedIndexesCommand } from "./unused-indexes/unused-indexes.command.ts";
import { inspectDbToastSizesCommand } from "./toast-sizes/toast-sizes.command.ts";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ NIT · style · source: claude

The new imports do not follow the established ordering in either the command registry or sibling command files.

Evidence: db.command.ts:22-27 places toast-sizes after unused-indexes rather than before the total-* imports in the otherwise grouped ordering. toast-sizes.command.ts:1-4 places its handler import last, while every other sibling imports its handler immediately after Effect's Command import.

Suggested fix: Move the registry import before total-index-size and move the handler import to the second line of toast-sizes.command.ts.

bruno222 and others added 2 commits September 8, 2026 23:33
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Use pg_total_relation_size for TOAST size (includes TOAST index, not
  just TOAST heap) and update ORDER BY to match
- Filter out tables with allocated but empty TOAST (size = 0)
- Shorten command description to match sibling command style
- Fix import order in db.command.ts (alphabetical)
- Update leaf-count comments 25→26 in db.layers.ts and inspect-db-command.ts
- Fix SIDE_EFFECTS.md: reference toast-sizes.query.ts, not toast_sizes.sql

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant