Skip to content

Define an app-wide convention for asserting domain and validation errors in tests #2174

Open

Description

@kmcginnes

Goal

How tests assert errors is currently decided per PR, and docs/agents/testing.md no longer matches what the codebase does. Since #2163 is deciding on a shared base type for domain errors, the assertion convention should fall out of that decision rather than be settled separately.

What is inconsistent today

docs/agents/testing.md:71 says to assert the full error instance and to "prefer this over toThrow(FooError) (type only)". Three departures from that now exist, none documented:

  • ZodError cannot use the instance form at all. Zod 4's parse/safeParse construct an internal ZodRealError whose _zod.constr differs from a hand-built z.ZodError, so toThrow(new z.ZodError(issues)) always fails with "Compared values have no visual difference". The working pattern is expect(error).toBeInstanceOf(z.ZodError) plus expect(z.prettifyError(error)).toMatchInlineSnapshot(...), now used in localDb.test.ts and exportedGraph.test.ts.
  • Class-level assertions are used deliberately in sparql/neighborCounts.test.ts and sparql/rawQuery.test.ts, because the alternative was exporting a module-private schema purely so a test could rebuild the expected error. The doc reads as forbidding this.
  • One assertion silently lost its class check. exportedGraph.test.ts moved from rejects.toThrow(new FileEnvelopeError(msg, issues)) to rejects.toMatchObject({ message, issues }). Vitest 4 deep-compares an expected error instance including its class, so the old form pinned the constructor and toMatchObject does not. It now passes for any object with that message and issues shape.

Separately, validationErrorFor (src/utils/testing/errorInstances.ts, 15 call sites) is absent from the "Key helpers" list, and nothing records that a helper which wraps expect() internally must be added to .oxlintrc.json's vitest/expect-expect assertFunctionNames allowlist.

Expected Outcome

One documented convention in docs/agents/testing.md covering domain errors, ZodError, and validation errors, aligned with whatever base type #2163 lands on, plus the drifted sites above brought into line. Decide explicitly whether class-level assertion is a sanctioned carve-out or something to eliminate.

The toBeInstanceOf(FileEnvelopeError) fix in exportedGraph.test.ts is a one-liner and does not need to wait for the convention.

Related Issues

Important

Internal only — this issue is maintained by the core team and is not accepting external contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationinternalSignals that the team will work on this issue internally.tech debtIssues, typically tasks, that are mainly about cleaning up code that is problematic in some way

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions