[FEATURE] ClickHouse: add trace query plugin

[erdincka]

Description

Adds ClickHouseTraceQuery, a TraceQuery plugin in the ClickHouse plugin module, so traces stored in ClickHouse can be displayed with the existing Trace Table and Tracing Gantt Chart panels. Until now the module had log and time series queries only.

Relates to perses/perses#4202 (ClickHouse traces as a datasource): this is the query plugin a ClickHouse trace explorer would build on.

How it works

The plugin reads the schema created by the OpenTelemetry Collector ClickHouse exporter, and follows the TraceData contract used by the Tempo plugin: a trace ID returns a trace, anything else returns search results.

• Trace ID (16 or 32 hex characters, after variable substitution): the plugin builds the SQL, reads every span of the trace from table (default otel_traces, optionally database.table), and converts the rows to OTLP, grouped by resource and instrumentation scope. Drill-down links such as ?var-traceId=${traceId} work as they do with Tempo.
• SQL: returns one row per span, with the exporter's column names (TraceId, Timestamp, Duration, ParentSpanId, SpanName, ServiceName, StatusCode). The plugin uses it as a subquery, and ClickHouse groups the spans into traces: one row per trace with the root span, the start and end time, and the span and error counts per service, newest first, with LIMIT limit + 1. The extra trace is how the panels know more traces match, as in the Tempo and Jaeger plugins. {start} and {end} are replaced as in the log and time series queries.

# Trace Table: traces that went through the payment service
kind: ClickHouseTraceQuery
spec:
  query: |
    SELECT TraceId, ParentSpanId, SpanName, ServiceName, Timestamp, Duration, StatusCode
    FROM otel.otel_traces
    WHERE Timestamp BETWEEN '{start}' AND '{end}'
      AND TraceId IN (
        SELECT TraceId FROM otel.otel_traces
        WHERE ServiceName = 'payment' AND Timestamp BETWEEN '{start}' AND '{end}'
      )
---
# Tracing Gantt Chart: the trace selected in the traceId variable
kind: ClickHouseTraceQuery
spec:
  query: $traceId
  table: otel.otel_traces

Implementation notes

• Timestamps. Both queries convert timestamps to nanosecond strings in SQL (toUnixTimestamp64Nano). Otherwise DateTime64 values are rendered in the server's timezone, and nanoseconds since the epoch exceed Number.MAX_SAFE_INTEGER, so they are summed and subtracted with BigInt.
• SQL built by the plugin. The trace ID is validated and normalized (lowercase, padded to 32 characters, as the exporter stores it), and table must be a plain identifier (checked in CUE and at runtime) before either goes into SQL. User-written search queries keep the existing behavior of the ClickHouse queries.
• Limit. limit is applied by ClickHouse, not after fetching: only the traces that are displayed cross the wire. On the verification data, the search panel returns 8 rows instead of 48 span rows, and a search over 48 traces with limit: 20 returns 21 rows instead of 168. The cost is that the search query is used as a subquery: all the documented columns are required, it has to be a single SELECT, and a query ending with a FORMAT clause is rejected with an explicit error. A trailing ; is removed.
• Errors. The ClickHouse client returns status: 'error' and logs the server's message to the console, so the plugin throws rather than showing an empty panel. An unknown trace ID also throws, as in the Jaeger plugin.
• Exporter details. SpanKind and StatusCode are accepted as the exporter writes them (Server, Error) and as OTLP enum names (SPAN_KIND_SERVER, STATUS_CODE_ERROR). The exporter stores attribute values as strings, so attributes are returned as stringValue. Duration is accepted as a JSON number or string, since ClickHouse quotes 64-bit integers or not depending on output_format_json_quote_64bit_integers.

Changes

Path	Change
clickhouse/src/queries/click-house-trace-query/	Plugin, options editor, data fetching and conversion, types, tests
clickhouse/schemas/queries/click-house-trace-query/	CUE schema, with valid and invalid test cases
clickhouse/sdk/go/query/trace/	Go SDK builder (Query, Datasource, Table, Limit) and tests
clickhouse/package.json, rsbuild.config.ts, src/queries/index.ts	Register and expose the plugin
docs/clickhouse/	Overview, data model (including the trace ID lookup and the search query columns), Go SDK page

Tests

• 22 vitest tests. Trace lookup: SQL, OTLP conversion (resource and scope grouping, both span kind and status spellings, nanosecond end times, events, links), trace ID normalization, variables, table validation, trace not found. Search: the generated query and its limit, the summary rows it returns, time range placeholders, more results than the limit, a 16 character SQL query running as a search, a trailing semicolon, a rejected FORMAT clause, unknown root service, ClickHouse errors. Editor: table committed on blur, limit.
• 2 Go tests for the SDK builder.
• 5 schema test cases: 2 valid, and 3 invalid (empty query, table that is not an identifier, non-positive limit).

Verification

CI-equivalent run. I ran the steps of react.yml, cue.yml, go.yml, doc.yml and ci.yml at this branch's commit, in a container with the versions CI pins (Go 1.27.1, Node 24, CUE v0.16.1, percli v0.54.0, golangci-lint v2.13.2, mdox). All 17 steps passed:

Steps	Result
npm ci, npm run lint, npm run format:check, npm run type-check, npm run test	Pass (clickhouse: 6 test files, 33 tests)
make checkformat-cue, make lint-plugins, make test-schemas-plugins, make tidy-modules with no cue.mod diff	Pass (the 2 valid and 3 invalid click-house-trace-query cases are evaluated)
make test, go test ./... in clickhouse, make golangci-lint, make checklicense	Pass
make checkdocs	Pass
percli plugin build for clickhouse, tracetable and tracingganttchart	Pass

The advisory React Doctor scan (react-doctor.yml) reports no findings in the new files.

End to end. OTLP/HTTP → OpenTelemetry Collector contrib 0.160.0 (ClickHouse exporter, create_schema: true) → ClickHouse 25.8 → Perses (main image of 2026-09-10), with a ClickHouseDatasource going through the Perses proxy and the plugin archives built from this branch. The data: 8 traces across three services (with errors, exception events and span links), plus 40 traces from telemetrygen.

• The ClickHouse plugin module loads with TraceQuery:ClickHouseTraceQuery, and a dashboard using it is provisioned.
• POST /api/validate/dashboards accepts valid specs, and rejects a table that is not an identifier, limit: 0, an empty query and an unknown field, each with the corresponding CUE error.
• The trace lookup SQL, run against the table created by the exporter, returns the types the conversion expects: maps as JSON objects, events and links as arrays, nanoseconds as strings.
• A Trace Table with a search query lists the traces with their span and error counts. With 48 traces and limit: 20, it shows the "Not all matching traces are currently displayed" notice. ClickHouse's query_log confirms the limit is applied there: 21 rows returned for that panel, and 8 for the search on the payment service.
• A Tracing Gantt Chart with query: $traceId renders the trace: span hierarchy, error statuses, attributes and events. Following a Trace Table link (?var-traceId=${traceId}) shows the selected trace.
• An unknown trace ID shows "Trace ffffffffffffffffffffffffffffffff was not found in otel.otel_traces." in the panel. It is the only error in the browser console.

I can share the Compose setup used for this (collector config, seed script, provisioning) if it helps the review.

Design decision to review: trace ID lookups are not bounded by time

Important

A trace ID lookup reads the spans with WHERE TraceId = '…' from table, with no time bound, so that a trace opens from a link whatever the time range of the dashboard. It relies on the bloom filter index the exporter creates on TraceId.

The alternative is to first read the start and end time of the trace from the exporter's <table>_trace_id_ts table (otel_traces_trace_id_ts by default), and bound the scan with them, which lets ClickHouse skip the partitions outside the trace on large tables. I did not do it because that table only exists when the exporter created the schema, so the lookup would fail on custom tables and views that otherwise have the right columns.

This is the part of the design most likely to need a change, for example an option to use that table, or using it by default. The choice is documented in the data model docs ("Trace ID lookup") and in a comment on the query builder, so it is easy to revisit. Happy to change it if you prefer the other way.

Noticed while working on this, not changed in this PR

• The existing ClickHouse Go SDK docs don't match the SDK. docs/clickhouse/go-sdk/datasource.md, log-query.md and timeseries-query.md import github.com/perses/perses-plugins/clickhouse/sdk/go/v1/..., while the SDK packages are github.com/perses/plugins/clickhouse/sdk/go/datasource, .../query/log and .../query/time-series. The two query pages also use builders that don't exist (query.LogQuery, query.TimeSeriesQuery, query.Format): the SDK has log.ClickHouseLogQuery and timeseries.ClickHouseTimeSeriesQuery, with Query and Datasource options only. Likewise, docs/clickhouse/model.md documents an optional format field for ClickHouseTimeSeriesQuery and ClickHouseLogQuery, which their CUE schemas (closed, with datasource and query only) reject. I didn't find an existing issue about it. I left these pages unchanged to keep this PR focused (the new trace-query.md uses the actual packages), and I'm happy to fix them in a follow-up.
• The Tracing Gantt Chart keeps the viewport and the selected span of the previous trace when its trace changes in place, for example after following a Trace Table link to the same dashboard: TracingGanttChart initializes both with useState, and the panel doesn't key the component by trace. The lower timeline then shows offsets from the previous trace, and the details pane the previously selected span. It happens with any trace query plugin, so it isn't addressed here. The drill-down screenshot below was taken after reloading the page.

Screenshots

Perses main with the plugin archives built from this branch, reading traces written by the OpenTelemetry Collector ClickHouse exporter.

Dashboard: two Trace Tables with search queries, and a Tracing Gantt Chart with query: $traceId

Span details converted from the exporter's columns: kind, status, attributes, resource, scope, events

Query editor: datasource, trace ID or SQL, trace table, max traces

Search limit: limit: 20 with 48 matching traces

A Trace Table link followed to the Gantt chart (after a page reload, see the Tracing Gantt Chart note above)

Unknown trace ID

Checklist

• Pull request has a descriptive title and context useful to a reviewer.
• Pull request title follows the [<catalog_entry>] <commit message> naming convention using one of the
  following catalog_entry values: FEATURE, ENHANCEMENT, BUGFIX, BREAKINGCHANGE, DOC,IGNORE.
• All commits have DCO signoffs.

UI Changes

• Changes that impact the UI include screenshots and/or screencasts of the relevant changes.
• Code follows the UI guidelines.

🤖 Generated with Claude Code

[jgbernalp]

Although the trade-off is documented, can a query limit be added and configured by a field in the query spec?, without it we just fetch all the results to remove them after fetching.

[erdincka]

Good point, thanks. limit was already in the spec (the "Max traces" field) but was only applied after fetching. I have tested 2 resolutions:

1. The search now runs the query as a subquery and does the grouping in ClickHouse: one row per trace (root span, start and end time, span and error counts per service), newest first, LIMIT limit + 1 to detect hasMoreResults as the Tempo plugin does. On my test data, a search over 48 traces now returns 21 rows instead of 168 span rows. The trade-off is that all seven columns are now required and the query has to be a single SELECT without its own FORMAT. Both are documented, and a FORMAT clause is rejected with an explicit error.
2. A {limit} placeholder the user writes into their own SQL, like {start}/{end}. It's a much smaller change, but the pushdown only works if the user puts it in the right place, which is the trace ID subquery (at span level it would cut traces short), and the browser would still fetch every span of those traces.
   Would you agree to go with Option 1?

[jgbernalp]

I'm not an expert in ClickHouse, so I lean on your expertise. But option 1 seems more reasonable as the user might not know which one is the limit they need to use.

[erdincka]

I've pushed a separate commit for this (including the test and doc update for the trace id thread). The search query is now used as a subquery and ClickHouse does the grouping and the limit (buildSearchQuery in get-click-house-trace-data.ts): one row per trace with the root span, start and end time, and span and error counts per service, newest first, with LIMIT limit + 1.

[Copilot]

🟡 Changes recommended

Search output formatting, datasource variables, JSON attributes, service aggregation, and unbounded JSON-schema lookups need correction.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds ClickHouse trace lookup and search support for Perses trace panels.

Changes:

• Implements trace retrieval, search aggregation, OTLP conversion, and editor UI.
• Adds CUE schemas, Go SDK builders, documentation, and tests.
• Registers and exposes ClickHouseTraceQuery.

File summaries

File	Description
docs/clickhouse/README.md	Introduces trace-query documentation.
docs/clickhouse/model.md	Documents configuration and data model.
docs/clickhouse/go-sdk/trace-query.md	Documents Go SDK usage.
clickhouse/src/queries/index.ts	Exports the trace query.
clickhouse/src/queries/click-house-trace-query/index.ts	Defines public exports.
clickhouse/src/queries/click-house-trace-query/get-click-house-trace-data.ts	Implements lookup, search, and conversion.
clickhouse/src/queries/click-house-trace-query/get-click-house-trace-data.test.ts	Tests trace data handling.
clickhouse/src/queries/click-house-trace-query/ClickHouseTraceQueryEditor.tsx	Adds query options UI.
clickhouse/src/queries/click-house-trace-query/ClickHouseTraceQueryEditor.test.tsx	Tests editor behavior.
clickhouse/src/queries/click-house-trace-query/ClickHouseTraceQuery.tsx	Defines the plugin.
clickhouse/src/queries/click-house-trace-query/ClickHouseTraceQuery.test.ts	Tests plugin options and dependencies.
clickhouse/src/queries/click-house-trace-query/click-house-trace-query-types.ts	Defines configuration and row types.
clickhouse/sdk/go/query/trace/trace.go	Adds the Go builder.
clickhouse/sdk/go/query/trace/trace_test.go	Tests Go serialization.
clickhouse/sdk/go/query/trace/options.go	Adds Go builder options.
clickhouse/schemas/queries/click-house-trace-query/query.cue	Defines the CUE contract.
clickhouse/schemas/queries/click-house-trace-query/tests/valid/trace-id.json	Covers valid trace lookup.
clickhouse/schemas/queries/click-house-trace-query/tests/valid/search.json	Covers valid search configuration.
clickhouse/schemas/queries/click-house-trace-query/tests/invalid/non-positive-limit.json	Rejects invalid limits.
clickhouse/schemas/queries/click-house-trace-query/tests/invalid/invalid-table.json	Rejects unsafe table names.
clickhouse/schemas/queries/click-house-trace-query/tests/invalid/empty-query.json	Rejects empty queries.
clickhouse/rsbuild.config.ts	Exposes the plugin bundle.
clickhouse/package.json	Registers the plugin metadata.

Review details

• Files reviewed: 23/23 changed files
• Comments generated: 5
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.