Skip to content

[DPCYHEALTH-1203] Update tracegen builder to Go 1.26 - #1908

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
lwen-tracegen-go-126-builder
Nov 4, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
lwen-tracegen-go-126-builder

Conversation

@luyingwen24

Copy link
Copy Markdown
Contributor

Summary

Update the tracegen Docker builder to Go 1.26. ADMS PR #1900 changes tracegen's go.mod directive to Go 1.26, but the current Dockerfile runs Go 1.23, so its image build fails at go mod download.

This builder version also supports the Go 1.24 directive in ADMS PR #1904.

Validation

Dockerfile owner: DataDog/agent-apm.
Jira: https://datadoghq.atlassian.net/browse/DPCYHEALTH-1203
Related ADMS PRs: #1900 and #1904

@luyingwen24
luyingwen24 requested a review from a team as a code owner November 4, 2026 17:07
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 8ef43a1 into main Nov 4, 2026
64 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the lwen-tracegen-go-126-builder branch November 4, 2026 18:15
gh-worker-dd-mergequeue-cf854d Bot pushed a commit to DataDog/datadog-agent that referenced this pull request Nov 10, 2026
…ources (#60002)

### What does this PR do?

Adds a custom resource definition and custom resources of it to the `benchmarkeks` workload, and makes the KSM checks collect those custom resources:

- `workload/crds.yaml`: the `widgets.benchmark.datadoghq.com` CRD, recreated every 5 minutes, and 5 `Widget` custom resources in `workload-fake`, recreated every minute.
- `datadog.kubeStateMetricsCore.collectCrMetrics` of both installations lists the `Widget` custom resources, after the `DatadogMetric` entry of the e2e framework defaults, repeated because Helm replaces lists.
- The shared `apps.Version` of the e2e test apps moves to `v0.0.10`, the first version whose `apps-churn` can churn CRDs and their custom resources. All the app images move with it: besides the CRD support of churn (DataDog/test-infra-definitions#1912), `v0.0.10` brings DataDog/test-infra-definitions#1887 (churn cleans up on exit and recreates leftover objects on start), dependency bumps of the churn and dogstatsd images, and the Go 1.26 builders of the dogstatsd and tracegen images (DataDog/test-infra-definitions#1907, DataDog/test-infra-definitions#1908).
- `how-it-works.md` lists the new objects and explains why.

**Functional changes**

| Before | After |
| --- | --- |
| KSM checks of benchmarkeks: their stores were only rebuilt while the Agents started, a number of times that varied between runs | A CRD recreation every 5 minutes, the length of a measurement block, makes both installations rebuild their stores once or twice per recreation |
| No custom resource churned; the KSM custom resource stores only held the (absent) `DatadogMetric` objects | The KSM checks also collect 5 `Widget` custom resources, whose store comes and goes with their definition |
| The Cluster Agents saw no CRD event after startup | They see a CRD deleted and created again every 5 minutes (workloadmeta CRD collection, orchestrator CRD collector) |
| All e2e test apps used the `v0.0.9` images of test-infra-definitions | They use the `v0.0.10` images |

Unchanged: the rest of the benchmarkeks workload.

### Motivation

The KSM check rebuilds all its stores after any event of its CRD informer. In the validation of #59901 (the revert of #48680, whose KSM check never cancelled the reflectors of its old stores), the cluster checks runner leaked about 3 extra sets of reflectors in phase 1 and 1 in phase 2. The benchmark caught it (+69% memory), but the size of the leak depended on how many rebuilds happened before the measurement. With a CRD recreated in every block, a leak of that kind grows at the same pace in every phase and every run.

It also exercises the CRD and custom resource handling of the KSM check and of the Cluster Agent, identically for both installations.

### Describe how you validated your changes

Not validated on a live cluster yet: the `v0.0.10` images don't exist (see below), and no image of the test-infra-definitions branch can be pulled from the EKS cluster.

Locally: `dda inv test --targets=./test/e2e-framework/components/datadog/apps/...`, `dda inv linter.go` on the changed packages, `bazel build` of the churn and scenario packages, `dda run docs build --check`, `dda inv linter.docs-links`. The churn change has its own unit tests, covering the API server behaviours it relies on.

### Additional Notes

This PR stays a draft until DataDog/test-infra-definitions#1912 is merged and the `v0.0.10` tag of test-infra-definitions is pushed, which publishes the `v0.0.10` images to ghcr.io and to the agent-qa registries. Until then, every e2e test using these apps would fail to pull its images. `v0.0.9` can't serve churn instead: it predates the support and exits at the first recreation of the CRD.

Once the images exist, the benchmark should run on this branch (manual job `new-e2e-containers-benchmarkeks-baseline`), to check the CRD recreations in the churn logs, the KSM rebuilds in the cluster checks runners, and the noise of the runs. A run right after a rebuild reads stores that are still filling, so `Test01ChecksMatchBaseline` may need its retries more often. The e2e tests of the other apps should also pass with `v0.0.10`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


Co-authored-by: lenaic.huard <lenaic.huard@datadoghq.com>
github-actions Bot pushed a commit to DataDog/datadog-agent that referenced this pull request Nov 10, 2026
…ources (#60002)

### What does this PR do?

Adds a custom resource definition and custom resources of it to the `benchmarkeks` workload, and makes the KSM checks collect those custom resources:

- `workload/crds.yaml`: the `widgets.benchmark.datadoghq.com` CRD, recreated every 5 minutes, and 5 `Widget` custom resources in `workload-fake`, recreated every minute.
- `datadog.kubeStateMetricsCore.collectCrMetrics` of both installations lists the `Widget` custom resources, after the `DatadogMetric` entry of the e2e framework defaults, repeated because Helm replaces lists.
- The shared `apps.Version` of the e2e test apps moves to `v0.0.10`, the first version whose `apps-churn` can churn CRDs and their custom resources. All the app images move with it: besides the CRD support of churn (DataDog/test-infra-definitions#1912), `v0.0.10` brings DataDog/test-infra-definitions#1887 (churn cleans up on exit and recreates leftover objects on start), dependency bumps of the churn and dogstatsd images, and the Go 1.26 builders of the dogstatsd and tracegen images (DataDog/test-infra-definitions#1907, DataDog/test-infra-definitions#1908).
- `how-it-works.md` lists the new objects and explains why.

**Functional changes**

| Before | After |
| --- | --- |
| KSM checks of benchmarkeks: their stores were only rebuilt while the Agents started, a number of times that varied between runs | A CRD recreation every 5 minutes, the length of a measurement block, makes both installations rebuild their stores once or twice per recreation |
| No custom resource churned; the KSM custom resource stores only held the (absent) `DatadogMetric` objects | The KSM checks also collect 5 `Widget` custom resources, whose store comes and goes with their definition |
| The Cluster Agents saw no CRD event after startup | They see a CRD deleted and created again every 5 minutes (workloadmeta CRD collection, orchestrator CRD collector) |
| All e2e test apps used the `v0.0.9` images of test-infra-definitions | They use the `v0.0.10` images |

Unchanged: the rest of the benchmarkeks workload.

### Motivation

The KSM check rebuilds all its stores after any event of its CRD informer. In the validation of #59901 (the revert of #48680, whose KSM check never cancelled the reflectors of its old stores), the cluster checks runner leaked about 3 extra sets of reflectors in phase 1 and 1 in phase 2. The benchmark caught it (+69% memory), but the size of the leak depended on how many rebuilds happened before the measurement. With a CRD recreated in every block, a leak of that kind grows at the same pace in every phase and every run.

It also exercises the CRD and custom resource handling of the KSM check and of the Cluster Agent, identically for both installations.

### Describe how you validated your changes

Not validated on a live cluster yet: the `v0.0.10` images don't exist (see below), and no image of the test-infra-definitions branch can be pulled from the EKS cluster.

Locally: `dda inv test --targets=./test/e2e-framework/components/datadog/apps/...`, `dda inv linter.go` on the changed packages, `bazel build` of the churn and scenario packages, `dda run docs build --check`, `dda inv linter.docs-links`. The churn change has its own unit tests, covering the API server behaviours it relies on.

### Additional Notes

This PR stays a draft until DataDog/test-infra-definitions#1912 is merged and the `v0.0.10` tag of test-infra-definitions is pushed, which publishes the `v0.0.10` images to ghcr.io and to the agent-qa registries. Until then, every e2e test using these apps would fail to pull its images. `v0.0.9` can't serve churn instead: it predates the support and exits at the first recreation of the CRD.

Once the images exist, the benchmark should run on this branch (manual job `new-e2e-containers-benchmarkeks-baseline`), to check the CRD recreations in the churn logs, the KSM rebuilds in the cluster checks runners, and the noise of the runs. A run right after a rebuild reads stores that are still filling, so `Test01ChecksMatchBaseline` may need its retries more often. The e2e tests of the other apps should also pass with `v0.0.10`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: lenaic.huard <lenaic.huard@datadoghq.com> e82c104
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants