Repository navigation
[DPCYHEALTH-1203] Update tracegen builder to Go 1.26 - #1908
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intoNov 4, 2026
Merged
Conversation
ajgajg1134
approved these changes
Nov 4, 2026
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
lwen-tracegen-go-126-builder
branch
November 4, 2026 18:15
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Nov 10, 2026
…ources (#60002) ### What does this PR do? Adds a custom resource definition and custom resources of it to the `benchmarkeks` workload, and makes the KSM checks collect those custom resources: - `workload/crds.yaml`: the `widgets.benchmark.datadoghq.com` CRD, recreated every 5 minutes, and 5 `Widget` custom resources in `workload-fake`, recreated every minute. - `datadog.kubeStateMetricsCore.collectCrMetrics` of both installations lists the `Widget` custom resources, after the `DatadogMetric` entry of the e2e framework defaults, repeated because Helm replaces lists. - The shared `apps.Version` of the e2e test apps moves to `v0.0.10`, the first version whose `apps-churn` can churn CRDs and their custom resources. All the app images move with it: besides the CRD support of churn (DataDog/test-infra-definitions#1912), `v0.0.10` brings DataDog/test-infra-definitions#1887 (churn cleans up on exit and recreates leftover objects on start), dependency bumps of the churn and dogstatsd images, and the Go 1.26 builders of the dogstatsd and tracegen images (DataDog/test-infra-definitions#1907, DataDog/test-infra-definitions#1908). - `how-it-works.md` lists the new objects and explains why. **Functional changes** | Before | After | | --- | --- | | KSM checks of benchmarkeks: their stores were only rebuilt while the Agents started, a number of times that varied between runs | A CRD recreation every 5 minutes, the length of a measurement block, makes both installations rebuild their stores once or twice per recreation | | No custom resource churned; the KSM custom resource stores only held the (absent) `DatadogMetric` objects | The KSM checks also collect 5 `Widget` custom resources, whose store comes and goes with their definition | | The Cluster Agents saw no CRD event after startup | They see a CRD deleted and created again every 5 minutes (workloadmeta CRD collection, orchestrator CRD collector) | | All e2e test apps used the `v0.0.9` images of test-infra-definitions | They use the `v0.0.10` images | Unchanged: the rest of the benchmarkeks workload. ### Motivation The KSM check rebuilds all its stores after any event of its CRD informer. In the validation of #59901 (the revert of #48680, whose KSM check never cancelled the reflectors of its old stores), the cluster checks runner leaked about 3 extra sets of reflectors in phase 1 and 1 in phase 2. The benchmark caught it (+69% memory), but the size of the leak depended on how many rebuilds happened before the measurement. With a CRD recreated in every block, a leak of that kind grows at the same pace in every phase and every run. It also exercises the CRD and custom resource handling of the KSM check and of the Cluster Agent, identically for both installations. ### Describe how you validated your changes Not validated on a live cluster yet: the `v0.0.10` images don't exist (see below), and no image of the test-infra-definitions branch can be pulled from the EKS cluster. Locally: `dda inv test --targets=./test/e2e-framework/components/datadog/apps/...`, `dda inv linter.go` on the changed packages, `bazel build` of the churn and scenario packages, `dda run docs build --check`, `dda inv linter.docs-links`. The churn change has its own unit tests, covering the API server behaviours it relies on. ### Additional Notes This PR stays a draft until DataDog/test-infra-definitions#1912 is merged and the `v0.0.10` tag of test-infra-definitions is pushed, which publishes the `v0.0.10` images to ghcr.io and to the agent-qa registries. Until then, every e2e test using these apps would fail to pull its images. `v0.0.9` can't serve churn instead: it predates the support and exits at the first recreation of the CRD. Once the images exist, the benchmark should run on this branch (manual job `new-e2e-containers-benchmarkeks-baseline`), to check the CRD recreations in the churn logs, the KSM rebuilds in the cluster checks runners, and the noise of the runs. A run right after a rebuild reads stores that are still filling, so `Test01ChecksMatchBaseline` may need its retries more often. The e2e tests of the other apps should also pass with `v0.0.10`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: lenaic.huard <lenaic.huard@datadoghq.com>
github-actions Bot
pushed a commit
to DataDog/datadog-agent
that referenced
this pull request
Nov 10, 2026
…ources (#60002) ### What does this PR do? Adds a custom resource definition and custom resources of it to the `benchmarkeks` workload, and makes the KSM checks collect those custom resources: - `workload/crds.yaml`: the `widgets.benchmark.datadoghq.com` CRD, recreated every 5 minutes, and 5 `Widget` custom resources in `workload-fake`, recreated every minute. - `datadog.kubeStateMetricsCore.collectCrMetrics` of both installations lists the `Widget` custom resources, after the `DatadogMetric` entry of the e2e framework defaults, repeated because Helm replaces lists. - The shared `apps.Version` of the e2e test apps moves to `v0.0.10`, the first version whose `apps-churn` can churn CRDs and their custom resources. All the app images move with it: besides the CRD support of churn (DataDog/test-infra-definitions#1912), `v0.0.10` brings DataDog/test-infra-definitions#1887 (churn cleans up on exit and recreates leftover objects on start), dependency bumps of the churn and dogstatsd images, and the Go 1.26 builders of the dogstatsd and tracegen images (DataDog/test-infra-definitions#1907, DataDog/test-infra-definitions#1908). - `how-it-works.md` lists the new objects and explains why. **Functional changes** | Before | After | | --- | --- | | KSM checks of benchmarkeks: their stores were only rebuilt while the Agents started, a number of times that varied between runs | A CRD recreation every 5 minutes, the length of a measurement block, makes both installations rebuild their stores once or twice per recreation | | No custom resource churned; the KSM custom resource stores only held the (absent) `DatadogMetric` objects | The KSM checks also collect 5 `Widget` custom resources, whose store comes and goes with their definition | | The Cluster Agents saw no CRD event after startup | They see a CRD deleted and created again every 5 minutes (workloadmeta CRD collection, orchestrator CRD collector) | | All e2e test apps used the `v0.0.9` images of test-infra-definitions | They use the `v0.0.10` images | Unchanged: the rest of the benchmarkeks workload. ### Motivation The KSM check rebuilds all its stores after any event of its CRD informer. In the validation of #59901 (the revert of #48680, whose KSM check never cancelled the reflectors of its old stores), the cluster checks runner leaked about 3 extra sets of reflectors in phase 1 and 1 in phase 2. The benchmark caught it (+69% memory), but the size of the leak depended on how many rebuilds happened before the measurement. With a CRD recreated in every block, a leak of that kind grows at the same pace in every phase and every run. It also exercises the CRD and custom resource handling of the KSM check and of the Cluster Agent, identically for both installations. ### Describe how you validated your changes Not validated on a live cluster yet: the `v0.0.10` images don't exist (see below), and no image of the test-infra-definitions branch can be pulled from the EKS cluster. Locally: `dda inv test --targets=./test/e2e-framework/components/datadog/apps/...`, `dda inv linter.go` on the changed packages, `bazel build` of the churn and scenario packages, `dda run docs build --check`, `dda inv linter.docs-links`. The churn change has its own unit tests, covering the API server behaviours it relies on. ### Additional Notes This PR stays a draft until DataDog/test-infra-definitions#1912 is merged and the `v0.0.10` tag of test-infra-definitions is pushed, which publishes the `v0.0.10` images to ghcr.io and to the agent-qa registries. Until then, every e2e test using these apps would fail to pull its images. `v0.0.9` can't serve churn instead: it predates the support and exits at the first recreation of the CRD. Once the images exist, the benchmark should run on this branch (manual job `new-e2e-containers-benchmarkeks-baseline`), to check the CRD recreations in the churn logs, the KSM rebuilds in the cluster checks runners, and the noise of the runs. A run right after a rebuild reads stores that are still filling, so `Test01ChecksMatchBaseline` may need its retries more often. The e2e tests of the other apps should also pass with `v0.0.10`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: lenaic.huard <lenaic.huard@datadoghq.com> e82c104
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Update the tracegen Docker builder to Go 1.26. ADMS PR #1900 changes tracegen's
go.moddirective to Go 1.26, but the current Dockerfile runs Go 1.23, so its image build fails atgo mod download.This builder version also supports the Go 1.24 directive in ADMS PR #1904.
Validation
go.modandgo.sumin a local validation worktree.go mod download,go mod verify, andgo build ./...passed.Dockerfile owner:
DataDog/agent-apm.Jira: https://datadoghq.atlassian.net/browse/DPCYHEALTH-1203
Related ADMS PRs: #1900 and #1904