The repo's first rule: never commit secrets (see AGENTS.md). Secrets
are generated on, or operator-provisioned to, the target host; the repo ships *.example
templates for them and commits non-secret config directly. There is no dedicated
secret scanner in the pipeline: keep secrets out by design (and rely on GitHub's push
protection).
Commits follow Conventional Commits. PRs are
squash-merged with the PR title as the commit subject, and scripts/release.sh renders
the subjects into the collections' and the chart's changelogs at release (see
RELEASING.md). Write a title an operator can read as a changelog entry;
the pr-title.yml workflow refuses a title that is not conventional, with a lowercase
type, one of feat fix perf refactor docs chore revert security ci
test style build. Never edit a released artifact's changelog by hand: every
section after its first is generated. Only a new artifact's first section is written by
hand, under [Unreleased] (see RELEASING.md).
pip install pre-commit # or: pipx install pre-commit
make hooks # installs the git pre-commit hookAfter this, every commit runs hygiene checks, shellcheck, yamllint (Ansible tree)
and markdownlint. CI runs the same hooks on every file, so skipping this only moves
the failure to the PR.
The root Makefile is the check driver (make help lists it); CI calls the same
targets, so a local pass means a CI pass. Deploy targets live in
ansible/Makefile and run from ansible/.
| Target | What it does |
|---|---|
make lint |
every pre-commit hook on every file (CI job pre-commit) |
make lint-ansible |
install Galaxy collections + ansible-lint (production profile, which includes the Ansible security rules) |
make molecule |
every ansible/molecule/*/ scenario in parallel, in privileged systemd containers (needs Docker and util-linux flock; pick some with SCENARIOS='a b'). The width defaults to one scenario per 32 of the host's fs.inotify.max_user_instances (4 on a stock 128): every container's systemd draws on that one pool, and past it the next container dies at boot. Raise it (sysctl -w fs.inotify.max_user_instances=1024) or set JOBS=<n>. make molecule-serial runs them one at a time for readable failures. Each scenario takes its own host-wide lock, so two runs can share a host as long as they share no scenario (a run that would refuses to start: runs of one scenario share its container names). make molecule-list prints the selection as JSON (CI's matrix). make molecule-changed runs only the scenarios your branch can affect since CHANGED_FROM (default origin/main), as CI selects them on a pull request (scripts/molecule-affected.py, which says what each changed path selected). The driver is scripts/molecule.sh. |
make lint-helm |
chart: helm lint --strict, positive/negative render tests, kubeconform and promtool check rules on the alert rules (digest-pinned images), the shared schema-key check (needs helm, yq, python3 ≥ 3.11, Docker). Set DECDN_CLI=<path to decdn> to also run the real decdn config validate; CI does, with the pinned release's (cli="$(scripts/fetch-decdn-cli.sh <dir>)" && DECDN_CLI="$cli" make lint-helm). |
make lint-alloy |
renders roles/grafana_alloy's templates and validates them with the real digest-pinned Alloy binary. The molecule stub exits 0 for everything, so this is the only gate that proves the config loads. It also checks that compose/alloy/config.alloy (the Compose alloy profile's render of the same template) is current, and runs it in Alloy: every component healthy, each daemon scraped only under the profiles that start it. ALLOY_BIN=<path> skips the download. |
make lint-compose |
reads compose/compose.yaml as written (yq) against compose/tests/bind-sources.jq (every bind mount's create_host_path), then renders it with every profile on (with its example env, without env files, with an empty .env) and asserts compose/tests/invariants.jq, inline-env.jq and fail-closed.jq |
make lint-cloud-init |
cloud-init schema on cloud-init/user-data-node.yaml and cloud-init/user-data-publisher.yaml, then cloud-init/tests/lint.py on each: only the templates' top-level modules and no YAML anchors, no secrets (only the bootstrap's own files, once each, plain content only, no secret-looking keys or assignments), no hardening skip, release installs verified against the vendored keys with a host-generated node wallet (trust knobs only in their own group's vars, no moved keyring or secret path), only localhost, in decdn_nodes and/or sponsord_hosts (an origin only beside decdn_nodes and with a backend, every backend key only in decdn_origin_nodes.vars, the onramp only beside sponsord), a keyed admin account, runcmd exactly stage 1, shellcheck-clean scripts, and a collection lock that covers ansible/requirements.yml (needs cloud-init, shellcheck, yq). CLOUD_INIT_FILE=<path> checks your own filled-in copy. |
make test-scripts |
tests/scripts-test.sh: the ansible/Makefile scoping guards (dry runs, and limit-guard.sh on a fixture inventory), the molecule driver's selection guards and locks (no containers), the split scenarios' shared inventories, the release gate, scripts/release.sh on a fixture repo and pr-title.yml's check (the former skipped without git-cliff or ssh-keygen on PATH, failed in CI), the negative cases of lint-compose and lint-cloud-init (the latter skipped without cloud-init on PATH), compose/decdn-compose's unit tests (compose/tests/test_decdn_compose.py), scripts/bump_decdn.py's (scripts/tests/test_bump_decdn.py), scripts/molecule-affected.py's against the real tree (scripts/tests/test_molecule_affected.py), that compose/alloy/config.alloy is a fresh render of the grafana_alloy template (scripts/render-compose-alloy.sh --check; needs ansible-core, failed in CI without it) and that Compose's grafana/alloy image is grafana_alloy_version, and the cloud-init bootstrap's contracts: baseline-plays.sh on fixtures and on the real playbooks (every node.yml, origin.yml and sponsord.yml play runs baseline tagged baseline), the two templates' shared stage 1, login hint and final_message, bootstrap.sh's groups against lint.py's, and lint.py's FORBIDDEN_VARS against the roles' keyrings and secret paths; and the Galaxy collections' shipped playbooks: every playbook in a playbooks.txt, galaxy/stage-playbooks.py on both collections and its refusals, that it changes only role names, decdn.node's backup.yml/decommission.yml against the fleet-wide node plays in both directions (needs yq, failed in CI without it), and the two roles' shared *_backup_local_dir default; and examples/collection-fleet/inventory/ against ansible/inventory/ (the same tracked files, byte for byte but for the hosts examples' comment headers; both hosts examples parse to the same inventory, needs ansible-core, failed in CI without it), and the ignore rules that keep a secret.* or a filled-in hosts.yml there out of git, and a vault.* out of this repo's git while a private copy can commit it, with negative cases on a fixture; examples/collection-fleet/scripts/check-vars-shadowing.sh and check-vault-encrypted.sh on fixtures, and that the example's vault-encrypted hook selects every vault file while check-json and yamllint skip them; and that the example's requirements.yml floors are the collections' galaxy.yml versions and its CI and pre-commit pin this repo's action SHAs and hook revs, with negative cases. UPSTREAM=<decdn checkout> adds the sync generators' exit codes. |
make security |
KICS IaC scan of ansible/, the rendered chart and compose/ (digest-pinned engine, fail on HIGH) |
make galaxy-check |
build the decdn.node and decdn.publisher collections, run galaxy-importer's checks on each, then install each and check its playbooks by FQCN, firewall holes included, and run examples/collection-fleet against it as a copy of it runs (its hosts example parses under its ansible.cfg, and every playbook it names ships and passes a syntax check) (ansible/galaxy/check-playbooks.sh; make -C ansible galaxy-check-node / galaxy-check-publisher for one) |
ansible-lint is not a per-commit hook (it needs the collections installed). Run it
with make lint-ansible, or pre-commit run ansible-lint --hook-stage manual.
Two things here are generated from decdn/decdn; regenerate, never hand-edit:
| Mirror | Regenerate with |
|---|---|
ansible/roles/{decdn_node,sponsord,sponsord_onramp,decdn_gateway}/vars/main/networks.yml (contract addresses per network) |
scripts/sync-network-profiles.py <decdn-checkout> |
ansible/molecule/schema/files/schema-keys.txt (node.toml keys) |
ansible/molecule/schema/files/gen-schema-keys.py <decdn-checkout> > … |
Both describe the newest release of the line the roles pin (decdn_node_version), so
generate them from its tag. scripts/sync-network-profiles.py reads git at --ref,
which defaults to that tag (the checkout's highest v<line>.N), whatever the
checkout's own branch; gen-schema-keys.py reads the checkout's working tree, so
check out the tag first.
The full bump checklist is in ansible/roles/decdn_node/README.md;
scripts/open-decdn-bump.sh does most of it (regenerating both) and opens the PR,
listing the rest in the PR body.
sync-network-profiles.py --check exits 1 for "stale" and 2 for "could not run";
the weekly upstream-drift workflow wraps gen-schema-keys.py in a generate-then-diff
check so it reports the two differently too.
The Grafana dashboards and Prometheus alert rules in monitoring/ are not a mirror:
they are maintained here. See its README.
ci.yml, path-filtered so heavy jobs skip unrelated PRs:- always:
pre-commit(every hook, every file),scripts(make test-scripts) andactionlint; - on
ansible/**:ansible-lint(plus a syntax-check of every playbook),galaxy-buildandalloy-config(make lint-alloy); - on
charts/**ormonitoring/**(or the shared schema files, the rootMakefile,ci.yml):helm(make lint-helm); - on
compose/**(or the rootMakefile,ci.yml):compose(make lint-compose), andalloy-config(it also validatescompose/alloy/config.alloy); - on
cloud-init/**(oransible/requirements.yml, the rootMakefile,ci.yml):cloud-init(make lint-cloud-init); - on the Ansible, chart or compose paths:
kics(make security). KICS has no cloud-init platform.
- always:
molecule.yml: onansible/**,cloud-init/**,scripts/molecule.shorscripts/molecule-affected.pychanges, one runner per scenario (make molecule SCENARIOS=<one>), the matrix read frommake molecule-list: every scenario on a push to main; on a pull request, those its changes can affect (scripts/molecule-affected.py; the run's summary says what each changed path selected, and a change no scenario reads runs none). Themoleculejob aggregates them: it is the one check name to require, since the per-scenario names follow the scenario list. Thecloud-initandcloud-init-publisherscenarios boot the real user-data templates, so they need network access to apt, PyPI and Galaxy. Keep each scenario well under its leg's timeout; when one grows, split it along its side effects (assponsord-install,sponsord-wallet,sponsord-onramp-nginx,sponsord-onramp-turnstile,sponsord-onramp-source,iroh-relay-gate,iroh-relay-access,source-build-recoveryand thevalidation-*scenarios are), or move a play into a short sibling that already shares its converge.source-build, its two siblings andsponsord-onramp-sourcedownload rustup and a Rust toolchain from static.rust-lang.org.pr-title.yml: on every PR, and again when its title is edited: the title (the squash-merge subject, whichscripts/release.shrenders into the changelogs) must be a Conventional Commit. It blocks a merge only as a required status check of themainruleset.release-collection.yml/release-chart.yml: onnode-collection-vX.Y.Zandpublisher-collection-vX.Y.Z/decdn-node-X.Y.Ztags, whichscripts/release.shpushes; see RELEASING.md.upstream-drift.yml: weekly, non-blocking; see "Upstream mirrors" above.- Every job is bounded by
timeout-minutes. The values are bounds sized off observed runtimes, not targets. Without one a hung job burns the 360-minute default, and combined withcancel-in-progresssome branch-protection setups read the resulting cancelled check as "not failed". - Caches.
ansible/collectionsis cached acrossansible-lint,galaxy-buildandmoleculeunder one shared key; a hit makesmake depsa no-op that never contactsgalaxy.ansible.com, which keeps a transient Galaxy error from failing an unrelated PR. It usesactions/cache's splitrestore/savewithsavegated on success, so a part-way Galaxy failure can't poison it. pip is cached bysetup-python, keyed on the workflow file (the repo has no pip manifest; installs stay unpinned, so that saves the download, not the PyPI round trip). pre-commit's hook environments are cached on.pre-commit-config.yaml.
- Third-party actions are pinned to a full commit SHA with a version comment — a mutable tag can be re-pointed to malicious code.
Checkmarx/kics-github-actionis deliberately not used. Its git tags were hijacked in the March 2026 TeamPCP attack (CISA KEV), and even post-remediation its entrypointapk addsnodejs/npmat run time inside its digest-pinned base image and executes the result — an unpinned fetch that defeats the pinning. (That fetch also breaks it outright today: Chainguard's current nodejs needs a newer glibc than the pinned base ships, so the action's Node reporter dies and the step fails regardless of findings.) CI instead runsmake security, which drives the Docker Hub KICS engine image — a different artifact from the hijacked action — pinned by a digest verified against Docker Hub, currentlyv2.1.20. The engine's--fail-on highexit code is the gate.- Dependabot (
.github/dependabot.yml) bumps the action SHAs and the pre-commit hook revs weekly. - Bump manually (Dependabot can't parse them): the
KICS_IMAGE,KUBECONFORM_IMAGEandPROMTOOL_IMAGEdigests in theMakefile; the molecule image digests inansible/molecule/*/molecule.yml(all together,docker buildx imagetools inspect); the fivesetup-helmversion:inputs (ci.yml'shelm,kicsandscriptsjobs, both jobs inrelease-chart.yml); the collection versions inansible/requirements.yml, and their exact pins incloud-init/collections.lock.yml(the full transitive set, from a freshmake depsresolve); ansible-core incloud-init/requirements.in, followed by a recompile of the hash-lockedrequirements.txt(command incloud-init/README.md); and the local yamllint hook'sadditional_dependenciespin. - Bump the
cache-epoch:counter inansible/requirements.ymlto make CI re-resolve the collections. Those are>=ranges, so a warm cache pins the resolved set — transitive collections likecommunity.cryptoincluded — until the file changes; the counter forces a fresh resolve without editing the requirements themselves. It is a comment, but a load-bearing one: the cache key is that file's hash. Keeping it in the hashed file is deliberate — an epoch duplicated across both workflows could drift, since neither workflow runs on a change to the other.