Skip to content

k8s 0.8.x: hashFiles() always returns an empty string, silently freezing every actions/cache key #448

Description

@remidebette

Summary

In kubernetes-novolume mode, and in any runner image built from main, ${{ hashFiles(...) }} always evaluates to the empty string — for every pattern, including a file that certainly exists such as README.md.

Nothing fails. The job stays green. But every cache key derived from hashFiles becomes a constant, and actions/cache writes only on an exact-key miss. So the entry is written once and never updates again.

Affected versions

Hook Workspace hashFiles()
0.7.0 shared volume works
0.8.0, 0.8.1, current main per-pod emptyDir + exec cp returns ""

actions/runner's images/Dockerfile pins RUNNER_CONTAINER_HOOKS_VERSION=0.7.0 for /home/runner/k8s and v0.8.1 for /home/runner/k8s-novolume. So users reach this by opting into containerMode: kubernetes-novolume, or by building a runner image from main.

Reproduction

Two otherwise identical scale sets on one cluster — same chart (gha-runner-scale-set 0.14.2), same pod template. The only difference is the hook bundle at /home/runner/k8s/index.js.

- name: Probe hashFiles
  run: echo "hashFiles(README.md) = '${{ hashFiles('README.md') }}'"
Hook Output
0.7.0 (stock image) a 64-character digest
main empty string

Every other pattern we tried behaved the same way.

Root cause

hashFiles() is an expression, so the runner evaluates it before the step runs — not the workflow pod.

Since #244 the workflow pod's /__w is a pod-local emptyDir:

// packages/k8s/src/k8s/index.ts
appPod.spec.volumes = [
  { name: WORK_VOLUME, emptyDir: {} },
  ...
]

actions/checkout runs as a step, so the repo lands in that emptyDir. runScriptStep copies only _temp in, and only _temp/_runner_file_commands back:

// packages/k8s/src/hooks/run-script-step.ts
await execCpToPod(state.jobPod, runnerTemp, containerTempSrc)
// ...
await execCpFromPod(state.jobPod, `${containerTemp}/_runner_file_commands`, `${workdir}/_temp`)

The workspace is never copied back, so the runner's $GITHUB_WORKSPACE stays empty and every glob matches zero files.

Same root cause as #280, reported there from the other end: "Looking at the runner, /home/runner/_work/<repo>/<repo> directory is empty." #299 / #300 (event.json) and #347 are further symptoms.

Why this one needs separate tracking

Every other symptom of this root cause is a hard error somebody notices — #280 Can't find 'action.yml', #299 / #300 missing $GITHUB_EVENT_PATH, #423 PrepareJob failure.

hashFiles() returning "" is a legal result. There is no warning. A workflow using

key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
restore-keys: npm-${{ runner.os }}-

pins itself to a cache entry written on its first ever run. And via the restore-keys prefix, that frozen entry can also be restored by jobs running on healthy 0.7.0 runners in the same repository.

How we hit it

We are affected by #228. #333 is merged but unreleased, so we built a runner image from main to pick it up. That also moved us from the 0.7.0 shared-volume workspace to the 0.8.x exec-cp one — our scale set still declares containerMode: kubernetes with a work volume claim, which the main hook ignores.

So the architecture change is easy to import unintentionally while chasing an unreleased fix.

Suggestion

#339 reverts to volumes and would fix this. Two smaller asks meanwhile:

  1. Document the constraint in packages/k8s/README.md. Under the exec-cp implementation the runner cannot see the workspace, so hashFiles(), local composite actions, and anything else resolved against GITHUB_WORKSPACE do not work. The README currently documents the opposite — "The runner pod should map a persistent volume claim into the _work directory".
  2. Consider whether hashFiles() matching zero files should warn instead of returning "". That is actions/runner territory, but the silence is what makes this dangerous.

Root-cause analysis assisted by Claude. The A/B measurement, the code references and the version pins were verified by hand.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions