You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
pnpm: read and update catalogs in files that pnpm-workspace.yaml includes - #16815
pnpm 12.15 lets pnpm-workspace.yaml delegate settings to other YAML files with include. A common use is moving the catalogs to their own file, so that routine catalog bumps don't touch the protected workspace file:
Dependabot only fetched and parsed pnpm-workspace.yaml, so it never saw the catalog entries in the included file. Their packages were only visible through the lockfile, with an empty requirement list, and version updates ended with Requirements to unlock update_not_possible / No update possible.
This PR:
Fetches the included files.FileFetcher fetches every local source named in include: keys that start with ./ or ../ and name a .yaml/.yml file, resolved from the workspace root as pnpm does. Config dependency sources live in node_modules and are skipped. A missing included file raises DependencyFileNotFound, because pnpm refuses to load the workspace without it. Included files are support files when the workspace file is one, so they behave the same when the workspace file comes from a parent directory. A file in a git submodule keeps the support flag fetch_file_from_host gives it.
Parses catalogs from them.FileParser reads catalog and catalogs from each included file, and the requirement points at that file. Updates therefore go to the file that owns the catalog, which is what pnpm add does.
Updates them.FileUpdater runs the existing PnpmWorkspaceUpdater on included files. It already filters requirements by file name. PnpmLockfileUpdater writes the updated included file before pnpm install --lockfile-only. pnpm-workspace.yaml stays in the set so it is still written, unchanged, next to it.
Writes them for dependency graph lockfile generation.LockfileGenerator runs pnpm in an empty temporary directory and wrote only package.json, .npmrc, .yarnrc* and pnpm-workspace.yaml. Without the included file, pnpm 12.15.0 stops with Failed to read the included file "./pnpm-catalogs.yaml": No such file or directory (os error 2).
The path rules live in one small module, PnpmWorkspaceIncludes, used by all four places.
Anything you want to highlight for special attention from reviewers?
Requirement file = owning file. I considered keeping pnpm-workspace.yaml as the requirement file and redirecting the write. Pointing the requirement at the included file instead reuses PnpmWorkspaceUpdater and DependencyFilesFilterer unchanged, because both already key on the requirement's file name. It also makes the PR touch exactly the file pnpm itself would edit.
Missing included file fails fast with DependencyFileNotFound naming the path, rather than being skipped. pnpm cannot load the workspace without it, so skipping would only move the failure into a less clear pnpm error later.
Side effect, with a spec: extracting the catalog parsing into one method that is also used for included files means a pnpm-workspace.yaml that is not a mapping is now skipped. An included file may legitimately be empty. Before, a workspace file with only comments raised:
The fetcher already accepted such a file ("treats the workspace as having no packages"); now the parser does too.
Several included files (from review):
PnpmWorkspaceUpdater now takes the previous requirement from the same file. It used to take the first one with matching groups, from any file.
The lockfile updater writes exactly the files that include names, even one that is itself called pnpm-workspace.yaml in a subdirectory.
The parser takes as the root the workspace file that no other file includes.
Paths above the project or repository (from review):
The fetcher raises DependencyFileNotFound for an included path that resolves above the repository root. Before, fetch_file_from_host silently read the same name at the root, and the later write would have gone outside the clone.
The graph LockfileGenerator no longer writes an included file that sits above the project, such as ../shared/catalog.yaml, into the parent of its temporary directory. This also holds for an included file named pnpm-workspace.yaml: the path check runs before the filename allow-list, which would otherwise match it. It skips that file, so pnpm stops on it as it did before this PR. Staging deeper was not done, because the existing handling of a pnpm-workspace.yaml fetched from a parent directory (../pnpm-workspace.yaml, already written verbatim) relies on the current layout. That pre-existing write is left for a separate change.
pnpm version.include needs pnpm 12.15 or later, and the updater image still ships pnpm 12.10.1 (ARG PNPM_VERSION in npm_and_yarn/Dockerfile). This works because Dependabot installs the exact version pinned in packageManager through corepack. I checked all three setups with the real updater image:
Repository
main
This PR
packageManager: pnpm@12.15.0
pnpm 12.15.0 runs; No update possible
pnpm 12.15.0 runs; update succeeds
No packageManager
No update possible
The image's pnpm 12.10.1 runs, warns The following settings in pnpm-workspace.yaml are not recognized by this version of pnpm and were ignored: "include", and fails with ERR_PNPM_CATALOG_ENTRY_NOT_FOUND_FOR_SPEC, reported as dependency_file_not_resolvable
Only engines.pnpm: ">=12.15.0"
No update possible
Same as no packageManager. Dependabot does not use engines as a minimum version here; that is existing behavior.
So unpinned repositories that use include weren't updated before and still aren't. Their failure is now visible instead of silent, but it is generic. This is a known limitation. Bumping PNPM_VERSION in the Dockerfile to 12.15 or later will be a separate change; a clearer "pnpm too old for include" error could follow.
Deliberately not changed:
packages delegated to an included file is not used for workspace package discovery. fetch_pnpm_workspace_package_jsons still reads packages from pnpm-workspace.yaml only.
Other settings delegated to included files are not read by Dependabot's own pnpm config checks, such as minimumReleaseAge for the cooldown floor and the lockfile-verification settings. Those checks still look only at pnpm-workspace.yaml and .npmrc. pnpm itself still applies them at install time, because the files are on disk in the cloned repository.
Config dependency sources are not fetched.
How will you know you've accomplished your goal?
Before/after with the real updater image (ghcr.io/dependabot/dependabot-updater-npm) and the Dependabot CLI, pnpm 12.15.0 from packageManager. "After" mounts this branch's npm_and_yarn/lib over the image's copy.
Project
Before
After
Minimal project, prettier: 3.3.0 in catalog: of pnpm-workspace.yaml (control)
Updating prettier from 3.3.0 to 3.9.10; PR changes pnpm-workspace.yaml and pnpm-lock.yaml
Same
Same project with the catalog moved to ./pnpm-catalogs.yaml (include: ./pnpm-catalogs.yaml: [catalog]), lockfile generated by pnpm 12.15.0
Fetched files package.json, pnpm-lock.yaml, pnpm-workspace.yaml; prettier 3.3.0 with requirements: []; Requirements to unlock update_not_possible, No update possible for prettier 3.3.0
pnpm-catalogs.yaml fetched; requirement 3.3.0 in pnpm-catalogs.yaml; Updating prettier from 3.3.0 to 3.9.10; PR changes pnpm-catalogs.yaml (prettier: 3.9.10) and pnpm-lock.yaml; pnpm-workspace.yaml untouched
meta/docusaurus at fd9f8d3 (the branch of meta/docusaurus#12815, which moves the catalogs with include: ./pnpm-catalogs.yaml: [catalogs]), --dep chokidar
chokidar 3.6.0 with requirements: []; No update possible for chokidar 3.6.0
Requirement ^3.6.0 in pnpm-catalogs.yaml; Updating chokidar from 3.6.0 to 6.0.0; PR changes one line of pnpm-catalogs.yaml (chokidar: ^3.6.0 → chokidar: ^6.0.0) and pnpm-lock.yaml (catalog specifier and resolution, chokidar 3's transitive packages removed)
pnpm-workspace.yaml containing only comments
NoMethodError (trace above)
Manifest dependencies parsed
is-even in two included catalogs, catalogs/legacy.yaml (0.1.1) and config/pnpm-workspace.yaml (0.1.2), --dep is-even (review follow-up; "before" is this PR's first commit)
Only catalogs/legacy.yaml updated; config/pnpm-workspace.yaml and the lockfile's current catalog stay at 0.1.2
Both files at is-even: 1.0.0; both lockfile catalogs at specifier: 1.0.0 / version: 1.0.0
The update job does not raise before the fix; it silently reports no update, as in the table. The new file updater spec, which hands the updater a requirement in an included file, fails on main with:
Dependabot::NpmAndYarn::FileUpdater::NoChangeError:
No files were updated! Package manager: pnpm
# ./lib/dependabot/npm_and_yarn/file_updater.rb:58:in 'Dependabot::NpmAndYarn::FileUpdater#updated_dependency_files'
Tests. These were written first, and each fails on main:
pnpm_workspace_includes_spec.rb (new): path resolution for ./ and ../ sources, a workspace file from a parent directory, config dependency and non-YAML sources ignored, and no include / not a mapping / invalid YAML. On main it cannot load the module.
file_fetcher_spec.rb: included file fetched and not a support file, but still a support file when it is in a git submodule; missing included file raises DependencyFileNotFound (/pnpm-catalogs.yaml); a workspace from a parent directory fetches ../../pnpm-catalogs.yaml with the workspace file's support flag. Uses a cloned fixture repo.
file_parser_spec.rb: catalog and named-catalog entries from the included file get it as their requirement file; an unparseable included file raises DependencyFileNotParseable; a comment-only workspace file parses.
file_updater_spec.rb: runs real pnpm 12.15.0 against the new pnpm/catalog_include fixture. Only pnpm-catalogs.yaml and pnpm-lock.yaml are updated, with prettier: 3.3.3 and specifier: 3.3.3 / version: 3.3.3.
lockfile_generator_spec.rb: the included file exists in the temporary directory when pnpm runs, and one above the project is not written outside it, including one named pnpm-workspace.yaml.
Review follow-up, using the new pnpm/catalog_include_multiple fixture (lockfile generated by pnpm 12.15.0): pnpm_workspace_updater_spec.rb replaces each file's own previous requirement; file_updater_spec.rb runs real pnpm and checks that both files and both lockfile catalogs are updated; file_parser_spec.rb detects the root workspace file when an included config/pnpm-workspace.yaml comes first; file_fetcher_spec.rb rejects an include above the repository root. Each failed on this PR's first commit.
The pnpm/catalog_include fixture lockfile was generated with pnpm 12.15.0.
Suites, run in the npm updater image with this branch mounted:
Full npm_and_yarn suite: 2595 examples, 0 failures, 4 pending (after the second review follow-up).
RuboCop on the changed files, from the repository root: no offenses.
Sorbet (srb tc): no errors, the same as a clean checkout of main.
Not verified:
Security updates and grouped updates with an included catalog. Only single-dependency version updates were replayed end to end.
The graph command end to end. The lockfile generator change is covered by its spec, and the failure it prevents was reproduced by running pnpm 12.15.0 by hand in a directory without the included file.
'*' sources. They go through the same code (every local included file is read for catalog/catalogs), but only [catalog], [catalogs] and [catalogs.<name>] were run end to end.
Checklist
I have run the complete test suite to ensure all tests and linters pass. (The full npm_and_yarn suite, RuboCop on the changed files and Sorbet; other ecosystems are untouched.)
I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
I have written clear and descriptive commit messages.
I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
I have ensured that the code is well-documented and easy to understand.
…udes
pnpm 12.15 lets pnpm-workspace.yaml delegate settings to other YAML
files with `include`, for example a catalog moved to its own file:
include:
./pnpm-catalogs.yaml: [catalogs]
Dependabot only fetched and parsed pnpm-workspace.yaml, so catalog
entries in an included file were only seen through the lockfile, with
no requirement, and version updates stopped with "No update possible".
- Fetch every local file that `include` names, resolved from the
workspace root. Config dependency sources live in node_modules and are
skipped. A missing included file raises DependencyFileNotFound, since
pnpm refuses to load the workspace without it.
- Parse catalog and catalogs from the included files, with the included
file as the requirement file, so updates go to the file that owns the
catalog, as pnpm add does.
- Update included files with PnpmWorkspaceUpdater and write them before
pnpm install --lockfile-only.
- Write included files when the dependency graph generates a lockfile.
Parsing the workspace catalogs now also skips a pnpm-workspace.yaml that
is not a mapping (for example only comments), which used to raise
NoMethodError.
Fixes#16766
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Parent-relative catalog includes can overwrite files outside the temporary staging area and survive cleanup, while file-selection bugs can leave catalog requirements and lockfiles stale.
Adds support for pnpm catalogs stored in locally included YAML files, allowing Dependabot to discover their dependencies and update the owning files rather than pnpm-workspace.yaml.
Changes:
Centralizes include-path resolution and fetches referenced local YAML files.
Parses catalog requirements from included files and writes updates before lockfile regeneration.
Stages included files for dependency graph generation and adds regression tests and fixtures.
…ject
Follow-up to review of the pnpm include support.
- PnpmWorkspaceUpdater takes the previous requirement from the same
file, so a dependency in catalogs in several included files is
updated in each of them, not only in the first.
- The pnpm lockfile updater writes exactly the files that include
names, so an included file that is itself named pnpm-workspace.yaml
in a subdirectory is written before pnpm install.
- The parser takes as root the workspace file that no other file
includes, whatever order the files come in.
- The dependency graph lockfile generator does not write an included
file above the project into the parent of its temporary directory.
- The fetcher rejects an included path above the repository root,
which was read from the root instead.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
An included workspace file can bypass path filtering and be written outside the temporary directory, while catalog handling can clear submodule support flags and allow unintended updates.
For cloned repositories, fetch_file_from_host marks files inside Git submodules as support files (common/lib/dependabot/file_fetchers/base.rb:275). When a non-support workspace includes a catalog in a submodule, this assignment clears that flag, allowing updates to be returned for a file owned by the submodule. Preserve the fetched file's support status and only set it to true when the workspace is also a support file.
Follow-up to review of the pnpm include support.
- The dependency graph lockfile generator checks an included file's path
before its filename allow-list. An included file that is itself named
pnpm-workspace.yaml and sits above the project used to pass that
allow-list and was written outside the temporary directory.
- The fetcher keeps the support flag that fetch_file_from_host sets for a
file in a git submodule, and only adds the workspace file's own flag,
so a catalog in a submodule is not treated as owned by the repository.
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
A catalog in a git submodule remains updatable, allowing its edit to be dropped while a lockfile change is retained; support-file includes need exclusion and regression coverage for parsing and update generation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What are you trying to accomplish?
pnpm 12.15 lets
pnpm-workspace.yamldelegate settings to other YAML files withinclude. A common use is moving the catalogs to their own file, so that routine catalog bumps don't touch the protected workspace file:Dependabot only fetched and parsed
pnpm-workspace.yaml, so it never saw the catalog entries in the included file. Their packages were only visible through the lockfile, with an empty requirement list, and version updates ended withRequirements to unlock update_not_possible/No update possible.This PR:
FileFetcherfetches every local source named ininclude: keys that start with./or../and name a.yaml/.ymlfile, resolved from the workspace root as pnpm does. Config dependency sources live innode_modulesand are skipped. A missing included file raisesDependencyFileNotFound, because pnpm refuses to load the workspace without it. Included files are support files when the workspace file is one, so they behave the same when the workspace file comes from a parent directory. A file in a git submodule keeps the support flagfetch_file_from_hostgives it.FileParserreadscatalogandcatalogsfrom each included file, and the requirement points at that file. Updates therefore go to the file that owns the catalog, which is whatpnpm adddoes.FileUpdaterruns the existingPnpmWorkspaceUpdateron included files. It already filters requirements by file name.PnpmLockfileUpdaterwrites the updated included file beforepnpm install --lockfile-only.pnpm-workspace.yamlstays in the set so it is still written, unchanged, next to it.LockfileGeneratorruns pnpm in an empty temporary directory and wrote onlypackage.json,.npmrc,.yarnrc*andpnpm-workspace.yaml. Without the included file, pnpm 12.15.0 stops withFailed to read the included file "./pnpm-catalogs.yaml": No such file or directory (os error 2).The path rules live in one small module,
PnpmWorkspaceIncludes, used by all four places.Fixes #16766
Anything you want to highlight for special attention from reviewers?
Requirement file = owning file. I considered keeping
pnpm-workspace.yamlas the requirement file and redirecting the write. Pointing the requirement at the included file instead reusesPnpmWorkspaceUpdaterandDependencyFilesFiltererunchanged, because both already key on the requirement's file name. It also makes the PR touch exactly the file pnpm itself would edit.Missing included file fails fast with
DependencyFileNotFoundnaming the path, rather than being skipped. pnpm cannot load the workspace without it, so skipping would only move the failure into a less clear pnpm error later.Side effect, with a spec: extracting the catalog parsing into one method that is also used for included files means a
pnpm-workspace.yamlthat is not a mapping is now skipped. An included file may legitimately be empty. Before, a workspace file with only comments raised:The fetcher already accepted such a file ("treats the workspace as having no packages"); now the parser does too.
Several included files (from review):
PnpmWorkspaceUpdaternow takes the previous requirement from the same file. It used to take the first one with matching groups, from any file.includenames, even one that is itself calledpnpm-workspace.yamlin a subdirectory.Paths above the project or repository (from review):
DependencyFileNotFoundfor an included path that resolves above the repository root. Before,fetch_file_from_hostsilently read the same name at the root, and the later write would have gone outside the clone.LockfileGeneratorno longer writes an included file that sits above the project, such as../shared/catalog.yaml, into the parent of its temporary directory. This also holds for an included file namedpnpm-workspace.yaml: the path check runs before the filename allow-list, which would otherwise match it. It skips that file, so pnpm stops on it as it did before this PR. Staging deeper was not done, because the existing handling of apnpm-workspace.yamlfetched from a parent directory (../pnpm-workspace.yaml, already written verbatim) relies on the current layout. That pre-existing write is left for a separate change.pnpm version.
includeneeds pnpm 12.15 or later, and the updater image still ships pnpm 12.10.1 (ARG PNPM_VERSIONinnpm_and_yarn/Dockerfile). This works because Dependabot installs the exact version pinned inpackageManagerthrough corepack. I checked all three setups with the real updater image:mainpackageManager: pnpm@12.15.0No update possiblepackageManagerNo update possibleThe following settings in pnpm-workspace.yaml are not recognized by this version of pnpm and were ignored: "include", and fails withERR_PNPM_CATALOG_ENTRY_NOT_FOUND_FOR_SPEC, reported asdependency_file_not_resolvableengines.pnpm: ">=12.15.0"No update possiblepackageManager. Dependabot does not useenginesas a minimum version here; that is existing behavior.So unpinned repositories that use
includeweren't updated before and still aren't. Their failure is now visible instead of silent, but it is generic. This is a known limitation. BumpingPNPM_VERSIONin the Dockerfile to 12.15 or later will be a separate change; a clearer "pnpm too old forinclude" error could follow.Deliberately not changed:
packagesdelegated to an included file is not used for workspace package discovery.fetch_pnpm_workspace_package_jsonsstill readspackagesfrompnpm-workspace.yamlonly.minimumReleaseAgefor the cooldown floor and the lockfile-verification settings. Those checks still look only atpnpm-workspace.yamland.npmrc. pnpm itself still applies them at install time, because the files are on disk in the cloned repository.How will you know you've accomplished your goal?
Before/after with the real updater image (
ghcr.io/dependabot/dependabot-updater-npm) and the Dependabot CLI, pnpm 12.15.0 frompackageManager. "After" mounts this branch'snpm_and_yarn/libover the image's copy.prettier: 3.3.0incatalog:ofpnpm-workspace.yaml(control)Updating prettier from 3.3.0 to 3.9.10; PR changespnpm-workspace.yamlandpnpm-lock.yaml./pnpm-catalogs.yaml(include: ./pnpm-catalogs.yaml: [catalog]), lockfile generated by pnpm 12.15.0package.json,pnpm-lock.yaml,pnpm-workspace.yaml;prettier 3.3.0withrequirements: [];Requirements to unlock update_not_possible,No update possible for prettier 3.3.0pnpm-catalogs.yamlfetched; requirement3.3.0inpnpm-catalogs.yaml;Updating prettier from 3.3.0 to 3.9.10; PR changespnpm-catalogs.yaml(prettier: 3.9.10) andpnpm-lock.yaml;pnpm-workspace.yamluntouchedmeta/docusaurusatfd9f8d3(the branch of meta/docusaurus#12815, which moves the catalogs withinclude: ./pnpm-catalogs.yaml: [catalogs]),--dep chokidarchokidar 3.6.0withrequirements: [];No update possible for chokidar 3.6.0^3.6.0inpnpm-catalogs.yaml;Updating chokidar from 3.6.0 to 6.0.0; PR changes one line ofpnpm-catalogs.yaml(chokidar: ^3.6.0→chokidar: ^6.0.0) andpnpm-lock.yaml(catalog specifier and resolution, chokidar 3's transitive packages removed)pnpm-workspace.yamlcontaining only commentsNoMethodError(trace above)is-evenin two included catalogs,catalogs/legacy.yaml(0.1.1) andconfig/pnpm-workspace.yaml(0.1.2),--dep is-even(review follow-up; "before" is this PR's first commit)catalogs/legacy.yamlupdated;config/pnpm-workspace.yamland the lockfile'scurrentcatalog stay at0.1.2is-even: 1.0.0; both lockfile catalogs atspecifier: 1.0.0 / version: 1.0.0The update job does not raise before the fix; it silently reports no update, as in the table. The new file updater spec, which hands the updater a requirement in an included file, fails on
mainwith:Tests. These were written first, and each fails on
main:pnpm_workspace_includes_spec.rb(new): path resolution for./and../sources, a workspace file from a parent directory, config dependency and non-YAML sources ignored, and noinclude/ not a mapping / invalid YAML. Onmainit cannot load the module.file_fetcher_spec.rb: included file fetched and not a support file, but still a support file when it is in a git submodule; missing included file raisesDependencyFileNotFound(/pnpm-catalogs.yaml); a workspace from a parent directory fetches../../pnpm-catalogs.yamlwith the workspace file's support flag. Uses a cloned fixture repo.file_parser_spec.rb: catalog and named-catalog entries from the included file get it as their requirement file; an unparseable included file raisesDependencyFileNotParseable; a comment-only workspace file parses.file_updater_spec.rb: runs real pnpm 12.15.0 against the newpnpm/catalog_includefixture. Onlypnpm-catalogs.yamlandpnpm-lock.yamlare updated, withprettier: 3.3.3andspecifier: 3.3.3 / version: 3.3.3.lockfile_generator_spec.rb: the included file exists in the temporary directory when pnpm runs, and one above the project is not written outside it, including one namedpnpm-workspace.yaml.pnpm/catalog_include_multiplefixture (lockfile generated by pnpm 12.15.0):pnpm_workspace_updater_spec.rbreplaces each file's own previous requirement;file_updater_spec.rbruns real pnpm and checks that both files and both lockfile catalogs are updated;file_parser_spec.rbdetects the root workspace file when an includedconfig/pnpm-workspace.yamlcomes first;file_fetcher_spec.rbrejects an include above the repository root. Each failed on this PR's first commit.The
pnpm/catalog_includefixture lockfile was generated with pnpm 12.15.0.Suites, run in the npm updater image with this branch mounted:
npm_and_yarnsuite: 2595 examples, 0 failures, 4 pending (after the second review follow-up).srb tc): no errors, the same as a clean checkout ofmain.Not verified:
graphcommand end to end. The lockfile generator change is covered by its spec, and the failure it prevents was reproduced by running pnpm 12.15.0 by hand in a directory without the included file.'*'sources. They go through the same code (every local included file is read forcatalog/catalogs), but only[catalog],[catalogs]and[catalogs.<name>]were run end to end.Checklist
npm_and_yarnsuite, RuboCop on the changed files and Sorbet; other ecosystems are untouched.)