Migrate from Yarn to pnpm

[awilfox]

PNPM_HOME is where the global binaries are installed - it is not where any of the modules are installed. Therefore it does not need to be, and absolutely should not be, writable by the app user's account.

The suggested installation method was curl|bash, which felt quite unauditable. I read the entire 400 line script, and besides signature validation (which was only possible on NPM-uploaded versions, i.e. 11.x) it's just a thick wrapper around figuring out how to download a tarball and extract it. The RUN I wrote pretty much does the same thing.

The pnpmVersion variable in that RUN should be set to the desired tag in the upstream pnpm repository. They have immutable tags and releases set, so once we have verified a version is not malicious, we can be confident in deploying it whenever we want - the binaries will not change.

--force is documented as only allowing PNPM_HOME to be overridden (which we want it to be), but also allows installation to continue without modifying the shell login file. By default, it wants to add bin-stub paths to the user's PATH by writing to .profile/.bash_profile/.zshrc - we don't want this in a Docker container, and it can't do it anyway because the shell detection heuristic fails due to empty $SHELL environment variable.

To upgrade Node, run pnpm runtime set node [version], i.e. pnpm runtime set node 24 is what I ran in the UCBEARS repo. The runtime version itself is saved to the lock file, so upon first fetch it will download the runtime as well. Since signatures are verified, that means we never have to worry about dealing with Node's repositories nor GPG keys again!

The -P flag to pnpm install ensures that only runtime dependencies are installed. This is all that is needed for the production target; devDependencies are unused. This keeps the container size smaller.

A node_modules volume is added to the Compose file. This is to keep the container's modules separate from your local development environment. Otherwise, when using pnpm from your local environment after running compose up or compose build, you will see something like:

[ERR_PNPM_UNEXPECTED_STORE] Unexpected store location

The dependencies at "/Users/awilcox/Code/UCB/UCBEARS/node_modules" are currently linked from the store at "/opt/app/node_modules/.pnpm-store/v11".

pnpm now wants to use the store at "/Users/awilcox/Library/pnpm/store/v11" to link dependencies.

Or

✔ The modules directory at "/Users/awilcox/Code/UCB/UCBEARS/node_modules" will be removed and reinstalled from scratch. Proceed?

If you see either of those messages, stop and ensure node_modules is a separate volume.

This change was solely from manually installing Node and using Yarn as a package manager to using PNPM as both a package manager and Node manager. This did not remove Webpack nor did it update any of the (in some cases, very stale) JS dependencies of the repo. It is hoped that this will be significantly easier with modern tooling, but was out of the scope of the ticket.

[anarchivist]

in general, so far i think it looks pretty good. per your comments do we need separate tickets to replace webpack, etc. for this app?

[awilfox]

Webpack itself is still maintained, it's the Ruby Gem connecting it to the asset pipeline that is deprecated and EOL. So I think we can actually keep using it just fine. (They even have plans for a new major release next year.)

[anarchivist]

Thanks! (I suppose what I meant is do we want to bother with switching to Propshaft, etc.?)

[awilfox]

Oh, yes, replacing Sprockets with Propshaft is probably its own ticket. I don't mind looking into that after this is merged, but I also don't see a huge emergency in it, either. We are already using jsbundling-rails and cssbundling-rails here, so we don't benefit a lot from the swap beyond "it's better maintained upstream".

Edit: I do realise that the original intent of the ticket was that change, but the more I dug in, the more I realised that our actual friction point is Yarn. I feel like switching to pnpm solves the issues we were having, and also provides us a clear pathway towards Propshaft if we want that later.

[anarchivist]

@awilfox OK, I've created AP-868 for that work.