"Hang loose" is for surfers, not developers [Why I pin dependency versions in Node.js packages]
A few days ago, I posted a response to a question I get asked about open-source project management. Here we go again - this time the topic is dependency versioning.
What is a package dependency?
In the Node.js ecosystem, packages (a.k.a. projects) can make use of other packages by declaring them as a dependency in package.json and specifying the range of supported versions.
When a package gets installed, the package manager (typically npm) makes sure appropriate versions of all dependencies are included.
How are dependency versions specified?
The Node community uses semantic versioning of the form major.minor.patch.
There are many ways to specify a version range, but the most common is to identify a specific version (typically the most recent) and prefix it with a tilde or caret to signify that later versions which differ only by patch or minor.patch are also acceptable.
For example: ~1.2.3 and ^1.2.3.
This is what is meant by "loose" versioning.
Why does the community use "loose" versioning?
The intent of loose versioning is to automatically benefit from bug fixes and non-breaking changes to dependent packages. Any time an install is run or an update is performed, the latest (allowable) version of such dependencies will be included and the user will seamlessly benefit from any bug fixes that were made since the named version was published.
What is a "pinned" dependency version?
A pinned dependency version specifies a particular major.minor.patch version and does not include any modifiers.
The only version that satisfies this range is the exact version listed.
For example: 1.2.3.
Bug fixes to such package dependencies will not be used until a new version of the package that references them is published (with updated references).
Why is pinning a better versioning strategy?
Pinning ensures that users only run a package with the set of dependencies it has been tested with. While this doesn't rule out the possibility of bugs, it's far safer and more predictable than loose versioning, which allows users to run with an unpredictable set of dependencies. In the loose versioning worst case, every install of a package could have a different set of dependencies. This is a nightmare for quality and reliability. With pinning, behavior changes only show up when the user decides to update versions. If anything breaks, the upgrade can be skipped while the issue is investigated. Loose versioning doesn't allow "undo"; when something breaks, you're stuck until a fix gets published.
What's so bad about running untested configurations?
As much as developers may try to ensure consistent behavior across minor- and patch-level version updates, any change - no matter how small - has the possibility of altering behavior and causing failures.
Worse, such behavior changes show up unexpectedly and unpredictably and can be difficult to track down, especially for users who may not even realize the broken package was being used.
I've had to investigate such issues on multiple occasions and think it is a waste of time for users and package maintainers alike.
Are popular projects safer to version loosely?
Well-run projects with thorough testing are probably less likely to cause problems then single-person hobby projects. But the underlying issue is the same: any change to dependency code can change runtime behavior and cause problems.
What about missing out on security bug fixes due to pinning?
While the urgency to include a security bug fix may be higher than a normal bug fix, the same challenges apply. There's no general-purpose way to identify a security fix from a normal fix from a breaking change.
Could pinning lead to larger install sizes?
Yes, because the package manager doesn't have as much freedom to choose among package versions that are shared by multiple dependencies. However, this is a speculative optimization with limited benefit in practice as disk space is comparatively inexpensive. Correctness and predictability are far more important.
Isn't pinning pointless if dependent packages version loosely?
No, though it's less effective because those transitive dependencies can change/break at any time. My opinion is that every package should use pinning, but I can only enforce that policy for my own packages. (But maybe by setting a good example, I can be the change I want to see in the world...)
Is there a way to force a dependency update for a pinned package?
Yes, by updating a project's package.json to use overrides (npm) or resolutions (yarn).
This means users who are worried about a specific dependency version can make sure that version is used in their scenario - and any resulting problems are their responsibility to deal with.
Does pinning versions create more work for a maintainer?
No, maintainers should already be updating package dependencies as part of each release.
This can be done manually or automatically through the use of a tool like Dependabot.