Centrally Managing NuGet package versions (CPVM)

Written by Marcin Krystianc, Open Source Software Developer

CPVM (“Central Package Version Management”) is a NuGet feature that simplifies the package version management. Instead of specifying the version of a package each time it is being referenced in a project file, versions of all packages are specified only once in a single `Directory.Packages.props` file. 

Moving version management to a single file is a great improvement and is especially valuable for large solutions. Although CPVM is available in NuGet for anyone to use, it is worth noting that it is still in an experimental phase and isn’t officially released yet. 

CPVM and performance

At G-Research we were looking for a workaround to a problem in which a specific solution, `nuget restore`, took too long. To our surprise, we discovered that enabling CPVM greatly reduced the time it took for a restore operation. To illustrate the scale of this change, we prepared two fixes,  SanitisedNetCoreApp3.1Centralised and SanitisedNetCoreApp3.1, in which one uses CPVM and the other doesn’t. The results of the benchmark are presented in the table below.

As we can clearly see, the time difference is mind blowing. The solution which uses CPVM restores six times faster than the original one!

So how exactly does CPVM improve performance ?

The performance improvement that comes with using CPVM is quite significant but it turns out to be an accidental side effect of its subfeature, called “transitive dependency pinning”.

At the beginning of the restore operation, NuGet constructs a dependency graph by recursively following all referenced projects and packages from a given project. This graph is used to detect circular dependencies and version downgrades and also to determine the flat list of packages referenced directly or transitively from it. Sadly, this graph construction algorithm has an exponential nature and for each project and package it may instantiate many nodes internally. Most of the time this is not a big problem, however for large solutions it can result in graphs containing millions of nodes, for which construction time is unacceptably slow.

The CPVM and its transitive dependency pinning mechanism changes how the graph construction algorithm works for packages which are managed centrally. So when the graph traversal algorithm encounters a node with a package version that is centrally managed, it doesn’t add it to the graph and doesn’t process it recursively as it would have normally done. Instead, it adds it to the global list of centrally managed packages and the processing stops at this point. That new behaviour dramatically reduces the number of internal nodes that are created by the algorithm, and thus leads to the performance improvements that we can see in our benchmarks.

Transitive dependency pinning

So what is transitive dependency pinning exactly? It is a feature that guarantees that all projects in the solution are being built and tested with the same package versions that are defined in the central file. Regardless of whether the package dependency is direct or transitive, it is handled in the same way. It also changes how dependencies are being handled when authoring new packages. Without transitive dependency pinning, only direct dependencies are included into the package dependency list; with transitive dependency pinning both direct and transitive dependencies are included. 

Although the performance improvement of CPVM and transitive dependency pinning are massive, the transitive dependency pinning itself has other effects which are undesirable and therefore was disabled by NuGet maintainers. We were unpleasantly surprised to discover this – the CPVM still works but it doesn’t speed-up the restore operation anymore.  

The NuGet community is divided on this issue. There are people claiming that transitive pinning for CPVM is essential and others for whom transitive pinning is a no-go. For us at G-Research, it is an important feature not for its primary functionality, but because of its positive impact on the restore time.

Recognising the conflicting expectations, we asked the NuGet maintainers whether they could implement transitive dependency pinning as an optional feature. This would allow projects which don’t want transitive dependency pinning to opt out and projects which do appreciate that feature to use it.

The maintainers agreed that it was a good idea but they did not have the capacity to work on it. So, we at G-Research took the initiative and submitted a change with the requested functionality. At the time of writing this post the change hasn’t yet been accepted, but we are optimistic that it eventually will be. And we hope that the unexpected benefits of transitive dependency pinning for CPVM will be available for others to use as well.