Embarking on a Epic Journey: Getting the Best out of NuGet

Written by Marcin Krystianc, Open Source Software Developer

At G-Research we tend to have very large .NET projects. As our projects grow in size, we have noticed that our NuGet restore times start to get out of control, sometimes taking 10 minutes or more.

I was curious whether I could find any performance improvements in the NuGet restore operation, so I started to run a series of experiments and investigations. What follows is a two-part series about my journey.

Different SDK Versions

NuGet was initially designed as a standalone Windows-only application. With the first release of the .NET Core, NuGet got fully integrated into the .NET SDK toolbox.

Because every new major release of .NET is advertised as faster than the previous ones, I wanted to run some performance tests across different .NET SDK versions to see whether the same improvements apply for NuGet.

Test Scenarios

The NuGet repository comes with scripts for running performance tests. I’ve performed the following test scenarios (repeated 50 times for better accuracy):

• Arctic – clean all local caches before running restore, restore time impacted by NuGet packages downloads from remote NuGet feeds
• Force – already restored, run again dotnet restore with --force
• NoOp – already restored, run again dotnet restore without --force

I used the SanitisedNet471 solution for my tests. It is a representative example of solutions that we are working on at G-Research. It consists of more than 300 projects, and references about 300 different NuGet packages.

Test Hardware:

• AWS m5.2xlarge instance 
• 32GB RAM
• Xeon 8175M, (4/8 virtual processors)

Results:

Conclusions & Next Steps:

From the chart from above we can draw several conclusions:

• There is a substantial slowdown between version v3.1.407 and v5.0.202 for the “Arctic” scenario.
• For other scenarios which don’t involve downloading NuGet packages from remote feeds, the fastest version is v5.0.202.
• Version v6.0.100-preview.2.21155.3 is slightly slower than version v5.0.202.

Speed matters a lot for this function, as G-Research has very large projects, and developers need to run restores often as part of their development process. Any incremental improvement in the algorithms translates into significant gains in our development cycle.

I had expected that each new release of .NET would be faster than the version before, but this is not always the case. Clearly, there is more work to be done in this arena. In upcoming blog entries, I plan to investigate various approaches to speeding things up, including:

• CPVM
• Static Graph
• Locked Mode
• Nearest Win

Looking forward to sharing those results with you soon!