I'm currently looking at NuGet to solve my dependency problems in TFS and what I wanted to do is to host my own NuGet server that would take care of internal dependencies. I also want to use NuGet to handle my 3rd party dependencies as well. I'm trying to set up automated builds for our company and this is one roadblock I'm trying to overcome with NuGet.
So my question is how do I handle this scenario in which I have to retrieve my dependencies from different servers?
Is there a better way to handle internal dependencies? How is everyone else doing this?
Also just as a note I intend on using NuGet without committing packages to TFS. I planned on using the method outline in this article:
http://blog.davidebbo.com/2011/08/easy-way-to-set-up-nuget-to-restore.html
Glad you're looking into the no commit scenario for NuGet packages on TFS. You can take a look at my blog post on this topic where I explain the concept.
EDIT (2012/06/13): NuGetPowerTools is replaced by NuGet's built-in package restore functionality. However, same concept of changing the PackageSources element in nuget.targets still applies.
You definitely should take a look at David Fowler's NuGetPowerTools.
After installing this package, you can Enable-PackageRestore (newly installed command in Package Manager Console), which will add...
Enabling package restore will add MSBuild targets to your project files. These MSBuild targets will trigger nuget.exe in a pre-build step and fetch any packages required by your project.
No need to check-in NuGet packages in source control, all you need is the packages.config and these msbuild tasks.
To configure multiple, different package sources, you need to set some settings to be used by these MSBuild tasks. One of them is PackageSources. You can set it by editing the NuGet.targets file, which you will find in the .nuget folder once you enabled package restore.
Regarding those package sources, you could set up different internal NuGet galleries, or simply set up different network shares to be used. This is a matter of requirements and preference, so you can choose. All you need to do, is to tell your msbuild targets to use these packagesources. The order in which you define them, will be the order of lookup of packages as well.
Good luck!
Xavier
Little update on accepted answer and question:
When using TFS as a buildmachine without visual studio installed on it, you can do the following so the buildmachine automatically uses your custom packageSources (more than 1 in the same solution) without any further configuration of packagesources in your solution.
Create a machine default config by placing a NuGet.Config in the root ( C:\NuGet.Config ) by using sample from: http://docs.nuget.org/docs/reference/nuget-config-file
Comment out the line with: <add key="repositorypath" value="$\External\Packages" />
Otherwise your packages gets expanded in C:\$\External\packages\'. When commented out, the config gets chained and the right directory will be used.
Config your needed packagesource(s).
For more Info about other options (e.g. user specifc) see: http://docs.nuget.org/docs/reference/nuget-config-file (bottom of the page).
Related
I have a build pipeline configured for a Service Fabric solution on Azure DevOps like this:
Everything was fine until a few days ago when the build started failing on a particular build agent (private), with the following error (for a few projects):
C:\Program Files\dotnet\sdk\2.1.200\Sdks\Microsoft.NET.Sdk\build\Microsoft.PackageDependencyResolution.targets(327,5): Error : Assets file 'F:\Agent03\w\84\s\src\MyProject.Sam.Tiles.Domain\obj\project.assets.json' not found. Run a NuGet package restore to generate this file.
The failing task is the Build solution $(PathToSolution) one.
The weird thing is that the build fails when running on some agents but with others the build is fine.
Some details:
Use NuGet 4.x task started using NuGet v4.9.1 very recently, I think. I tried using v4.8.1 with no luck;
Most of the projects use the PackageReference format, but the .sfproj project uses the packages.config file
I tried using the dotnet restore task but there is an error when trying to restore the packages for the .sfproj project:
`Error : Unable to find the
'....\packages\Microsoft.VisualStudio.Azure.Fabric.MSBuild.1.6.7\build\Microsoft.VisualStudio.Azure.Fabric.Application.props'
file. Please restore the
'Microsoft.VisualStudio.Azure.Fabric.MSBuild' Nuget package
Any idea on what might be causing this issue?
Some of the projects use the PackageReference format but the .sfproj project uses the packages.config file.
I still don't understand why the build started failing, but I was able to find a workaround. Given that PackageReference is not yet supported in Service Fabric projects, my workaround was to use both restore tasks as follows:
My problem turned out to be a solution that didn't include all the necessary projects.
I have a master solution file that includes all my projects, and a number of smaller solution files with only some of the projects. The master solution built fine in Azure DevOps, but the partial solution failed.
I realized that the missing project.assets.json file belonged to a project that needed to be included in this failing solution.
Trevor's comment on 2/20 gave me the clue. You likely don't have the complete set of projects referenced by the solution. (ProjectReferences may go to other projects, which are not in the solution).
Here is why this crazy workaround (run dotnet.exe and nuget.exe restore tasks) worked:
dotnet restore will walk project references by default to ensure they are restored also.
--no-dependencies switch can turn that off.
nuget.exe restore has the opposite default, because we didn't want to break old users.
-recursive can turn this on.
The right solution is to make your solution contain all the projects.
-Rob Relyea
NuGet Client Team, Engineering Manager
We're using TeamCity (9.0) as our CI server to build, test and deploy several applications. Recently we are seeing occassional (one in every 30/40 builds or so) NuGet (2.8.3) errors as follows:
[restore] The process cannot access the file 'C:\BuildAgent\work\e32cbd0940f38bf.....\packages\Newtonsoft.Json.5.0.6\Newtonsoft.Json.5.0.6.nupkg' because it is being used by another process.
where the actual package seems to differ from time to time.
We suspected it has something to do with the same package being referenced in multiple projects within the same solution, but I would expect NuGet to be able to handle this correctly by filtering out duplicates instead of attempting to retrieve the same package multiple times, thereby ending up with write-locks when restoring the packages to the work folder.
As a first step of each Build Configuration we have a 'NuGet Installer' step set to 'restore'. I've tried fiddling with its settings (different 'Update modes', '-NoCache', older NuGet version (2.8.0)), but to no avail.
Has anyone else experienced similar issues, and if so, has any suggestions on how to ensure this error does not occur.
Any help would be greatly appreciated!
I had the same issue with Jenkins and fixed that by adding "-DisableParallelProcessing" to the nuget restore command, the final command would look like:
nuget restore "%WORKSPACE%\Solutions\App\App.sln" -DisableParallelProcessing
Excluding NuGet package files from our anti-malware products resolved this issue for us.
I used the SysInternals Process Explorer utility on the build agents to search for file handles for any *.nupkg files while the builds were running. After several builds I observed the anti-malware products briefly locking these files during the NuGet restore operations. Adding an exclusion to the anti-malware scanning rules prevented these locks as the files were no longer being scanned.
In our environment we use two different anti-malware products on different build agent servers. We encountered this issue with both products.
As far as the error message is concerned, I also came across it.
I debugged the “nuget restore” process, breaking at the point where the .nupkg is copied to the local repository, and then freezing the thread while the file was opened for writing. And sure enough I got the exception in another task, due to the fact that the two packages had Ids where one was a prefix of the other. I filed an issue for this : https://nuget.codeplex.com/workitem/4465.
However, this is probably not exactly your problem, since the error in my case is on reading the .nupkg of the package with the “long” name, and I don’t think there is a package with an Id that is a prefix of NewtonSoft.Json (whereas it is very possible the other way around : there are for instance NewtonSoft.JsonResult of NewtonSoft.Json.Glimpse).
I installed new Newtonsoft.Json and problem disappear
You can turn on build feature Swabra with option "Locking processes" (requires handle.exe). And check are there any files locked after build's finish or not.
If there are no locked files then try to run Nuget using command line build step instead of NuGet Installer. If the issue is reproduced then most probably it means that the issue is related NuGet.
Is there a way to add a package to a project when this package is already added to another project? I would like to avoid depending on the net because sometimes we are require to work in a non connected scenario.
I know I can simple add a reference and browse, but I suspect I will loose the ability to update automatically the reference in the future.
One way to be able to work with NuGet in a non-connected state, is to supply an alternative package source, either locally on your own computer, or on a machine on your internal network. You can add new package sources in Visual Studio through Tools > Options > Package Manager > Package Sources.
We do this ourselves for two reasons:
With a local package source you can work without an internet connection.
With a local package source and the official NuGet package source disabled, we have better control of which packages are available. This way will we for instance avoid undesired updates in our development group until we've approved them.
My aim is to have package restore working on a build server so that I don't have to check in binaries. At the moment, I'm simply trying to get it to work on my own machine using Visual Studio.
Here's what I've done so far:
Followed the instructions here http://docs.nuget.org/docs/workflows/using-nuget-without-committing-packages, including both setting the Tools-Options flag and the environment variable (belt and braces)
Installed the NuGetEnablePackageRestore package as suggested here NuGet package restore consent without NuGet
Checked everything in (the .nuget solution folder and its contents), but not the binaries I want to reference, because that's the whole point of the exercise
Here's what I'm doing:
Check out solution
Verify that nunit.framework.dll and moq.dll are not present in the checked out solution
Build the solution
Visual Studio complains that Moq is missing. I search for the dlls in the solution directory and find that:
nunit.framework.dll is present in the appropriate bin folders
Moq.dll is nowhere to be found
But there's more. This is truly mysterious, but if I do a fresh checkout, disconnect from the internet and build, I get precisely the same results - nunit.framework.dll is there, but moq.dll is not. The build process has conjured nunit.framework.dll literally from nowhere.
So it's something of an understatement to say that I am completely baffled. Can anyone suggest answers to the following questions:
Why is package restore not downloading Moq?
Where on earth is the build process getting nunit.framework.dll, if not the internet?
In vs, Options, Package Manager... there's a section "Package Cache", if you click on the "Browse" button it will take you to the location of the nuget cache in your machine.
Okay, I noticed in the documentation that enabling package restore was supposed to modify project files in order to add a new target. My project files did not have this change. Right-clicking the solution title in VS and selecting 'Manage NuGet packages...' then added the required changes and everything built as it should.
I checked, and package restore still appears to work when I have no internet access, so I'm still mystified about that. Does NuGet maintain some kind of cache of binaries outside the solution?
I've done a fair bit of reading on NuGet, and I can't seem to find what I want. Essentially, I'm hoping that it will work like Apache Ivy, where you can just check in your config file (without any binaries) and tell NuGet to fetch all the DLLs -- thus saving you from versioning tons of DLLs.
Hence: is there a command in NuGet to fetch and configure all dependencies mentioned in packages.config?
Again, the case for this is that I only checked packages.config into source control, not the actual DLLs, and I need to re-fetch everything. (Preferably without fetching packages one by one by name).
This has been covered recently in blog posts:
Inbuilt functionality for this is coming in a future version of NuGet: http://feeds.haacked.com/~r/haacked/~3/x8g_kFzD4eA/feedback-request-for-using-nuget-without-committing-packages.aspx
(Linked from above) How to do this today using command line NuGet.exe (available from the NuGet pages on CodePlex): http://blog.davidebbo.com/2011/03/using-nuget-without-committing-packages.html
EDIT: Now also covered on NuGet's Documentation Pages