I tried to follow the best practices by using a single feed from the client. So I've setup a Nuget.config file with a single entry to my feed, with a <clear/> tag as stated in the doc.
On the devops server side, I've set up the feed with an Upstream source to the public Nuget Gallery (in order to cache, as this sounded nice in case of public package managers outages.
When I directly use the nuget.exe client on any machine, I can install any nuget.org package in any case. But at build time, when a public package isn't already in my serve's upstream cache, it won't be found by the build agent... (it seems that in this case, the upstreams isn't been used to feed the cache).
Is this normal? Is it limited to already cached packages at build time?
In our develoment team, we are used to add nuget packages through the VS UI (right click project "manage nuget packages"). In this mode, we can not see those upstreams packages, so we toggle the source to nuget directly at dev time.
The solutions that I found could be either:
to add a second entry to the Nuget.config to declare nuget.org for build (but we loose the caching capability)
to systematically use the command line nuget.exe to install the packages at development time?
What did I miss? Do you have any other ideas/solutions?
EDIT: it seems that it failed due to the following error:
BTW: I'm quite new to Azure Devops (using an on premise version "Dev18.M170.6").
Is this normal? Is it limited to already cached packages at build time?
This is not normal. It is not limited to the cached packages at build time.
For example, I create a new test feed with enable Upstream source to the public Nuget Gallery:
Now only a few packages I tested before are cached, then I add another test package log4net and restore it with my following nuget.config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="NewFeed" value="https://pkgs.dev.azure.com/<MyOrgName>/_packaging/NewFeed/nuget/v3/index.json" />
</packageSources>
<config>
</config>
<packageSourceCredentials>
<NewFeed>
<add key="Username" value="XXX" />
<add key="ClearTextPassword" value="XXX" />
</NewFeed>
</packageSourceCredentials>
</configuration>
The nuget restore result:
The feed result:
So, It is not limited to the cached packages at build time. The reason why this method is invalid for you requires specific analysis of your nuget restore task log.
As a reminder, please check if project Build Services have permission to access your feed:
Feed settings->Permissions->...->Allow Project-scoped build:
In our develoment team, we are used to add nuget packages through the
VS UI (right click project "manage nuget packages"). In this mode, we
can not see those upstreams packages, so we toggle the source to nuget
directly at dev time.
You could add the upstream source nuget.org as nuget scource in Visual Studio, which you could dd nuget packages through the VS UI. When you build the pipeline in Azure devops, nuget will restore the package from the custom feed and cache the package from upstream source:
Related
I'm trying to push and then consume a NuGet package using GitHub packages (documentation here). As a newcomer, after experimenting a bit I managed to push a sample NuGet package to a public GitHub repository.
Yet, I'm missing the final part, i.e. consume the package from some Visual Studio project. When I try to add the package, I first get a "Restoring packages for..." notification, and then the error "Unable to load the service index for source... : The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters".
So, it seems my endpoint for NuGet is not configured as expected, yet I could not find a clear direction about this. To help newbies like me starting with GPR, and detail my procedure so that readers can spot my errors, here is what I learnt until now:
Setup
Before using GPR, you must create a token in your GitHub account.
Creating and Publishing Packages
set the RepositoryUrl and RepositoryType properties in your .csproj file to your target repository URL and git respectively, e.g.:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<!-- omissis ... -->
<RepositoryUrl>https://github.com/USERNAME/PROJECTNAME</RepositoryUrl>
<RepositoryType>git</RepositoryType>
</PropertyGroup>
</Project>
open the GitHub bash in your project folder, and create the package like this:
dotnet pack NAME.csproj -c Release -p:IncludeSymbols=true -p:SymbolPackageFormat=snupkg
where -c selects the configuration. See creating snupkg for more, and the dotnet pack reference.
open the GitHub bash in your project folder, and publish like this: dotnet nuget push "bin/Release/NAME.1.0.0.nupkg" --source "github".
Note: you first need to register (once) the GPR feed with:
nuget sources add -name "github" -Source https://nuget.pkg.github.com/YOURGITHUBUSERNAME/index.json -Username YOURGITHUBUSERNAME -Password YOURGITHUBTOKEN
If you need to install nuget.exe, download it from https://www.nuget.org/downloads. If you place it e.g. in C:\Exe, you can invoke it from the Windows Git Bash with /c/Exe/nuget.exe.
Also, you need to set the nuget API key:
nuget setapikey YOURGITHUBTOKEN -Source https://nuget.pkg.github.com/YOURGITHUBUSERNAME/index.json
This encrypts the key and saves it in a config file under your %APPDATA% folder. e.g. mine ends up in C:\Users\USERNAME\AppData\Roaming\NuGet\NuGet.Config.
Using Packages
documentation
for each client project, add nuget.config to your project: add a nuget.config file in your project folder with a content like this:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="github" value="https://nuget.pkg.github.com/OWNER/index.json" />
</packageSources>
<packageSourceCredentials>
<github>
<add key="Username" value="YOURUSERNAME" />
<add key="ClearTextPassword" value="YOURTOKEN" />
</github>
</packageSourceCredentials>
</configuration>
This comes from the documentation sample. Yet, to avoid storing token in this file, which would be saved in the repo, I rather use a per-user or per-machine NuGet setting (reference): e.g. per-user:
nuget config -set GITHUB_PACKAGES_TOKEN=YOURTOKEN
This saves the setting per-user (the default option). Now, consume it like this (see Setting an environment variable in a NuGet.Config file):
<add key="Password" value="%GITHUB_PACKAGES_TOKEN%" />
to use a package, execute dotnet add YOURPROJECT.csproj package YOURPACKAGE --version YOURPACKAGEVERSION.
Yet, at this last point I get the above error. Where is my NuGet source config wrong?
The documentation for pushing NuGet packages to Github is outdated. The steps that worked for me:
Go to GitHub
Click your avatar (top-right)
Settings
Developer settings
Personal access tokens
Generate
write:packages
read:packages
delete:packages
This will automatically check the repo permissions for your OAuth token
Click Generate token
Open cmd
Navigate to your project directory or the directory containing your NuGet package
Add a new nuget source:
dotnet nuget add source --username [GithubUserName] --password [YourApiKey] --name github https://nuget.pkg.github.com/[UsernameOrOrganizationName]/index.json
Push the package to the github source
dotnet nuget push --source github bin\Release\MyAwesomePackage.1.0.0.nupkg
Verify that the Github API key is not stored inside a Nuget.config file inside your solution before committing your code to source control.
I've set up an artifact feed in Azure Devops and pushed some of our private packages to it using "nuget.exe push".
The problem I have is some packages have multiple versions (e.g. 1.1, 1.2, 1.3 etc) and we have a number of projects where some will use 1.1, some will use 1.2 etc.
After uploading versions 1.1, 1.2 and 1.3 to the artifact feed, only 1.3 is available to my projects as it is the latest version. If I click into the uploaded package in the Devops interface I can see the previous versions, but none of the devops projects that use 1.1 or 1.2 will build as they can't find the older versions.
I've read in a few places that a way around it would be to have one artifact feed per project, with the required versions pushed to that feed. The issue I have with that is that I've simplified the scale of my problem as in reality we have around 20 packages, each one could have up to 30 different versions, and I have around 50 projects. To create a feed for each one would be massively time consuming and would involve duplicating a lot of packages when I push them.
If I add any package to a project from nuget, I get to choose which version I want to add but it seems like I can't replicate this when using an artifact feed. Am I doing this incorrectly or is there a better way to achieve what I need?
EDIT:
I'm not using a project-scoped feed, it is listed in Devops as a organisation-scoped feed.
The packages.config file specifically targets a certain version e.g.
<package id="CommonResourceAssembly" version="2.17.60.0" targetFramework="net451" />
and the error log shows that is can't find this version:
##[error].nuget\NuGet.targets(103,9): Error : Unable to find version '2.17.60.0' of package 'CommonResourceAssembly'.
The feed itself shows that version 2.17.61 is the current one, but 2.17.60 is what is needed for this particular project and does appear in the version history:
Am I doing this incorrectly or is there a better way to achieve what I need?
I am afraid you may have made some incorrect settings. That because the Azure Artifact supports multiple versions of a packages. That also is reason why you can see the previous versions in the Devops interface.
When we use the nuget restore in the Azure devops pipeline, nuget restore task will restore the package according to the version of the package specified in the packages.config/PackageReference, like:
Packages.config:
<packages>
<package id="Newtonsoft.Json" version="8.0.3" targetFramework="net46" />
</packages>
PackageReference:
<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="8.0.3" />
</ItemGroup>
So, please check the following possibilities that may cause this issue:
Check if your Azure Artifact is project scoped feeds(If your Azure Artifact feeds are created before 11/4/2019, please go to second point):
Check if you use wildcards in the packages.config/PackageReference:
<PackageReference Include="Packagename" Version="1.*" />
Check if there is a nuget.config in your solution with following settings:
<configuration>
<config>
<add key="dependencyversion" value="Highest" />
</config>
</configuration>
If above not help you, please share us more info about this issue, the info/image about the packages.config/PackageReference for the build failed project, info/image about the Azure Artifact including the package version 1.1, 1.2, the latest but not important, the error log.
Hope this helps.
Thanks to the help from #Leo Liu-MSFT I worked out what needed to be modified:
1 Delete the nuget.exe and nuget.targets files from the .nuget folder
2 Delete the references in all .csproj files that pointed to the nuget.targets file:
<Import Project="$(SolutionDir)\.nuget\NuGet.targets" Condition="Exists('$(SolutionDir)\.nuget\NuGet.targets')" /> <Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
<PropertyGroup>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
</PropertyGroup>
<Error Condition="!Exists('$(SolutionDir)\.nuget\NuGet.targets')" Text="$([System.String]::Format('$(ErrorText)', '$(SolutionDir)\.nuget\NuGet.targets'))" /> </Target>
3 Add a step in the pipeline YAML before the build that looks at the nuget.config file to restore the packages:
task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
restoreSolution: '**\*.sln'
feedsToUse: config
nugetConfigPath: '.nuget/NuGet.config'
This now builds the solution correctly and can reference any previous version of a package
I have a Xamarin Forms project for which I am using a private Telerik nuget feed (we're using their UI controls).
In the Azure DevOps build pipeline I have added a new service connection to the Telerik nuget feed.
In my build pipeline I have a nuget task that restores these packages.
However my build fails with the error "Unable to resolve Telerik.UI.for.Xamarin".
My understanding of configuring a private nuget feed may be incorrect. Do I need to specify a nuget.config file that specifies the Telerik nuget package? Do I need to remove the reference to the Telerik package from the .csproj? (i.e. do I need to separate out my private nuget packages to prevent the build from attempting to download them?)
After some trial and error I have eventually managed to fix this issue by creating a nuget.config file for my project and adding the following entries.
<?xml version="1.0"?>
<configuration>
<packageSources>
<add key="nuget.org" value="https://www.nuget.org/api/v2/" />
</packageSources>
<packageRestore>
<add key="enabled" value="true" />
<add key="automatic" value="true" />
</packageRestore>
</configuration>
These are just default values that allow my project to fetch the packages from nuget.org. The service credentials that I have configured for fetching packages from the private Telerik nuget feed are then merged into this nuget.config file during the build. The build is therefore able to fetch both the default packages from nuget.org as well as the private Telerik feed.
In our case, the problem was a trailing slash. In our nuget.config file, we had the endpoint specified as https://nuget.telerik.com/nuget/, but in the Service Connection in Azure DevOps it was specified as https://nuget.telerik.com/nuget (note the lack of a trailing slash!).
It's a bit annoying that Azure DevOps is that particular about it, but once we eventually saw the difference and removed the trailing slash from nuget.config, everything works as expected.
If you have multiple packageSources in your NuGet.config:
<packageSources>
<clear />
<add key="dev" value="http://server2/branches/feature1/nuget" />
<add key="release" value="http://server1/nuget" />
</packageSources>
Is the order in the XML used when running nuget install, etc? If I have packageA at version 1.0.0-SNAPSHOT in both, will the dev channel version on feature1 branch win?
The order of your feeds in nuget.config determines the order that they show up in the drop down menus from the various views into NuGet from Visual Studio. In all cases, you have the choice of which feed you install from. See the screenshots below.
Visual Studio doesn't keep track of where your package came from, so the package ID is really in a global namespace that you have to know which feed it's a part of. Visual Studio will show you any package from the selected feed with a matching ID, so it's possible that you'd overwrite a package you installed from one feed with another from a different feed.
For a NuGet feed that I manage at my company, I prefix our internal packages with <CompanyName>. so as to reduce the risk for a naming conflict now and in the future.
I've set up an in-house NuGet repository (a shared folder on one of our servers), and have added it to the "Available Sources" list in the Visual Studio NuGet settings on my PC. I'm able to successfully install a package into my solution from this repository.
However on the TFS server the build is failing because it can't restore this particular package (message: "Unable to find version 'xxx' of package 'xxx'"). The solution is configured for "enable NuGet package restore", which works fine for packages installed via nuget.org. It's just not restoring the package from our internal repository.
Permissions on the repository folder/share are fine. What am I missing?
I'm guessing I have to configure Visual Studio's "available sources" on the TFS server too, but presumably these settings are user-specific? The build service runs as NETWORK SERVICE, so how can I configure NuGet for this user?
You can add the source to a machine-wide config on the TFS build server and it will be included in the list of sources when doing package restores.
What I did was create a file %ProgramData%\NuGet\Config\sources.config that contained:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="internal source" value="http://internal_server/nuget/" />
</packageSources>
</configuration>
The name of the file doesn't matter, it's the location of the file. Nuget picks up any .config files under that folder.
http://docs.nuget.org/docs/reference/NuGet-Config-File#NuGet_config_extensibility_point
UPDATE 2016-11-18
According to http://blog.nuget.org/20161121/introducing-nuget4.0.html (under breaking changes), VS 2017+ and NuGet 4.0+ changes the location of the machine-wide config folder to %ProgramFiles(x86)%\NuGet\Config\.
change your build to run as a specific account. you can then logon with that account and prove off all of these little snags.
depending on which way you are doing the NuGet restore, have you updated the targets file with the location of your local feed?
<ItemGroup Condition=" '$(PackageSources)' == '' ">
<!-- Package sources used to restore packages. By default, registered sources under %APPDATA%\NuGet\NuGet.Config will be used -->
<!-- The official NuGet package source (https://www.nuget.org/api/v2/) will be excluded if package sources are specified and it does not appear in the list -->
<!--
<PackageSource Include="https://www.nuget.org/api/v2/" />
<PackageSource Include="https://my-nuget-source/nuget/" />
-->
<PackageSource Include="http://MyLocalFeed/nuget/nuget"/>
</ItemGroup>