My build creates Source Link packages, in the form of .snupkg files, and I'd like to push those packages to MyGet. From what I understand, tools such as Rider and Visual Studio can then download the symbols automatically. I can get this nuget push command working from my local machine, but I cannot figure out how to push those symbols using a DevOps build task. Here's my YAML:
- task: NuGetCommand#2
displayName: 'Publish Symbols ($(Build_Major).$(Build_Minor).$(Build_Patch))'
condition: succeeded()
inputs:
command: 'push'
packagesToPush: 'projects/${{ parameters.projectName }}/src/**/bin/**/*.$(Build_Major).$(Build_Minor)*.snupkg'
nuGetFeedType: 'external'
publishFeedCredentials: 'MyGet - Test'
I'm absolutely sure that MyGet - Test is correct, since I'm using the exact URL and API key that the MyGet web UI tells me to use for publishing symbols, as well as the same ones I have working locally. However, in the build logs I get:
C:\hostedtoolcache\windows\NuGet\6.4.0\x64\nuget.exe sources Add -NonInteractive -Name redacted_source -Source https://company.myget.org/F/myfeed/api/v3/index.json -ConfigFile D:\a\1\Nuget\tempNuGet_311668.config
Package source with Name: redacted_source added successfully.
Using authentication information for the following URI: https://company.myget.org/F/myfeed/api/v3/index.json
C:\hostedtoolcache\windows\NuGet\6.4.0\x64\nuget.exe setapikey *** -NonInteractive -Source redacted_source -ConfigFile D:\a\1\Nuget\tempNuGet_311668.config
The API Key '***' was saved for 'https://company.myget.org/F/myfeed/api/v3/index.json'.
C:\hostedtoolcache\windows\NuGet\6.4.0\x64\nuget.exe push D:\a\1\s\src\myproject\bin\Release\MyProject.1.10.24-beta.snupkg -NonInteractive -Source https://company.myget.org/F/myfeed/api/v3/index.json -ApiKey *** -ConfigFile D:\a\1\Nuget\tempNuGet_311668.config -Verbosity Detailed
Unable to load the service index for source https://company.myget.org/F/myfeed/api/v3/index.json.
NuGet Version: 6.4.0.123
Response status code does not indicate success: 401 (Unauthorized).
This does work a bit different than how I can successfully do this from the command line, which is:
nuget push .\MyProject.1.10.100-mike-beta.nupkg b848309c-0000-0000-0000-a893d8fc5d56 -Source https://company.myget.org/F/myfeed/api/v3/index.json
However, I'm not quite sure how to get the NuGetCommand task to behave like that. It seems like both should work anyway. Any ideas?
I figured this out! Though, the solution is kinda stupid. You must use a pre-authenticated URL to push packages or symbols using the v3 API:
Authentication seems to work fine using the v2 API, but the v2 API doesn't let you push symbols. I'm not sure if this is a limitation of MyGet or the DevOps NuGet task or what. However, since the pre-authenticated URL can still live as a secret in the service connection, it's a fairly easy and secure workaround.
I'll leave this question open in case anyone has more insight or a better solution.
You can first create a service connection credential for the feed.
Then add NuGet authenticate task before NuGetCommand#2 task in the pipeline.
steps:
- task: NuGetAuthenticate#1
displayName: 'NuGet Authenticate'
inputs:
nuGetServiceConnections: feed
Please refer to the document Publish NuGet packages with Azure Pipelines.
Related
Having the same issue on a self hosted agent but I'm not specifying password in the yml. just specifying the vstsFeed
- checkout: self
submodules: true
persistCredentials: true
- task: NuGetToolInstaller#1
inputs:
versionSpec: 6.2.1
- task: UseDotNet#2
displayName: Using Dotnet Version 6.0.400
inputs:
packageType: 'sdk'
version: '6.0.400'
- task: DotNetCoreCLI#2
displayName: Restore Nuget packages
inputs:
command: 'restore'
projects: '**/*.sln'
feedsToUse: 'select'
vstsFeed: 'ba05a72a-c4fd-43a8-9505-a97db9bf4d00/6db9ddb0-5c18-4a24-a985-75924292d079'
and it fails with following error error NU1301: Unable to load the service index for source
Nuget feed is on another project of the same organization. I can see that pipeline produces a temp nuget config where it specifies username and password for this feed during run. Been breaking my head for the last 72 hours non-stop to find what is the issue. Azure pipelines and nuget sucks. 99% of the problems we had so far was with nuget not working smoothly with azure pipelines. Microsoft has to take a step back and resolve pipelines and nuget ssues.
Just to make sure: The NuGet feed is on the same Azure instance as the agent is registered with, right?
I remember similar issues on my on-premise Azure DevOps server, but also sometimes on the paid cloud variant. Sometimes it was flaky service state, sometimes the agent itself..
Kevin did give a good point with the permissions - if those are set, you're good to go from a permissions point of view - actually reader permission is enough for a restore - make sure to check the views panel too.
If after permissions check you still got issues, you might try my "just-making-sure-lines for your .yml file:
# NuGet Authentication (safety step, normally not required as all within the same organization/project)
- task: NuGetAuthenticate#1
displayName: "Nuget Authentication"
It shouldn't be required but I have it on all my pipes since I had such issues, and it reduced the occurence of the error line you posted in my cases (Hybrid devops architecture).
Another thing I ended up with is to specifiy the feeds explicitly in a repository-wide "NuGet.Config" file - and using this file within my yml files or with script lines instead of tasks now.
If nothing helps, enable diagnostics/verbose logging to get more error details. In the worst case: Log in to your agent machine, open a terminal in the same agent work folder and manually issue a dotnet restore command to see whats going on.
Post the additional results if still no progress.
Good luck
From your description, you are using the Nuget feed on another project of the same organization.
You need to check the following points:
Check the permission of the Build Service account.
Here are the steps:
Step1: Navigate to Artifacts ->Target Feed ->Feed Settings -> Permission.
Step2: Grant the Build service account Contributor Role . Build service account name: ProjectnamethePipelinelocated Build Service (Organization name)
For example:
Check the Limit job authorization scope to current project for non-release pipelines option is Enabled in Project Settings -> Settings.
If yes, you need to disable the option and then the pipeline can use the resource outside the project.
Note: To disable this option, you need to disable the option in Organization Settings-> Settings first. Then you could disable the option in Project level.
The documentation here says
To publish to an external NuGet feed, you must first create a service connection to point to that feed....
It then provides this YAML:
- task: NuGetAuthenticate#0
inputs:
nuGetServiceConnections: '<Name of the NuGet service connection>'
- task: NuGetCommand#2
inputs:
command: push
nuGetFeedType: external
versioningScheme: byEnvVar
versionEnvVar: <VersionVariableName>
This fails with
"Error: The service connection for 'https://api.nuget.org/v3/index.json' is not valid. ApiKey service connections are not supported in this task. Instead, use -ApiKey (NuGet) or --api-key (dotnet) when invoking the tool itself. See the task documentation for more details."
I have created a service connection that points to nuget.org with my ApiKey.
I do not understand this portion of the error message: "Instead, use -ApiKey (NuGet) or --api-key (dotnet) when invoking the tool itself. See the task documentation for more details."
I have also seen this question which refers to a deficiency which has since been implemented.
Edit: Turns out the deficiency mentioned above is a red herring. The real problem is a limitation of nuget which still exists as of this writing.
According to the NuGet Authenticate task document:
Some package sources such as nuget.org use API keys for authentication
when pushing packages, rather than username/password credentials. Due
to limitations in NuGet, this task cannot be used to set up a NuGet
service connection that uses an API key.
Instead:
Configure a secret variable containing the ApiKey
Perform the package
push using nuget push -ApiKey $(myNuGetApiKey) or dotnet nuget push
--api-key $(myNuGetApiKey), assuming you named the variable myNuGetApiKey
"az pipelines runs artifact download --run-id 10976432 --artifact-name "Android/Android" --path "c:/foo/" fails with error "Artifact with name 'Android' was found, but is of type 'Container'"
I publish artifacts from our build runs to several different repositories like NPM, Nuget, PyPi, CoCoPods, and this process requires "two factor authentication", so at least for now it will remain a manual post build process for us.
But even though this is a manual process, I want to automate as much of it as I can, e.g. automating the download a dozen or more build run artifacts to a local staging directory that I then use to publish from.
I have found several community post that have done this with CURL and the REST apis, but I want to get this working with the az devops extention.
As far as I can tell from the azure extension this command should work, but it does not!
How can I download build run artifacts (not public feed artifacts), so that I can manually or automatically publish them outside for the normal DevOps pipeline?
Thanks,
Brian.
"Artifact with name 'Android' was found, but is of type 'Container'"
I guess the publish task you are using is Publish Build Artifacts task, right? If yes, as the message shown, we indeed does not support download container type artifacts until now.
As of today, you can just download artifacts of type -'Pipeline Artifact' from az cli.
In order to use az cli to download artifacts successfully , if possible, please go replace the publish task with Publish Pipeline Artifact task:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
artifact: 'AzLogicApp'
publishLocation: 'pipeline'
Now, you will see you can succeed to download the artifact with az cli.
In Azure DevOps Server (version 2019.0.1) running on a Windows Server 2019 agent, with the 'Allow duplicates to be skipped' option selected for NuGet push task, a warning is displayed:
The 'Allow duplicates to be skipped' option is currently only available on Azure Pipelines. If NuGet.exe encounters a conflict, the task will fail.
The task results in the following error that causes the task to fail indicating that the above warning applies:
Response status code does not indicate success: 409 (Conflict - The feed already contains 'MyPackage X.Y.Z'. (DevOps Activity ID: 1A57312F-3C56-4E4D-9E78-73C7072A288F)).
I'm wondering if this issue is particular to Azure DevOps Server (rather than Azure DevOps Services), or if I'm doing something wrong, or if there is another workaround. I noticed someone else has the same issue from this comment on another question where it was mentioned that the option was available after someone asked how to ignore error 409 (duplicate package).
I would like to ignore duplicate packages using the NuGet task and ideally the 'Allow duplicates to be skipped' option on Azure DevOps Server. I'm aware that it could be resolved using scripting, but I'd prefer to avoid that if possible. Any help appreciated.
I don't know about the Azure DevOps task, but if you upgrade to nuget.exe 5.1, you can use the new -SkipDuplicate option. This should work for any NuGet server that correctly implements the NuGet protocol and on any CI server/agent.
If you're using the NuGetCommand#2 Azure Pipelines task, you can use the allowPackageConflicts parameter.
allowPackageConflicts
It allows the task to report success even if some of your packages are rejected with 409 Conflict errors.
This option is currently only available on Azure Pipelines and using Windows agents. If NuGet.exe encounters a conflict, the task will fail. This option will not work and publish will fail if you are within a proxy environment.
— https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/package/nuget
If you switch to azure pipelines (it seems it's the new way of doing things) you can use dotnet commands.
the option --skip-duplicate will be available in dotnet core 3.1 (still in preview) for the dotnet nuget push command (no need to use NuGet command as it's already available in dotnet).
But you can use it now if you install the latest .NET Core.
For example, this is a stage that will grab whichever nuGet you've got in a specific folder, install the latest dotnet core that supports the skip duplicates and push it to the repository feed.
- stage:
displayName: 'Release'
condition: succeeded()
jobs:
- job: 'Publish'
displayName: 'Publish nuGet Package'
steps:
- download: current
artifact: $(PIPELINE_ARTIFACT_NAME)
displayName: 'Download pipeline artifact'
- script: ls $(PATH_PIPELINE_ARTIFACT_NAME)
displayName: 'Display contents of downloaded articacts path'
- task: NuGetAuthenticate#0
displayName: 'Authenticate in NuGet feed'
- task: UseDotNet#2
displayName: 'Use .NET Core sdk 3.1 (preview)'
inputs:
packageType: sdk
version: '3.1.100-preview2-014569'
installationPath: $(Agent.ToolsDirectory)/dotnet
- script: dotnet nuget push $(PATH_PIPELINE_ARTIFACT_NAME)/**/*.nupkg --source $(NUGET_FEED) --api-key $(NUGET_API_KEY) --skip-duplicate
displayName: 'Uploads nuGet packages'
I have an Azure DevOps project (just one).
I have a Build Pipeline set to run in the "Hosted VS2017" Agent Pool. This Agent Pool appears to be in the [MyProject]\Build Administrators, Contributors, Project Administrators, and Release Administrators roles.
I also have an Artifacts nuget feed in the DevOps project. It has [MyProject]\Project Valid Users set as "Reader" role. It appears that Project Valid Users has all of the Agent Pool's roles mentioned above as members.
I have an azure-pipelines.yml script that adds that adds the artifacts feed as a nuget source right at the beginning:
# Add nuget source
- powershell: Invoke-RestMethod "https://dist.nuget.org/win-x86-commandline/latest/nuget.exe" -OutFile "$env:UserProfile/nuget.exe"
- script: '%UserProfile%\nuget.exe sources Add -Name "devops" -Source "https://pkgs.dev.azure.com/MyProject/_packaging/feed/nuget/v3/index.json"'
The build yml then dot a dotnet build but fails inside NuGet.targets with:
Unable to load the service index for source https://pkgs.dev.azure.com/MyProject/_packaging/feed/nuget/v3/index.json.
Response status code does not indicate success: 401 (Unauthorized).
how can I make this work? My build needs packages from other builds that are on that artifacts feed...
There is a better alternative imo. You can continue using your script to dotnet restore.
Simply add a task just before that with NuGetAuthenticate#0
steps:
- task: NuGetAuthenticate#0
- script: dotnet restore --no-cache --force
this task will authenticate the pipeline with the nuget feeds that require so and are found at the NuGet.config file.
More info here
Notice that when the nuGet feed is within Azure DevOps there is nothing else required. If the feed is external you can configure at your Azure DevOps a nuGet Service Connections (at the link there is further info).
Use the built-in tasks for installing and running NuGet and you won't have authentication problems.
Use the dotnet task's restore command. If you're using a single Azure Artifacts feed, just select it from the dropdown in the task (instead of the PowerShell you mentioned). If multiple feeds (doesn't look like it from your question, but just in case), you'll need to check in a NuGet.config that references all of those feeds, then point the task to that config.
You may also need to pass the '--no-restore' flag to your 'dotnet build' command.
If you still encounter issues, ensure the correct build identity has access to your feed.