In Azure DevOps Server (version 2019.0.1) running on a Windows Server 2019 agent, with the 'Allow duplicates to be skipped' option selected for NuGet push task, a warning is displayed:
The 'Allow duplicates to be skipped' option is currently only available on Azure Pipelines. If NuGet.exe encounters a conflict, the task will fail.
The task results in the following error that causes the task to fail indicating that the above warning applies:
Response status code does not indicate success: 409 (Conflict - The feed already contains 'MyPackage X.Y.Z'. (DevOps Activity ID: 1A57312F-3C56-4E4D-9E78-73C7072A288F)).
I'm wondering if this issue is particular to Azure DevOps Server (rather than Azure DevOps Services), or if I'm doing something wrong, or if there is another workaround. I noticed someone else has the same issue from this comment on another question where it was mentioned that the option was available after someone asked how to ignore error 409 (duplicate package).
I would like to ignore duplicate packages using the NuGet task and ideally the 'Allow duplicates to be skipped' option on Azure DevOps Server. I'm aware that it could be resolved using scripting, but I'd prefer to avoid that if possible. Any help appreciated.
I don't know about the Azure DevOps task, but if you upgrade to nuget.exe 5.1, you can use the new -SkipDuplicate option. This should work for any NuGet server that correctly implements the NuGet protocol and on any CI server/agent.
If you're using the NuGetCommand#2 Azure Pipelines task, you can use the allowPackageConflicts parameter.
allowPackageConflicts
It allows the task to report success even if some of your packages are rejected with 409 Conflict errors.
This option is currently only available on Azure Pipelines and using Windows agents. If NuGet.exe encounters a conflict, the task will fail. This option will not work and publish will fail if you are within a proxy environment.
— https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/package/nuget
If you switch to azure pipelines (it seems it's the new way of doing things) you can use dotnet commands.
the option --skip-duplicate will be available in dotnet core 3.1 (still in preview) for the dotnet nuget push command (no need to use NuGet command as it's already available in dotnet).
But you can use it now if you install the latest .NET Core.
For example, this is a stage that will grab whichever nuGet you've got in a specific folder, install the latest dotnet core that supports the skip duplicates and push it to the repository feed.
- stage:
displayName: 'Release'
condition: succeeded()
jobs:
- job: 'Publish'
displayName: 'Publish nuGet Package'
steps:
- download: current
artifact: $(PIPELINE_ARTIFACT_NAME)
displayName: 'Download pipeline artifact'
- script: ls $(PATH_PIPELINE_ARTIFACT_NAME)
displayName: 'Display contents of downloaded articacts path'
- task: NuGetAuthenticate#0
displayName: 'Authenticate in NuGet feed'
- task: UseDotNet#2
displayName: 'Use .NET Core sdk 3.1 (preview)'
inputs:
packageType: sdk
version: '3.1.100-preview2-014569'
installationPath: $(Agent.ToolsDirectory)/dotnet
- script: dotnet nuget push $(PATH_PIPELINE_ARTIFACT_NAME)/**/*.nupkg --source $(NUGET_FEED) --api-key $(NUGET_API_KEY) --skip-duplicate
displayName: 'Uploads nuGet packages'
Related
My build creates Source Link packages, in the form of .snupkg files, and I'd like to push those packages to MyGet. From what I understand, tools such as Rider and Visual Studio can then download the symbols automatically. I can get this nuget push command working from my local machine, but I cannot figure out how to push those symbols using a DevOps build task. Here's my YAML:
- task: NuGetCommand#2
displayName: 'Publish Symbols ($(Build_Major).$(Build_Minor).$(Build_Patch))'
condition: succeeded()
inputs:
command: 'push'
packagesToPush: 'projects/${{ parameters.projectName }}/src/**/bin/**/*.$(Build_Major).$(Build_Minor)*.snupkg'
nuGetFeedType: 'external'
publishFeedCredentials: 'MyGet - Test'
I'm absolutely sure that MyGet - Test is correct, since I'm using the exact URL and API key that the MyGet web UI tells me to use for publishing symbols, as well as the same ones I have working locally. However, in the build logs I get:
C:\hostedtoolcache\windows\NuGet\6.4.0\x64\nuget.exe sources Add -NonInteractive -Name redacted_source -Source https://company.myget.org/F/myfeed/api/v3/index.json -ConfigFile D:\a\1\Nuget\tempNuGet_311668.config
Package source with Name: redacted_source added successfully.
Using authentication information for the following URI: https://company.myget.org/F/myfeed/api/v3/index.json
C:\hostedtoolcache\windows\NuGet\6.4.0\x64\nuget.exe setapikey *** -NonInteractive -Source redacted_source -ConfigFile D:\a\1\Nuget\tempNuGet_311668.config
The API Key '***' was saved for 'https://company.myget.org/F/myfeed/api/v3/index.json'.
C:\hostedtoolcache\windows\NuGet\6.4.0\x64\nuget.exe push D:\a\1\s\src\myproject\bin\Release\MyProject.1.10.24-beta.snupkg -NonInteractive -Source https://company.myget.org/F/myfeed/api/v3/index.json -ApiKey *** -ConfigFile D:\a\1\Nuget\tempNuGet_311668.config -Verbosity Detailed
Unable to load the service index for source https://company.myget.org/F/myfeed/api/v3/index.json.
NuGet Version: 6.4.0.123
Response status code does not indicate success: 401 (Unauthorized).
This does work a bit different than how I can successfully do this from the command line, which is:
nuget push .\MyProject.1.10.100-mike-beta.nupkg b848309c-0000-0000-0000-a893d8fc5d56 -Source https://company.myget.org/F/myfeed/api/v3/index.json
However, I'm not quite sure how to get the NuGetCommand task to behave like that. It seems like both should work anyway. Any ideas?
I figured this out! Though, the solution is kinda stupid. You must use a pre-authenticated URL to push packages or symbols using the v3 API:
Authentication seems to work fine using the v2 API, but the v2 API doesn't let you push symbols. I'm not sure if this is a limitation of MyGet or the DevOps NuGet task or what. However, since the pre-authenticated URL can still live as a secret in the service connection, it's a fairly easy and secure workaround.
I'll leave this question open in case anyone has more insight or a better solution.
You can first create a service connection credential for the feed.
Then add NuGet authenticate task before NuGetCommand#2 task in the pipeline.
steps:
- task: NuGetAuthenticate#1
displayName: 'NuGet Authenticate'
inputs:
nuGetServiceConnections: feed
Please refer to the document Publish NuGet packages with Azure Pipelines.
Having the same issue on a self hosted agent but I'm not specifying password in the yml. just specifying the vstsFeed
- checkout: self
submodules: true
persistCredentials: true
- task: NuGetToolInstaller#1
inputs:
versionSpec: 6.2.1
- task: UseDotNet#2
displayName: Using Dotnet Version 6.0.400
inputs:
packageType: 'sdk'
version: '6.0.400'
- task: DotNetCoreCLI#2
displayName: Restore Nuget packages
inputs:
command: 'restore'
projects: '**/*.sln'
feedsToUse: 'select'
vstsFeed: 'ba05a72a-c4fd-43a8-9505-a97db9bf4d00/6db9ddb0-5c18-4a24-a985-75924292d079'
and it fails with following error error NU1301: Unable to load the service index for source
Nuget feed is on another project of the same organization. I can see that pipeline produces a temp nuget config where it specifies username and password for this feed during run. Been breaking my head for the last 72 hours non-stop to find what is the issue. Azure pipelines and nuget sucks. 99% of the problems we had so far was with nuget not working smoothly with azure pipelines. Microsoft has to take a step back and resolve pipelines and nuget ssues.
Just to make sure: The NuGet feed is on the same Azure instance as the agent is registered with, right?
I remember similar issues on my on-premise Azure DevOps server, but also sometimes on the paid cloud variant. Sometimes it was flaky service state, sometimes the agent itself..
Kevin did give a good point with the permissions - if those are set, you're good to go from a permissions point of view - actually reader permission is enough for a restore - make sure to check the views panel too.
If after permissions check you still got issues, you might try my "just-making-sure-lines for your .yml file:
# NuGet Authentication (safety step, normally not required as all within the same organization/project)
- task: NuGetAuthenticate#1
displayName: "Nuget Authentication"
It shouldn't be required but I have it on all my pipes since I had such issues, and it reduced the occurence of the error line you posted in my cases (Hybrid devops architecture).
Another thing I ended up with is to specifiy the feeds explicitly in a repository-wide "NuGet.Config" file - and using this file within my yml files or with script lines instead of tasks now.
If nothing helps, enable diagnostics/verbose logging to get more error details. In the worst case: Log in to your agent machine, open a terminal in the same agent work folder and manually issue a dotnet restore command to see whats going on.
Post the additional results if still no progress.
Good luck
From your description, you are using the Nuget feed on another project of the same organization.
You need to check the following points:
Check the permission of the Build Service account.
Here are the steps:
Step1: Navigate to Artifacts ->Target Feed ->Feed Settings -> Permission.
Step2: Grant the Build service account Contributor Role . Build service account name: ProjectnamethePipelinelocated Build Service (Organization name)
For example:
Check the Limit job authorization scope to current project for non-release pipelines option is Enabled in Project Settings -> Settings.
If yes, you need to disable the option and then the pipeline can use the resource outside the project.
Note: To disable this option, you need to disable the option in Organization Settings-> Settings first. Then you could disable the option in Project level.
I'm trying to set up a simple CI pipeline in Azure Devops
I have found it super easy to to do a Build -> Publish from within VS2019 to an Azure App Service, and it works flawlessly.
However, when trying to set up a CI pipeline which does the exact same thing from Azure Devops, I am presented with a YAML file, and options which I can drop into it.
I managed to get my pipeline to reach the "build" part, but then it couldn't find my custom nuget feed, so I updated my yaml to include this nuget restore:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
buildConfiguration: 'Release'
steps:
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '**/*.sln'
feedsToUse: 'select'
vstsFeed: 'feed-id'
- script: dotnet build --configuration $(buildConfiguration)
displayName: 'dotnet build $(buildConfiguration)'
But the build fails again, with this error:
##[error]The nuget command failed with exit code(1) and error(Unable to load the service index for source https://MYVSTSADDRESS.pkgs.visualstudio.com/_packaging/THE_GUID/nuget/v3/index.json.
My question is, can't I just import the exact process that I follow from vs 2019?
Why is it so annoyingly complicated to do the exact same thing from devops?
Why would I need to complete this YAML document in devops, if its not required from VS??
##[error]The nuget command failed with exit code(1) and error(Unable to load the service index for source https://MYVSTSADDRESS.pkgs.visualstudio.com/_packaging/THE_GUID/nuget/v3/index.json.
From the URL in the error message, it seems that the task is using the Organization Scope Feed.
Based on my test, some pipeline settings will affect the use of this feed.
First, please check if your error message is similar to the following：
If yes, you could check the Limit job authorization scope to current project for non-release pipelines option is Enabled in Project Settings -> Pipelines.
You could try to disable the option.
Note: To disable this option, you need to disable the option in Organization Settings-> Settings first. Then you could disable the option in Project level.
Then you could navigate to Artifacts -> Feed Settings -> Permissions->... to enable the option Allow Project-Scoped builds.
In my company earlier we have executed RF scripts in Jenkins via Github. Now they have moved to Azure Devops.
I am new to Azure Devops. Could some provide the steps for the configuration and how to run
From your description , it seems that your Source Code is on Github.
Here are the steps to run the RF scripts:
Create a pipeline and use the repo from Github resource. You may need to create the connection to Github, then you could get the repos.
Add Use Python Version task to specify the required python version.
Install the Robot framework dependencies with Powershell task or Command Line Task.
Run the RF script with Powershell task or Command Line Task.
For example: Yaml Editor
trigger:
- master
jobs:
- job: 'Test'
pool:
vmImage: 'vs2017-win2016'
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '3.7'
architecture: 'x64'
- script: pip install robotframework robotframework-seleniumlibrary && npm install chromedriver
displayName: 'Install dependencies'
- powershell: robot --pythonpath . -x outputxunit.xml TestCases.robot
displayName: 'Run Robot Scripts'
If you want to use Pipeline Classic Editor, you could refer to this blog.
Hope this helps.
"az pipelines runs artifact download --run-id 10976432 --artifact-name "Android/Android" --path "c:/foo/" fails with error "Artifact with name 'Android' was found, but is of type 'Container'"
I publish artifacts from our build runs to several different repositories like NPM, Nuget, PyPi, CoCoPods, and this process requires "two factor authentication", so at least for now it will remain a manual post build process for us.
But even though this is a manual process, I want to automate as much of it as I can, e.g. automating the download a dozen or more build run artifacts to a local staging directory that I then use to publish from.
I have found several community post that have done this with CURL and the REST apis, but I want to get this working with the az devops extention.
As far as I can tell from the azure extension this command should work, but it does not!
How can I download build run artifacts (not public feed artifacts), so that I can manually or automatically publish them outside for the normal DevOps pipeline?
Thanks,
Brian.
"Artifact with name 'Android' was found, but is of type 'Container'"
I guess the publish task you are using is Publish Build Artifacts task, right? If yes, as the message shown, we indeed does not support download container type artifacts until now.
As of today, you can just download artifacts of type -'Pipeline Artifact' from az cli.
In order to use az cli to download artifacts successfully , if possible, please go replace the publish task with Publish Pipeline Artifact task:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
artifact: 'AzLogicApp'
publishLocation: 'pipeline'
Now, you will see you can succeed to download the artifact with az cli.