trigger:
- develop
pool:
vmImage: windows-2019
- task: MSBuild#1
inputs:
solution: '**/*.sln'
- task: DownloadBuildArtifacts#1
inputs:
buildType: 'current'
downloadType: 'single'
itemPattern: '**/*.exe'
downloadPath: '$(System.ArtifactsDirectory)'
I get an error with this YAML in both Azure DevOps and using the YAML extension for VS Code. I'm trying to build a Windows Service and then put the .exe file somewhere that I can download it.
Azure DevOps:
VSCode
Error:
Implicit keys need to be on a single line, Implicit map keys need to
be followed by map values
Although the error looks some what confusing, your are missing the keyword steps.
trigger:
- develop
pool:
vmImage: windows-2019
steps:
- task: MSBuild#1
inputs:
solution: '**/*.sln'
- task: DownloadBuildArtifacts#1
inputs:
buildType: 'current'
downloadType: 'single'
itemPattern: '**/*.exe'
downloadPath: '$(System.ArtifactsDirectory)'
Related
Per https://learn.microsoft.com/en-us/azure/devops/pipelines/process/resources?view=azure-devops&tabs=schema#pipeline-resource-variables, resource pipeline variable names are camel-case:
resources.pipeline.<Alias>.projectID
resources.pipeline.<Alias>.pipelineName
resources.pipeline.<Alias>.pipelineID
resources.pipeline.<Alias>.runName
resources.pipeline.<Alias>.runID
resources.pipeline.<Alias>.runURI
resources.pipeline.<Alias>.sourceBranch
resources.pipeline.<Alias>.sourceCommit
resources.pipeline.<Alias>.sourceProvider
resources.pipeline.<Alias>.requestedFor
resources.pipeline.<Alias>.requestedForID
I have a pipeline template that follows this, for example:
stages:
- template: deploy-pipeline-job-template.yml
parameters:
canRun: and(not(or(failed(), canceled())), in(variables['resources.pipeline.buildPipeline.sourceBranch'], 'refs/heads/master', 'refs/heads/main'))
stageName: Development
...
But that template (as can be seen above) invokes another one, which has
steps:
- task: DownloadBuildArtifacts#0
displayName: 'Download artifacts from build'
inputs:
buildType: 'specific'
project: '17be1886-bedf-4a25-b77b-3a427a7ca351'
pipeline: ${{ parameters.buildPipelineId }}
specificBuildWithTriggering: true
buildVersionToDownload: specific
buildId: '$(resources.pipeline.buildPipeline.RunId)'
downloadType: 'specific'
itemPattern: 'drop/**'
downloadPath: '$(System.ArtifactsDirectory)'
I, apparently, used Pascal-casing for "RunId" in the buildId parameter. Now, I've been having trouble in some cases getting the correct final result from this pipeline setup--but usually it's turning out correctly. However, I don't understand why it works at all. Since buildType is "specific" and buildVersionToDownload is "specific, buildId is required. Either this template should be rejected altogether because there's no such thing as resources.pipeline.buildPipeline.RunId, or else it should fail at this step because that variable is null or empty or undefined or whatever, where it needs to be set to the ID of a real, existing build.
Is RunId being treated case-insensitively, as though I'd coded it as "runId", or is something else going on here?
Is RunId being treated case-insensitively, as though I'd coded it as
"runId", or is something else going on here?
Yes, it should don't matter about the case-sensitive. I think the issue comes from other places.
Now, I've been having trouble in some cases getting the correct final result from this pipeline setup
Could you please explain about this, how is the 'performances/behaves' of the pipeline on your side?
Do you mean the task wasn't be processed, or do you mean the artifacted downloaded was not expected?
Any way, the information you provided is not enough
#--------------------------------------------------------------
I think I need to point out some issues based on the information you provided (if I understand your description correctly.):
stages:
- template: deploy-pipeline-job-template.yml
parameters:
canRun: and(not(or(failed(), canceled())), in(variables['resources.pipeline.buildPipeline.sourceBranch'], 'refs/heads/master', 'refs/heads/main'))
stageName: Development
...
1, I notice the 'canRun' is defined by runtime expressions, this should not been used in parameters of template, parameters of template are compile time. You should use hardcode value or ${{parameters.somekey}} (parameter somekey is define in current YAML.).
steps:
- task: DownloadBuildArtifacts#0
displayName: 'Download artifacts from build'
inputs:
buildType: 'specific'
project: '17be1886-bedf-4a25-b77b-3a427a7ca351'
pipeline: ${{ parameters.buildPipelineId }}
specificBuildWithTriggering: true
buildVersionToDownload: specific
buildId: '$(resources.pipeline.buildPipeline.RunId)'
downloadType: 'specific'
itemPattern: 'drop/**'
downloadPath: '$(System.ArtifactsDirectory)'
2, The usage of $(resources.pipeline.buildPipeline.RunId) in this place needs you really defined resources in the YAML, otherwise the value will be NaN.
3, I notice you are using:
buildId: '$(resources.pipeline.buildPipeline.RunId)'
downloadType: 'specific'
I think it can transfer to:
buildVersionToDownload: 'latest'
#--------------------------------------------------------------
A demo on my side that always works:
pipeline for trigger purpose
trigger:
- none
pool:
vmImage: ubuntu-latest
steps:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(System.DefaultWorkingDirectory)'
artifact: 'drop'
publishLocation: 'pipeline'
the pipeline be triggered by resources:
trigger:
- none
resources:
pipelines:
- pipeline: SmartHotel
project: BowmanCP
source: Multiple_Repo_Trigger
trigger:
branches:
include:
- '*'
pool:
# vmImage: ubuntu-latest
vmImage: windows-latest
stages:
- template: deploy-pipeline-job-template.yml
parameters:
canRun: and(not(or(failed(), canceled())), in(variables['resources.pipeline.buildPipeline.sourceBranch'], 'refs/heads/master', 'refs/heads/main'))
stageName: Development
buildPipelineId: Multiple_Repo_Trigger
deploy-pipeline-job-template.yml
stages:
- stage: A
jobs:
- job: Job_1
steps:
- script: echo $(resources.pipeline.buildPipeline.RunId)
displayName: 'Run a one-line script'
- task: DownloadBuildArtifacts#0
displayName: 'Download artifacts from build'
inputs:
buildType: 'specific'
project: 'BowmanCP'
pipeline: '${{ parameters.buildPipelineId }}'
specificBuildWithTriggering: true
buildVersionToDownload: 'latest'
downloadType: 'specific'
itemPattern: 'drop/**'
downloadPath: '$(System.ArtifactsDirectory)'
y)'
Works fine on my side:
I want the version numbers of the Nuget package to be incremented and human readable with every version pushed to the Artifacts store. This is my yaml.
variables:
feedName : 'MyNugets'
buildConfiguration: 'release'
name: $(Year:yyyy).$(Month).$(DayOfMonth).$(Rev:.r)
pool:
vmImage: windows-latest
steps:
- task: DotNetCoreCLI#2
displayName: 'Build'
inputs:
command: 'build'
projects: '**/*.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: 'Pack'
inputs:
command: pack
versioningScheme: byBuildNumber
- task: NuGetAuthenticate#1
displayName: 'NuGet Authenticate'
- task: NuGetCommand#2
displayName: 'NuGet push'
inputs:
command: push
publishVstsFeed: '$(feedName)'
allowPackageConflicts: true
Whatever suggestion I could find on SO or other google result. None of them seems to be working. I always get this error.
##[error]Could not find version number data in the following environment variable: BUILD_BUILDNUMBER. The value of the variable should contain a substring with the following formats: X.Y.Z or X.Y.Z.A where A, X, Y, and Z are positive integers.
Any suggestions ?
Don't store the name in a variable, put in in a separate section above variables.
name: $(Year:yyyy).$(Month).$(DayOfMonth).$(Rev:.r)
variables:
feedName : 'MyNugets'
buildConfiguration: 'release'
See: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/run-number?view=azure-devops&tabs=yaml
I have the Unity project code in Azure DevOps Repos and configured the below yaml pipeline to build the Unity project.
trigger:
- none
stages:
- stage: Build
displayName: Unity Build
jobs:
- job: 'UnityBuild'
displayName: 'Build the Unity application'
pool:
name: XXXXXXXXX
steps:
- checkout: none
- script: "git config system core.longpaths true"
- checkout: self
- task: UnityBuildTask#3
inputs:
buildTarget: 'standalone'
unityProjectPath: 'XXXXXXXXXX'
outputPath: '$(Build.BinariesDirectory)'
outputFileName: 'Standalone'
- task: UnityGetProjectVersionTask#1
inputs:
unityProjectPath: 'XXXXXXXXXX'
- task: CopyFiles#2
inputs:
SourceFolder: '$(Build.BinariesDirectory)'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
Whenever I ran the yaml build pipeline, the build failed before it even executed unity build tasks due to file path name length restrictions.
How to fix the issue of file path names being too long in the Azure DevOps YAML pipeline?
You can run a script before "checkout" that tells Git.exe how to handle long paths (i.e. git config --system core.longpaths true).
See here.
If the agent is running on your own Windows server, then you'll need to configure the server to Enable Long Paths support.
I am using Azure Devops YAML build pipeline for the build and test of my SQL SSDT project. I am trying to amend my YAML code so that the test part will run on multiple Microsoft hosted agents (I have purchased 2). When I run the VSTest task as part of the first job that is run on the single agent, it runs no problems. However, when I run it as part of the second job, the unit tests do not run, with an error saying it cannot find the requisite test files. Is there something I need to add to my second job to make this work?
Many thanks.
trigger:
branches:
include:
- develop
- stage
- master
pr:
- master
- stage
- develop
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
jobs:
- job: BuildPublish
steps:
- task: NuGetToolInstaller#1
inputs:
versionSpec: '2.*'
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '$(solution)'
feedsToUse: 'select'
restoreDirectory: '$(Build.SourcesDirectory)/DIP_UsqlSln/packages'
- task: VSBuild#1
inputs:
solution: '**/*.sln'
msbuildArgs: '/p:USQLSDKPath=$(Build.SourcesDirectory)/DIP_UsqlSln/packages/Microsoft.Azure.DataLake.USQL.SDK.1.4.190703/build/runtime;USQLTargetType=SyntaxCheck;DataRoot=$(Build.SourcesDirectory) /p:EnableDeployment=true'## Heading ##
createLogFile: true
logFileVerbosity: 'detailed'
- task: CopyFiles#2
inputs:
Contents: '**'
TargetFolder: '$(build.artifactstagingdirectory)'
CleanTargetFolder: true
OverWrite: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
- job: Test
dependsOn: BuildPublish
strategy:
parallel: 2
steps:
- task: DownloadBuildArtifacts#0
displayName: 'Download Build Artifacts'
inputs:
artifactName: 'drop'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
runOnlyImpactedTests: true
runInParallel: true
How to find unit test dll in Azure Devops when copying test artifact to run on multiple agents (YAML)
That because the default Destination directory for the task DownloadBuildArtifacts is $(System.ArtifactsDirectory), however, the default Search folder for Visual Studio Test task is $(System.DefaultWorkingDirectory).
When you use the those two task in the build pipeline, the value of those two predefined variables are not same:
The value of $(System.ArtifactsDirectory) should be C:\agent\_work\r1\a.
The value of $(System.DefaultWorkingDirectory) should be c:\agent_work\1\s.
So, it will cause this issue, could not found the requisite test files.
To resolve this issue, we just need to change the default value of those two predefined variables to use same value:
- task: DownloadBuildArtifacts#0
displayName: 'Download Build Artifacts'
inputs:
artifactName: drop
downloadPath: '$(System.DefaultWorkingDirectory)'
Or
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
searchFolder: '$(System.ArtifactsDirectory)'
runOnlyImpactedTests: true
runInParallel: true
Hope this helps.
I am using Azure DevOps to Build and deploy my Azure Synapse
I have two YAML files. First one works well and I need to separate Build and Release steps. because of that I wrote second YAML file. Second YAML file has this error :
##[error]No files were found to deploy with search pattern
d:\a\1\s\SQL_ASynapse\bin\Release\SQL_ASynapse.dacpacCheck out how to troubleshoot failures at
https://aka.ms/sqlazuredeployreadme#troubleshooting-
First YAML file that works well:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: AzureKeyVault#1
inputs:
azureSubscription: 'XXX'
KeyVaultName: 'XXX-Dev'
SecretsFilter: '*'
- task: VSBuild#1
inputs:
solution: '$(solution)'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: SqlAzureDataWarehouseDacpacDeployment#1
inputs:
azureSubscription: 'XXX'
AuthenticationType: 'server'
ServerName: 'XXX'
DataWarehouse: '$(SynapseName)'
SqlUsername: '$(SynapseSQLUsername)'
SqlPassword: '$(SynapseSQLPassword)'
deployType: 'DacpacTask'
DeploymentAction: 'Publish'
DacpacFile: 'SQL_ASynapse\bin\Release\SQL_ASynapse.dacpac'
IpDetectionMethod: 'AutoDetect'
Second YAML file who has error:
trigger:
- master
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
# Agent VM image name
vmImageName: 'windows-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
restoreSolution: '$(solution)'
- task: VSBuild#1
inputs:
solution: '$(solution)'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- task: VSTest#2
inputs:
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'development'
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureKeyVault#1
inputs:
azureSubscription: 'XXX-Dev'
KeyVaultName: 'XXX-Dev'
SecretsFilter: '*'
- task: SqlAzureDataWarehouseDacpacDeployment#1
inputs:
azureSubscription: 'XXX-Dev'
AuthenticationType: 'server'
ServerName: 'XXX'
DataWarehouse: '$(SynapseName)'
SqlUsername: '$(SynapseSQLUsername)'
SqlPassword: '$(SynapseSQLPassword)'
deployType: 'DacpacTask'
DeploymentAction: 'Publish'
DacpacFile: 'SQL_ASynapse\bin\Release\SQL_ASynapse.dacpac'
IpDetectionMethod: 'AutoDetect'
UPDATE #1 FOR #Levi Lu-MSFT ANSWER
I have changed my second script like below code :
...
- task: VSBuild#1
inputs:
solution: '$(solution)'
platform: '$(buildPlatform)'
configuration: '$(buildConfiguration)'
- publish: $(system.defaultworkingdirectory)\SQL_ASynapse\bin\Release ## path to a file or folder
artifact: MyBuildOutputs
...
- download: current # refers to artifacts published by current pipeline
artifact: MyBuildOutputs
- task: SqlAzureDataWarehouseDacpacDeployment#1
inputs:
azureSubscription: 'xxx-Dev'
AuthenticationType: 'server'
ServerName: 'xxx.database.windows.net'
DataWarehouse: '$(SynapseName)'
SqlUsername: '$(SynapseSQLUsername)'
SqlPassword: '$(SynapseSQLPassword)'
deployType: 'DacpacTask'
DeploymentAction: 'Publish'
DacpacFile: 'SQL_ASynapse\bin\Release\SQL_ASynapse.dacpac'
IpDetectionMethod: 'AutoDetect'
But I have still same error.
Upload task result
Download task result
SqlAzureDataWarehouseDacpacDeployment error :
##[error]No files were found to deploy with search pattern
d:\a\1\s\SQL_ASynapse\bin\Release\SQL_ASynapse.dacpacCheck out how to
troubleshoot failures at https://aka.ms/sqlazuredeployreadme#troubleshooting-
If you use Microsoft-hosted agent. Every job defined in the yaml pipeline will run on a fresh new virtual machine. The virtual machine is discarded after one use.
So that the build artifacts from Build job of Build stage doesnot exist on the agent machine of the deploy job of Deploy stage. That's why you got above error.
You need to add a publish task after Vsbuild task in Build job to publish the build results to azure devops server. You should specify the correct path(where the build artifacts reside) to publish.
- publish: $(system.defaultworkingdirectory)\SQL_ASynapse\bin\Release ## path to a file or folder
artifact: MyBuildOutputs
Then add a download task before qlAzureDataWarehouseDacpacDeployment task in Deploy job to download the build results to deploy job agent. So that the build results will be available to the deployment tasks.
- download: current # refers to artifacts published by current pipeline
artifact: MyBuildOutputs
To publish and download build artifacts, You can also use Publish Build Artifacts task, Download Build Artifacts task.
Update:
When you use download task: See here for more information.
By default, files are downloaded to $(Pipeline.Workspace)/{artifact}, where artifact is the name of the artifact. The folder structure of the artifact is always preserved.
And you can see from above updated screenshot. The artifacts is downloaded to folder D:\a\1\MyBuildOutputs
So you should set DacpacFile attribute as below:
DacpacFile : $(Pipeline.Workspace)/MyBuildOutputs/SQL_ASynapse.dacpac