Azure pipelines failing stating Incorrect task refrence - azure-devops

My Azure pipeline is as below:
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: terraform init
displayName: 'terraform init'
inputs:
provider: aws
backendServiceAWS: 'tcp-aws-aa'
backendAWSBucketName: 'terraform-backend-20200102'
backendAWSKey: dev.plan
- task: terraform fmt
displayName: 'terraform fmt'
inputs:
provider: aws
command: fmt
- task: terraform validate
displayName: 'terraform validate'
inputs:
provider: aws
command: validate
- task: terraform plan
displayName: 'terraform plan'
inputs:
provider: aws
command: plan
environmentServiceNameAWS: 'tcp-aws-aa'
- task: tflint check
inputs:
script: tflint .
- task: tfsec check
inputs:
script: tfsec .
However, it produces an error as like below
How to have it resolved?

Well it looks like you want to refer to task: TerraformTaskV1#0 (based on the syntax) and the you should use as this:
- task: TerraformTaskV1#0
inputs:
provider: 'azurerm'
command: 'init'
backendAWSKey:
backendAWSBucketName:
It support these commands:
And of course to use it you need to install this extension. I guessed that this is the one you should use based on the input settings. They are exactly the same like this extension has.
You also have there tflint and tfsec but I didn't found extensions or native solution for them so assuming that you installed them on agent you should rather use them like this:
- script: |
tflint .
displayName: 'tflint check'
- script: |
tfsec .
displayName: 'tfsec check'

Related

Question about Azure Devops pipeline question about task conditions

I've been trying to setup an Azure Devops pipeline for testing purposes and i'm struggling to understand why one of my tasks runs the script line despite being skipped.
Here's the pipeline yaml code:
## Example azure-pipelines.yml
## Event (branch to trigger the pipeline execution)
trigger:
branches:
include:
- main
exclude:
- My-branch # Will not run
# Configures pipeline execution on pull requests
pr:
branches:
include:
- main
exclude:
- My-branch # Will not run
# Environment variables created
variables:
- group: my-keys
## OS where the pipeline will run
pool:
vmImage: 'ubuntu-latest'
# List of stages for your application
stages:
- stage: Test
displayName: Application Testing
# List of jobs the pipeline stage will run
jobs:
- job: MyJob
displayName: Install packages and and publishes
variables:
# Sets the environment variable to cache the application packages
npm_config_cache: $(Pipeline.Workspace)/.npm
# List of steps for the job
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.x'
displayName: 'Install Node.js'
- task: Cache#2
displayName: Install and cache packages
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
path: $(npm_config_cache)
- script: npm ci
condition: ne(variables.CACHE_RESTORED, 'true')
- task: Npm#1
displayName: Publish and auto accept
condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/main'))
- script: npx my-package --with-token=${my-keys} --auto-publish-changes
- task: Npm#1
displayName: Publish
condition: eq(variables['Build.Reason'], 'PullRequest')
- script: npx my-package --with-token=${my-keys}
- script: echo ${{variables['Build.Reason']}} ${{eq(variables['Build.Reason'], 'PullRequest')}}
A example, for instance when a push is made into the main branch it runs Publish and auto accept followed by the Publish, when it technically should only run the first one. One other thing that i saw was that when a pull request is incoming to one other branch rather than main it shouldn't trigger the script associated to Publish and auto accept but instead jump over that and run only the script in Publish, but instead it runs the scripts in both.
If anyone could provide some help with this i would appreciate it.
Thanks in advance
I think the problem is that you run 4 tasks instead of two
Take a look at NPM task syntax, it has no 'script' parameter
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/package/npm?view=azure-devops
'script' task that you are using is indeed shortcut of another task 'CmdLine#2'
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/command-line?view=azure-devops&tabs=yaml
Firstly you run NPM task with specified condition, but it does nothing
task: Npm#1
displayName: Publish and auto accept
condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/main'))
Then you run script task without condition(so it will run always), this task does npm stuff
script: npx my-package --with-token=${my-keys} --auto-publish-changes
Then you run again npm task without desired parameters but with conditions
task: Npm#1
displayName: Publish
condition: eq(variables['Build.Reason'], 'PullRequest')
And finally you run fourth task doing stuff, without conditions so it runs always.
script: npx my-package --with-token=${my-keys}
So as to fix this problem, you need to use Npm#1 task with parameters specified in provided documentation. Or just add conditions to your script tasks(CmdLine#2).
I think that below snippet should work
- task: CmdLine#2
displayName: 'Publish and auto accept'
condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/main'))
inputs:
script: 'npx my-package --with-token=${my-keys} --auto-publish-changes'
- task: CmdLine#2
displayName: 'Publish'
condition: eq(variables['Build.Reason'], 'PullRequest')
inputs:
script: 'npx my-package --with-token=${my-keys}'

Azure pipeline - unzip artefact, copy one directory into Azure blob store YAML file

I am getting stuck with Azure pipelines.
I have an existing node SPA project that needs built for each environment (TEST and PRODUCTION). This i can do, but need to have a manual step when pushing to PROD. I am using Azure Dev-op pipeline environments with Approval and Checks to mandate this.
The issue is using a 'deploy job' to take an artefact from a previous step I am unable to find the right directory. This is my YAML file have so far:
variables:
# Agent VM image name
vmImageName: 'ubuntu-latest'
trigger:
- master
# Don't run against PRs
pr: none
stages:
- stage: Development
displayName: Devlopment stage
jobs:
- job: install
displayName: Install and test
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '12.x'
displayName: 'Install Node.js'
- script: |
npm install
displayName: Install node modules
- script: |
npm run build
displayName: 'Build it'
# Build creates a ./dist folder. The contents will need to be copied to blob store
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.BinariesDirectory)'
includeRootFolder: true
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
replaceExistingArchive: true
verbose: true
- deployment: ToDev
environment: development
dependsOn: install
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact#2
inputs:
buildType: 'current'
targetPath: '$(Pipeline.Workspace)'
- task: ExtractFiles#1
inputs:
archiveFilePatterns: '**/*.zip'
cleanDestinationFolder: true
destinationFolder: './cpDist/'
# Somehow within a deploy job retrieve the .zip artefact, unzip, copy the ./dist folder into the blob store
- task: AzureCLI#2
inputs:
azureSubscription: MYTEST-Development
scriptLocation: "inlineScript"
scriptType: "bash"
inlineScript: |
az storage blob upload-batch -d \$web --account-name davey -s dist --connection-string 'DefaultEndpointsProtocol=https;AccountName=davey;AccountKey=xxxxxxx.yyyyyyyyy.zzzzzzzzzz;EndpointSuffix=core.windows.net'
displayName: "Copy build files to Development blob storage davey"
- script: |
pwd
ls
cd cpDist/
pwd
ls -al
displayName: 'list'
- bash: echo "Done"
If you are confused with the folder path, you could add few debug steps to check the location of know system variables to understand what was going on using a powershell script as below:
- task: PowerShell#2
displayName: 'Degug parameters'
inputs:
targetType: Inline
script: |
Write-Host "$(Build.ArtifactStagingDirectory)"
Write-Host "$(System.DefaultWorkingDirectory)"
Write-Host "$(System.ArtifactsDirectory)"
Write-Host "$(Pipeline.Workspace)"
Write-Host "$(System.ArtifactsDirectory)"
You should simply publish the build generated artifacts to drop folder.
Kindly check this official doc -- Artifact selection , in there is explaining that you can define the path which to download the artifacts to with the following task:
steps:
- download: none
- task: DownloadPipelineArtifact#2
displayName: 'Download Build Artifacts'
inputs:
patterns: '**/*.zip'
path: '$(Build.ArtifactStagingDirectory)'
Please be aware that the download happens automatically to $(Pipeline.Workspace), so if you don’t want you deployment to download the files twice, you need to specify the “download: none” in your steps.

File from previous step cannot be found in Azure DevOps-Pipeline

In a pipeline I have two different steps. The first one generates some files, the second should take these files as an input.
the Yaml for that pipeline is the following:
name: myscript
stages:
- stage: Tes/t
displayName: owasp-test
jobs:
- job: owasp_test
displayName: run beasic checks for site
pool:
name: default
demands: Agent.OS -equals Windows_NT
steps:
- task: DotNetCoreCLI#2
inputs:
command: 'build'
projects: '**/*.sln'
- task: dependency-check-build-task#5
inputs:
projectName: 'DependencyCheck'
scanPath: '**/*.dll'
format: 'JUNIT'
- task: PublishTestResults#2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/*-junit.xml'
the dependency-check-build-task returns an XML-File:
File upload succeed.
Upload 'P:\Azure-Pipelines-Agent\_work\2\TestResults\dependency-check\dependency-check-junit.xml' to file container: '#/11589616/dependency-check'
Associated artifact 53031 with build 21497
The following step (PublishTestResults) SHOULD take that file but returns
##[warning]No test result files matching **/*-junit.xml were found.
instead. I can see that file in the artifact after the pipeline is run.
This is because your report is written to Common.TestResultsDirectory which is c:\agent_work\1\TestResults (for Microsoft Hosted agents), and publish test task looks in System.DefaultWorkingDirectory which is c:\agent_work\1\s.
Please try:
- task: PublishTestResults#2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/*-junit.xml'
searchFolder: '$(Common.TestResultsDirectory)'
I had the same trouble:
I fixed changing the Agent Specification

Azure DevOps Multi Stage Pipeline Error: No package found with specified pattern: /home/vsts/work/1/s/**/*.zip - How do I fix?

I have an Azure DevOps Build (yaml) and Release Pipeline (Classic) successfully deploying to Azure.
I am trying to convert these 2 separate steps in a Multi Stage Yaml Pipeline.
On the Azure App Service Deploy task (AzureRmWebAppDeployment#4), I am getting the following error:
No package found with specified pattern: /home/vsts/work/1/a/*.zip
Below is my Multi Stage Yaml Pipeline
stages:
- stage: Build
jobs:
- job: 'Build'
pool:
vmImage: 'windows-latest'
variables:
buildConfiguration: 'Release'
steps:
- task: DotNetCoreCLI#2
displayName: Restore
inputs:
command: restore
projects: '**/*.csproj'
vstsFeed: 'dd55642d-8943-411f-8856-9714dd0da8af'
- task: DotNetCoreCLI#2
displayName: Build
inputs:
projects: '**/*.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: Test
inputs:
command: test
projects: '**/*[Tt]ests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: Publish
inputs:
command: publish
publishWebProjects: false
projects: '**/Tools.Client.Blazor.ServerApp.csproj'
arguments: '--configuration $(buildConfiguration) --output $(build.artifactstagingdirectory)'
- task: PublishSymbols#2
displayName: 'Publish symbols path'
inputs:
SearchPattern: '**\bin\**\*.pdb'
PublishSymbols: false
continueOnError: true
- task: CopyFiles#2
displayName: 'Copy Files to: $(build.artifactstagingdirectory)\AzureDeploy'
inputs:
SourceFolder: AzureDeploy
TargetFolder: '$(build.artifactstagingdirectory)\AzureDeploy'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()
- stage: Systest
jobs:
- job: 'Systest'
variables:
resourceGroupName: '$(appName)-rg-$(environment)'
location: 'East US'
appServiceName: '$(appName)-svc-$(environment)'
appInsightsName: '$(appName)-ins-$(environment)'
appServicePlanName: '$(appName)-asp-$(environment)'
appName: 'tools'
owner: 'Pod'
environment: 'systest'
steps:
- task: AzureResourceManagerTemplateDeployment#3
displayName: 'ARM Template deployment: Resource Group scope'
inputs:
azureResourceManagerConnection: 'Dev/Test Connection'
subscriptionId: ''
resourceGroupName: '$(resourceGroupName)'
location: '$(location)'
csmFile: '$(System.DefaultWorkingDirectory)/AzureDeploy/Tools.azureDeploy.json'
csmParametersFile: '$(System.DefaultWorkingDirectory)/AzureDeploy/Tools.azureDeploy.parameter.json'
overrideParameters: '-appServiceName "$(appServiceName)" -appInsightsName "$(appInsightsName)" -appServicePlanName "$(appServicePlanName)" -owner "$(owner)" -environment "$(environment)" -location "$(location)"'
- task: AzureRmWebAppDeployment#4
displayName: 'Azure App Service Deploy: $(appServiceName)'
inputs:
ConnectionType: 'AzureRM'
azureSubscription: ''
appType: 'webApp'
WebAppName: '$(appServiceName)'
packageForLinux: '$(Build.ArtifactStagingDirectory)/*.zip'
Any help / suggestions would be appreciated.
Because it's 2 stages the second stage doesn't have the file you published in the first stage, you need to download it.
You can use Pipeline artifacts instead of build artifacts.
Pipeline artifacts provide a way to share files between stages in a
pipeline or between different pipelines. They are typically the output
of a build process that needs to be consumed by another job or be
deployed. Artifacts are associated with the run they were produced in
and remain available after the run has completed.
To publish (upload) an artifact for the current run:
steps:
- publish: $(build.artifactstagingdirectory)
artifact: drop
And in the second stage, you download the artifact:
steps:
- download: current
artifact: drop
You can also achieve it with build artifacts and download with DownloadBuildArtifacts#0 task.
During Publish it will not work like this. Instead of using path "/home/vsts/work/1/a/.zip", this path can be used "$(System.DefaultWorkingDirectory)/_Releasepipelinename/drop/.zip"

Azure Pipeline Task inputs won't accept variables

In the azure pipeline yaml files, the variable imgRepoName is trimmed from the gitRepoName. An bash echo for gitRepoName shown core/cqb-api; bash echo for imgRepoName shown cqb-api
variables:
vmImageName: 'ubuntu-18.04'
gitRepoName: $(Build.Repository.Name)
imgRepoName: $(basename $(gitRepoName))
- job: build_push_image
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and Push image
inputs:
repository: imgRepoName
command: buildAndPush
containerRegistry: "coreContainerRegistry"
tags: test2
Issues:
When I wrote repository: "cqb-api" as the input for the docker task it works just fine, while use the variable directly as shown above won't create any images in the container registry.
PS, I also tried repository: $(imgRepoName) it give out the following error
invalid argument "***/$(basenamecore/cqb-api):test2" for "-t, --tag" flag: invalid reference format
It looks that it is executed at runtime. So gitreponame is replaced but basename function is not recognized in this context. You can check this:
variables:
gitRepoName: $(Build.Repository.Name)
steps:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
$name = $(basename $(gitRepoName))
Write-Host "##vso[task.setvariable variable=imgRepoName]$name"
- task: Docker#2
displayName: Build and Push
inputs:
repository: $(imgRepoName)
command: build
Dockerfile: docker-multiple-apps/Dockerfile
tags: |
build-on-agent
It works for me.