PowerShell script location on release job task - azure-devops

In my local source repo, I have a bunch of power shell scripts that will need to execute as part of a release job.
In order to get them to be executed by the deployment agent, what is the correct way to get the scripts where they need to be and reference them? Do I need to copy the scripts into an artifact and retrieve that artifact on the deploy agent side, or is there another way to do this? All MS documentation says use $(System.DefaultWorkingDirectory) as my base path but on the deployment side the scripts aren't copied to there since it does not download the repo. I am assuming making a artifact and downloading it is the correct way to do this, then they would be in that location.

You can use a download Repository artifact on your release pipeline and then use the predefined release variables in order to cd into that directory.System.DefaultWorkingDirectory
change dir
cd $(System.DefaultWorkingDirectory)/$(Release.PrimaryArtifactSourceAlias)
ls
Then you can locate code.
Release variables documentation:
https://learn.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch

In build pipeline:
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)/xxx.ps1' # Put all of the powershell script files here.
includeRootFolder: true
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
In release pipeline:
steps:
- task: ExtractFiles#1
displayName: 'Extract files '
inputs:
destinationFolder: '$(System.ArtifactsDirectory)'
cleanDestinationFolder: false
steps:
- powershell: |
# Write your PowerShell commands here.
Write-Host "Hello World"
cd $(System.ArtifactsDirectory)
dir
displayName: 'PowerShell Script'
Get the ps1 file:

Related

Azure AzureRmWebAppDeployment step cant find zip file from PublishArtifacts

I had an application that was on .NET 5 and was hooked up to an Azure DevOps pipeline that built and deployed fine. The relevant parts are:
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
- task: AzureRmWebAppDeployment#4
inputs:
ConnectionType: 'AzureRM'
azureSubscription: 'mySub'
appType: 'webAppLinux'
WebAppName: 'myApp'
packageForLinux: '$(Build.ArtifactStagingDirectory)/s.zip'
RuntimeStack: 'DOTNETCORE|5.0'
I when I moved to .NET 6 I recreated the project from scratch and ported the code over including the .yaml file, but when the pipeline runs, when it gets to the Deployment step it says it can't find 'D:\a\1\a\s.zip'. When i wildcard it it says more than 1 zip file exists.
- task: AzureRmWebAppDeployment#4
inputs:
ConnectionType: 'AzureRM'
azureSubscription: 'mySub'
appType: 'webAppLinux'
WebAppName: 'myApp'
packageForLinux: '$(Build.ArtifactStagingDirectory)/*.zip'
RuntimeStack: 'DOTNETCORE|6.0'
TLDR: How can I determine what the Publish step output as the zip file?
There can be three approaches.
Use artifactName while publishing the artifact, that way the artifacts would be published with your desired name.
Reference https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/publish-build-artifacts?view=azure-devops
ArtifactName: Specify the name of the artifact that you want to create. It can be whatever you want.
Default value: drop
Use a bash task to list the content of the directory.
- bash: ls $(Build.ArtifactStagingDirectory)
displayName: "List the content of staging directory"
Once the artifact is published and your pipeline run is completed, navigate to Summary to explore or download your artifact.
Reference: https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/build-artifacts?view=azure-devops&tabs=yaml#download-artifacts

Git Checkout fails due to long file path names in the Azure DevOps yaml build pipeline

I have the Unity project code in Azure DevOps Repos and configured the below yaml pipeline to build the Unity project.
trigger:
- none
stages:
- stage: Build
displayName: Unity Build
jobs:
- job: 'UnityBuild'
displayName: 'Build the Unity application'
pool:
name: XXXXXXXXX
steps:
- checkout: none
- script: "git config system core.longpaths true"
- checkout: self
- task: UnityBuildTask#3
inputs:
buildTarget: 'standalone'
unityProjectPath: 'XXXXXXXXXX'
outputPath: '$(Build.BinariesDirectory)'
outputFileName: 'Standalone'
- task: UnityGetProjectVersionTask#1
inputs:
unityProjectPath: 'XXXXXXXXXX'
- task: CopyFiles#2
inputs:
SourceFolder: '$(Build.BinariesDirectory)'
Contents: '**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
Whenever I ran the yaml build pipeline, the build failed before it even executed unity build tasks due to file path name length restrictions.
How to fix the issue of file path names being too long in the Azure DevOps YAML pipeline?
You can run a script before "checkout" that tells Git.exe how to handle long paths (i.e. git config --system core.longpaths true).
See here.
If the agent is running on your own Windows server, then you'll need to configure the server to Enable Long Paths support.

Azure Pipeline Copy Secure file into build folder

I've a vite/svelte project which uses .env files for environment settings. I also have an Azure Pipeline which contains a secure file .env.staging this is on the .gitignore list of the associated repo. I'd like to download this secure file, copy it to my build directory and then have it's contents read when I run vite build --mode staging (well, npm run build:staging which includes vite build...)
When run locally from my machine npm run build:staging works as expected and reads the .env.staging file, however it seems to get ignored when used in the pipeline, am I doing anything wrong?
Here's my yml.
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- task: DownloadSecureFile#1
name: "dotenvStaging"
inputs:
secureFile: '.env.staging'
displayName: "Download .env.staging"
- task: NodeTool#0
inputs:
versionSpec: 14.15.4
displayName: "Install Node.JS"
- task: CopyFiles#2
inputs:
contents: "$(Agent.TempDirectory)/.env.staging"
targetFolder: "$(Agent.BuildDirectory)"
displayName: "Import .env.staging"
- script: npm install
displayName: "npm install"
- script: npm run build:staging
displayName: "npm run build:staging"
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: 'dist'
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
#replaceExistingArchive: true
#verbose: # Optional
#quiet: # Optional
displayName: "Create archive"
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
ArtifactName: 'drop'
publishLocation: 'Container'
displayName: "Publish archive"
I'm not sure if CopyFiles#2 is doing what I expect or not as it just matches the content parameter to copy whatever files match, which could be 0 if I'm writing it wrong...
Another note, I also tried using $(dotenvStaging.secureFilePath) as the content parameter, but it doesn't seem to do anything either.
Naturally I figured it out as soon as I posted, I needed to update the CopyFiles part to specify sourceFolder, clearly it didn't like my absolute file path for content.
- task: CopyFiles#2
inputs:
sourceFolder: "$(Agent.TempDirectory)"
contents: ".env.staging"
targetFolder: "$(Agent.BuildDirectory)"
displayName: "Import .env.staging"

CopyFiles Task not picking up files

Using Azure DevOps YAML in a database project build and release pipeline
This bit of code correctly picks up my four dacpac files, I can see these being copied in the console
- task: CopyFiles#2
displayName: Copy build output to artifacts staging
inputs:
SourceFolder: "$(Build.SourcesDirectory)"
flattenFolders: true
Contents: '**\bin\**\*.dacpac'
TargetFolder: "$(Build.ArtifactStagingDirectory)"
This bit of code correctly picks up my publish files, I can see these being copied in the console
- task: CopyFiles#2
displayName: Copy build output to artifacts staging
inputs:
SourceFolder: "$(Build.SourcesDirectory)"
flattenFolders: true
Contents: '**\PublishProfile\*.publish.xml'
TargetFolder: "$(Build.ArtifactStagingDirectory)"
This bit of code reports "zero files found"
- task: CopyFiles#2
displayName: Copy build output to artifacts staging
inputs:
SourceFolder: "$(Build.SourcesDirectory)"
flattenFolders: true
Contents: |
'**\bin\**\*.dacpac'
'**\PublishProfile\*.publish.xml'
TargetFolder: "$(Build.ArtifactStagingDirectory)"
This pipe multiline syntax is all over the examples
https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/copy-files?view=azure-devops&tabs=yaml#examples
I've also used Get-ChildItem to doubly confirm that the files exist.
It seems like | / multiline doesn't work as described.
As usual, as I write this I checked in detail and the one difference between my code and the example was single quotes.
So it works if you remove single quotes.
Does anyone even QA this stuff?
- task: CopyFiles#2
displayName: Copy build output to artifacts staging
inputs:
SourceFolder: "$(Build.SourcesDirectory)"
flattenFolders: true
Contents: |
# NOTE THESE PATHS ARE NOT SURROUNDED BY SINGLE QUOTES
# EVEN THOUGH THIS WORKS IN THE SINGLE LINE VERSION
**\bin\**\*.dacpac
**\PublishProfile\*.publish.xml
TargetFolder: "$(Build.ArtifactStagingDirectory)"
Other hot tips to save you hours:
Use this to list files to help troubleshoot missing files
- task: Bash#3
inputs:
targetType: inline
workingDirectory: $(PIPELINE.WORKSPACE)
script: ls -R
Remember Linux is CASE SENSITIVE - get the case wrong and it won't find your files
As of right now, you can't parameterise service connections. Maybe that will change in future
It's possible to get indentation wrong in YAML and it gives you no clues
This code makes all the variables in the variable group TST available (these are under "Library" not "Environment" - go figure)
variables:
- group: TST
This code (with extra indentation) doesn't throw an error or give any clues, it just doesn't make any variables available. All your variables like $(MyVariable) will be treated as literals
variables:
- group: TST

how to convert classic build job to yaml build in AzureDevops

We have a working classic build job in azure Devops with an self hosted agent pool. But when we tried to convert this build job to yaml method, while executing no agents are getting assigned and its hanging. Could you please correct me here if i am doing something task.
Error
"All eligible agents are disabled or offline"
below is the converted yaml file from classic build - agent job
pool:
name: MYpool
demands: maven
#Your build pipeline references an undefined variable named ‘Parameters.mavenPOMFile’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972
steps:
- task: Maven#3
displayName: 'Maven pom.xml'
inputs:
mavenPomFile: '$(Parameters.mavenPOMFile)'
- task: CopyFiles#2
displayName: 'Copy Files to: $(build.artifactstagingdirectory)'
inputs:
SourceFolder: '$(system.defaultworkingdirectory)'
Contents: '**/*.war'
TargetFolder: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: Root'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: Root
condition: succeededOrFailed()
- task: CopyFiles#2
displayName: 'Copy wars to build directory'
inputs:
SourceFolder: '$(build.artifactstagingdirectory)/target'
TargetFolder: '/home/myadmin/builds/$(build.buildnumber)'
- task: CopyFiles#2
displayName: 'copying docker file to Build Directory'
inputs:
SourceFolder: Admin
TargetFolder: '/home/myadmin/builds/$(build.buildnumber)'
- bash: |
# Write your commands here
mv /home/myadmin/builds/$(build.buildnumber)/mypack0.0.1.war /home/myadmin/builds/$(build.buildnumber)/ROOT.war
displayName: 'Name war file Root.war'
- task: Docker#2
displayName: 'Build the docker image'
inputs:
repository: 'mycontainerregistry.azurecr.io/myservice'
command: build
Dockerfile: '/home/myadmin/builds/$(build.buildnumber)/Dockerfile'
tags: '$(Build.BuildNumber)-DEV'
- bash: |
# Write your commands here
docker login mycontainerregistry.azurecr.io
docker push mycontainerregistry.azurecr.io/myservice:$(Build.BuildNumber)-DEV
displayName: 'Push Docker Image'
- task: CopyFiles#2
displayName: 'Copy Deployment file'
inputs:
SourceFolder: /home/myadmin/kubernetes
TargetFolder: '/home/myadmin/builds/$(build.buildnumber)'
- task: qetza.replacetokens.replacetokens-task.replacetokens#3
displayName: 'Replace image in deployment file'
inputs:
rootDirectory: '/home/myadmin/builds/$(build.buildnumber)'
targetFiles: '**/*.yml'
In my previous answer, I said when I wait for nearly 20-30 mins, the interface of agent will prompt below message.
In fact, this is a process which upgrade the agent to latest version automatically.
Yes, when you using YAML with private agent, the agent version must be the latest one. No matter you add the demands or not.
For our system, the agent version is a implicit demand that your agent must satisfied with the latest one when you applying it in YAML.
If it is not satisfied, it will be blocked and the agent upgrade process will be forced to be performed automatically by system after some times.
So, to execute the private agent in YAML successfully, please upgrade the agent to latest one manually.
Since what my colleague and I talked are all private to microsoft in this ticket, sorry you could not get visible on this summary. So, here I take the screenshots about it, and you can refer to it: https://imgur.com/a/4OnzHp3
We are still working on why the system prompting so confusing message like: All eligible agents are disabled or offline. And, am trying to do some contribution to let this message more clear, for example: no agents meet demands: agent version xxx.