Pass variable set in PowerShell task to another task (YAML) - azure-devops

I have a yaml file with following tasks:
parameters:
steps:
- task: AzurePowerShell#4
displayName: 'script'
inputs:
azureSubscription:
ScriptPath:
ScriptArguments:
azurePowerShellVersion: LatestVersion
- task: AzureResourceGroupDeployment#2
displayName: 'deployment'
inputs:
azureSubscription:
resourceGroupName:
location:
overrideParameters: '-abc $(var1) -def $(var2)'
deploymentMode: 'Incremental'
In the Powershell Script, I'm setting 2 variables as follows:
$ABC = 1
$DEF = 2
Write-Host "##vso[task.setvariable variable=var1;isOutput=true]$ABC"
Write-Host "##vso[task.setvariable variable=var2;isOutput=true]$DEF"
On trying to use these variables in the 2nd task (in overrideParameters section), I see the following error:
[error]InvalidContentLink: Unable to download deployment content from 'xxxx$(var1)'
[error]InvalidContentLink: Unable to download deployment content from 'xxxx$(var2)'
Am I setting variables in PowerShell script wrong?

You can try to add a reference name to the first task. For example:
- task: AzurePowerShell#4
displayName: 'script'
inputs:
azureSubscription:
ScriptPath:
ScriptArguments:
azurePowerShellVersion: LatestVersion
name: test
Then in the second task, get the variable value in the form of $(test.var1).
This is because in definition editor, downstream tasks won't get variable name intellisense for output variables that were published by an ad-hoc script. You can refer to this document for details.
In addition, here is a blog with some examples on how to pass variables in Azure Pipelines YAML tasks.

Related

AzureCLI task in Azure DevOps pipeline cannot find the location of scripts

I am using Azure DevOps pipeline and in one of my tasks I need to run a bash script which contains some Azure CLI scripts. I have put this script in a folder called scripts, and my pipeline is running in pipelines folder. Pipelines and script folders are at the same level in root directory. The following shows the part of my pipeline where I run the AzureCLI#2 task, but when the pipeline runs it raises the error that it cannot find the file!
I have already pushed everything in the repository and I can see the files. However, the pipeline cannot find it. I am using AzureCLI#2 documentation link to provide values for this task. The part of pipeline that uses AzureCLI is as follows:
pool:
vmImage: ubuntu-20.04
trigger:
branches:
include:
- "feature/ORGTHDATAMA-4810"
exclude:
- "main"
- "release"
paths:
include:
- "dip-comma-poc/**"
variables:
- group: proj-comma-shared-vg
stages:
- stage: DownloadArtifact
displayName: "Download python whl from artifactory"
jobs:
- job: DownloadArtifactJob
steps:
- checkout: self
## To download from devops artifactory with AZ CLI
## https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/azure-cli-v2?view=azure-pipelines
- task: AzureCLI#2
inputs:
azureSubscription: "sc-arm-pa042-man"
scriptType: 'bash'
scriptLocation: 'scriptPath'
scriptPath: 'dip-comma-poc/deployment-pipelines/scripts/sp-login.sh'
arguments: '$(SVCApplicationID) $(SVCSecretKey) $(SVCDirectoryID)'
displayName: "Download python whl from artifactory"
This caused the following error:
To resolve the error I tried using relative path in scriptPath as following but it caused the same error:
- task: AzureCLI#2
inputs:
azureSubscription: "sc-arm-pa042-man"
scriptType: 'bash'
scriptLocation: 'scriptPath'
scriptPath: './scripts/sp-login.sh'
arguments: '$(SVCApplicationID) $(SVCSecretKey) $(SVCDirectoryID)'
displayName: "Download python whl from artifactory"
I also tried inlineScript but again it cannot find the file.
- task: AzureCLI#2
inputs:
azureSubscription: "sc-arm-pa042-man"
scriptType: 'bash'
scriptLocation: 'inlineScript'
arguments: '$(SVCApplicationID) $(SVCSecretKey) $(SVCDirectoryID)'
inlineScript: './scripts/sp-login.sh $1 $2 $3'
displayName: "Download python whl from artifactory"
This also raised the same error:
How can I refer to my script in the pipeline yaml file so that it does not raise "No such file or directory error" as shown above? Thank you.
Open the Git repository on the web UI of your Azure DevOps and then check whether its file structure is looking like as below image shows.
If it is same as this file structure. You need to change the file path set on the Azure CLI task to be "deployment-pipelines/scripts/sp-login.sh" instead of "dip-comma-poc/deployment-pipelines/scripts/sp-login.sh".
- task: AzureCLI#2
inputs:
azureSubscription: "sc-arm-pa042-man"
scriptType: 'bash'
scriptLocation: 'scriptPath'
scriptPath: 'deployment-pipelines/scripts/sp-login.sh'
arguments: '$(SVCApplicationID) $(SVCSecretKey) $(SVCDirectoryID)'
displayName: "Download python whl from artifactory"

Deploy to another workspace in synapse with CI/CD

Currently I am able to deploy to Synapse via the Azure DevOps portal pipeline that I made with the UI. I am trying to achieve the same result via a yml file but I am encountering the problem [error]Encountered with exception:Error: No file found with this pattern and I have the following code:
trigger:
branches:
include:
- workspace_publish
pool:
name: Azure Pipelines
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Deploying
steps:
- task: CopyFiles#2
displayName: 'Copy ARM Template Files to: $(Build.ArtifactStagingDirectory)'
inputs:
SourceFolder: testsaws
Contents: '*json'
archiveFile: '$(Build.ArtifactStagingDirectory)'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishPipelineArtifact#1
displayName: 'Publish Pipeline Artifact'
inputs:
#PathtoPublish: ./testsaws
targetPath: '$(Build.ArtifactStagingDirectory)'
artifact: 'ASA_Drop'
- task: AzureSynapseWorkspace.synapsecicd-deploy.synapse-deploy.Synapse workspace deployment#1
displayName: 'Synapse deployment task for workspace: qasaws'
inputs:
azureSubscription: 'dataplatform-qa-group-SPN'
subscriptionId: 'XXX'
action: 'Create Or Update Resource Group'
resourceGroupName: 'dataplatform-qa-group'
location: 'West US 2'
TemplateFile: '$(Build.ArtifactStagingDirectory)/ASA_Drop/ARM/TemplateForWorkspace.json'
ParametersFile: '$(Build.ArtifactStagingDirectory)/ASA_Drop/ARM/TemplateParametersForWorkspaceQA.json'
Any help or suggestion in finding the problem is welcome and appreciated.
When in doubt, print out the contents. I used the following command to print out the content of the current working directory and found out the artifact didn't contain any file.
# PowerShell
# Run a PowerShell script on Linux, macOS, or Windows
- task: PowerShell#2
inputs:
#targetType: 'filePath' # Optional. Options: filePath, inline
#filePath: # Required when targetType == FilePath
#arguments: # Optional
#script: '# Write your PowerShell commands here.Write-Host Hello World'
# Required when targetType == Inline
#errorActionPreference: 'stop' # Optional. Options: default, stop, continue, silentlyContinue
#warningPreference: 'default' # Optional. Options: default, stop, continue, silentlyContinue
#informationPreference: 'default' # Optional. Options: default, stop, continue, silentlyContinue
#verbosePreference: 'default' # Optional. Options: default, stop, continue, silentlyContinue
#debugPreference: 'default' # Optional. Options: default, stop, continue, silentlyContinue
#failOnStderr: false # Optional
#ignoreLASTEXITCODE: false # Optional
#pwsh: false # Optional
#workingDirectory: # Optional
Then I changed the following in the yaml file:
- task: PublishPipelineArtifact#1
displayName: 'Publish Pipeline Artifact'
inputs:
#PathtoPublish: ./testsaws
targetPath: '$(Build.ArtifactStagingDirectory)'
artifact: 'ASA_Drop'
- task: AzureSynapseWorkspace.synapsecicd-deploy.synapse-deploy.Synapse workspace deployment#1
displayName: 'Synapse deployment task for workspace: qasaws'
inputs:
azureSubscription: 'qa-group-SPN'
subscriptionId: 'XXX'
action: 'Create Or Update Resource Group'
resourceGroupName: 'qa-group'
location: 'West US 2'
TemplateFile: 'testsaws/TemplateForWorkspace.json'
ParametersFile: 'testsaws/TemplateParametersForWorkspaceQA.json'

Pass Variable Group as Dictionary To Python Script

I have a variable group that i'm using from a python script. Something like this:
- task: PythonScript#0
inputs:
scriptSource: 'inline'
script: |
print('variableInVariableGroup: $(variableInVariableGroup)')
I'd like to write my script so that I can iterate over the variable group without explicitly referencing individual variables. Is there a way to feed in the entire variable group to the script as a dictionary or something similar?
You could not do that directly, for the workaround is to get the vars via azure cli and set with task variable, then get them in the python script task.
Something like below:
# 'Allow scripts to access the OAuth token' was selected in pipeline. Add the following YAML to any steps requiring access:
# env:
# MY_ACCESS_TOKEN: $(System.AccessToken)
# Variable Group 'vargroup1' was defined in the Variables tab
resources:
repositories:
- repository: self
type: git
ref: refs/heads/testb2
jobs:
- job: Job_1
displayName: Agent job 1
pool:
vmImage: ubuntu-20.04
steps:
- checkout: self
persistCredentials: True
- task: PowerShell#2
name: TestRef
displayName: PowerShell Script
inputs:
targetType: inline
script: >-
echo $(System.AccessToken) | az devops login
$a=az pipelines variable-group variable list --org 'https://dev.azure.com/orgname/' --project testpro1 --group-id 3 --only-show-errors --output json
echo "$a"
echo "##vso[task.setvariable variable=allvars;isOutput=true]$a"
- task: PythonScript#0
displayName: Run a Python script
inputs:
scriptSource: inline
script: "b=$(TestRef.allvars)\nprint(b)\n\n "
...

azure pipeline variable created in powershell script not displaying value

In an Azure Pipeline, I want to set a variable in a Powershell script and then use that variable in a later build step. The variable doesn't print out a value.
In my Powrshell script I hardcode a value for testing purposes:
$packageFolder = 'dotnet/TestPackage/'
##vso[task.setvariable variable=changedPackage;isOutput=true]$packageFolder
Here is my YAML file:
steps:
- task: PowerShell#2
displayName: 'Detect Subfolder Changes'
name: setvarStep
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)\detectchanges.ps1
failOnStderr: true
- script: echo "$(setvarStep.changedPackage)"
- task: DotNetCoreCLI#2
displayName: 'dotnet build'
condition: ne('$(setvarStep.changedPackage)', '')
inputs:
projects: '$(setvarStep.changedPackage)'
When the pipeline gets to the script step it just prints out this:
Script contents:
echo "$(setvarStep.changedPackage)"
I don't think the variable is actually set. When the pipeline gets to the dotnet build step it throws an error: ##[error]Project file(s) matching the specified pattern were not found.
While having debugging turned on I noticed this in the dotnet build step logs: ##[debug]findPath: 'C:\agents\librarywin-1\_work\2\s\$(setvarStep.changedPackage)'
I think my issue is the pipeline variable syntax is wrong even though I'm following the example on Microsoft's website. I can't figure it out.
You need double quote the logging command:
detectchanges.ps1:
$packageFolder = 'dotnet/TestPackage/'
"##vso[task.setvariable variable=changedPackage;isOutput=true]$packageFolder"
YAML:
steps:
- task: PowerShell#2
displayName: 'Detect Subfolder Changes'
name: setvarStep
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)\detectchanges.ps1
failOnStderr: true
- script: echo "$(setvarStep.changedPackage)"
You should have Write-Host
$packageFolder = 'dotnet/TestPackage/'
Write-Host ##vso[task.setvariable variable=changedPackage;isOutput=true]$packageFolder
like here

How to use output variable value from one powershell script to another powershell script

Hi All I'm working on ymal release pipeline where I have two PowerShell script.
test1.ps1
test2.ps1
step 1)
In test1.ps1 I'm setting a output variable using :
$link="google.com"
Write-Host "##vso[task.setvariable variable=packageurl]$link"
step 2)
Now I want to use this pacakgeurl variable value in test2.ps1 script.
Ymal code will look like :
- task: AzurePowerShell#3
displayName: 'Query Latest Package'
inputs:
azureSubscription: 'test1'
ScriptType: FilePath
ScriptPath: 'source\test1.ps1'
- task: PowerShell#2
displayName: 'Download package'
inputs:
targetType: filePath
filePath: 'source\test2.ps1'
So basically I have to use the output variable value from 1 task to 2 task via PowerShell script.
I also tried to follow this link : https://developercommunity.visualstudio.com/content/problem/676342/how-to-use-output-variable-from-a-powershell-scrip.html
Can anyone help me on this ..?
Thanks in advance.
This works as you expect:
trigger: none
pr: none
pool:
vmImage: 'windows-latest'
steps:
- task: AzurePowerShell#3
displayName: 'Query Latest Package'
inputs:
azureSubscription: 'rg-the-code-manual'
ScriptType: FilePath
ScriptPath: 'stackoverflow\94-variables\script-1.ps1'
azurePowerShellVersion: LatestVersion
- task: PowerShell#2
displayName: 'Download package'
inputs:
targetType: filePath
filePath: 'stackoverflow\94-variables\script-2.ps1'
Yaml file is as you have it.
script-1.ps1 sets variable:
$link="google.com"
Write-Host "##vso[task.setvariable variable=packageurl]$link"
And script-2.ps1 uses that variable:
Write-Host $env:PACKAGEURL