my question is closely releted to this one, but I this one doesnt seem to help me :( How to retrieve value from nested object?
I am trying to loop over an object that i get as a parameter, I use a default value as an example, the values in this object are also objects.
I have a template:
parameters:
- name: fruits
type: object
default:
- Banana:
name: " acuminata"
url: "www.banana.com/ acuminata"
- Apple:
name: "jonaGold"
url: "www.apple.com"
steps:
- bash: echo ${{parameters.fruits[0].url}}
- ${{ each fruit in parameters.fruits }}:
- script: echo ${{fruit.name}}
Both the first bash and the second bash are not working. Is it possible to use objects in azure pipelines like this?
This is my output for both bash steps:
2022-04-13T13:01:21.8035586Z ##[section]Starting: CmdLine
2022-04-13T13:01:21.8178179Z ==============================================================================
2022-04-13T13:01:21.8178533Z Task : Command line
2022-04-13T13:01:21.8178874Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2022-04-13T13:01:21.8179171Z Version : 2.201.1
2022-04-13T13:01:21.8179614Z Author : Microsoft Corporation
2022-04-13T13:01:21.8179954Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2022-04-13T13:01:21.8180355Z ==============================================================================
2022-04-13T13:01:23.2732262Z Generating script.
2022-04-13T13:01:23.2840010Z Script contents: shell
2022-04-13T13:01:23.2849739Z echo
2022-04-13T13:01:23.3217106Z ========================== Starting Command Output ===========================
2022-04-13T13:01:23.3517504Z ##[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a\_temp\7a91b1cc-432d-4a20-b39c-7c6b0c78724b.cmd""
2022-04-13T13:01:23.3655983Z ECHO is off.
2022-04-13T13:01:23.3990627Z ##[section]Finishing: CmdLine
I just found a solution. The other thread does provide an answer, I just need to read better :P.
You can access the values with the 'value' keyword. I also removed the '-' before the keywords so its not an array anymore. The code now looks like this:
parameters:
- name: fruits
type: object
default:
Banana:
name: " acuminata"
url: "www.banana.com/ acuminata"
Apple:
name: "jonaGold"
url: "www.apple.com"
steps:
- ${{ each fruit in parameters.fruits }}:
- script: echo ${{fruit.value.name}}
And I get this output:
I did not yet get the first one working, but i don't really need this for my project
- bash: echo ${{parameters.fruits[0].value.url}}
Related
If have created the Pipeline for One of our CRM Solution in Yaml
I have Created variables SolutionExit,SolutionName,SolutionVersion
#back-up.yaml
parameters:
- name: Solution
displayName: Solution Need to be backed up
type: object
default: [solution1,Solution2]
trigger:
- none
variables:
SolutionExit: ''
SolutionVersion: ''
SolutionName: ''
pool:
vmImage: windows-2019
steps:
- task: MSCRMToolInstaller#12
inputs:
nugetFeed: 'official'
psFeed: 'official'
- ${{each value in parameters.Solution}}:
- task: MSCRMGetSolution#12
inputs:
crmConnectionString: '$(ConnectStr)'
solutionName: '${{value}}'
existsVariableName: 'SolutionExit' #replace existing variable with output value True or False
versionVariableName: 'SolutionVersion' #replace existing variable with output value
displayVariableName: 'SolutionName' #replace existing variable with output value
- template: template/validate-solution.yml
parameters:
SolutionExit: $(SolutionExit)
#validate-solution.yml
parameters:
SolutionExit: ""
steps:
- ${{if eq(parameters.SolutionExit, 'True')}}:
- script: echo "Solution exist $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
- ${{else}}:
- bash: echo "solution doesnt exit $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
But runtime parameters are not accepted If condition never satisfies. How to pass the runtime parameter
This Is the result for both cases
If the solution available
Starting: Bash
==============================================================================
Task : Bash
Description : Run a Bash script on macOS, Linux, or Windows
Version : 3.214.0
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
Script contents:
echo "solution doesnt exit True,1.0.0.11,Solution1,True"
"C:\Program Files\Git\bin\bash.exe" -c pwd
/d/a/_temp
========================== Starting Command Output ===========================
"C:\Program Files\Git\bin\bash.exe" /d/a/_temp/237f7dc2-debc-4c47-ba88-41a6a97c49d9.sh
solution doesnt exit True,1.0.0.11,soultion1,True
Finishing: Bash
if solution not available
Starting: Bash
==============================================================================
Task : Bash
Description : Run a Bash script on macOS, Linux, or Windows
Version : 3.214.0
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
Script contents:
echo "solution doesnt exit False,,,False"
"C:\Program Files\Git\bin\bash.exe" -c pwd
/d/a/_temp
========================== Starting Command Output ===========================
"C:\Program Files\Git\bin\bash.exe" /d/a/_temp/49e5bea1-b380-4a74-a0ce-b89d0dfb3a07.sh
solution doesnt exit False,,,False
Finishing: Bash
How to fix this issue. There is compile time and Runtime variables,as per microsoft I tried ${{if eq(variables['SolutionExit'], 'True')}}: this too didnt work. We cannot use the macro variable in conditions like $(SolutionExit)
From your YAML definition, the variable will generate during the process of Pipeline Run.
The Variable should be Runtime variable. It will be expand at runtime. But the If expression and parameters will be expand at Compile time.
In this case, If expression will not work.
To solve this issue, you need to change to use Condition to replace the If expression.
At the same time, you don't need to use parameters to pass the variable value. You can directly use the variable in the condition.
For example:
Main YAML :
parameters:
- name: Solution
displayName: Solution Need to be backed up
type: object
default: [solution1,Solution2]
trigger:
- none
variables:
SolutionExit: ''
SolutionVersion: ''
SolutionName: ''
pool:
vmImage: windows-2019
steps:
- task: MSCRMToolInstaller#12
inputs:
nugetFeed: 'official'
psFeed: 'official'
- ${{each value in parameters.Solution}}:
- task: MSCRMGetSolution#12
inputs:
crmConnectionString: '$(ConnectStr)'
solutionName: '${{value}}'
existsVariableName: 'SolutionExit' #replace existing variable with output value True or False
versionVariableName: 'SolutionVersion' #replace existing variable with output value
displayVariableName: 'SolutionName' #replace existing variable with output value
- template: template/validate-solution.yml
Template YAML:
steps:
- script: echo "Solution exist $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
condition: eq(variables['SolutionExit'], 'true')
- bash: echo "solution doesnt exit $(SolutionExit),$(SolutionVersion),$(SolutionName),${{parameters.SolutionExit}}"
condition: ne(variables['SolutionExit'], 'true')
My issue
I am testing this YAML script in Azure DevOps:
parameters:
- name: myStep
type: step
default:
script: echo my step
- name: mySteplist
type: stepList
default:
- script: echo step one
- script: echo step two
jobs:
- job: myStep
steps:
- ${{ parameters.myStep }}
- job: stepList
steps: ${{ parameters.mySteplist }}
But lines with ${{ }} are rejected with this error message:
Incorrect type. Expected "object"
What I need
Why am I getting this message? What can I do to fix it?
What I tested
I found this sample in the official documentation. This is what puzzle me. I would expect this to be a valid YAML script given the source.
I found what happend.
This is a bug in Visual Studio Code. The script works as expected when launch
I'm looking for a way to define a variable in my azure-pipelines.yml file where I can substring the 'Build.SourceVersion' -> Use the first 7 characters only.
Seems like there is no build-in function that can do such string operations in the documentation. Is there something I'm missing?
My other approach would be to use a bash task and overwrite the variable there but finding something build-in that can do this would be way better solution.
My other approach would be to use a bash task and overwrite the variable there but finding something build-in that can do this would be way better solution.
I agree with Lucas. There is no such built-in task to get the first 7 characters of $(Build.SourceVersion) in Azure DevOps.
We could use the command line/powershell task to split long sha into short sha:
echo $(Build.SourceVersion)
set TestVar=$(Build.SourceVersion)
set MyCustomVar=%TestVar:~0,7%
echo %MyCustomVar%
echo ##vso[task.setvariable variable=ShortSourceVersion]%MyCustomVar%
In this case, we could get the short versions of Build.SourceVersion and set it as environment variable.
Then we could set this command line task as a task group:
So, we could use this task to set the ShortSourceVersion directly.
Hope this helps.
Here is the shortest version that I use for this job;
- bash: |
longcommithash=$(Build.SourceVersion)
echo "##vso[task.setvariable variable=shorthash;]$(echo ${longcommithash::9})"
It gives you the output as shown below;
On Azure devops it's possible to be done using batch technique indeed - like other answers propose in here - however I wanted to remove string from the end of string - and with batch this gets more complex ( See following link ).
Then I've concluded to use Azure's built-in expressions, for example like this:
- name: sourceBranchName
value: ${{ replace(variables['Build.SourceBranch'], 'refs/heads/') }}
replace will have similar effect to substring - so it will remove unnecessary part from original string.
If however you want to concatenate some string and then remove it from original name - it's possible to be done using for instance format function:
- name: originBranchName
value: ${{ replace(variables.sourceBranchName, format('-to-{0}', variables.dynamicvar ) ) }}
If dynamicvar is a - then function will remove -to-a from branch name.
You are right, there is no native way to do it. You will have to write a script to transform the variable.
Here is an example:
trigger:
- master
resources:
- repo: self
stages:
- stage: Build
displayName: Build image
jobs:
- job: Build
displayName: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: CmdLine#2
inputs:
script: ' x=`echo "$(Build.SourceVersion)" | head -c 7`; echo "##vso[task.setvariable variable=MyVar]$x"'
- task: CmdLine#2
inputs:
script: 'echo "$(MyVar)"'
Sure you can! If you absolutely must. Here is a runtime computation taking the first 7 characters of the Build.SourceVersion variable.
variables:
example: ${{ format('{0}{1}{2}{3}{4}{5}{6}', variables['Build.SourceVersion'][0], variables['Build.SourceVersion'][1], variables['Build.SourceVersion'][2], variables['Build.SourceVersion'][3], variables['Build.SourceVersion'][4], variables['Build.SourceVersion'][5], variables['Build.SourceVersion'][6]) }}
NB: I can't get it to work with $[...] syntax as the variable is apparently empty at the initial.
For (mainly) pedagogical reasons, I'm trying to run this workflow in GitHub actions:
name: "We 🎔 Perl"
on:
issues:
types: [opened, edited, milestoned]
jobs:
seasonal_greetings:
runs-on: windows-latest
steps:
- name: Maybe greet
id: maybe-greet
env:
HEY: "Hey you!"
GREETING: "Merry Xmas to you too!"
BODY: ${{ github.event.issue.body }}
run: |
$output=(perl -e 'print ($ENV{BODY} =~ /Merry/)?$ENV{GREETING}:$ENV{HEY};')
Write-Output "::set-output name=GREET::$output"
produce_comment:
name: Respond to issue
runs-on: ubuntu-latest
steps:
- name: Dump job context
env:
JOB_CONTEXT: ${{ jobs.maybe-greet.steps.id }}
run: echo "$JOB_CONTEXT"
I need two different jobs, since they use different context (operating systems), but I need to get the output of a step in the first job to the second job. I am trying with several combinations of the jobs context as found here but there does not seem to be any way to do that. Apparently, jobs is just the name of a YAML variable that does not really have a context, and the context job contains just the success or failure. Any idea?
Check the "GitHub Actions: New workflow features" from April 2020, which could help in your case (to reference step outputs from previous jobs)
Job outputs
You can specify a set of outputs that you want to pass to subsequent jobs and then access those values from your needs context.
See documentation:
jobs.<jobs_id>.outputs
A map of outputs for a job.
Job outputs are available to all downstream jobs that depend on this job.
For more information on defining job dependencies, see jobs.<job_id>.needs.
Job outputs are strings, and job outputs containing expressions are evaluated on the runner at the end of each job. Outputs containing secrets are redacted on the runner and not sent to GitHub Actions.
To use job outputs in a dependent job, you can use the needs context.
For more information, see "Context and expression syntax for GitHub Actions."
To use job outputs in a dependent job, you can use the needs context.
Example
jobs:
job1:
runs-on: ubuntu-latest
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: echo "test=hello" >> $GITHUB_OUTPUT
- id: step2
run: echo "test=world" >> $GITHUB_OUTPUT
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo ${{needs.job1.outputs.output1}} ${{needs.job1.outputs.output2}}
Note the use of $GITHUB_OUTPUT, instead of the older ::set-output now (Oct. 2022) deprecated.
To avoid untrusted logged data to use set-state and set-output workflow commands without the intention of the workflow author we have introduced a new set of environment files to manage state and output.
Jesse Adelman adds in the comments:
This seems to not work well for anything beyond a static string.
How, for example, would I take a multiline text output of step (say, I'm running a pytest or similar) and use that output in another job?
either write the multi-line text to a file (jschmitter's comment)
or base64-encode the output and then decode it in the next job (Nate Karasch's comment)
Update: It's now possible to set job outputs that can be used to transfer string values to downstream jobs. See this answer.
What follows is the original answer. These techniques might still be useful for some use cases.
Write the data to file and use actions/upload-artifact and actions/download-artifact. A bit awkward, but it works.
Create a repository dispatch event and send the data to a second workflow. I prefer this method personally, but the downside is that it needs a repo scoped PAT.
Here is an example of how the second way could work. It uses repository-dispatch action.
name: "We 🎔 Perl"
on:
issues:
types: [opened, edited, milestoned]
jobs:
seasonal_greetings:
runs-on: windows-latest
steps:
- name: Maybe greet
id: maybe-greet
env:
HEY: "Hey you!"
GREETING: "Merry Xmas to you too!"
BODY: ${{ github.event.issue.body }}
run: |
$output=(perl -e 'print ($ENV{BODY} =~ /Merry/)?$ENV{GREETING}:$ENV{HEY};')
Write-Output "::set-output name=GREET::$output"
- name: Repository Dispatch
uses: peter-evans/repository-dispatch#v1
with:
token: ${{ secrets.REPO_ACCESS_TOKEN }}
event-type: my-event
client-payload: '{"greet": "${{ steps.maybe-greet.outputs.GREET }}"}'
This triggers a repository dispatch workflow in the same repository.
name: Repository Dispatch
on:
repository_dispatch:
types: [my-event]
jobs:
myEvent:
runs-on: ubuntu-latest
steps:
- run: echo ${{ github.event.client_payload.greet }}
In my case I wanted to pass an entire build/artifact, not just a string:
name: Build something on Ubuntu then use it on MacOS
on:
workflow_dispatch:
# Allows for manual build trigger
jobs:
buildUbuntuProject:
name: Builds the project on Ubuntu (Put your stuff here)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: some/compile-action#v99
- uses: actions/upload-artifact#v2
# Upload the artifact so the MacOS runner do something with it
with:
name: CompiledProject
path: pathToCompiledProject
doSomethingOnMacOS:
name: Runs the program on MacOS or something
runs-on: macos-latest
needs: buildUbuntuProject # Needed so the job waits for the Ubuntu job to finish
steps:
- uses: actions/download-artifact#master
with:
name: CompiledProject
path: somewhereToPutItOnMacOSRunner
- run: ls somewhereToPutItOnMacOSRunner # See the artifact on the MacOS runner
It is possible to capture the entire output (and return code) of a command within a run step, which I've written up here to hopefully save someone else the headache. Fair warning, it requires a lot of shell trickery and a multiline run to ensure everything happens within a single shell instance.
In my case, I needed to invoke a script and capture the entirety of its stdout for use in a later step, as well as preserve its outcome for error checking:
# capture stdout from script
SCRIPT_OUTPUT=$(./do-something.sh)
# capture exit code as well
SCRIPT_RC=$?
# FYI, this would get stdout AND stderr
SCRIPT_ALL_OUTPUT=$(./do-something.sh 2>&1)
Since Github's job outputs only seem to be able to capture a single line of text, I also had to escape any newlines for the output:
echo "::set-output name=stdout::${SCRIPT_OUTPUT//$'\n'/\\n}"
Additionally, I needed to ultimately return the script's exit code to correctly indicate whether it failed. The whole shebang ends up looking like this:
- name: A run step with stdout as a captured output
id: myscript
run: |
# run in subshell, capturiing stdout to var
SCRIPT_OUTPUT=$(./do-something.sh)
# capture exit code too
SCRIPT_RC=$?
# print a single line output for github
echo "::set-output name=stdout::${SCRIPT_OUTPUT//$'\n'/\\n}"
# exit with the script status
exit $SCRIPT_RC
continue-on-error: true
- name: Add above outcome and output as an issue comment
uses: actions/github-script#v5
env:
STEP_OUTPUT: ${{ steps.myscript.outputs.stdout }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
// indicates whather script succeeded or not
let comment = `Script finished with \`${{ steps.myscript.outcome }}\`\n`;
// adds stdout, unescaping newlines again to make it readable
comment += `<details><summary>Show Output</summary>
\`\`\`
${process.env.STEP_OUTPUT.replace(/\\n/g, '\n')}
\`\`\`
</details>`;
// add the whole damn thing as an issue comment
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: comment
})
Edit: there is also an action to accomplish this with much less bootstrapping, which I only just found.
2022 October update: GitHub is deprecating set-output and recommends to use GITHUB_OUTPUT instead. The syntax for defining the outputs and referencing them in other steps, jobs.
An example from the docs:
- name: Set color
id: random-color-generator
run: echo "SELECTED_COLOR=green" >> $GITHUB_OUTPUT
- name: Get color
run: echo "The selected color is ${{ steps.random-color-generator.outputs.SELECTED_COLOR }}"
I'm trying to execute script by passing variable from the azure pipeline. Here is my simple test pipeline:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
major: 1.2
minor: $[counter(variables['major'], 1)]
version: $[format('{0}.{1}', variables.major, variables.minor)]
name: $[format('{0} v{1}', 'Yaml Testing', variables['version'])]
steps:
- script: |
echo variables['version']
echo $(variables.version)
echo '$(variables.version)'
echo "$(variables.version')"
echo $[ variables['version'] ]
echo ${{ variables['version'] }}
echo $(Build.BuildNumber)
displayName: 'Run a multi-line script'
- script: $[format('{0} {1}', 'echo', variables['version'])]
displayName: 'Echo Formatted String'
Outputs of scripts are:
Generating script.
========================== Starting Command Output ===========================
##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "d:\a\_temp\3cb45b74-f6cd-4d2f-bf65-f635779b9d86.cmd""
variables['version']
$(variables.version)
'$(variables.version)'
"$(variables.version')"
$[ variables['version'] ]
$[format('{0}.{1}', variables.major, variables.minor)]
Yaml Testing v1.2.11
##[section]Finishing: Run a multi-line script
and
Generating script.
Script contents:
$[format('{0} {1}', 'echo', variables['version'])]
========================== Starting Command Output ===========================
##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "d:\a\_temp\5e42dc54-e027-4b9a-9af4-0db02e879b0f.cmd""
'$[format' is not recognized as an internal or external command,
operable program or batch file.
##[error]Cmd.exe exited with code '1'.
##[section]Finishing: Echo Formatted String
Strangely, code work fine in the name, but not when trying to use in the script.
What am I doing wrong?
$[] is evaluated at runtime, that is why it is not working. You can pass ${{expression}} to script like below:
- script: ${{format('{0} {1}', 'echo', '$(version)')}}
displayName: 'Echo Formatted String'
Expression in ${{}} will be evaluated at parse time. Before the -script is actually executed, expression in
${{}} is parsed into a valid command.
You can directly refer to the self-defined variables like this '$(variableName)' instead of $(variables.Name)