Passing variables between PS stages on different agents in jenkins pipeline - powershell

I have a jenkins pipeline with two (powershell) stages, each running on a different node/slave.
I'm trying to set a variable during the first stage (on node 1) and use it afterwards on the next stage (on node 2) - however, I couldn't find a way to pass this variable between them.
It's worth clarifying that this variable is being set dynamically in stage 1, meaning that solutions like declaring an env:var using script or def or environment didn't help (I tried many combinations) - the value can vary every time I run the pipeline.
I tried setting a new value for env var in stage 1, but it looks like it only affects its value in the context of the running stage.
(yes, it must be in powershell and on different nodes)
this is the basic idea of the pipeline (which doesn't work)
pipeline {
agent { label 'Node1' }
stages{
stage('Calculate'){
steps {
git changelog: false, credentialsId: 'user', poll: false, url: 'https://mygit.com/myrepo.git'
powershell '''
$currentVersion=[version]$(git describe --tags)
$newVersion=New-Object -TypeName System.Version($currentVersion.Major, $currentVersion.Minor, $currentVersion.Build, ($currentVersion.Revision + 1))
'''
}
}
stage('Act'){
agent { label 'Node2' }
steps {
powershell "Write-Host $env:newVersion"
}
}
}
}
Any ideas?
Thanks!

In my scripts I define a var above the pipeline tag and then I use
returnValue = powershell returnStdout: true, script: '...'
to generate my Powershell output.
'returnStdout: true'
is the key to receive your output from your PowerShell script.
In another stage I use bat to call a PowerShell script and sned my var to it:
bat "powershell -File script.ps1 ${returnValue}"
Hope that helps.
Greetings

Related

Azure DevOps Pipeline - Create a Synapse managed-private-endpoints to a Azure Storage Account

I am trying to create a 'Synapse Managed private endpoint' to an Azure storage account via a 'Azure cli' task as a step in a pipeline. I want to create the MPE automatically.
The pipeline step calls a power-shell script with parameters. The script is located in source control. Calling the script and passing in parameter values is working fine.
Within the powershell script the following happens...
Get the json template (see below) from source control - this step works.
In the powershell script subsitute the json fields enclosed in <...> with a the parameter values passed in to the power-shell script - this works fine. The converted json is shown in the below screenshot.NB: sensitive values have been readacted here but look correct...
The line in the power-shell that's raising the error is...
az synapse managed-private-endpoints create --workspace-name "$pSynapseWorkspaceName" --pe-name "$pPrivateEndpointName" --file $mpeArmJson --debug --verbose
I think it's to do with the json-string parmater $mpeArmJson and double-quotes - this is what I need help solving ???
The value of $mpeArmJson which the value at this point is (note sensitive values have been readacted here but look correct)...
{
"name": "dds2-datalake-endpoint",
"properties": {
"privateLinkResourceId": "/subscriptions/<redacted subscription id>/resourceGroups/dds2-data-tst-rg/providers/Microsoft.Storage/storageAccounts/dds2datatstdlksa",
"groupId": "dfs",
"fqdns": [
"<redacted-storage-account>.dfs.core.windows.net"
]
}
}
In my Azure devOps pipeline I have created a 'Azure Powershell' task. The task calls a PowerShell script stored in source-control - taking in parameters.
Inside the powershell script I am calling...
New-AzSynapseManagedPrivateEndpoint -WorkspaceName "$pSynapseWorkspaceName" -Name "$pPrivateEndpointName" -DefinitionFile "$tmpDir"
NB: the devOps pipeline runs under a Az 'service principal' which creates the Synapse workspace and in doing so gets the 'owner' and also a 'synapse admin.' permissions automatically set in IAM.
The 'service principal' also needs 'blob storage data contributor' on the main storage account linked to Synapse.

How to pass jenkinsfile parameters to a powershell script

I have a jenkinsfile, where I am setting a parameter and wants to use it in another powershell script.
Here is my jenkinsfile
parameters {
choice( name: 'DeploymentEnvironment',choices: "None\nDev\nQA\nProd",description: 'Deployment Choice')
}
stage('Publishing to S3') {
steps {
withCredentials() {
echo "${params.DeploymentEnvironment}"
powershell(". '.\\packages.ps1' ${params.DeploymentEnvironment}")
}
}
}
echo "${params.DeploymentEnvironment}" = QA
My packages.ps1 script
...
Write-Output $DeploymentEnvironment
Write-Output $env.DeploymentEnvironment
Write-Output $env:DeploymentEnvironment
Write-Output ($params.DeploymentEnvironment)
...
I am unable to get DeploymentEnvironment=QA here in packages.ps1. Above every Write-Output command is printing nothing.
How can I pass and use DeploymentEnvironment variable declared in Jenkinsfile in my packages.ps1 script.
Thanks.
You need to pass your input parameter as an environment parameter to the PowerShell runtime environment, the easiest way to do so is using the environment directive that is a part of the declarative pipeline syntax, which sets parameters (in addition to default ones) that will be loaded as environments variables to the powershell (or cmd) execution environment.
The environment directive can be used in the pipeline level or in the stage level.
In your case you can use something like:
parameters {
choice( name: 'DeploymentEnvironment',choices: ['None', 'Dev', 'QA', 'Prod' ,description: 'Deployment Choice')
}
stage('Publishing to S3') {
environment {
DEPLOYMENT_ENVIRONMENT = "${params.DeploymentEnvironment}"
}
steps {
withCredentials() {
echo DEPLOYMENT_ENVIRONMENT
powershell(". '.\\packages.ps1'")
// DEPLOYMENT_ENVIRONMENT will be available as an environment parameter to the powershell script
}
}
}
then in your poweshell script use the DEPLOYMENT_ENVIRONMENT parameter.
A different approach will be to modify your powershell script to receive a parameter, and pass the parameter like you are doing now.
Btw, if your are only publishing things to s3 you can use the Pipeline: AWS Steps plugin for this task and avoid the powershell script

How to refer previous task and stop the build in azure devops if there is no new data to publish an artifact

Getsolution.exe will give New data available or no new data available, if new data available then next jobs should be executed else nothing should be executed. How should i do it? (i am working on classic editor)
example: i have set of tasks, consider 4 tasks:
task-1: builds the solution
task-2: runs the Getstatus.exe which get the status of data available or no data available
task-3: i should be able to use the above task and make a condition/use some api query and to proceed to publish an artifact if data is available if not cleanly break out of the task and stop the build. it Shouldn't proceed to publish artifact or move to the next available task
task-4:publish artifact
First what you need is to set a variable in your task where you run Getstatus.exe:
and then set condition in next tasks:
If you set doThing to different valu than Yes you will get this:
How to refer previous task and stop the build in azure devops if there is no new data to publish an artifact
Since we need to execute different task based on the different results of Getstatus.exe running, we need set the condition based on the result of Getstatus.exe running.
To resolve this, just like the Krzysztof Madej said, we could set variable(s) based on the return value of Getstatus.exe in the inline powershell task:
$dataAvailable= $(The value of the `Getstatus.exe`)
if ($dataAvailable -eq "True")
{
Write-Host ("##vso[task.setvariable variable=Status]Yes")
}
elseif ($dataAvailable -eq "False")
{
Write-Host ("##vso[task.setvariable variable=Status]No")
}
Then set the different condition for next task:
You could check the document Specify conditions for some more details.

Azure DevOps: Getting variable value by concatenating other variables'value as task input

I have my release pipeline Variables tab set like:
I would like to access my123 variable in task's display name by concatenating initialVariable's result.
Outputs
I have tried so far referencing only initialVariable and it returned proper value in Job's display name.
But when I try to create my123 value by using initialVariable(=123), I am not getting proper value (was hoping that $(initialVariable) would convert to 123 and $(my123) would get proper "finalValue").
Azure DevOps: Getting variable value by concatenating other variables'value as task input
This is a known issue. Because the value of nested variables (like $(my$(initialVariable)) are not yet supported in the build/release pipelines.
Check my other thread for some details.
The workaround is add a Run Inline Powershell task to set the variable based on the input pipeline variables, just like Josh answered.
For you case, I test it by following Powershell scripts:
if ($(initialVariable)-eq "123")
{
Write-Host "##vso[task.setvariable variable=my123]finalvalue"
}
else
{
Write-Host "##vso[task.setvariable variable=my123]otherValue"
}
Then we could get the variable my123 based on the value of variable initialVariable in following task, I add command line task to display the value:
In the result, the value in the command line task is correct finalvalue. But the display name is still $(my123):
Important:
That is also the question in your comment. This behavior is expected. That because the variable in the display name is just to get the predefined value. It's static acquisition, not dynamic. The variable my123 is assigned when running powershell. The static variable my123 in the display name does not go in to the environment where the powershell code is running.
So, the variable my123 in the title could not get the value in the task powershell. But other task could use it very well.
Hope this answer clean your puzzle.
It's ugly, but...
Like I mentioned in my comment, I don't think you're going to get this to work in the UI by default.
Luckily you can use PowerShell to hack this together if your REALLY need the ability to address a variable name based on the value of another variable.
All the variables (secrets are handled a little differently) in your build or release pipeline definition are made available to your powershell script FILE (not inline) via environment variables (ie. $env:initialVariable).
Suppose your situation is thus:
selector = selectable1 //this is the value that can change
selectable1 = theFirstSelection
selectable2 = theSecondSelection
selectable3 = theThirdSelection
In this case (assuming I understand your request) you want to be able to change the value of the selector and force tasks to access the appropriate selectable variable.
So...
Define a new variable in your pipeline.
selector = selectable1 //this is the value that can change
selected = "" //this is the variable you use in your tasks
selectable1 = theFirstSelection
selectable2 = theSecondSelection
selectable3 = theThirdSelection
Write a VariableSelection.ps1 script. This powershell script will be what you need to run to assign the value of $(selected) before it gets used.
# VariableSelection.ps1
Write-Host "select variable: $env:selector"
$selectedValue = (gci env:"$env:selector").value
Write-Host "##vso[task.setvariable variable=selected]$selectedValue"
Note: it is my observation that if you write this script inline, it will not work b/c the environment variable functionality is different for scripts run from a file.
Given the value of $(selector) is selectable2, when the script is run, then the value of the $(selected) will be theSecondSelection.
Example in a Pipeline
Powershell
YAML
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
pool:
name: Hosted VS2017
variables:
- name: "selector"
value: "var1"
- name: "selected"
value: ""
- name: "var1"
value: "var1_value"
- name: "var2"
value: "var2_value"
steps:
- task: PowerShell#2
inputs:
filePath: '$(build.sourcesdirectory)/varSelector.ps1'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "env:selected: $env:selected"
Write-Host "selected: $(selected)"
Results

Setting a Property Value in 1 task and use the updated value in another

In my psake build script, I have a property called $build_mode that I set to "Release".
I have 2 tasks; "Compile_Debug", "Compile_Release". In Compile_Debug, I change $build_mode to "Debug" and it works fine while that task executes; however, if I have a have another task execute that uses $build_mode afterwards, $build_mode is back to "Release".
Is there a way to globally change or set a variable in a Psake build script so that an updated value can be used between tasks?
(I'm trying to have 1 "test" or 1 "package" task instead of a "Test_Debug", etc.)
Code:
properties {
$build_mode = "Release"
}
task default -depends Compile_Debug, Test
task Compile_Debug {
$build_mode = "Debug"
# Compilation tasks here that use the Debug value
}
task Test {
# Test related tasks that depend on $build_mode being updated.
}
I usually set the build mode as #manojlds suggested, passing in as a paramenter in the Invoke-Psake call. But, if you find yourself again in a situation that you want to modify the value of a object in Task A and have access to the modified value in Task B, here is how to do it:
The fact that the modified value of $build_mode is not accessible in Task B is due to powershell scoping. When you set a value for the $buildMode variable in a Task A, that change is made within Task A's scope, therefore outside it the variable value keeps unchanged.
One way to achieve what you want is to use a hashtable scoped to the whole script to store your objects:
Code:
properties {
#initializes your global hash
$script:hash = #{}
$script:hash.build_mode = "Release"
}
task default -depends Compile_Debug, Test
task Compile_Debug {
$script:hash.build_mode = "Debug"
# Compilation tasks here that use the Debug value
}
task Test {
# Test related tasks that depend on $script:hash.build_mode being updated.
}
The only caveat is that every time you want to refer to your build mode you have to use the long $script:hash.build_mode name instead of simply $build_mode
Why don't you pass the build mode as parameter to the tasks from the Invoke-Psake?
Invoke-Psake "$PSScriptRoot\Deploy\Deploy.Tasks.ps1" `
-framework $conventions.framework `
-taskList $tasks `
-parameters #{
"build_mode" = "Debug"
}
And in the tasks you can now use $build_mode