How to capture who ran the build pipeline in Azure DevOps as a variable?
Is there any predefined variable to capture that?
From the docs:
Build.QueuedBy - See "How are the identity variables set?".
Note: This value can contain whitespace or other invalid label characters. In these cases, the label format will fail.
Build.QueuedById - See "How are the identity variables set?".
Yes, its called: "Build.QueuedBy"
https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#build-variables
Related
I have a mutli-step azure pipeline used to trigger the execution of a certain job based on keywords I have in azure devops work items.
First step executed is a powershell script that stores into a 'validTags' variable a comma-separated list of strings:
Write-Host "##vso[task.setvariable variable=validTags]$csTags"
After this step, I correctly see the list formatted as I expect:
string1,string2,string3
The 'validTags' variable is then passed as a parameter to another pipeline in which I should split this list and trigger separate jobs:
- template: run.yml
parameters:
tags: $(validTags)
directory: 'path\to\tests'
platforms: 'platform1,platform2'
In the 'run' pipeline I defined this 'tags' parameter:
parameters:
- name: tags
type: string
default: 'someDefaultValue'
and I try to split the parameter:
- ${{each t in split(parameters.tags, ',')}}:
- script: |
echo 'starting job for ${{t}}'
but when I execute the pipeline, I have in 't' still the full string (string1,string2,string3) not splitted.
I have noticed that if I try to perform the split on the "platforms" parameter which is passed along with "tags" to the run.yml pipeline, it works, so it seems that the problem is related to the fact that I am trying to split a string stored in an external variable?
Anyone with a similar issue? Any help on this is much appreciated.
Thanks
For those interested in the outcome of this issue:
I tested several possible alternate solutions, including the use of global variables and group variables, but without success.
I submitted a request to MSFT engineering support to get some insight on this and their response is:
The pipeline does not support splitting the runtime variable with
template syntax ${{ }} currently, and we are not able to find other
workarounds to meet your request. Sorry for the inconvenience. Hope
you can understand.
So, to overcome the issue I removed the split at the pipeline level, as initially planned, but rather passed the comma-separated value's string to the template and added there the necessary processing in Powershell.
Another option would have been to perform all the operations from within the first PowerShell script step:
transform the 'run.yml' template in a separate pipeline
in the script, after getting the tags, loop over their values and trigger the 'run.yml' pipeline passing the single tag as a parameter.
I avoided this solution to keep the operations separate and have more control over the execution flow.
I have n number of variables that I need to assign as Azure DevOps variables in a Release pipeline, and it doesn't seem like I'm getting the syntax right.
The variables may have different values (variable names) such that they could be:
- {guid 1}
- {guid 2}
...
So I won't know them prior to runtime. The problem is that it seems all of the examples of vso[task.setvariable] use static variable names, but I need to set it dynamically.
Here's what should work but doesn't:
Write-Host "##vso[task.setvariable variable=$($myVariable)]$($myValue)"
I've also tried just using [Environment]::SetEnvironmentVariable (with user) and it doesn't seem to persist across two different tasks in the same job.
[Environment]::SetEnvironmentVariable($myVariable, $myValue, 'User')
(Is null in subsequent task)
Is there some way that I can dynamically create release variables that persist between tasks? I've tried to search and found one question on the developer community but no answer to it.
It actually looks like the issue isn't that the variable isn't set, but that after using task.setvariable, the variable will only be available in subsequent tasks (and not the current one).
So I would say this is the best way to set variables in Azure DevOps:
When needing to use variables in the same task/script step, use:
[Environment]::SetEnvironmentVariable(...)
Or just use a variable in PowerShell.
When needing to use variables with multiple steps, use:
$myVariable = "some name"
$myValue = "some value"
# Note that passing in $($variableName) should work with this call
Write-Host "##vso[task.setvariable variable=$($myVariable)]$($myValue)"
# Note that trying to use a Write-Host for $env:myVariable will return empty except in tasks after this one
Write-Host "Setting $($myVariable) to $($myValue)
It works. This is example from my build task:
$myVariableNewValue = '##vso[task.setvariable variable=myVariable]' + $newValue
Write-Host $myVariableNewValue
I have a concat expression defined in the Function Name setting of an Azure Function in my pipeline, where it concatenates the API Query with the current filename that I want to run on this function. When I debug the pipeline, it fails without giving me any feedback. It just says "AzureFunction failed:"
If I manually insert the string, it works fine.
the concat expression is:
#concat('HttpTrigger?filename=', variables('filename'))
I'm new to Azure, any way I can debug this?
try this way:
#concat(variables('FirstName') ,variables('LastName'))
You could use Set Variable Activity with your Azure Function Activity.
In the Variable Activity, set the value of the variable.
Then refer the variable in the Azure Function Activity:
#concat('HttpTriggerJS1?name=',variables('name'))
I'm doing some experimentation with Kubeflow Pipelines and I'm interested in retrieving the run id to save along with some metadata about the pipeline execution. Is there any way I can do so from a component like a ContainerOp?
You can use kfp.dsl.EXECUTION_ID_PLACEHOLDER and kfp.dsl.RUN_ID_PLACEHOLDER as arguments for your component. At runtime they will be replaced with the actual values.
I tried to do this using the Python's DSL but seems that isn't possible right now.
The only option that I found is to use the method that they used in this sample code. You basically declare a string containing {{workflow.uid}}. It will be replaced with the actual value during execution time.
You can also do this in order to get the pod name, it would be {{pod.name}}.
Since kubeflow pipeline relies on argo, you can use argo variable to get what you want.
For example,
#func_to_container_op
def dummy(run_id, run_name) -> str:
return run_id, run_name
#dsl.pipeline(
name='test_pipeline',
)
def test_pipeline():
dummy('{{workflow.labels.pipeline/runid}}', '{{workflow.annotations.pipelines.kubeflow.org/run_name}}')
You will find that the placeholders will be replaced with the correct run_id and run_name.
For more argo variables: https://github.com/argoproj/argo-workflows/blob/master/docs/variables.md
To Know what are recorded in the labels and annotation in the kubeflow pipeline run, just get the corresponding workflow from k8s.
kubectl get workflow/XXX -oyaml
create_run_from_pipeline_func which returns RunPipelineResult, and has run_id attribute
client = kfp.Client(host)
result = client.create_run_from_pipeline_func(…)
result.run_id
Your component's container should have an environment variable called HOSTNAME that is set to its unique pod name, from which you derive all necessary metadata.
I have initialized a pipeline parameter, lets say a flag, with a value ('true'). Now inside the pipeline I want to update the flag value based on some condition.
So lets say I have a web activity inside the pipeline, I want to update the value of that flag to 'false' if i dont get the response from the web activity.
Any idea how to do that?
You could try #empty(body('web1'))
So if I understand correctly you want to call a Web activity from your pipeline and if there is no response then set a variable?
In this case, could you not just set a low timeout and use a "Set Variable" activity on the Fail condition? So if there was no response it would fail and set the variable