I am having a runtime argument set at namespace which is business_date: ${logicalStartTime(yyyy-MM-dd)} . I am using this argument in my pipeline and want to use the same in other pipeline.
There are many pipelines back to back and I want to the value to be same throughout the pipelines once calculated in the first pipeline.
suppose the value is calculates as '2020-08-20 20:14:11' and once the pipeline one succeeded i am passing this argument to pipeline 2, but as this arguments are defined at namespace level it is getting overrided when pipeline2 starts.
How can I have prevent this value to be calculated again ?
As it was commented before, you could set up one pipeline to trigger another pipeline; you can set a runtime variable in the first pipeline and this variable will be set in the triggered pipelines. You can create inbound trigger by following the next steps:
Once you have created your pipelines, select the last pipeline you want to be run. In my case I have DataFusionQuickstart2 pipeline.
Into the pipeline application, on the left side, click on "Inbound triggers" -> "Set pipeline triggers" and you will see the pipes which you can be triggered. Check the event that will trigger the DataFusionQuickstart2 pipeline from DataFusionQuickstart and enable it.
If you take a look to the previous pipeline DataFusionQuickstar you will see, into the outbound trigger option (right side), the pipelines that will be triggered by DataFusionQuickstar.
Finally set your runtime argument.
Additional information
In this post, it was mentioned that there are three ways you can set the runtime argument of a pipeline:
Argument Setter plugin (You can write down that value in a file into the first pipeline. In all subsequent pipelines, create a parameter to read that file.)
Passing runtime argument when starting a pipeline (The one described above)
Setting Preferences (It provides the ability to save configuration information at various levels of the system, including the CDAP instance, namespace, application, and program levels.)
You can write down that value in a file in first pipeline. In all subsequent pipelines, create a parameter to read that file. That way, objective should be achieved.
#Sudhir, you may explore PREFERENCES. https://cdap.atlassian.net/wiki/spaces/DOCS/pages/477561058/Preferences+HTTP+RESTful+API
You have set the variable at namespace level and as per your finding it is getting evaluated each time its being used.
Can you try setting it at Application level?
And pass it to next pipeline. I believe in that case, it should be evaluated only once in that specific application (pipeline) and thereafter value would be passed.
Preference is also available at program level.
Related
I am trying to look for a solution where our developers want to deploy a different version of their build. My train of thought was to add an env variable to our cicd configs, and then in our cd playbook evaluate if that var is None, and if so override the version we normally get from the package or pom file.
This is turn will grab that XXX version from our helm and docker container registry allowing them to roll back without or redeploy an older version quickly and efficiently.
I see that Azdo provides environment variables, but I wanted something that would only set or store the env variable for the run from the pipelines and would not be persistant.
You can configure so-called queue time variables. The idea is that you set a default value, like latest, and check the for to change the value at queue time:
If a variable appears in the variables block of a YAML file, its value is fixed and can't be overridden at queue time. Best practice is to define your variables in a YAML file but there are times when this doesn't make sense. For example, you may want to define a secret variable and not have the variable exposed in your YAML. Or, you may need to manually set a variable value during the pipeline run.
You have two options for defining queue-time values. You can define a variable in the UI and select the option to Let users override this value when running this pipeline or you can use runtime parameters instead. If your variable is not a secret, the best practice is to use runtime parameters.
To set a variable at queue time, add a new variable within your pipeline and select the override option.
Set a variable at queue time.
To allow a variable to be set at queue time, make sure the variable doesn't also appear in the variables block of a pipeline or job. If you define a variable in both the variables block of a YAML and in the UI, the value in the YAML will have priority.
See:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/variables?WT.mc_id=DOP-MVP-5001511&view=azure-devops&tabs=yaml%2cbatch#allow-at-queue-time
I am trying to create a simple Runtime Variable in my Release Pipeline that carries the BuildNumber of the Triggering Artifact.
I tried adding a variable called alias and in the value column I wrote $(Release.TriggeringArtifact.Alias) And I set the Settable at Release Time to true.
Then I added another variable called BuildNumber and I wrote this in the value column: $(Release.Artifacts{alias}.BuildNumber} and set the Settable at Release Time to true.
I have a powershell task that Echoes the value of my variable BuildNumber. I was expecting the build number that was assigned to my artifact, but what I see is the git repository's name instead.
How can I have release time variables with pre-defined variables substituted in them?
What you are trying to do is to use nested variable which is not supported:
Thanks for your patience on this. I checked with the product team and there are places where variables will expand when nested, and the behavior will be slightly different if you specify variables with macro syntax (example: $(var)). However, this is not an officially supported feature and their recommendation is to not document macro variables as expanding recursively.
If you would like to reach out the product team directly, please post in the Azure DevOps Developer Community. The product team actively monitors that space.
You can also vote on this on developer community
I have a release pipeline with a variable, but there doesn't seem to be any way to set the value of that variable to something that's evaluated at release time. For example, another variable.
Here's a real example:
All I want to do is set the value of MyExpressionBasedVariable to the value of MyOtherVariable.
All the docs and examples online seem to suggest it's possible, but I can't get it to work. I always end up with the literal string rather than the evaluated value.
I've tried using these different syntaxes:
$(MyOtherVariable)
$[variables['MyOtherVariable']]
${{variables['MyOtherVariable']}}
I've seen that you can define custom tasks to set variable names as part of the pipeline but this seems massively overkill.
Essentially all I want to do is rename a key vault secret to a different variable name for convention-based XML variable replacement in config files.
E.g. I have a secret called this-is-a-secret-name-which-is-a-different-naming-convention-to-my-connectionstrings but I need it in a variable called MySecret-ConnectionString.
How do I use the value of another variable in a release pipeline variable?
How do I use the value of another variable in a release pipeline variable?
As I test, what you set should be work. You can try to follow below steps to check if you still have this issue:
Create a new release pipeline without link any variable group.
Set the Variable like following:
Add a Run Inline Powershell task to output the value of the Variable:
Write-Output 'The value of MyExpressionBasedVariable is $(MyExpressionBasedVariable)'
Write-Output 'The value of $(MyOtherVariable) is $(MyOtherVariable)'
Then we could get the log:
So, what you set should be work, if this still does not work for you, then you need to make sure that the variable you describe in the question is the variable your actual test.
Besides, at this moment, the value of nested variables (like $(TestVar_$(Release.Reason))) are not yet supported in the build/release pipelines, check this thread for some details, so make sure there are no such nested variables in your project.
Hope this helps.
With designer/class build pipeline, you can define pipeline variables with default values to be passed into the tasks. How do I do the same for a YAML-based pipeline?
I want to create three build pipelines, each would have a single variable set to a different value. All three point to a single YAML file. The documentation states:
You can choose which variables are allowed to be set at queue time and which are fixed by the pipeline author. If a variable appears in the variables block of a YAML file, it is fixed and cannot be overridden at queue time. To allow a variable to be set at queue time, make sure it doesn't appear in the variables block of a pipeline or job. You can set a default value in the editor, and that value can be overridden by the person queuing the pipeline.
It's not clear how to do this for YAML file.
I can create a template YAML file, and an individual YAML file for each config value that calls the template file, but then I can't set configuration value at run-time.
when you edit the build definition (not when you create it, at least with default experience). you'd need to click on 3 dots and pick variables from the list:
and there you would be able to define variables, and they would have a checkbox - Settable at queue time.
I have a web application that I build and release using TFS. The release definition contains a task group with all the necesarry steps for the deployment of the web application.
What I want to do is to determine a certain value and store it in a variable to use later on in the release proccess in the other release tasks.
At this moment, the first step in de task group is a Powershell task that determines the necessary value and stores it in an environment variable using:
Write-Output ("##vso[task.setvariable variable=MyVar;]$var")
When I use this environment variable in the next Task (Again, a PowerShell task), it runs perfectly as expected using:
Write-Host "Doing stuf for: $env:MyVar"
It's when I want to use the variable as a parameter for (multiple) different task(s) when it gets weird. When the environment variable has no default value, the calculated value from the first PowerShell task is used and all is well. But TFS doesn't like it when environment variables have no default value and forces to provide one before being able to save the release definition again. When I provide a default value, the task that uses the variable as Task parameter uses the default value, instead of the calculated value. I would expect that the calculated value should be used, as the second PowerShell task ensures that the calculated value is properly stored.
So, the symptoms I see:
When not providing a default value, the code in scripts as wel as the task parameters use the calculated value
When providing a default value, the code in scripts use the calculated value and the task parameters use the default value
Is there something I do wrong, or am I using environment variables wrong and should I use a different method?
As I understand it, you want to define your variable in a powershell-task within the task group, but you are also being forced to provide a value in the environment variable, causing the variable sometimes to have the value set in the powershell, sometimes to have the value defined in the environment variable.
The solution is to just provide the name of the environment variable itself, so it always gets overwritten by the value you set in the powershell task.
Source: https://blogs.msdn.microsoft.com/harshillodhi/2016/11/29/vststfs-understanding-task-groups-and-its-various-use-cases-with-setvariable-logging-command/
Under the header "What is a setvariable logging command and how to use it with Task Groups?", three scenario's are referenced. It sounds like the second scenario may fit your needs.