I have an Nx monorepo project set up. I want to set some environment variables to use throughout my app. I'm looking for a solution where I can define a value for these variables during the release process to each environment (QA, Test, Dev).
Example:
NX_API_URL is my environment variable in my .env file.
If I release to QA env the variable should be NX_API_URL=api-url-qa.com.
If I release to Test env the variable should be NX_API_URL=api-url-test.com
I have found solutions during the build process but that's not going to work, it needs to be at the release phase. How can I accomplish this?
If the file that you'd like to modify during the deployment process is either XML or JSON you can define the URLs as tokens and use standard 'FileTransform' task to replace them: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/file-transform-v2?view=azure-pipelines
For more details regarding file transformation please refer to documentation: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/transforms-variable-substitution?view=azure-devops&tabs=Classic
If none of the links above will help you with this, then I'd suggest to create PS script to do whatever modifications are required, like adding a line after specific line definition: https://social.technet.microsoft.com/Forums/office/en-US/fabc9790-0ffe-43b2-9d6b-bc483cd28bb6/add-line-to-a-text-file-just-after-a-specific-line-with-powershell?forum=winserverpowershell
Related
is it possible to set sort of a project-variable in rundeck?
I am organizing jobs in projects, but the jobs are mostly the same and use the same values over and over again. If I could set variables per project, I could just copy jobs from project to project without having to adapt the same parameters over and over again...
like... path=/path/to/project
and use the variable path in jobs
Thanks, regards
Jochen
You can define a global variable for that. You can define it on framework.properties file like framework.globals.myvar=myvalue or at project level (project.properties config) like project.globals.myvar=myvalue, to access it just use ${globals.myvar} for steps, #globals.myvar# for inline-scripts or $RD_GLOBALS_MYVAR for scripts.
UPDATE:
In Rundeck 3.1/3.2/3.3/4.X the "project.properties" isn't a "file", is a config reachable/editable following this.
I'd like to update a variable in a linked variable group with $(Build.BuildId) from a Linux pipeline. I see a few examples using the #echo ##vso[task.setvariable command, but can't get it to work because I don't think I'm referencing the source or destination right.
The linked variable group is NightlyBuildID and the variable is LinuxBuildID.
Here's one of my many attempts:
#echo ##vso[task.setvariable variable=LinuxBuildID]$(Build.BuildId)
The $(variable) syntax is only valid within the build editor interface. Within a script, you have to reference it as an environment variable. Periods are replaced with underscores.
Thus, in Linux, $(Build.BuildId) would be accessed as $BUILD_BUILDID.
Apparently, this can't be done, according to the link below. I'll have to come up with a work around, bummer.
https://visualstudio.uservoice.com/forums/330519-visual-studio-team-services/suggestions/32083207-allow-variables-in-variable-groups-to-be-settable
I have a build step in Azure Pipelines that takes the variables from Azure Pipelines and uploads them somewhere equally secret. Currently I have about 50 builds, and each build has anywhere between 5-20 variables.
Some are secret and some are not. So for non secret ones I enumerate all the set ones and off i go; but for secret ones I need to add them to the build step manually; further, because I am writing them with the same keys i need to:
Declare variable in the group e.g. MyPrefix.MyVar
Edit the build step to say /specialtool --vars=MyPrefix.MyVar=$(MyPrefix.MyVar) which is rather mundane.
I found that I can get a list of variables using the Azure DevOps api, so i thought i could just modify the next build step as the build is running.
However, if I update the same build definition that is currently running (to dynamically write the command), it is not sent to the agent (rather, it feels like all arguments for tasks are captured when the whole build is triggered). Any thoughts on how i can dynamically enumerate secret vars to feed to my tool?
You can use VSTS Logging Commands to update variable value during the build. This will make the updated variable to be available in next build task.
Write-Host "##vso[task.setvariable variable=testvar;]testvalue"
When you create a Typescript custom task (NodeJS based), you can access all the build variables that are available to the build at that point in time through the getVariable api.
This function returns an array of VariableInfo:
/** Snapshot of a variable at the time when getVariables was called. */
export interface VariableInfo {
name: string;
value: string;
secret: boolean;
}
When you create a PowerShell3 custom task, you can access all the build variables that are available to the build at that point in time through the Get-VstsTaskVariable function.
Which returns a similar object structure as the Node version:
New-Object -TypeName psobject -Property #{
Name = $info.Name
Value = Get-TaskVariable -Name $info.Name
Secret = $info.Secret
}
If you need to support TFS 2015 and the 1.x build agents as well, you can use (now deprecated) PowerShell handler and enumerate the secrets using a custom powershell function I describe here.
Each task SDK (Typescript and Powershell), supports a function to set variables as well. Here is an example of setting the variable value in Typescript:
tl.setVariable(variable, value, isSecret);
And on PowerShell3:
Set-VstsTaskVariable -name $VariableName -value $Value -Secret $IsSecret
And on PowerShell (deprecated):
Write-Host "##vso[task.setvariable variable=$($VariableName);issecret=$($IsSecret)]$Value"
My suspicion is that you'd want to create a single task that reads the variables and invokes the command you mentioned in your original post to then post these variables to the other secret store. It's not recommended to read all the secrets and either store them in non-secret variables or to somehow pass them along to the next task.
So I have been looking at a solution for this too. It appears the only way to do this at the moment is to write a custom task. Within a custom task you can get hold of secret values dynamically.
An example is the 'vsts-replacetokens-task' (https://github.com/qetza/vsts-replacetokens-task/blob/master/task/index.ts)
Internally it uses the vsts task library (vsts-task-lib/task)
(https://github.com/Microsoft/azure-pipelines-task-lib/blob/master/node/task.ts)
This vsts task library exposes methods like GetVariables() and GetVariable() etc. which can provide what you need. Unfortunately bit long winded, but the only way that I can see.
How can I get a current date-time and pass it as a variable to some Deployment task?
You can define a variable with any value, and then modify the variable as current date. Detail steps as below:
Define a variable in release
Assume the variable name is time, and we set the value as none. If you need to use the variable for a environment, you can define it in environment variables. Else you should define it in variables Tab.
Add a power shell task at the begin of deploy tasks:
Type: Inline Script.
Inline script:
$date=$(Get-Date -Format g);
Write-Host "##vso[task.setvariable variable=time]$date"
Note:
I use the date format as MM/DD/YYYY HH:MM AM/PM here. You can use other date formats.
For the subsequent deploy task, if you want to use current date time, you can direct use $(time).
Update
Documentation for Defining Variables: Set Variables Using Expressions has a nugget of gold for the answer to this question in the example for creating a counter value that is reset daily.
a: $[counter(format('{0:yyyyMMdd}', pipeline.startTime), 100)]
The pipeline.startTime variable used here is subtle and not mentioned anywhere in the Pipeline Predefined Variables documentation, even when being careful to land on the correct documentation based on the pipeline method being used. As is suggested HERE and in some of the answers on this thread, certain variables may have different values or not exist at all depending on where you are while trying to access them.
There is now a variable specific to a release stage named "Release.Deployment.StartTime" or if you use it in powershell the environment variable is "Release_Deployment_StartTime".
It's in UTC and the format is "yyyy-MM-dd HH:mm:ssZ"
ex: "2018-11-09 21:23:27Z"
NOTE: This variable is set at the time the deployment stage is started, so if you have pre-deployment approvals the time will be set before any approvals are completed. From my testing if you have multiple stages that execute at the same time it will be the same between them, even if one stage waits for the other due to limited agent availability.
I'm using Azure DevOps online, unsure if local TFS installations will have this.
For those who use Linux on tfs:
Define variable
Make sure it has "Settable at queue time set"
Create a script in root of your repository
set-build.date.sh:
#!/usr/bin/env bash
DATE=$(date '+%d/%m/%Y %H:%M:%S')
echo "##vso[task.setvariable variable=BUILD_DATE;]$DATE"
Other options are listed here.
Add shell script right after get sources
Type bash to find this task.
Done, you can use BUILD_DATE variable in later tasks :)
An easier way is
$(Date:MMddyy)
Some options are only available in the Build Definition options section. The date formatting is one of them. However, if you were to go into the options section, set the build number format as $(Date:yyyyMMdd-HHmmss), you could then use the $(Build.BuildNumber) variable in your tasks.
More info here - https://learn.microsoft.com/en-us/vsts/build-release/concepts/definitions/build/variables?tabs=batch
Based on answer of Marina Liu (here) for quick copy and paste.
Define variable on top:
variables:
buildTimeStamp: # will be set by script
# ...
Add this as first task (and change format as needed):
- task: PowerShell#2
displayName: set variable buildTimeStamp
inputs:
targetType: 'inline'
script: |
$date=$(Get-Date -Format yyyy-MM-dd_HH.mm);
Write-Host "##vso[task.setvariable variable=buildTimeStamp]$date"
Now you can use variable $(buildTimeStamp) in your code below.
I'm new to Chef and seeking help here. I'm looking into using Chef to deploy our builds to Chef node servers (Windows Server 2012 machines). I have a cookbook called copy_builds that goes out to a central repository and selects the build we want to deploy and copies it out to the node server. The recipe I have contains basic steps that perform the copy steps, and this recipe could be used for all builds we want to deploy except for one thing: the build name.
Here is an example of the recipe:
powershell_script 'Copy build files' do
code '
$Project = "Dev3_SomeCoolBuild"
net use "\\\\server\\build_share\\drop\\$Project"
$BuildNum = GC "\\\\server\\build_share\\drop\\$Project\\buildlabel.txt"
robocopy \\\\server\\build_share\\drop\\$Project\\bin W:\\binroot\\$BuildNum'
end
As you can see, the variable $Project contains the name of the build in this recipe. If we have 100 different builds, all with different names, then what is the best way to handle this without creating 100 different recipes for my copy_builds cookbook?
BTW: this is how I'm currently calling Chef to deploy, which is in a PowerShell script that's external to Chef:
knife node run_list set $Node "recipe[copy_builds::$ProjectName],recipe[install_build]"
This command (from the external PowerShell script) contains the project/build name info within it's own $ProjectName variable. In this case $ProjectName contains the value of 'Dev3_SomeCoolBuild', to reference the recipe Dev3_SomeCoolBuild.rb.
What I'd like is have just one default recipe under copy_builds cookbook, and pass in the build/project name. Is this possible? And what is the best way to do it? I've read about data bags, attributes, and providers, but not sure if they would work for what I want.
Please advise.
Thanks,
Keith
The best approach for you is likely to use a single recipe that gets a list of projects to deploy from a databag or node attributes (or both). So basically take what you have now and put it in a loop, and then use either roles to set node attributes or put the project mapping into a databag item.
I ended up using attributes here to solve my problem. I updated my script to write the build name to the attributes/default.rb file for the copy_builds recipe and upload the cookbook to Chef each time a deployment is run.
My recipe now includes a call to the attributes file to get the build name, like so:
powershell_script 'Copy build files' do
code <<-EOH
$BuildNum = GC \\\\hqfas302002c\\build_share\\drop\\"#{node['copy_builds']['build']}"\\buildlabel.txt
robocopy \\\\hqfas302002c\\build_share\\drop\\"#{node['copy_builds']['build']}"\\webbin W:\\binroot\\$BuildNum /E
EOH
end
And now my call to Chef looks like this:
knife node run_list set $Node "recipe[copy_builds],recipe[install_build]"