I am creating Azure devops build pipeline through terraform and want my build variables to get passed through an external file while the build gets created. The same terraform code will be used to create pipelines of different templates, so accordingly the variables will also change, so inorder to make it reusable for all pipelines i want my terraform codes to take the build pipelines variables from external file and create it. Can't go with variable groups as well. So if any other process please advice.
Any help or advice would be really appreciated.
resource "azuredevops_build_definition" "build" {
project_id = azuredevops_project.project.id
name = "Sample Build Definition"
path = "\\ExampleFolder"
ci_trigger {
use_yaml = true
}
variable {
name = "PipelineVariable"
value = "Go Microsoft!"
}
variable {
name = "PipelineSecret"
secret_value = "ZGV2cw"
is_secret = true
}
the variables that above passed should instead get passed from external file.
You just need to define terraform variables for the values you require and use them in the resource:
variable "buildinfo"{ type = string default = "my value" }
and in the resource you just use:
.... value = var.buildinfo ....
you can pass the variable value for each run with a tfvars file or in your pipeline variables if you use a CI
you could do it like this:
Terraform:
variable "PipelineVariable"" {
}
Then define variables in the Pipeline like TF_VAR_PipelineVariable or in any stage you call a file (example file: .env) with source .env:
.env:
export TF_VAR_PipelineVariable=CONTENT
Related
I have terraform Module for example to create a VM on Azure and it works when I trigger the Pipeline.
But When I trigger the Pipeline twice it fails to create two VM's. How do I manipulate terraform State file ? Only way I can think of is two run multiple pipeline in different agents, does that work ?
What we have done is create terraform "common" modules (basically a subdirectory with tf files), which we source into a terraform environment multiple times with different parameters.
These we usually put into a list with a loop.
In your environments terraform:
locals {
azure_vms = [
{ name = "vm1", size = "Standard_B2s" },
{ name = "vm2", size = "Standard_B4s" }
]
}
module "my_azure_vm" {
source = "./common/my_azure_vm"
for_each = { for vm in local.azure_vms : vm.name => vm }
size = each.value.size
name = each.value.name
}
In common my_azure_vm, you can define inputs for size and name, then use those to create the VM's with your standard parameters.
I am new to terraform and devops in general. First I need to get ssh key from url to known host to later use for Flux.
data "helm_repository" "fluxcd" {
name = "fluxcd"
url = "https://charts.fluxcd.io"
}
resource "helm_release" "flux" {
name = "flux"
namespace = "flux"
repository = data.helm_repository.fluxcd.metadata[0].name
chart = "flux"
set {
name = "git.url"
value = "git.project"
}
set {
name = "git.secretName"
value = "flux-git-deploy"
}
set {
name = "syncGarbageCollection.enabled"
value = true
}
set_string {
name = "ssh.known_hosts"
value = Need this value from url
}
}
Then I need to generate key and use it to create kubernetes secret to communicate with gitlab repository.
resource "kubernetes_secret" "flux-git-deploy" {
metadata {
name = "flux-git-deploy"
namespace = "flux"
}
type = "Opaque"
data = {
identity = tls_private_key.flux.private_key_pem
}
}
resource "gitlab_deploy_key" "flux_deploy_key" {
title = "Title"
project = "ProjectID"
key = tls_private_key.flux.public_key_openssh
can_push = true
}
I am not sure if I am on the right track. Any advice will help.
There are few approaches you could use. These can be divided into "two categories":
generate manually the ssh_known_hosts and use the output through variables or files
create the file on the machine where you're running terraform with the command ssh-keyscan <git_domain> and set the path as value for ssh.known_hosts.
You can also use the file function directly in the variable or use the file output directly as env variable. Personally I would not recommend it because the value is saved directly in the terraform state but in this case it is not a critical issue. Critical would be if you're using ssh_keys or credentials.
Another approach would be to use the local-exec provisioner with a null_resource before you create the helm resource for flux and create the file directly in terraform. But additional to that you have to take care of accessing the file you created and also managing the triggers to run the command if a setting is changed.
In general, I would not use terraform for such things. It is fine to provide infrastructure like aws resources or services which are directly bound to the infrastructure but in order to create and run services you need a provisioning tool like ansible where you can run commands like "ssh-keyscan" directly as module. At the end you need a stable pipeline where you run ansible (or your favorite provisioning tool) after a terraform change.
But if you want to use only terraform you're going to right way.
I have my release pipeline Variables tab set like:
I would like to access my123 variable in task's display name by concatenating initialVariable's result.
Outputs
I have tried so far referencing only initialVariable and it returned proper value in Job's display name.
But when I try to create my123 value by using initialVariable(=123), I am not getting proper value (was hoping that $(initialVariable) would convert to 123 and $(my123) would get proper "finalValue").
Azure DevOps: Getting variable value by concatenating other variables'value as task input
This is a known issue. Because the value of nested variables (like $(my$(initialVariable)) are not yet supported in the build/release pipelines.
Check my other thread for some details.
The workaround is add a Run Inline Powershell task to set the variable based on the input pipeline variables, just like Josh answered.
For you case, I test it by following Powershell scripts:
if ($(initialVariable)-eq "123")
{
Write-Host "##vso[task.setvariable variable=my123]finalvalue"
}
else
{
Write-Host "##vso[task.setvariable variable=my123]otherValue"
}
Then we could get the variable my123 based on the value of variable initialVariable in following task, I add command line task to display the value:
In the result, the value in the command line task is correct finalvalue. But the display name is still $(my123):
Important:
That is also the question in your comment. This behavior is expected. That because the variable in the display name is just to get the predefined value. It's static acquisition, not dynamic. The variable my123 is assigned when running powershell. The static variable my123 in the display name does not go in to the environment where the powershell code is running.
So, the variable my123 in the title could not get the value in the task powershell. But other task could use it very well.
Hope this answer clean your puzzle.
It's ugly, but...
Like I mentioned in my comment, I don't think you're going to get this to work in the UI by default.
Luckily you can use PowerShell to hack this together if your REALLY need the ability to address a variable name based on the value of another variable.
All the variables (secrets are handled a little differently) in your build or release pipeline definition are made available to your powershell script FILE (not inline) via environment variables (ie. $env:initialVariable).
Suppose your situation is thus:
selector = selectable1 //this is the value that can change
selectable1 = theFirstSelection
selectable2 = theSecondSelection
selectable3 = theThirdSelection
In this case (assuming I understand your request) you want to be able to change the value of the selector and force tasks to access the appropriate selectable variable.
So...
Define a new variable in your pipeline.
selector = selectable1 //this is the value that can change
selected = "" //this is the variable you use in your tasks
selectable1 = theFirstSelection
selectable2 = theSecondSelection
selectable3 = theThirdSelection
Write a VariableSelection.ps1 script. This powershell script will be what you need to run to assign the value of $(selected) before it gets used.
# VariableSelection.ps1
Write-Host "select variable: $env:selector"
$selectedValue = (gci env:"$env:selector").value
Write-Host "##vso[task.setvariable variable=selected]$selectedValue"
Note: it is my observation that if you write this script inline, it will not work b/c the environment variable functionality is different for scripts run from a file.
Given the value of $(selector) is selectable2, when the script is run, then the value of the $(selected) will be theSecondSelection.
Example in a Pipeline
Powershell
YAML
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
trigger:
- master
pool:
name: Hosted VS2017
variables:
- name: "selector"
value: "var1"
- name: "selected"
value: ""
- name: "var1"
value: "var1_value"
- name: "var2"
value: "var2_value"
steps:
- task: PowerShell#2
inputs:
filePath: '$(build.sourcesdirectory)/varSelector.ps1'
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
Write-Host "env:selected: $env:selected"
Write-Host "selected: $(selected)"
Results
I'm trying to release my buildartifacts to a specific folder based on the name of the sourcebranch which upon creating a pull request triggered the build and therefor the release.
I've managed to so far to get:
write-host $env:RELEASE_TRIGGERINGARTIFACT_ALIAS
$triggerAlias = $env:RELEASE_TRIGGERINGARTIFACT_ALIAS
This alias (from my point of view) is the primary artifcat alias which I need to access
Release.Artifacts.{Primary artifact alias}.SourceBranchName
based on this documentation. So how do I combine the alias to get the sourcebranchname
$env:RELEASE_ARTIFACTS_{$triggerAlias}_SOURCEBRANCHNAME
This doesn't seem to be working and neither does
$env:RELEASE_ARTIFACTS_$($triggerAlias)_SOURCEBRANCHNAME
Any advice is much appreciated.
You can read the variable in this way:
$triggerAlias = $env:RELEASE_TRIGGERINGARTIFACT_ALIAS
$branchNameVariable = "RELEASE_ARTIFACTS_$($triggerAlias)_SOURCEBRANCHNAME"
#Get the value of the environment variable Release.Artifacts.{alias}.SourceBranchName
$branchName = (Get-item env:$branchNameVariable).Value
I'm stuck with a release variable substitution of an angular project. I have a settings.json file which I would like to replace some variables:
{
test : "variable to replace"
}
I tried to find some custom task on the marketplace but all of the tasks seems to work only with xml files for the web.config.
I use the "Replace tokens" from the Marketplace https://marketplace.visualstudio.com/items?itemName=qetza.replacetokens
You define the desired values as variables in the Release Definition and then you add the Replace Tokens task and configure a wildcard path for all target text files in your repository where you want to replace values (for example: **/*.json). The token that gets replaced has configurable prefix and postfix (default are '#{' and '}#'). So if you have a variable named constr you can put in your config.json
{
"connectionstring": "#{constr}#"
}
and it will deploy the file like
{
"connectionstring": "server=localhost,user id=admin,password=secret"
}
The IIS Web App Deploy Task in VSTS Releases has JSON variable substitution under *File Transforms & Variable Substitution Options.
Provide a list of json files and JSONPath expressions for the variables that need replacing
For example, to replace the value of ‘ConnectionString’ in the sample below, you need to define a variable as ‘Data.DefaultConnection.ConnectionString’ in the build/release definition (or release definition’s environment).
{
"Data": {
"DefaultConnection": {
"ConnectionString": "Server=(localdb)\SQLEXPRESS;Database=MyDB;Trusted_Connection=True"
}
}
}
You can add a variable in release variables Tab, and then use PowerShell task to update the content of your settings.json.
Assume the original content is
{
test : "old
}
And you want to change it to
{
test : "new"
}
So you can replace the variable in json file with below steps:
1. Add variable
Define a variable in release variable tab with the value you want to replace with (variable test with value new):
2. Add PowerShell task
Settings for powershell task:
Type: Inline Script.
Inline Script:
# System.DefaultWorkingDirectory is the path like C:\_work\r1\a, so you need specify where your appsettings.json is.
$path="$(System.DefaultWorkingDirectory)\buildName\drop\WebApplication1\src\WebApplication1\appsettings.json"
(Get-Content $path) -replace "old",$(test) | out-file $path