Azure DevOps file transform with vault secrets - azure-devops

Question

I need to deploy linux azure app service via azure devops. My configuration is stored in appsettings file and I need to substitute configuration values to the values stored in azure vault.
So I created variable group in artifacts, linked it to variables in pipeline and used FileTransform#2 to substitute appsettings values.
But it substitutes to null values. If I explicitly define variable value in pipeline by assigning some string value - it works fine.
Also I cannot use AzureRmWebAppDeployment#4 with JSONFiles, it does not work for linux deployment
What is the way of solving this?
here is pipeline code:
trigger:
branches:
include:
- master
- develop
- release/*
paths:
include:
- src/ConsumerBackEnd/*
- src/ConsumerShared/*
variables:
- name: poolName
value: 'Private-Windows10'
- name: azureRegisteredApp
value: 'portal-devops'
- name: workingDirectory
value: '$(System.DefaultWorkingDirectory)/src/ConsumerBackEnd'
- name: solutionDirectory
value: '$(System.DefaultWorkingDirectory)/src'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
name: $(poolName)
variables:
- group: ConsumerDevVariableGroup
- name: 'Graph.GraphAppTenantId'
value: '**************' #works fine
- name: 'Graph.GraphAppClientId'
value: '$[variables.GraphAppClientId]' #should take value from vault but injects null
- task: FileTransform#2
inputs:
folderPath: '$(workingDirectory)'
xmlTransformationRules:
jsonTargetFiles: '**/appsettings.json'
- task: DotNetCoreCLI#2
displayName: Nuget Restore
inputs:
command: 'restore'
projects: '$(workingDirectory)/*.csproj'
feedsToUse: 'config'
nugetConfigPath: '$(solutionDirectory)/NuGet.config'
- task: DotNetCoreCLI#2
displayName: Build
inputs:
command: 'build'
projects: |
$(workingDirectory)/ConsumerBackEnd.csproj
arguments: --output $(System.DefaultWorkingDirectory)/output
- task: DotNetCoreCLI#2
displayName: Publish
inputs:
command: 'publish'
publishWebProjects: false
projects: '$(workingDirectory)/ConsumerBackEnd.csproj'
arguments: '-c Release -r linux-x64 --self-contained true --output $(System.DefaultWorkingDirectory)/publish_output'
#requires approval on pipeline
- stage: DeployDev
displayName: DeployDev
dependsOn: Build
condition: succeeded()
jobs:
- deployment: DeployConsumerBackendAPIDev
displayName: DeployConsumerBackendAPIDev
environment: ConsumerBackendAPIDev
pool:
name: $(poolName)
strategy:
runOnce:
deploy:
steps:
- task: AzureRmWebAppDeployment#4
inputs:
ConnectionType: 'AzureRM'
azureSubscription: '$(azureRegisteredApp)'
appType: 'webAppLinux'
WebAppName: 'my-backend-dev'
packageForLinux: '$(System.DefaultWorkingDirectory)/publish_output/**/*.zip'
RuntimeStack: 'DOTNETCORE|LTS --configuration Release'

Answer 1

it appears that referencing group variable using runtime expression does not work with file transform task but macro syntax works fine
Microsoft documentation does not describe it well
so here is how it should be defined:
variables:
#secrets
- group: ConsumerDevVariableGroup
- name: Graph.GraphAppTenantId
value: $(GraphAppTenantId) #works fine
- name: 'Graph.GraphAppClientId'
value: '$[variables.GraphAppClientId]' #does not work

Answer 2

Azure DevOps file transform with vault secrets
To achieve this, you could use the task Replace Tokens to update the key's values with Azure vault secrets.
As test, I created a vault secret LeoVar1 in the Azure portal with test value 123456:
Then connect Azure key vault into azure devops pipeline.
Now, I use the Replace Tokens to replace the value of key webpages:Version in the web.config file with the vault secret in the Azure vault #{LeoVar1}#:
My test web.config file：
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<connectionStrings>
<add name="DefaultConnection"
connectionString="Data Source=(LocalDb)\\MSDB;DbFilename=aspcore-local.mdf;" />
</connectionStrings>
<appSettings>
<add key="webpages:Version" value="#{LeoVar1}#" />
<add key="webpages:Enabled" value="false" />
</appSettings>
<system.web>
<authentication mode="None" />
<compilation targetFramework="4.5" debug="true" />
</system.web>
</configuration>
You will see the replace are finished successfully after this task ended：
You could check this document Store the app secrets in Azure Key Vault and use during Azure Pipelines for some details.
Update:
I just wandering why FileTransform#2 does not work with secrets
To make it work, you should make sure following things:
The path of appsettings.json is correct. Makre sure it under the folder $(workingDirectory).
The important is that make sure the name of the vault secret is the combination of the attribute name and its parent node.
For example, if we want change the DebugMode from enabled to disable in following json file:
{
"Data": {
"DefaultConnection": {
"ConnectionString": "Data Source=(LocalDb)\\MSDB;AttachDbFilename=aspcore-local.mdf;"
},
"DebugMode": "enabled",
"DBAccess": {
"Administrators": ["Admin-1", "Admin-2"],
"Users": ["Vendor-1", "vendor-3"]
},
"FeatureFlags": {
"Preview": [
{
"newUI": "AllAccounts"
},
{
"NewWelcomeMessage": "Newusers"
}
]
}
}
}
We need defined the name of the vault secret to Data.DebugMode instead of DebugMode.
However, the point . is not supported by vault secret, it will give the error:
Please provide a valid secret name. Secret names can only contain
alphanumeric characters and dashes.
If we test it with Variables in the pipeline, it works without any error:
Since the point . is not supported by vault secret, we could NOT replace the properties with parent node. That the reason why I suggest you use replace token task at first.