Deploy only certain resources into Azure using DevOps Pipeline - azure-devops

I have a main.bicep template with multiple resources that I deploy to Azure via my DevOps pipeline. My template is currently configured to either deploy all modules to Azure or none. However, I would like to customize this in my pipeline so that only certain resources show up in my resource group so that it acts like a selection catalog.
However, I need advice and help on how to solve the problem as I can’t find the right approach. Attached you can see a snippet from my bicep template below.
– main.bicep–
param deployAppInsights bool = true param deployNetworkWatcher bool = true param deploySentinel bool = true
Example for a module
// Deploy App Insights resource
module appInsights 'AzureMonitor/AppInsights/appInsights-temp.bicep' = if(deployAppInsights) { name: 'appInsightsDeployment' params: { appName: appInsightsName regionId: regionId // tagsArray: tagsArray requestSource: requestSource workspaceResourceId: workspace.outputs.resourceWorkspaceIdOutput } }
This snippet is in my pipeline - yml
(I would like to shorten the parameters block by using override parameters.)
parameters:
- name: appInsights
type: boolean
default: True
values:
- True
- False
inputs: overrideParameters: -deployAppInsights false -deployNetworkWatcher true -deploySentinel true
Currently, my override parameters are hardcoded, but I’d like to keep it in a general way, like for example:
overrideParameters: -deployAppInsights ${{appInsights}} -deployNetworkWatcher ${{networkWatcher}} -deploySentinel ${{sentinel}}
However, this does not unfortunately work. Could anyone please help me in this regard, how I can solve this or how the variables need to look like?
Thank you very much!
Best regards
Jennifer

As far as I understand, the syntax you are using is wrong in the below code
overrideParameters: -deployAppInsights ${{appInsights}} -deployNetworkWatcher ${{networkWatcher}} -deploySentinel ${{sentinel}}
Since you have already defined parameters section in the Yml file. The correct syntax is
overrideParameters: -deployAppInsights ${{parameters.appInsights}} -deployNetworkWatcher ${{parameters.networkWatcher}} -deploySentinel ${{parameters.sentinel}}
A sample code snippet for your reference
parameters:
- name: var1
type: string
- name: var2
type: string
- name: bool1
type: bool
...
omitted
...
- task: AzureResourceManagerTemplateDeployment#3
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: ''
subscriptionId: ''
action: 'Create Or Update Resource Group'
resourceGroupName: rg-${{parameters.var1}}-${{parameters.var2}}
location: ''
templateLocation: 'Linked artifact'
csmFile: $(System.DefaultWorkingDirectory)/arm.json
csmParametersFile: $(System.DefaultWorkingDirectory)/arm.parameters.json
deploymentMode: 'Incremental'
overrideParameters:
-string1 ${{parameters.var1}}
-string2 ${{parameters.var2}}
-flag ${{parameters.bool1}}
For more, please refer to the link from Microsoft. Azure Devops Template
Edit: After the first comment
Please update your 'appInsights' parameter as follows
parameters:
- name: appInsights
type: boolean
default: true

Related

Azure Devops Yaml Python Script task: PythonScriptV0 , how to pass arrays as the field supports only a strings

I am trying to pass to a python scripts 3 parameters one of them is an array, this works when i run the script locally, i am using sys.argv to achieve this.
However the argument field support only strings as far i can see. How can i go around this any ideas? Thanks
the array is ${{ parameters.packageVersion }}
Code:
- task: PythonScript#0
displayName: 'Modify ansible inventory files (wms_common.yml) with deployed versions'
inputs:
scriptSource: filePath
scriptPath: deployment/s/azure-devops/scripts/script.py
arguments: |
../../inventories/${{ variables.inventory }}/group_vars/all/wms_common.yml
../../inventories/central/${{ variables.inventory }}/group_vars/all/wms_common.yml
${{ parameters.packageVersion }}
Error:
/azure-devops/wms.full.pipeline.yml (Line: 95, Col: 18): Unable to convert from Array to String. Value: Array
Edit: Reframed question
I think the below YAML will help you convert the array to String objects and use them.
variables:
myVariable1: 'value1'
myVariable2: 'value2'
system.debug: true
parameters:
- name: InstanceArgs
type: object
default: [1,2]
steps:
- task: PythonScript#0
inputs:
scriptSource: 'inline'
script: |
import argparse
parse = argparse.ArgumentParser(description="Test")
parse.add_argument("test1")
parse.add_argument("test2")
args = parse.parse_args()
print(args.test1)
print(args.test2)
print('this is a test.')
# arguments: $(myVariable1) $(myVariable2)'
arguments: ${{join(' ',parameters.InstanceArgs)}}
You need to use the join expression in YAML to get the elements:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops#join

Unable to use | character in AzureFunctionApp appSettings:

I am setting a load of appSettings in my AzureFunctionApp#1 deployment task - but whenever I try to put each on a new line using the | character I get the error:
##[error]Error: Failed to update App service '{{functionName}}' application settings. Error: BadRequest - Parameter name cannot be empty. (CODE: 400)
The output above that seems to show that it has indeed built the JSON with an empty parameter name. But I don't know why? I've tested with the values on separate lines, and still in a single line, so neither of these work:
appSettings: |
'-Values:Setting1 "$(SettingVal1)"
-Values:Setting2 "$(SettingVal2)"'
appSettings: |
'-Value:Setting1 "$(SettingVal1)" -Values:Setting2 "$(SettingVal2)"'
But this does:
appSettings: '-Value:Setting1 "$(SettingVal1)" -Values:Setting2 "$(SettingVal2)"'
I've also tried without the ' - but that made no difference either.
As per your feedback - Converting my comment as an answer, also tried locally in my system.
Multi-line json input works for setting the multiple values in the app settings as this is the closest way.
appSettings: |
[
{
"name": "APPINSIGHTS_INSTRUMENTATIONKEY",
"value": "$(Key)",
"slotSetting": false
},
{
"name": "MYSQL_DATABASE_NAME",
"value": "$(DB_Name)",
"slotSetting": false
}
]
Multiline JSON doesn't work with the AzureFunctionApp#1 task's appSettings parameter (for some reason).
If you try to use the multiline JSON appSettings with the AzureFunctionApp#1 task, you will get an error: BadRequest - Parameter name cannot be empty. (CODE: 400)
To use the multiline JSON appSettings, you need to use a separate AzureAppServiceSettings#1 task - as mentioned in this document.
I can confirm this works after the AzureFunctionApp#1 task. So my pipeline now has:
steps:
...
- task: AzureFunctionApp#1
displayName: Deploy the Function App
condition: succeeded()
inputs:
azureSubscription: "${{parameters.AppAzureSubscription}}"
appName: "${{parameters.functionAppName}}"
package: "$(Pipeline.Workspace)/drop/$(Build.BuildId).zip"
- task: AzureAppServiceSettings#1
displayName: Update app settings
inputs:
azureSubscription: "${{parameters.AppAzureSubscription}}"
appName: "${{parameters.functionAppName}}"
appSettings: |
[
{
"name": "Values:DbConnectionString",
"value": "$(DbConnectionString)",
"slotSetting": false
},
...
]

AZP: Is there a best practice to be able to "namespace" script tasks in yaml templates for usage of variables?

In Azure Pipelines: my main problem is, if I create a yml template and have some logic inside that template in a script task where I want to set a variable, i need the
name: "pseudonamespace" to reference that variable further down in that template via
$(pseudonamespace.variablename)
An example, where the script part does nothing overtly useful, but should demonstrate my problem:
mytemplate.yml:
parameters:
- name: isWindowsOnTarget
type: boolean
default: true
steps:
- script: |
if [ "${{lower(parameters.isWindowsOnTarget)}}" == "true" ]; then
delimiter="\\"
else
delimiter="/"
fi
echo "##vso[task.setvariable variable=myCoolVariable;isOutput=true]$delimiter"
name: MyFakeNameSpace
...
- task: SomeTask#0
inputs:
myInput: $(MyFakeNameSpace.myCoolVariable)
This codeblock works; but only, if, in a job, I only instanciate it once:
- template: mytemplate.yml#templates
parameters:
isWindowsOnTarget: true
If I would need that template twice, differently parameterized, I get the error that the name of the script block needs to be unique.
Is there any useful possibility I'm not currently thinking about other than to have an extra parameter for the template that I could basically just call "UniqueNamespace"?
There is no much space to move. Your task needs a unique name as later as you mention for output parameters it works like a namespace. So the best and the only way you have is to provide another parameter which would be task name.
parameters:
- name: isWindowsOnTarget
type: boolean
default: true
- name: taskName
type: string
steps:
- script: |
if [ "${{lower(parameters.isWindowsOnTarget)}}" == "true" ]; then
delimiter="\\"
else
delimiter="/"
fi
echo "##vso[task.setvariable variable=myCoolVariable;isOutput=true]$delimiter"
name: ${{ parameters.taskName }}
...
- task: SomeTask#0
inputs:
myInput: $(MyFakeNameSpace.myCoolVariable)
and then:
- template: mytemplate.yml#templates
parameters:
isWindowsOnTarget: true
taskName: MyFakeNameSpace
- template: mytemplate.yml#templates
parameters:
isWindowsOnTarget: true
taskName: MyFakeNameSpace2
In fact when you do not provide a name Azure DevOps assign a unique name. However, in this way you don't know the name till runtime.

How to pass Azure ARM template object value from YAML file?

Azure ARM Template parameter.json file has the below object property.
"appInsightsObject": {
"value": {
"name": "appInsghtName",
"id": "appInsgID"
}
}
I have to replace these values from build.yaml file. My build.yaml file has the below
- task: AzureResourceManagerTemplateDeployment#3
displayName: APIM Development CI
inputs:
ConnectedServiceName: My-Service-Name
subscriptionName: My-Subs-Values
resourceGroupName: My-Rg-Name
location:$(locationName)
csmFile: template.json
csmParametersFile: parameters.json
overrideParameters: '-appInsightsObject.name $(appInsightNameValue) '
How to pass appInsightsObject object value?
Update:
I found one way of passing the value as JSON object like '{"name":"name-goes-here", "id":"id-value-goes-here"}'. Is there any better option?
overrideParameters: '-appInsightsObject $(appInsightValue) '
This one works and allows to store values individually for any type of object.
overrideParameters: '-appInsightsObject {"name": "$(dev.insightName)","id": "$(dev.insightId)"} '

Azure Data Factory - Update environment variable in continues integration pipeline

I have recently started with ARM and Azure Data Factory, and now I have faced a problem when I try to deploy ADF into other environments. Because the source connection string are different in each environment and I can not keep the static value in adf.content.parameters.json file.
So I have created three YML files for each environment as Dev.yml, Test.yml, and Prod.yml. I have three files
adf.content.json
adf.content.parameters.json
Dev.yml, test.yml and prod.yml
In the file, adf.content.json, I have a connection string as my source. This value is changing in each environment. Here is the adf.content.parameters.json
"parameters": {
"source_connectionString": {
"type": "secureString",
"metadata": "Secure string for 'connectionString' of 'source-db'"
},
and I removed this parameter from adf.content.parameters.json but instead, I have added that to Dev.yml file which looks like this one (Test.yml and prod.yml are the same just different values)
variables:
- name: source-connectionstring
value: <some value>
I have a ci-build.yml and ci-deploy.yml files which will be used for the CI pipeline. In the ci-build.yml file, I use the same name in the adf.content.json
variables:
SourceConnectionString: $(source_connectionString:?You need to set the source_connectionString environment variable)
- stage: DevDeploy
displayName: Deploy to development (D)
variables:
- template: ../.../.../dev.yml
- group: ...
dependsOn: Build
jobs:
- template: cd_deploy.yml
parameters:
environment: dev
azureServiceConnection: '...'
containerRegistryDomain: ...
apiResourceGroup: '...'
webAppName: '...
azureSubscriptionName: ...
apimResourceGroup: ...
apimName: ...
And in my ci-deply.yml file looks like this
task: AzureResourceGroupDeployment#2
displayName: "Deploy Azure Data Factory Content"
inputs:
azureSubscription: '...'
action: '...'
resourceGroupName: '...'
location: '...'
templateLocation: '...'
csmFile: '$(System.ArtifactsDirectory)/.../arm/adf.content.json'
csmParametersFile: '$(System.ArtifactsDirectory)/../arm/adf.content.parameters.json'
overrideParameters: ' -source_connectionString$(SourceConnectionString)
deploymentMode: 'Incremental'
But I get an error " did not find expected key" on overrideParameters: ' -source_connectionString$(SourceConnectionString)
I do not know if this is the right approach for this? and if anyone can get why I can not get the key?
This error is usually caused by wrong syntax.
In ci-deply.yml, the script overrideParameters: ' -source_connectionString$(SourceConnectionString) loses a '.
So your task in ci-deply.yml should look like this:
- task: AzureResourceGroupDeployment#2
displayName: "Deploy Azure Data Factory Content"
inputs:
azureSubscription: '...'
action: '...'
resourceGroupName: '...'
location: '...'
templateLocation: '...'
csmFile: '$(System.ArtifactsDirectory)/.../arm/adf.content.json'
csmParametersFile: '$(System.ArtifactsDirectory)/../arm/adf.content.parameters.json'
overrideParameters: ' -source_connectionString$(SourceConnectionString) '
deploymentMode: 'Incremental'