Azure nested template deployment: Using template element (not templateLink) with PowerShell - powershell

In an attempt to make life easier (in the long run), I'm trying to use properties.template, as opposed to the well documented properties.templateLink. The former has very little documentation by passing the contents of child.json template file into the parent.json template, as a template' parameter.
From the MS documentation for Microsoft.Resources/deployments:
The template content. You use this element when you want to pass the template syntax directly in the request rather than link to an existing template. It can be a JObject or well-formed JSON string. Use either the templateLink property or the template property, but not both.
In my parent template, I am declaring the parameter childTemplates and referencing it in properties.template:
"parameters": {
"childTemplates": {
"type": "object",
"metadata": {
"description": "Child template"
}
}
}
other stuff...
"resources": [
{
"name": "[concat('linkedTemplate-VM-Net-',copyIndex(1))]",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2017-06-01",
"dependsOn": [],
"copy": {
"name": "interate",
"count": "[parameters('vmQty')]"
},
"properties": {
"mode": "Incremental",
"template": "[parameters('childTemplates')]",
"parameters": {
"sharedVariables": { "value": "[variables('sharedVariables')]" },
"sharedTemplate": { "value": "[variables('sharedTemplate')]" },
"artifactsLocationSasToken": { "value": "[parameters('artifactsLocationSasToken')]" },
"adminPassword": { "value": "[parameters('adminPassword')]" },
"copyIndexValue": { "value": "[copyIndex(1)]" }
},
"debugSetting": {
"detailLevel": "both"
}
}
}
],
I then pass the child template to New-AzureRmResourceGroupDeployment -TemplateParameterObject to deploy the parent template:
$TemplateFileLocation = "C:\Temp\templates\parent.json"
$JsonChildTemplate = Get-Content -Raw (Join-Path ($TemplateFileLocation | Split-Path -Parent) "nestedtemplates\child.json") | ConvertFrom-Json
$TemplateParameters = #{
childTemplates = $JsonChildTemplate
...Other parameters...
}
New-AzureRmResourceGroupDeployment -TemplateParameterObject $TemplateParameters
This produces the following error:
Code : InvalidTemplate
Message : The nested deployment 'linkedTemplate-VM-Net-1' failed validation: 'Required property '$schema' not found in JSON. Path 'properties.template'.'.
Target :
Details :
If I look at $JsonChildTemplate, it gives me:
$schema : https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#
contentVersion : 1.0.0.0
parameters : #{sharedVariables=; sharedTemplate=; vhdStorageAccountName=; artifactsLocationSasToken=; adminPassword=; copyIndexValue=}
variables : #{seqNo=[padleft(add(parameters('copyIndexValue'),3),3,'0')]; nicName=[concat('NIC-',parameters('sharedVariables').role,'-', variables('seqNo'),'-01')];
subnetRef=[parameters('sharedVariables').network.subnetRef]; ipConfigName=[concat('ipconfig-', variables('seqNo'))]}
resources : {#{apiVersion=2016-03-30; type=Microsoft.Network/networkInterfaces; name=[variables('nicName')]; location=[resourceGroup().location]; tags=; dependsOn=System.Object[];
properties=}}
outputs : #{nicObject=; vmPrivateIp=; vmNameSuffix=; vmPrivateIpArray=}
To me, it looks like the $schema is there.
I have also tried removing | ConvertFrom-Json with the same error.
Above, I am showing the latest API version, but I have tried with others such as 2016-09-01, just in case there's a bug.
In my search for a solution, I found this issue on GitHub. The recomendation is to remove $schema and contentVersion, although this flies in the face of the error. I tried this with the following:
Function Get-ChildTemplate
{
$TemplateFileLocation = "C:\Temp\templates\nestedtemplates\child.json"
$json = Get-Content -Raw -Path $TemplateFileLocation | ConvertFrom-Json
$NewJson = #()
$NewJson += $json.parameters
$NewJson += $json.variables
$NewJson += $json.resources
$NewJson += $json.outputs
Return $NewJson | ConvertTo-Json
}
$JsonChildTemplate = Get-ChildTemplate
$TemplateParameters = #{
childTemplates = $JsonChildTemplate
...Other parameters...
}
$JsonChildTemplate returns:
[
{
"sharedVariables": {
"type": "object",
"metadata": "#{description=Object of variables from master template}"
}...
My guess is that I have done something wrong passing child.json's contents to New-AzureRmResourceGroupDeployment. That or it's not actually possible to do what I'm trying to do.
P.S.
get-command New-AzureRmResourceGroupDeployment
CommandType Name Version Source
----------- ---- ------- ------
Cmdlet New-AzureRmResourceGroupDeployment 4.1.0 AzureRM.Resources

First of all, what you are doing makes 0 sense what so ever, that being said, lets try to help you.
Try splatting. so do New-AzureRmResourceGroupDeployment ... #TemplateParameters instead of what you are doing. (no idea, but somehow it works better in my experience)
If that doesn't work directly try simplifying you nested template to the bare minimum and see if it works if it does, check if your nested template is fine.
Try creating a deployment with -Debug switch and see where that goes.
Try the same deployment using Azure Cli (maybe it converts json to input object in a proper way)
Skip items 1-4 and do it the proper way. I would advice never do preprocessing\in flight generation of ARM Templates. They have enough of features already to accomplish anything if you are smart hacky enough. I have no idea what you are trying to achieve but I can bet my life on it you don't need that monstrosity you are trying to create
small template example:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"resources": []
}
EDIT: I dug a bit more and found a solution. one way to do it would be using the json() function of the arm template that accepts a string and converts to valid json.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"inp": {
"type": "string"
}
},
"resources": [
{
"name": "NestedDeployment1",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2015-01-01",
"properties": {
"mode": "Incremental",
"template": "[json(parameters('inp'))]",
"parameters": {}
}
}
]
}
To deploy use something like this:
New-AzureRmResourceGroupDeployment ... -inp ((get-content path\to.json -raw) -replace '\s','')
# we minify the string so it gets properly converted to json
This is a bit of a hack, but the problem lies with how powershell converts your input to what it passes to the template, and you cannot really control that.
Another way to do that: (if you need output you can add another parameter)
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"input-param": {
"type": "object"
},
"input-resource": {
"type": "array"
}
},
"resources": [
{
"name": "NestedDeployment1",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2015-01-01",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": "[parameters('input-param')]",
"resource": "[parameters('input-resource')]"
},
"parameters": {}
}
}
]
}
and deploying like so:
New-AzureRmResourceGroupDeployment -ResourceGroupName zzz -TemplateFile path\to.json -input-param #{...} -input-resource #(...)
ps. don't mind walter, each time he says something can't be done or is impossible it actually is possible.

Related

issues PowerShell POST request with body

I have a bit issue with powershell + invoke-webrequest
This is my body
$pram = #{
"name": "MDE.Windows",
"id": "$resourceId/extensions/MDE.Windows",
"type": "Microsoft.Compute/virtualMachines/extensions",
"location": "westeurope",
"properties": {
"autoUpgradeMinorVersion": true,
"publisher": "Microsoft.Azure.AzureDefenderForServers",
"type": "MDE.Windows",
"typeHandlerVersion": "1.0",
"settings": {
"azureResourceId": "$resourceId",
"defenderForServersWorkspaceId": "$subscriptionId",
"vNextEnabled": "true",
"forceReOnboarding": true,
"provisionedBy": "Manual"
},
"protectedSettings": {
"defenderForEndpointOnboardingScript": "$defenderForEndpointOnboardingScript"
}
}
}
I don't get watch wrong with my body because by looking examples from google this should be right but it still ouputs red
I have tried also with #"{ }"#, #'{ }'#, { }, "{ }" but no matter what I do it is more or less red.
I think your mistaking powershell hash tables for json. Normally you would create a hashtable using powershell syntax, then convert that object into Json. eg
$pram = #{
name= "MDE.Windows";
id= "$resourceId/extensions/MDE.Windows";
} | ConvertTo-Json
You can now pass the json encoded value of $parm to Invoke-WebRequest.
The other option is to create a String and write the JSON yourself:
$paramString = '{"id": "/extensions/MDE.Windows", "name": "MDE.Windows"}'
(but the first solution is probably the solution your looking for).

ARM InvalidTemplateDeployment: Required property '$schema' not found in JSON

I receive the following errors when attempting to deploy an exported Azure Resource Template:
"The template is invalid. Error: 'Required property '$schema' not found in
JSON.
"Domain name is already taken"
Error Content:
The error content is as follows:
{"code":"InvalidTemplateDeployment","details":[{"code":"DeploymentPreflightValidationFailed","target":"/subscriptions/<SOME_GUID_VALUE>/resourceGroups/<SOME_RESOURCE_GROUP_NAME>-Resource/providers/Microsoft.Resources/templateSpecs/<SOME_ARM_TEMPLATE_NAME>/versions/0.0","message":"The
template is invalid. Error: 'Required property '$schema' not found in
JSON. Path '', line 1, position
2.'"},{"code":"BadArgument","message":"Domain name is already taken"}],"message":"The template deployment
'Microsoft.Template-20220817090827' is not valid according to the
validation procedure. The tracking id is
'c27e8ad4-6d96-46e9-92ed-ac8d5f462bf6'. See inner errors for
details."}
Observations:
I have observed the following:
The latest schema path appears to match what I have listed.
Template:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vulnerabilityAssessments_Default_storageContainerPath": {
"type": "SecureString"
},
...
}
The error regarding the $schema property is not referring to the $schema of the template itself, but is instead coming back from preflight (validation) of one of the Microsoft.Resources/templateSpecs/versions resources in your template.
I can repro this myself with the following file I exported (note that Export is stripping out the contents of mainTemplate, which I believe is the root cause of the issue, as it doesn't contain the expected $schema property).
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"templateSpecs_asdgasdg_name": {
"defaultValue": "asdgasdg",
"type": "String"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Resources/templateSpecs",
"apiVersion": "2022-02-01",
"name": "[parameters('templateSpecs_asdgasdg_name')]",
"location": "centralus",
"properties": {}
},
{
"type": "Microsoft.Resources/templateSpecs/versions",
"apiVersion": "2022-02-01",
"name": "[concat(parameters('templateSpecs_asdgasdg_name'), '/v1')]",
"location": "centralus",
"dependsOn": [
"[resourceId('Microsoft.Resources/templateSpecs', parameters('templateSpecs_asdgasdg_name'))]"
],
"properties": {
"mainTemplate": {}
}
}
]
}
My team at Microsoft owns the Export Template API - I will follow up internally on this.

Why is password type of AzureKeyVaultSecret dropped when creating LinkedService via powershell,

I'm attemping to create a LinkedService via the powershell command
New-AzureRmDataFactoryV2LinkedService -ResourceGroupName rg -DataFactoryName df -Name n -DefinitionFile n.json
the result is that the LinkedService is created, however the reference to the password type of AzureKeyVaultSecret is removed rendering it non-operational
The config file n.json was extracted from the DataFactory code tab and has the syntax below...
{
"name": "<name>",
"type": "Microsoft.DataFactory/factories/linkedservices",
"properties": {
"type": "Oracle",
"typeProperties": {
"connectionString": "host=<host>;port=<port>;serviceName=<serviceName>;user id=<user_id>",
"password": {
"type": "AzureKeyVaultSecret",
"store": {
"referenceName": "Prod_KeyVault",
"type": "LinkedServiceReference"
},
"secretName": "<secretname>"
}
},
"connectVia": {
"referenceName": "<runtimename>",
"type": "IntegrationRuntimeReference"
}
}
}
When the new LinkedService is created, the code looks exactly the same except properties->typeProperties->password is removed and requires manual configuration - which I'm trying to avoid if possible.
Any thoughts?
If you have tried using "Update-Module -Name AzureRm.DataFactoryV2" to update your powershell to the latest version, and it is still the same behavior, then the possible root cause is that password is not support as Azure Key Value yet in Powershell. As far as I know, it is a new feature added recently. So it may take some time to rollout it to Powershell.
In that case, the workaround is to use UI to create linked service for now.

Override template parameter in the script

I am deploying VMs using a template and parameter file. I would like to override the parameter files virtualMachineName, so I can dynamically name each VM without needing 10 different parameter files. What is the correct syntax to do so?
Currently I have:
New-AzureRmResourceGroupDeployment -ResourceGroupName "myRSG" -TemplateFile $server1TmpFile -TemplateParameterFile $serverParamFile -virtualMachineName "AS1FTW12" -networkInterfaceName "AS1FTW12NIC" -networkSecurityGroupName "MyNSGName"
This will produce an error:
New-AzureRmResourceGroupDeployment : A parameter cannot be found that matches parameter name 'virtualMachineName'
I am authoring the script using PowerShell Tools for Visual Studio 2017. My installed PowerShell AzureRM version is 5.1.1 according to Get-Module AzureRM.
This Works with AzureRM 5.1.1, you can test with the following setup:
template:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"paramTest": {
"type": "string"
}
},
"resources": [],
"outputs": {
"result": {
"type": "string",
"value": "[parameters('paramTest')]"
}
}
}
parameters:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"paramTest": {
"value": "GEN-UNIQUE"
}
}
}
Powershell:
New-AzureRmResourceGroupDeployment -ResourceGroupName rg -TemplateFile template.json -TemplateParameterFile param.json -paramTest somevalue
results in output value equal to the one passed from powershell
Your error indicates that the parameter name is wrong, doublé check it

Apply tags to Azure resource group within resource template file

I'm using Powershell to create an Azure resource group from a template file and parameter file.
New-AzureResourceGroup -Name $groupName `
-TemplateFile $templateFile `
-TemplateParameterFile $paramFile `
-Location $location `
-Force -Verbose
Within the template I'm setting some tags on the resources within the group.
resourcegroup.json
"parameters": {
"environment": {
"type": "string",
"allowedValues": [
"Dev",
"Test",
"QA",
"Prod"
]}
}
....
"resources": [
{
...
"tags": {
"Environment": "[parameters('environment')]",
}
}
I'd like to apply the same tag values to the resource group itself, but I don't see a way to do that within the schema of the template file. Am I missing something?
Now at least, this is possible:
{
"type": "Microsoft.Resources/tags",
"name": "default",
"apiVersion": "2020-06-01",
"dependsOn": [
"resource1"
],
"properties": {
"tags": {
"Tag1": "Value1",
"Tag2": "Value2"
}
}
}
Documentation reference here
AFAIK, you have to use the azure Powershell command-let to add tags to your Azure Resource group. It cannot be done through the Template file.