Is there a way to apply my CloudFormation resources tags retrieved from a parameter store? I have stored the below json as type string in parameter store.
{
"APP_ID": "APP_1",
"Owner": "Test",
"TIER": 1
}
I want to be able to retrieve this SSM parameter as CloudFormation parameter and use the json value to apply to my Cloudformation resource tags. I have tried this but throws an error:
"Transform": "AWS::LanguageExtensions",
"Parameters": {
"apptagsParameter": {
"Type": "AWS::SSM::Parameter::Value<String>",
"Default": "/apps/dcs/crm"
},
},
"Resources": {
"ExampleQueue7618E31B": {
"Type": "AWS::SQS::Queue",
"Properties": {
"Tags": [
{
"Fn::ToJsonString": {
"Ref": "apptagsParameter"
}
}
],
},
Results in error:
Properties validation failed for resource ExampleQueue7618E31B with message: #/Tags/0: expected type: JSONObject, found: String
Related
I have the below mappings for a Copy activity in ADF:
"translator": {
"type": "TabularTranslator",
"mappings": [
{
"source": {
"path": "$['id']"
},
"sink": {
"name": "TicketID"
}
},
{
"source": {
"path": "$['summary']"
},
"sink": {
"name": "TicketSummary"
}
},
{
"source": {
"path": "$['status']['name']"
},
"sink": {
"name": "TicketStatus"
}
},
{
"source": {
"path": "$['company']['identifier']"
},
"sink": {
"name": "CustomerAccountNumber"
}
},
{
"source": {
"path": "$['company']['name']"
},
"sink": {
"name": "CustomerName"
}
},
{
"source": {
"path": "$['customFields'][74]['value']"
},
"sink": {
"name": "Landlord"
}
},
{
"source": {
"path": "$['customFields'][75]['value']"
},
"sink": {
"name": "Building"
}
}
],
"collectionReference": "",
"mapComplexValuesToString": false
}
The challenge I need to overcome is that the array indexes of the custom fields of the last two sources might change. So I've created an Azure Function which calculates the correct array index. However I can't work out how to use the Azure Function output value in the source path string - I have tried to refer to it using an expression like #activity('Get Building Field Index').output but as it's expecting a JSON path, this doesn't work and produces an error:
JSON path $['customFields'][#activity('Get Building Field Index').outputS]['value'] is invalid.
Is there a different way to achieve what I am trying to do?
Thanks in advance
I have a slightly similar scenario that you might be able to work with.
First, I have a JSON file that is emitted that I then access with Synapse/ADF with Lookup.
I next have a For each activity that runs a copy data activity.
The for each activity receives my Lookup and makes my JSON usable, by setting the following in the For each's Settings like so:
#activity('Lookup').output.firstRow.childItems
My JSON roughly looks as follows:
{"childItems": [
{"subpath": "path/to/folder",
"filename": "filename.parquet",
"subfolder": "subfolder",
"outfolder": "subfolder",
"origin": "A"}]}
So this means in my copy data activity within the for each activity, I can access the parameters of my JSON like so:
#item()['subpath']
#item()['filename']
#item()['folder']
.. etc
Edit:
Adding some screen caps of the parameterization:
https://i.stack.imgur.com/aHpWk.png
I'm trying to assign RBAC role "Reader" to a list of users in subscription level. When I try to do it for one user ARM template works. But for list of users it gives this error.
InvalidRoleAssignmentId: The role assignment ID 'u4ttmsjymtpe21' is not valid. The role assignment ID must be a GUID.
InvalidRoleAssignmentId: The role assignment ID 'u4ttmsjymtpe20' is not valid. The role assignment ID must be a GUID.
Here's the code I used:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"principalId": {
"type": "array"
},
"builtInRoleType": {
"type": "string"
},
"guidValue": {
"type": "string"
}
},
"variables": {
"unique_string":"[uniqueString(parameters('guidValue'))]",
"Reader": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', 'acdd72a7-3385-48ef-bd42-f606fba81ae7')]"
},
"resources": [
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2018-09-01-preview",
"name": "[concat(variables('unique_string'),copyIndex())]",
"copy": {
"name": "useridLoop",
"count": "[length(parameters('principalId'))]"
},
"properties": {
"roleDefinitionId": "[variables(parameters('builtInRoleType'))]",
"principalId": "[parameters('principalId')[copyIndex()]]"
}
}
]
}
This is the parameter file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"principalId": {
"value": [
"b5*****c-****-****-****-c*****0*****",
"e******d-****-****-****-b*****b*****"
]
},
"builtInRoleType": {
"value": "Reader"
},
"guidValue": {
"value": "[newGuid()]"
}
}
}
InvalidRoleAssignmentId: The role assignment ID 'u4ttmsjymtpe21' is
not valid. The role assignment ID must be a GUID.
InvalidRoleAssignmentId: The role assignment ID 'u4ttmsjymtpe20' is
not valid. The role assignment ID must be a GUID.
Instead of passing the [newGuid()] as value to the parameter, you need to pass it as a default value to the parameter. Because of this, you were landed up with the above error message.
We have made some changes to the above-shared template & tried
deploying the modified template, we are able to assign the users as 'Reader' to the subscription.
Here is the Modified ARM template:
{
"$schema":"https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion":"1.0.0.0",
"parameters":{
"principalId":{
"type":"array"
},
"name":{
"type":"string",
"defaultValue":"[newGuid()]"
}
},
"variables":{
"Reader":"[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', 'acdd72a7-3385-48ef-bd42-f606fba81ae7')]"
},
"resources":[
{
"type":"Microsoft.Authorization/roleAssignments",
"apiVersion":"2018-09-01-preview",
"name":"[guid(concat(parameters('name'),copyIndex()))]",
"copy":{
"name":"useridLoop",
"count":"[length(parameters('principalId'))]"
},
"properties":{
"roleDefinitionId":"[variables('Reader')]",
"principalId":"[parameters('principalId')[copyIndex()]]"
}
}
]
}
Here is the parameter.json file:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"principalId": {
"value": [
<object-id of the users>
]
}
}
}
Here is the sample output for reference:
Hey I am doing a CI/CD deployment for a logic app, I have a table storage where I store some data, I have two table storage for test and prod environment. I created a parameter called *table_storage_name" in ARM template :
"parameters": {
// ....
"connections_azuretables_1_externalid": {
"defaultValue": "/subscriptions/e5..../resourceGroups/myrg.../providers/Microsoft.Web/connections/azuretables-1",
"type": "String"
},
"table_storage_name": {
"defaultValue": "testdevops",
"type": "String"
}
}
The error comes from when I reference the parameter here in template.json file:
// ...
"Insert_Entity": {
"runAfter": {
"Initialize_variable": [
"Succeeded"
]
},
"type": "ApiConnection",
"inputs": {
"body": {
"PartitionKey": "#body('Parse_JSON')?['name']",
"RowKey": "#body('Parse_JSON')?['last']"
},
"host": {
"connection": {
"name": "#parameters('$connections')['azuretables_1']['connectionId']"
}
},
"method": "post",
// problem occur after this line
"path": "/Tables/#{encodeURIComponent('[parameters('table_storage_name')]')}/entities"
}
}
but get this error:
InvalidTemplate: The template validation failed: 'The template action 'Insert_Entity' at line '1' and column '582' is not valid: "Unable to parse template language expression 'encodeURIComponent([parameters('table_storage_name')])': expected token 'Identifier' and actual 'LeftSquareBracket'.".'.
I tried escaping the quote with a backslash like: encodeURIComponent(\'[parameters('table_storage_name')]\') or encodeURIComponent('[parameters(''table_storage_name'')]') but all of them raise an error. How can I reference a paramter inside encodeURIComponent in an ARM template ?
As discussed in the comments. credits: #marone
"path": "/Tables/#{encodeURIComponent(parameters('table_storage_name'))}/entities"
Found the solution from this link https://platform.deloitte.com.au/articles/preparing-azure-logic-apps-for-cicd
but here are the steps to reference a parameter logic app:
create an ARM parameter table_storage_name_armparam in template.json, in order to use it's value to reference the value of the ARM parameter (yes it's confusing but follow along you'll understand):
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"table_storage_name_armparam": {
"type": "String"
}
},
"variables": {},
"resources": [
{
......
}
Now in the logic app parameter value (in the bottom of json file) create the logic app parameter table_storage_name and the value of this parameter will be the ARM parameter created in step 1:
.......
"parameters": {
"$connections": {
"value": {
"azuretables": {
"connectionId": "[parameters('connections_azuretables_externalid')]",
"connectionName": "azuretables",
"id": "/subscriptions/xxxxx-xxxx-xxxx-xxxxxxxx/providers/Microsoft.Web/locations/francecentral/managedApis/azuretables"
}
}
},
"table_storage_name": {
"value": "[parameters('table_storage_name_armparam')]"
}
}
}
}
]
}
finally, reference the logic app parameter value as follow:
"path": "/Tables/#{encodeURIComponent(parameters('table_storage_name'))}/entities"
I'm trying to drive the columnMapping property from a database configuration table. My first activity in the pipeline pulls in the rows from the config table. My copy activity source is a Json file in Azure blob storage and my sink is an Azure SQL database.
In copy activity I'm setting the mapping using the dynamic content window. The code looks like this:
"translator": {
"value": "#json(activity('Lookup1').output.value[0].ColumnMapping)",
"type": "Expression"
}
My question is, what should the value of activity('Lookup1').output.value[0].ColumnMapping look like?
I've tried several different json formats but the copy activity always seems to ignore it.
For example, I've tried:
{
"type": "TabularTranslator",
"columnMappings": {
"view.url": "url"
}
}
and:
"columnMappings": {
"view.url": "url"
}
and:
{
"view.url": "url"
}
In this example, view.url is the name of the column in the JSON source, and url is the name of the column in my destination table in Azure SQL database.
The issue is due to the dot (.) sign in your column name.
To use column mapping, you should also specify structure in your source and sink dataset.
For your source dataset, you need specify your format correctly. And since your column name has dot, you need specify the json path as following.
You could use ADF UI to setup a copy for a single file first to get the related format, structure and column mapping format. Then change it to lookup.
And as my understanding, your first format should be the right format. If it is already in json format, then you may not need use "json" function in your expression.
There seems to be a disconnect between the question and the answer, so I'll hopefully provide a more straightforward answer.
When setting this up, you should have a source dataset with dynamic mapping. The sink doesn't require one, as we're going to specify it in the mapping.
Within the copy activity, format the dynamic json like the following:
{
"structure": [
{
"name": "Address Number"
},
{
"name": "Payment ID"
},
{
"name": "Document Number"
},
...
...
]
}
You would then specify your dynamic mapping like this:
{
"translator": {
"type": "TabularTranslator",
"mappings": [
{
"source": {
"name": "Address Number",
"type": "Int32"
},
"sink": {
"name": "address_number"
}
},
{
"source": {
"name": "Payment ID",
"type": "Int64"
},
"sink": {
"name": "payment_id"
}
},
{
"source": {
"name": "Document Number",
"type": "Int32"
},
"sink": {
"name": "document_number"
}
},
...
...
]
}
}
Assuming these were set in separate variables, you would want to send the source as a string, and the mapping as json:
source: #string(json(variables('str_dyn_structure')).structure)
mapping: #json(variables('str_dyn_translator')).translator
VladDrak - You could skip the source dynamic definition by building dynamic mapping like this:
{
"translator": {
"type": "TabularTranslator",
"mappings": [
{
"source": {
"type": "String",
"ordinal": "1"
},
"sink": {
"name": "dateOfActivity",
"type": "String"
}
},
{
"source": {
"type": "String",
"ordinal": "2"
},
"sink": {
"name": "CampaignID",
"type": "String"
}
}
]
}
}
I have a master Cloudformation template which invokes two child templates. I have my first template run and the Outputs captured in the Outputs section of the resource. I have given lot of tries in using the ChildStack01 Output values in the Second Template which is nested and I am not sure why I get Template format error: Unresolved resource dependencies [XYZ] in the Resources block of the template. Here is my master template.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"LambdaStack": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/bucket1/cloudformation/Test1.json",
"TimeoutInMinutes": "60"
}
},
"PermissionsStack": {
"Type": "AWS::CloudFormation::Stack",
"Properties": {
"TemplateURL": "https://s3.amazonaws.com/bucket1/cloudformation/Test2.json",
"Parameters": {
"LambdaTest": {
"Fn::GetAtt": ["LambdaStack", "Outputs.LambdaTest"]
}
},
"TimeoutInMinutes": "60"
}
}
}
}
Here is my Test1.json Template
{
"Resources": {
"LambdaTestRes": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Description": "Testing AWS cloud formation",
"FunctionName": "LambdaTest",
"Handler": "lambda_handler.lambda_handler",
"MemorySize": 128,
"Role": "arn:aws:iam::3423435234235:role/lambda_role",
"Runtime": "python2.7",
"Timeout": 300,
"Code": {
"S3Bucket": "bucket1",
"S3Key": "cloudformation/XYZ.zip"
}
}
}
},
"Outputs": {
"LambdaTest": {
"Value": {
"Fn::GetAtt": ["LambdaTestRes", "Arn"]
}
}
}
}
Here is My Test2.json which has to use the output of Test1.json.
{
"Resources": {
"LambdaPermissionLambdaTest": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"Action": "lambda:invokeFunction",
"FunctionName": {
"Ref": "LambdaTest"
},
"Principal": "apigateway.amazonaws.com",
"SourceArn": {
"Fn::Join": ["", ["arn:aws:execute-api:", {
"Ref": "AWS::Region"
}, ":", {
"Ref": "AWS::AccountId"
}, ":", {
"Ref": "TestAPI"
}, "/*"]]
}
}
}
},
"Parameters": {
"LambdaTest": {
"Type": "String"
}
}
}
It is not enough to just have output, you need to export that output.
Look here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-stack-exports.html
So you need something like:
"Outputs": {
"LambdaTest": {
"Value": {
"Fn::GetAtt": ["LambdaTestRes", "Arn"]
}
"Export": {
"Name": "LambdaTest"
}
}
}
You have two unresolved Ref resource dependencies in Test2.json, one to LambdaTest and one to TestAPI.
For LambdaTest, it looks like you're trying to pass this as a parameter from the parent stack, but you haven't specified it as an input Parameter in the child Test2.json template. Add an entry in Test2.json's Parameters section, like this:
"Parameters": {
"LambdaTest": {
"Type": "String"
}
},
Regarding TestAPI, this reference doesn't seem to appear anywhere else in your templates, so you should either specify this as a fixed string directly, or add another input Parameter in your Test2.json stack (see above) and then provide it from the parent stack.
The error is coming from test1.json(LambdaStack).
Logical ID
An identifier for the current output. The logical ID must be alphanumeric (a-z, A-Z, 0-9) and unique within the template.
It seems you have two logical ID with the same name "LambdaTest", one in resource section and other in output section.