Attaching latest content of blob storage to SendGrid - sendgrid

How to attach latest blob content in sendgrid service using logic app?

You can create a Blob trigger LA with SendGrid Action as shown below:
{
"$connections": {
"value": {
"azureblob": {
"connectionId": "/subscriptions/<SubscriptionId>/resourceGroups/<ResourceGroup>/providers/Microsoft.Web/connections/azureblob",
"connectionName": "azureblob",
"id": "/subscriptions/<SubscriptionId>/providers/Microsoft.Web/locations/southeastasia/managedApis/azureblob"
},
"sendgrid": {
"connectionId": "/subscriptions/<SubscriptionId>/resourceGroups/<ResourceGroup>/providers/Microsoft.Web/connections/sendgrid",
"connectionName": "sendgrid",
"id": "/subscriptions/<SubscriptionId>/providers/Microsoft.Web/locations/southeastasia/managedApis/sendgrid"
}
}
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Get_blob_content_using_path": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "get",
"path": "/datasets/default/GetFileContentByPath",
"queries": {
"inferContentType": true,
"path": "#triggerBody()?['Path']",
"queryParametersSingleEncoded": true
}
},
"runAfter": {},
"type": "ApiConnection"
},
"Send_email_(V2)": {
"inputs": {
"body": {
"file": "#{base64(body('Get_blob_content_using_path'))}",
"filename": "#triggerBody()?['Name']",
"from": "<FromEmail>",
"ishtml": true,
"subject": "test attach met",
"text": "PFA",
"to": "<ToEmail>"
},
"host": {
"connection": {
"name": "#parameters('$connections')['sendgrid']['connectionId']"
}
},
"method": "post",
"path": "/mail/send"
},
"runAfter": {
"Get_blob_content_using_path": [
"Succeeded"
]
},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"When_a_blob_is_added_or_modified_(properties_only)": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "get",
"path": "/datasets/default/triggers/batch/onupdatedfile",
"queries": {
"folderId": "JTJmbGFjb250YWluZXI=",
"maxFileCount": 10
}
},
"metadata": {
"JTJmbGFjb250YWluZXI=": "/lacontainer"
},
"recurrence": {
"frequency": "Minute",
"interval": 3
},
"splitOn": "#triggerBody()",
"type": "ApiConnection"
}
}
}
}

Related

How to make the file sent only once and not duplicated? Function App and blob storage container

I have this function App and I get the file once when I put one file on blob storage container!
when I put 2 files, I get 4 files in my email addressee and the same file twice ! when I put 3 files I get 9 files , the same file duplicated 3 times , and when I put 4 files I get 16 files and the same file duplicated 4 times , how to resolve this ?
this my function app :
As the Send an email (V2) connector is in for each loop of Lists blob's value, you are receiving everything in duplicate as it iterates all over the folder/container that you have set. After reproducing from our end this was working when we have changed Get Blob's Blob path to List of Files Path of When a blob is added or modified (properties only) (V2) and also use When a blob is added or modified (properties only) (V2) content in Send email too, we could receive emails on current item one time.
Here is the schema of my logic app
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Get_blob_content_(V2)_2": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "get",
"path": "/v2/datasets/#{encodeURIComponent(encodeURIComponent('AccountNameFromSettings'))}/files/#{encodeURIComponent(encodeURIComponent(triggerBody()?['Path']))}/content",
"queries": {
"inferContentType": true
}
},
"runAfter": {
"Lists_blobs_(V2)": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"Lists_blobs_(V2)": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "get",
"path": "/v2/datasets/#{encodeURIComponent(encodeURIComponent('AccountNameFromSettings'))}/foldersV2/#{encodeURIComponent(encodeURIComponent('JTJmY29udGFpbmVyMQ=='))}",
"queries": {
"nextPageMarker": "",
"useFlatListing": false
}
},
"metadata": {
"JTJmY29udGFpbmVyMQ==": "/container1"
},
"runAfter": {},
"type": "ApiConnection"
},
"Send_an_email_(V2)": {
"inputs": {
"body": {
"Body": "<p>#{triggerBody()}</p>",
"Subject": "#triggerBody()?['DisplayName']",
"To": "<To email id>"
},
"host": {
"connection": {
"name": "#parameters('$connections')['office365']['connectionId']"
}
},
"method": "post",
"path": "/v2/Mail"
},
"runAfter": {
"Get_blob_content_(V2)_2": [
"Succeeded"
]
},
"type": "ApiConnection"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"When_a_blob_is_added_or_modified_(properties_only)_(V2)": {
"evaluatedRecurrence": {
"frequency": "Second",
"interval": 1
},
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['azureblob']['connectionId']"
}
},
"method": "get",
"path": "/v2/datasets/#{encodeURIComponent(encodeURIComponent('AccountNameFromSettings'))}/triggers/batch/onupdatedfile",
"queries": {
"checkBothCreatedAndModifiedDateTime": false,
"folderId": "JTJmY29udGFpbmVyMQ=="
}
},
"metadata": {
"JTJmY29udGFpbmVyMQ==": "/container1"
},
"recurrence": {
"frequency": "Second",
"interval": 1
},
"splitOn": "#triggerBody()",
"type": "ApiConnection"
}
}
},
"parameters": {
"$connections": {
"value": {
"azureblob": {
"connectionId": "/subscriptions/***/resourceGroups/***/providers/Microsoft.Web/connections/azureblob",
"connectionName": "azureblob",
"id": "/subscriptions/***/providers/Microsoft.Web/locations/centralus/managedApis/azureblob"
},
"office365": {
"connectionId": "/subscriptions/***/resourceGroups/***/providers/Microsoft.Web/connections/office365",
"connectionName": "office365",
"id": "/subscriptions/***/providers/Microsoft.Web/locations/centralus/managedApis/office365"
}
}
}
}
}

NULL being passed from pipeline to linked service baseURL

I have a pipeline that will iterate over files and copy them to a storage location.
The baseURL and relativeURL are stored in a json file.
I can read in this file and it is valid.
I have parameterized the linked service baseURL and this works when testing from the linked service, and from the dataset.
When I try to debug the pipeline however, I get an error:
"code":"BadRequest"
"message":null
"target":"pipeline//runid/310b8ac1-2ce6-4c7c-a1ad-433ee9019e9b"
"details":null
"error":null
It appears that from the activity in the pipeline, a null value is being passed instead of the baseURL.
I have iterated over the values from my config file and it is being read and the values are correct. It really seems like the pipeline is not passing in the correct value for baseURL.
Do I have to modify the json code behind the pipeline to get this to work?
If it helps, the json for the linked service, data set and pipeline are below:
--Linked Service:
{
"name": "ls_http_opendata_ecdc_europe_eu",
"properties": {
"parameters": {
"baseURL": {
"type": "string"
}
},
"annotations": [],
"type": "HttpServer",
"typeProperties": {
"url": "#linkedService().baseURL",
"enableServerCertificateValidation": true,
"authenticationType": "Anonymous"
}
}
}
--dataset
{
"name": "ds_ecdc_raw_csv_http",
"properties": {
"linkedServiceName": {
"referenceName": "ls_http_opendata_ecdc_europe_eu",
"type": "LinkedServiceReference"
},
"parameters": {
"relativeURL": {
"type": "string"
},
"baseURL": {
"type": "string"
}
},
"annotations": [],
"type": "DelimitedText",
"typeProperties": {
"location": {
"type": "HttpServerLocation",
"relativeUrl": {
"value": "#dataset().relativeURL",
"type": "Expression"
}
},
"columnDelimiter": ",",
"escapeChar": "\\",
"firstRowAsHeader": true,
"quoteChar": "\""
},
"schema": []
}
}
--pipeline
{
"name": "pl_ingest_ecdc_data",
"properties": {
"activities": [
{
"name": "lookup ecdc filelist",
"type": "Lookup",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "JsonSource",
"storeSettings": {
"type": "AzureBlobStorageReadSettings",
"recursive": true,
"enablePartitionDiscovery": false
},
"formatSettings": {
"type": "JsonReadSettings"
}
},
"dataset": {
"referenceName": "ds_ecdc_file_list",
"type": "DatasetReference"
},
"firstRowOnly": false
}
},
{
"name": "execute copy for every record",
"type": "ForEach",
"dependsOn": [
{
"activity": "lookup ecdc filelist",
"dependencyConditions": [
"Succeeded"
]
}
],
"userProperties": [],
"typeProperties": {
"items": {
"value": "#activity('lookup ecdc filelist').output.value",
"type": "Expression"
},
"activities": [
{
"name": "Copy data1",
"type": "Copy",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "DelimitedTextSource",
"storeSettings": {
"type": "HttpReadSettings",
"requestMethod": "GET"
},
"formatSettings": {
"type": "DelimitedTextReadSettings"
}
},
"sink": {
"type": "DelimitedTextSink",
"storeSettings": {
"type": "AzureBlobFSWriteSettings"
},
"formatSettings": {
"type": "DelimitedTextWriteSettings",
"quoteAllText": true,
"fileExtension": ".txt"
}
},
"enableStaging": false,
"translator": {
"type": "TabularTranslator",
"typeConversion": true,
"typeConversionSettings": {
"allowDataTruncation": true,
"treatBooleanAsNumber": false
}
}
},
"inputs": [
{
"referenceName": "DelimitedText1",
"type": "DatasetReference",
"parameters": {
"sourceBaseURL": {
"value": "#item().sourceBaseURL",
"type": "Expression"
},
"sourceRelativeURL": {
"value": "#item().sourceRelativeURL",
"type": "Expression"
}
}
}
],
"outputs": [
{
"referenceName": "ds_ecdc_raw_csv_dl",
"type": "DatasetReference",
"parameters": {
"fileName": {
"value": "#item().sinkFileName",
"type": "Expression"
}
}
}
]
}
]
}
}
],
"concurrency": 1,
"annotations": []
}
}
I reproduced your error.
{"code":"BadRequest","message":null,"target":"pipeline//runid/abd35329-3625-490b-85cf-f6d0de3dac86","details":null,"error":null}
It is because you didn't pass your baseURL to link service in Source Dataset. Please do this:
And Dataset JSON code should be like this:
{
"name": "ds_ecdc_raw_csv_http",
"properties": {
"linkedServiceName": {
"referenceName": "ls_http_opendata_ecdc_europe_eu",
"type": "LinkedServiceReference",
"parameters": {
"baseURL": {
"value": "#dataset().baseURL",
"type": "Expression"
}
}
},
"parameters": {
"relativeURL": {
"type": "string"
},
"baseURL": {
"type": "string"
}
},
"annotations": [],
"type": "DelimitedText",
"typeProperties": {
"location": {
"type": "HttpServerLocation",
"relativeUrl": {
"value": "#dataset().relativeURL",
"type": "Expression"
}
},
"columnDelimiter": ",",
"escapeChar": "\\",
"firstRowAsHeader": true,
"quoteChar": "\""
},
"schema": []
}
}

Powershell Invoke-RestMethod in LogicApps

For this line of code in Powershell I used an HTTP connector in Logic Apps using Joey Cai's advice.
$body_login = #{"method"="login";"username"="qq";"password"="qqq"} | ConvertTo-Json
Now, I have this line of code in Powershell. How do I do the equivalent in LogicApps?
$Conn = Invoke-RestMethod -Method Post $uri_login -Headers $header -Body $body_login
Do I use the same HTTP connector or do I need something else? It's the Invoke-RestMethod syntax that I'm unsure of in Logic Apps.
I will need the output in JSON format, so I can parse it.
Thanks for the first answer. I need to know what to put in the uri, header and body. Here is the rest of the code which I should have provided before.
$baseuri = "https://test"
$header = #{
"Accept" = "text/json"
"Content-Type" = "text/json"
}
$G_header = #{"Accept" = "text/json"}
Write-Output "Login ..."
$uri_login = $baseuri + "SPDEDJSONSERVICE.LOGIN"
$body_login = #{"method"="login";"username"="qqq";"password"="qqq"} | ConvertTo-Json
$Conn = Invoke-RestMethod -Method Post $uri_login -Headers $header -Body $body_login
$SessionID = $conn.sessionID</code>
How do I do the equivalent in LogicApps?
As I have provided before, use HTTP connector.
I will need the output in JSON format, so I can parse it.
You could use Compose to work with data in JSON format.
1.Add Headers/Body which you want into Compose.
2.Add Outputs into Parse JSON. Copy the HTTP response Headers/Body info, and click use sample payload to generate schema, then parse Headers in it.
3.Use Initialize variable to get info what you want such as Date.
The result:
With Azure Logic Apps and the built-in HTTP action, you can create automated tasks and workflows that regularly send requests to any HTTP or HTTPS endpoint.
Sign in to the Azure portal. Open your logic app in Logic App Designer.
Under the step where you want to add the HTTP action, select New step.
To add an action between steps, move your pointer over the arrow between steps. Select the plus sign (+) that appears, and then select Add an action.
Under Choose an action, in the search box, enter "http" as your filter. From the Actions list, select the HTTP action.
Select HTTP action
For your scenarion you can use Basic Authentication.
This seems to work, but I could not have done it without Joey
<code>
{
"definition": {
"$schema":
"https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-
01/workflowdefinition.json#",
"actions": {
"HTTP_2": {
"inputs": {
"body": {
"FORMAT": "payload",
"FROM": 0,
"GRIDID": "PROP",
"GRIDVIEW": "1",
"HITS": 100,
"ORDERBY": "PR_DATESOLD",
"PROFILE": [
{
"PR_NAME": "G*",
"PR_USER1": "GENERATED"
}
],
"sessionID": "#body('Parse_JSON3')['sessionID']"
},
"headers": {
"Accept": "text/json",
"Content-Type": "text/json"
},
"method": "POST",
"uri": "#variables('uri_DefGrid')"
},
"runAfter": {
"Parse_JSON3": [
"Succeeded"
]
},
"type": "Http"
},
"Initialize_Header": {
"inputs": {
"variables": [
{
"name": "Header",
"type": "string",
"value": "{\"Accept\":\"text/json\",\"Content-
Type\":\"text/json\"}"
}
]
},
"runAfter": {
"Initialize_body_login": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Initialize_body_DefGrid": {
"inputs": {
"variables": [
{
"name": "body_DefGrid",
"type": "string",
"value": "json(#{body('HTTP_2')})"
}
]
},
"runAfter": {
"HTTP_2": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Initialize_body_login": {
"inputs": {
"variables": [
{
"name": "body_login",
"type": "string",
"value": "json(#{triggerBody()})"
}
]
},
"runAfter": {},
"type": "InitializeVariable"
},
"Initialize_uri_DefGrid": {
"inputs": {
"variables": [
{
"name": "uri_DefGrid",
"type": "string",
"value": "https://test/SPDEDMHAPI.GRIDGET"
}
]
},
"runAfter": {
"Initialize_uri_login": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Initialize_uri_login": {
"inputs": {
"variables": [
{
"name": "uri_login",
"type": "string",
"value": "https://test/SPDEDJSONSERVICE.LOGIN"
}
]
},
"runAfter": {
"Initialize_Header": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Initialize_uri_logout": {
"inputs": {
"variables": [
{
"name": "uri_logout",
"type": "string",
"value": "https://test/SPDEDJSONSERVICE.LOGOUT"
}
]
},
"runAfter": {
"Initialize_body_DefGrid": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Logout": {
"inputs": {
"body": {
"method": "logout",
"sessionID": "#body('Parse_JSON3')['sessionID']"
},
"headers": {
"Accept": "text/json",
"Content-Type": "text/json"
},
"method": "POST",
"uri": "#variables('uri_logout')"
},
"runAfter": {
"Initialize_uri_logout": [
"Succeeded"
]
},
"type": "Http"
},
"Parse_JSON3": {
"inputs": {
"content": "#triggerBody()",
"schema": {
"properties": {
"RLS_WHERE": {
"$id": "#/properties/RLS_WHERE",
"type": "string"
},
"contact": {
"type": "string"
},
"error": {
"type": "string"
},
"errorId": {
"type": "string"
},
"fullName": {
"type": "string"
},
"labellanguage": {
"type": "string"
},
"language": {
"type": "string"
},
"message": {
"type": "string"
},
"params": {
"properties": {
"WOPARTSOPT": {
"type": "string"
}
},
"required": [
"WOPARTSOPT"
],
"title": "The Params Schema",
"type": "object"
},
"role": {
"type": "string"
},
"sessionID": {
"type": "string"
},
"success": {
"type": "string"
},
"userEmail": {
"$id": "#/properties/userEmail",
"type": "string"
}
},
"required": [
"success",
"message",
"sessionID",
"language",
"labellanguage",
"error",
"errorId",
"fullName",
"role",
"contact",
"RLS_WHERE",
"userEmail",
"params"
],
"title": "The Root Schema",
"type": "object"
}
},
"runAfter": {
"Initialize_uri_DefGrid": [
"Succeeded"
]
},
"type": "ParseJson"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {},
"triggers": {
"HTTP": {
"inputs": {
"body": {
"method": "login",
"password": "qqq",
"username": "qqq"
},
"headers": {
"Accept": "text/json",
"Content-Type": "text/json"
},
"method": "POST",
"uri": "https://test/SPDEDJSONSERVICE.LOGIN"
},
"recurrence": {
"frequency": "Minute",
"interval": 4
},
"type": "Http"
}
}
},
"parameters": {}
}
</code>

AWS CloudFormation/API Gateway gives 'Invalid Resource identifier specified'

I have been trying to use CloudFormation to deploy to API Gateway, however, I constantly run into the same issue with my method resources. The stack deployments keep failing with 'Invalid Resource identifier specified'.
Here is my method resource from my CloudFormation template:
"UsersPut": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"ResourceId": "UsersResource",
"RestApiId": "MyApi",
"ApiKeyRequired": true,
"AuthorizationType": "NONE",
"HttpMethod": "PUT",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["", ["arn:aws:apigateway:", {
"Ref": "AWS::Region"
}, ":lambda:path/2015-03-31/functions/", {
"Fn::GetAtt": ["MyLambdaFunc", "Arn"]
}, "/invocations"]]
}
},
"MethodResponses": [{
"StatusCode": 200
}]
}
}
Is anyone able to help me figure out why this keeps failing the stack deployment?
UPDATE: I forgot to mention that I had also tried using references to add the resource ID, that also gave me the same error:
"UsersPut": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"ResourceId": {
"Ref": "UsersResource"
},
"RestApiId": "MyApi",
"ApiKeyRequired": true,
"AuthorizationType": "NONE",
"HttpMethod": "PUT",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["", ["arn:aws:apigateway:", {
"Ref": "AWS::Region"
}, ":lambda:path/2015-03-31/functions/", {
"Fn::GetAtt": ["MyLambdaFunc", "Arn"]
}, "/invocations"]]
}
},
"MethodResponses": [{
"StatusCode": 200
}]
}
}
Here is the full CloudFormation template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"LambdaDynamoDBRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}]
},
"Path": "/",
"Policies": [{
"PolicyName": "DynamoReadWritePolicy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Sid": "1",
"Action": [
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:Query",
"dynamodb:Scan",
"dynamodb:UpdateItem"
],
"Effect": "Allow",
"Resource": "*"
}, {
"Sid": "2",
"Resource": "*",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Effect": "Allow"
}]
}
}]
}
},
"MyFirstLambdaFn": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": "myfirstlambdafn",
"S3Key": "lambda_handler.py.zip"
},
"Description": "",
"FunctionName": "MyFirstLambdaFn",
"Handler": "lambda_function.lambda_handler",
"MemorySize": 512,
"Role": {
"Fn::GetAtt": [
"LambdaDynamoDBRole",
"Arn"
]
},
"Runtime": "python2.7",
"Timeout": 3
},
"DependsOn": "LambdaDynamoDBRole"
},
"MySecondLambdaFn": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": "mysecondlambdafn",
"S3Key": "lambda_handler.py.zip"
},
"Description": "",
"FunctionName": "MySecondLambdaFn",
"Handler": "lambda_function.lambda_handler",
"MemorySize": 512,
"Role": {
"Fn::GetAtt": [
"LambdaDynamoDBRole",
"Arn"
]
},
"Runtime": "python2.7",
"Timeout": 3
},
"DependsOn": "LambdaDynamoDBRole"
},
"MyApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Name": "Project Test API",
"Description": "Project Test API",
"FailOnWarnings": true
}
},
"FirstUserPropertyModel": {
"Type": "AWS::ApiGateway::Model",
"Properties": {
"ContentType": "application/json",
"Name": "FirstUserPropertyModel",
"RestApiId": {
"Ref": "MyApi"
},
"Schema": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "FirstUserPropertyModel",
"type": "object",
"properties": {
"Email": {
"type": "string"
}
}
}
}
},
"SecondUserPropertyModel": {
"Type": "AWS::ApiGateway::Model",
"Properties": {
"ContentType": "application/json",
"Name": "SecondUserPropertyModel",
"RestApiId": {
"Ref": "MyApi"
},
"Schema": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "SecondUserPropertyModel",
"type": "object",
"properties": {
"Name": {
"type": "string"
}
}
}
}
},
"ErrorCfn": {
"Type": "AWS::ApiGateway::Model",
"Properties": {
"ContentType": "application/json",
"Name": "ErrorCfn",
"RestApiId": {
"Ref": "MyApi"
},
"Schema": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Error Schema",
"type": "object",
"properties": {
"message": {
"type": "string"
}
}
}
}
},
"UsersResource": {
"Type": "AWS::ApiGateway::Resource",
"Properties": {
"RestApiId": {
"Ref": "MyApi"
},
"ParentId": {
"Fn::GetAtt": ["MyApi", "RootResourceId"]
},
"PathPart": "users"
}
},
"UsersPost": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"ResourceId": {
"Ref": "UsersResource"
},
"RestApiId": "MyApi",
"ApiKeyRequired": true,
"AuthorizationType": "NONE",
"HttpMethod": "POST",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["", ["arn:aws:apigateway:", {
"Ref": "AWS::Region"
}, ":lambda:path/2015-03-31/functions/", {
"Fn::GetAtt": ["MyFirstLambdaFn", "Arn"]
}, "/invocations"]]
}
},
"MethodResponses": [{
"ResponseModels": {
"application/json": {
"Ref": "FirstUserPropertyModel"
}
},
"StatusCode": 200
}, {
"ResponseModels": {
"application/json": {
"Ref": "ErrorCfn"
}
},
"StatusCode": 404
}, {
"ResponseModels": {
"application/json": {
"Ref": "ErrorCfn"
}
},
"StatusCode": 500
}]
}
},
"UsersPut": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"ResourceId": {
"Ref": "UsersResource"
},
"RestApiId": "MyApi",
"ApiKeyRequired": true,
"AuthorizationType": "NONE",
"HttpMethod": "PUT",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["", ["arn:aws:apigateway:", {
"Ref": "AWS::Region"
}, ":lambda:path/2015-03-31/functions/", {
"Fn::GetAtt": ["MySecondLambdaFn", "Arn"]
}, "/invocations"]]
}
},
"MethodResponses": [{
"ResponseModels": {
"application/json": {
"Ref": "SecondUserPropertyModel"
}
},
"StatusCode": 200
}, {
"ResponseModels": {
"application/json": {
"Ref": "ErrorCfn"
}
},
"StatusCode": 404
}, {
"ResponseModels": {
"application/json": {
"Ref": "ErrorCfn"
}
},
"StatusCode": 500
}]
}
},
"RestApiDeployment": {
"Type": "AWS::ApiGateway::Deployment",
"Properties": {
"RestApiId": {
"Ref": "MyApi"
},
"StageName": "Prod"
},
"DependsOn": ["UsersPost", "UsersPut"]
}
},
"Description": "Project description"
}
ResourceId must be a reference to a cloudformation resource, not a simple string.
e.g.
ResourceId:
Ref: UsersResource
I figured that actually it was the RestApiId which needed to be a reference too:
"RestApiId": {
"Ref": "MyApi"
},
When you create an API resource(1), a default root resource(2) for the API is created for path /. In order to get the id for the MyApi resource(1) root resource(2) use:
"ResourceId": {
"Fn::GetAtt": [
"MyApi",
"RootResourceId"
]
}
(1) The stack resource
(2) The API resource
Try
{
"$ref": "https://apigateway.amazonaws.com/restapis/YOUR_API_GATEWAY_IT/models/YOUR_MODEL_NAME"
}
In addition, you can also reference another model schema defined in an external model file by setting that model's URL as the value of the $ref property: "$ref": "https://apigateway.amazonaws.com/restapis/{restapi_id}/models/{model_name}".
See here for more details

Service fabric, AD and client certificate security

service fabric cluster communication fail when using my domainname.com certificate. I'm getting this error using the portal and/or using ARM template:
Failed to communicate with the cluster (get cluster health: Client certificate required).
Template (variables were removed):
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"resources": [
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('supportLogStorageAccountName')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"kind": "Storage",
"sku": {
"name": "[parameters('supportLogStorageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('applicationDiagnosticsStorageAccountName')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"kind": "Storage",
"sku": {
"name": "[parameters('applicationDiagnosticsStorageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('vNetApiVersion')]",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('virtualNetworkName')]",
"location": "[parameters('computeLocation')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('addressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnet0Name')]",
"properties": {
"addressPrefix": "[parameters('subnet0Prefix')]"
}
}
]
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('publicIPApiVersion')]",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[concat(parameters('lbIPName'),'-','0')]",
"location": "[parameters('computeLocation')]",
"properties": {
"dnsSettings": {
"domainNameLabel": "[parameters('dnsName')]"
},
"publicIPAllocationMethod": "Dynamic"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('lbApiVersion')]",
"type": "Microsoft.Network/loadBalancers",
"name": "[concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'))]",
"location": "[parameters('computeLocation')]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/',concat(parameters('lbIPName'),'-','0'))]"
],
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerIPConfig",
"properties": {
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]"
}
}
}
],
"backendAddressPools": [
{
"name": "LoadBalancerBEAddressPool",
"properties": {}
}
],
"loadBalancingRules": [
{
"name": "LBRule",
"properties": {
"backendAddressPool": {
"id": "[variables('lbPoolID0')]"
},
"backendPort": "[parameters('nt0fabricTcpGatewayPort')]",
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPort": "[parameters('nt0fabricTcpGatewayPort')]",
"idleTimeoutInMinutes": "5",
"probe": {
"id": "[variables('lbProbeID0')]"
},
"protocol": "tcp"
}
},
{
"name": "LBHttpRule",
"properties": {
"backendAddressPool": {
"id": "[variables('lbPoolID0')]"
},
"backendPort": "[parameters('nt0fabricHttpGatewayPort')]",
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPort": "[parameters('nt0fabricHttpGatewayPort')]",
"idleTimeoutInMinutes": "5",
"probe": {
"id": "[variables('lbHttpProbeID0')]"
},
"protocol": "tcp"
}
}
],
"probes": [
{
"name": "FabricGatewayProbe",
"properties": {
"intervalInSeconds": 5,
"numberOfProbes": 2,
"port": "[parameters('nt0fabricTcpGatewayPort')]",
"protocol": "tcp"
}
},
{
"name": "FabricHttpGatewayProbe",
"properties": {
"intervalInSeconds": 5,
"numberOfProbes": 2,
"port": "[parameters('nt0fabricHttpGatewayPort')]",
"protocol": "tcp"
}
}
],
"inboundNatPools": [
{
"name": "LoadBalancerBEAddressNatPool",
"properties": {
"backendPort": "3389",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPortRangeEnd": "4500",
"frontendPortRangeStart": "3389",
"protocol": "tcp"
}
}
]
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[variables('uniqueStringArray0')[copyIndex()]]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"copy": {
"name": "storageLoop",
"count": 5
},
"kind": "Storage",
"sku": {
"name": "[parameters('storageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('vmssApiVersion')]",
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[parameters('vmNodeType0Name')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('virtualNetworkName'))]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[0])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[1])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[2])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[3])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[4])]",
"[concat('Microsoft.Network/loadBalancers/', concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name')))]",
"[concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName'))]",
"[concat('Microsoft.Storage/storageAccounts/', parameters('applicationDiagnosticsStorageAccountName'))]"
],
"properties": {
"overprovision": "[parameters('overProvision')]",
"upgradePolicy": {
"mode": "Automatic"
},
"virtualMachineProfile": {
"extensionProfile": {
"extensions": [
{
"name": "[concat(parameters('vmNodeType0Name'),'_ServiceFabricNode')]",
"properties": {
"type": "ServiceFabricNode",
"autoUpgradeMinorVersion": false,
"protectedSettings": {
"StorageAccountKey1": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('supportLogStorageAccountName')),'2015-05-01-preview').key1]",
"StorageAccountKey2": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('supportLogStorageAccountName')),'2015-05-01-preview').key2]"
},
"publisher": "Microsoft.Azure.ServiceFabric",
"settings": {
"clusterEndpoint": "[reference(parameters('clusterName')).clusterEndpoint]",
"nodeTypeRef": "[parameters('vmNodeType0Name')]",
"dataPath": "D:\\\\SvcFab",
"durabilityLevel": "Bronze",
"certificate": {
"thumbprint": "[parameters('certificateThumbprint')]",
"x509StoreName": "[parameters('certificateStoreValue')]"
}
},
"typeHandlerVersion": "1.0"
}
},
{
"name": "[concat('VMDiagnosticsVmExt','_vmNodeType0Name')]",
"properties": {
"type": "IaaSDiagnostics",
"autoUpgradeMinorVersion": true,
"protectedSettings": {
"storageAccountName": "[parameters('applicationDiagnosticsStorageAccountName')]",
"storageAccountKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('applicationDiagnosticsStorageAccountName')),'2015-05-01-preview').key1]",
"storageAccountEndPoint": "https://core.windows.net/"
},
"publisher": "Microsoft.Azure.Diagnostics",
"settings": {
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": "50000",
"EtwProviders": {
"EtwEventSourceProviderConfiguration": [
{
"provider": "Microsoft-ServiceFabric-Actors",
"scheduledTransferKeywordFilter": "1",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableActorEventTable"
}
},
{
"provider": "Microsoft-ServiceFabric-Services",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableServiceEventTable"
}
}
],
"EtwManifestProviderConfiguration": [
{
"provider": "cbd93bc2-71e5-4566-b3a7-595d8eeca6e8",
"scheduledTransferLogLevelFilter": "Information",
"scheduledTransferKeywordFilter": "4611686018427387904",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricSystemEventTable"
}
}
]
}
}
},
"StorageAccount": "[parameters('applicationDiagnosticsStorageAccountName')]"
},
"typeHandlerVersion": "1.5"
}
}
]
},
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[concat(parameters('nicName'), '-0')]",
"properties": {
"ipConfigurations": [
{
"name": "[concat(parameters('nicName'),'-',0)]",
"properties": {
"loadBalancerBackendAddressPools": [
{
"id": "[variables('lbPoolID0')]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[variables('lbNatPoolID0')]"
}
],
"subnet": {
"id": "[variables('subnet0Ref')]"
}
}
}
],
"primary": true
}
}
]
},
"osProfile": {
"adminPassword": "[parameters('adminPassword')]",
"adminUsername": "[parameters('adminUsername')]",
"computernamePrefix": "[parameters('vmNodeType0Name')]",
"secrets": [
{
"sourceVault": {
"id": "[parameters('sourceVaultValue')]"
},
"vaultCertificates": [
{
"certificateStore": "[parameters('certificateStoreValue')]",
"certificateUrl": "[parameters('certificateUrlValue')]"
}
]
}
]
},
"storageProfile": {
"imageReference": {
"publisher": "[parameters('vmImagePublisher')]",
"offer": "[parameters('vmImageOffer')]",
"sku": "[parameters('vmImageSku')]",
"version": "[parameters('vmImageVersion')]"
},
"osDisk": {
"vhdContainers": [
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[0]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[1]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[2]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[3]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[4]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]"
],
"name": "vmssosdisk",
"caching": "ReadOnly",
"createOption": "FromImage"
}
}
}
},
"sku": {
"name": "[parameters('vmNodeType0Size')]",
"capacity": "[parameters('nt0InstanceCount')]",
"tier": "Standard"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "2016-09-01",
"type": "Microsoft.ServiceFabric/clusters",
"name": "[parameters('clusterName')]",
"location": "[parameters('clusterLocation')]",
"dependsOn": [
"[concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName'))]"
],
"properties": {
"azureActiveDirectory": {
"clientApplication": "55dae335-8436-4900-8f33-abbff16e8036",
"clusterApplication": "d0392358-9fa2-4f9a-aa2a-101859e31c34",
"tenantId": "c8656f45-daf5-42c1-9b29-ac27d3e63bf3"
},
"certificate": {
"thumbprint": "[parameters('certificateThumbprint')]",
"x509StoreName": "[parameters('certificateStoreValue')]"
},
"clientCertificateCommonNames": [],
"clientCertificateThumbprints": [
{
"certificateThumbprint": "C5EFB021F5D8BA8966B43B523B2A6BF8EE8202C5",
"isAdmin": true
},
{
"certificateThumbprint": "C5EFB021F5D8BA8966B43B523B2A6BF8EE8202C5",
"isAdmin": false
}
],
"clusterState": "Default",
"diagnosticsStorageAccountConfig": {
"blobEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.blob]",
"protectedAccountKeyName": "StorageAccountKey1",
"queueEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.queue]",
"storageAccountName": "[parameters('supportLogStorageAccountName')]",
"tableEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.table]"
},
"fabricSettings": [
{
"parameters": [
{
"name": "ClusterProtectionLevel",
"value": "[parameters('clusterProtectionLevel')]"
}
],
"name": "Security"
}
],
"managementEndpoint": "[concat('https://',reference(concat(parameters('lbIPName'),'-','0')).dnsSettings.fqdn,':',parameters('nt0fabricHttpGatewayPort'))]",
"nodeTypes": [
{
"name": "[parameters('vmNodeType0Name')]",
"applicationPorts": {
"endPort": "[parameters('nt0applicationEndPort')]",
"startPort": "[parameters('nt0applicationStartPort')]"
},
"clientConnectionEndpointPort": "[parameters('nt0fabricTcpGatewayPort')]",
"durabilityLevel": "Bronze",
"ephemeralPorts": {
"endPort": "[parameters('nt0ephemeralEndPort')]",
"startPort": "[parameters('nt0ephemeralStartPort')]"
},
"httpGatewayEndpointPort": "[parameters('nt0fabricHttpGatewayPort')]",
"isPrimary": true,
"vmInstanceCount": "[parameters('nt0InstanceCount')]"
}
],
"provisioningState": "Default",
"security": {
"metadata": "The Credential type X509 indicates this is cluster is secured using X509 Certificates.",
"ClusterCredentialType": "X509",
"ServerCredentialType": "X509",
"CertificateInformation": {
"ClusterCertificate": {
"Thumbprint": "[parameters('certificateThumbprint')]",
"X509StoreName": "My"
},
"ServerCertificate": {
"Thumbprint": "[parameters('certificateThumbprint')]",
"X509StoreName": "My"
},
"ClientCertificateThumbprints": [
{
"CertificateThumbprint": "[parameters('certificateThumbprint')]",
"IsAdmin": false
},
{
"CertificateThumbprint": "[parameters('certificateThumbprint')]",
"IsAdmin": true
}
]
}
},
"reliabilityLevel": "Bronze",
"upgradeMode": "Automatic",
"vmImage": "Windows"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
}
],
"outputs": {
"clusterProperties": {
"value": "[reference(parameters('clusterName'))]",
"type": "object"
}
}
}
hint?
It might be as simple as the client certificate is not imported (installed) in the machine from where you are trying to access the explorer. When you open the Service Fabric explorer, it might pop up to select one of the certificates installed in your machine. You have to choose the client certificate at this point.
I compared the ARM template with my (working) template and it looks OK. No obvious issues.
Your ARM template shows you are missing the client certificate within the VMSS OS Profile, which means it will not be installed to each of the nodes during deployment then it is missing when Service Fabric attempts to use it.
Here is an example from one of my ARM templates
"osProfile": {
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]",
"computernamePrefix": "[parameters('vmNodeType0Name')]",
"secrets": [
{
"sourceVault": {
"id": "[parameters('sourceVault')]"
},
"vaultCertificates": [
{
"certificateStore": "My",
"certificateUrl": "[parameters('clusterCertificateUrl')]"
},
{
"certificateStore": "My",
"certificateUrl": "[parameters('adminCertificateUrl')]"
}
]
}
]
},
EDIT:
Where you have two client certificates, I would expect your OS Profile to have three vault certificates for installation on deployment