Cloudformation Template validation error? - aws-cloudformation

I am receiving the follow error but do not know where it is coming from, hopefully someone can help.
Template validation error: Template format error: Any Properties member must be a JSON object.
The cloudformation script
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "The AWS CloudFormation template for this Serverless application's resources outside of Lambdas and Api Gateway",
"Resources": {
"KMSKey": {
"Type": "AWS::KMS::Key",
"Properties": {
"Description": "KMS Key Dev",
"Enabled": "True",
"EnableKeyRotation": "True",
"KeyPolicy": {
"Version": "2012-10-17",
"Id": "key-default-1",
"Statement": {
"Effect": "Allow",
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
}
}
}
},
"IamRoleLambda": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "*"
}
},
"IamPolicyLambda": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "dev-lambda",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:us-east-1:*"
},
{
"Effect": "Allow",
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": {
"Ref": "KMSKey"
}
},
{
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:DetachNetworkInterface",
"ec2:DeleteNetworkInterface",
"elastiCache:*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::{domain.com}/",
"arn:aws:s3:::{domain.com}/Serverless/*"
]
}
]
},
"Roles": [
{
"Ref": "IamRoleLambda"
}
]
}
},
"RedisCluster": {
"Type": "AWS::ElastiCache::CacheCluster",
"Properties": {
"AutoMinorVersionUpgrade": "False",
"AZMode": "cross-az",
"CacheNodeType": "cache.m3.medium",
"VpcSecurityGroupIds": {
"Ref": "VpcSecurityGroup"
},
"ClusterName": "Dev",
"Engine": "redis",
"EngineVersion": "2.8",
"NumCacheNodes": "1",
"Tags": [
{
"Key": "CostCenter",
"Value": "0000000000000000"
},
{
"Key": "Application",
"Value": "Appname"
},
{
"Key": "Function",
"Value": "cache"
},
{
"Key": "Environment",
"Value": "dev"
}
]
}
},
"VpcSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "DEV VPC Security group form",
"SecurityGroupEgress": {
"Ref": "SecurityGroupEgress"
},
"SecurityGroupInress": {
"Ref": "SecurityGroupIngress"
},
"Tags": [
{
"Key": "CostCenter",
"Value": "0000000000000000"
},
{
"Key": "Application",
"Value": "Appname"
},
{
"Key": "Function",
"Value": "cache"
},
{
"Key": "Environment",
"Value": "dev"
}
],
"VpcId": "vpc-8c3113e2"
}
},
"SecurityGroupEgress": {
"Type": "AWS::EC2::SecurityGroupEgress",
"Properties": [
{
"CidrIp": "0.0.0.0/0",
"FromPort": "443",
"ToPort": "443",
"IpProtocol": "tcp"
},
{
"CidrIp": "0.0.0.0/0",
"FromPort": "80",
"ToPort": "80",
"IpProtocol": "tcp"
},
{
"DestinationSecurityGroupId": {
"Fn:GetAtt": [
"VpcSecurityGroup"
]
},
"FromPort": "6379",
"ToPort": "6379",
"IpProtocol": "tcp"
}
]
},
"SecurityGroupIngress": {
"Type": "AWS::EC2::SecurityGroupIngress",
"Properties": [
{
"CidrIp": "0.0.0.0/0",
"FromPort": "443",
"ToPort": "443",
"IpProtocol": "tcp"
},
{
"CidrIp": "0.0.0.0/0",
"FromPort": "80",
"ToPort": "80",
"IpProtocol": "tcp"
},
{
"DestinationSecurityGroupId": {
"Fn:GetAtt": [
"VpcSecurityGroup"
]
},
"FromPort": "6379",
"ToPort": "6379",
"IpProtocol": "tcp"
}
]
}
},
"Outputs": {
"IamRoleArnLambda": {
"Description": "ARN of the lambda IAM role",
"Value": {
"Fn::GetAtt": [
"IamRoleLambda",
"Arn"
]
}
}
}
}

The error describes that the Properties member requires a JSON object. Looking at your code, two of your Properties members are defined as JSON arrays. The underlying issue is that AWS::EC2::SecurityGroupIngress and AWS::EC2::SecurityGroupEgress resources define a single security group rule, whereas you're attempting to define multiple rules within a single resource.
I see that you're trying to self-reference the security group for a custom port (6379), in addition to fully-open ports 80 and 443. As noted in the documentation, this is the correct use-case for using AWS::EC2::SecurityGroupEgress and AWS::EC2::SecurityGroupIngress resources:
Important
If you want to cross-reference two security groups in the ingress and egress rules of those security groups, use the AWS::EC2::SecurityGroupEgress and AWS::EC2::SecurityGroupIngress resources to define your rules. Do not use the embedded ingress and egress rules in the AWS::EC2::SecurityGroup. If you do, it causes a circular dependency, which AWS CloudFormation doesn't allow.
To accomplish this without defining several additional resources, you can leave the fully-open port rules inline with the resource (since they don't cause a circular dependency), and just create additional AWS::EC2::SecurityGroup[In|E]gress resources for the ones you want locked down to the security group:
"VpcSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "DEV VPC Security group form",
"SecurityGroupEgress": [
{
"CidrIp": "0.0.0.0/0",
"FromPort": "443",
"ToPort": "443",
"IpProtocol": "tcp"
},
{
"CidrIp": "0.0.0.0/0",
"FromPort": "80",
"ToPort": "80",
"IpProtocol": "tcp"
}
],
"SecurityGroupIngress": [
{
"CidrIp": "0.0.0.0/0",
"FromPort": "443",
"ToPort": "443",
"IpProtocol": "tcp"
},
{
"CidrIp": "0.0.0.0/0",
"FromPort": "80",
"ToPort": "80",
"IpProtocol": "tcp"
}
],
"Tags": [
{
"Key": "CostCenter",
"Value": "0000000000000000"
},
{
"Key": "Application",
"Value": "Appname"
},
{
"Key": "Function",
"Value": "cache"
},
{
"Key": "Environment",
"Value": "dev"
}
],
"VpcId": "vpc-8c3113e2"
}
},
"SecurityGroupEgress": {
"Type": "AWS::EC2::SecurityGroupEgress",
"Properties": {
"DestinationSecurityGroupId": {
"Fn::GetAtt": [
"VpcSecurityGroup",
"GroupId"
]
},
"FromPort": "6379",
"ToPort": "6379",
"IpProtocol": "tcp"
}
},
"SecurityGroupIngress": {
"Type": "AWS::EC2::SecurityGroupIngress",
"Properties": {
"SourceSecurityGroupId": {
"Fn::GetAtt": [
"VpcSecurityGroup",
"GroupId"
]
},
"FromPort": "6379",
"ToPort": "6379",
"IpProtocol": "tcp"
}
}
[...etc...]
You'll also want to make sure to be careful with the spelling and syntax of the Fn::GetAtt function, which I've corrected above for you.

Related

destinationArn for vendor firehose cannot be used with roleArn when creating AWS::Logs::SubscriptionFilter via CloudFormation

My CloudFormation template that fails to create AWS::Logs::SubscriptionFilter resource:
{
"Resources": {
"Bucket83908E77": {
"Type": "AWS::S3::Bucket",
"UpdateReplacePolicy": "Delete",
"DeletionPolicy": "Delete"
},
"MyFirehoseServiceRoleFD019CCC": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "firehose.amazonaws.com"
}
}
],
"Version": "2012-10-17"
}
}
},
"MyFirehoseS3DestinationRoleDE043A9B": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "firehose.amazonaws.com"
}
}
],
"Version": "2012-10-17"
}
}
},
"MyFirehoseS3DestinationRoleDefaultPolicyF2D4C970": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:GetObject*",
"s3:GetBucket*",
"s3:List*",
"s3:DeleteObject*",
"s3:PutObject",
"s3:Abort*"
],
"Effect": "Allow",
"Resource": [
{
"Fn::GetAtt": [
"Bucket83908E77",
"Arn"
]
},
{
"Fn::Join": [
"",
[
{
"Fn::GetAtt": [
"Bucket83908E77",
"Arn"
]
},
"/*"
]
]
}
]
},
{
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Effect": "Allow",
"Resource": {
"Fn::GetAtt": [
"MyFirehoseLogGroupE92127AD",
"Arn"
]
}
}
],
"Version": "2012-10-17"
},
"PolicyName": "MyFirehoseS3DestinationRoleDefaultPolicyF2D4C970",
"Roles": [
{
"Ref": "MyFirehoseS3DestinationRoleDE043A9B"
}
]
}
},
"MyFirehoseLogGroupE92127AD": {
"Type": "AWS::Logs::LogGroup",
"Properties": {
"RetentionInDays": 731
},
"UpdateReplacePolicy": "Retain",
"DeletionPolicy": "Retain"
},
"MyFirehoseLogGroupS3Destination06C9B080": {
"Type": "AWS::Logs::LogStream",
"Properties": {
"LogGroupName": {
"Ref": "MyFirehoseLogGroupE92127AD"
}
},
"UpdateReplacePolicy": "Retain",
"DeletionPolicy": "Retain"
},
"MyFirehoseFCA2F9D3": {
"Type": "AWS::KinesisFirehose::DeliveryStream",
"Properties": {
"DeliveryStreamType": "DirectPut",
"ExtendedS3DestinationConfiguration": {
"BucketARN": {
"Fn::GetAtt": [
"Bucket83908E77",
"Arn"
]
},
"CloudWatchLoggingOptions": {
"Enabled": true,
"LogGroupName": {
"Ref": "MyFirehoseLogGroupE92127AD"
},
"LogStreamName": {
"Ref": "MyFirehoseLogGroupS3Destination06C9B080"
}
},
"RoleARN": {
"Fn::GetAtt": [
"MyFirehoseS3DestinationRoleDE043A9B",
"Arn"
]
}
}
},
"DependsOn": [
"MyFirehoseS3DestinationRoleDefaultPolicyF2D4C970"
]
},
"MyFirehoseCloudWatchLogsCanPutRecordsIntoKinesisFirehose30DECEBA": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": {
"Fn::Join": [
"",
[
"logs.",
{
"Ref": "AWS::Region"
},
".amazonaws.com"
]
]
}
}
}
],
"Version": "2012-10-17"
}
}
},
"MyFirehoseCloudWatchLogsCanPutRecordsIntoKinesisFirehoseDefaultPolicyF5730531": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyDocument": {
"Statement": [
{
"Action": [
"firehose:PutRecord",
"firehose:PutRecordBatch"
],
"Effect": "Allow",
"Resource": {
"Fn::GetAtt": [
"MyFirehoseFCA2F9D3",
"Arn"
]
}
}
],
"Version": "2012-10-17"
},
"PolicyName": "MyFirehoseCloudWatchLogsCanPutRecordsIntoKinesisFirehoseDefaultPolicyF5730531",
"Roles": [
{
"Ref": "MyFirehoseCloudWatchLogsCanPutRecordsIntoKinesisFirehose30DECEBA"
}
]
}
},
"LogGroupF5B46931": {
"Type": "AWS::Logs::LogGroup",
"Properties": {
"RetentionInDays": 731
},
"UpdateReplacePolicy": "Delete",
"DeletionPolicy": "Delete"
},
"Subscription391C9821": {
"Type": "AWS::Logs::SubscriptionFilter",
"Properties": {
"DestinationArn": {
"Fn::GetAtt": [
"MyFirehoseFCA2F9D3",
"Arn"
]
},
"FilterPattern": "",
"LogGroupName": {
"Ref": "LogGroupF5B46931"
},
"RoleArn": {
"Fn::GetAtt": [
"MyFirehoseCloudWatchLogsCanPutRecordsIntoKinesisFirehose30DECEBA",
"Arn"
]
}
}
}
}
}
Cryptic error message:
Subscription (Subscription391C9821) destinationArn for vendor firehose cannot be used with roleArn (Service: AWSLogs; Status Code: 400; Error Code: InvalidParameterException; Request ID: 0e598426-5fcb-4fde-b9d3-11b14c129eb6; Proxy: null)
Stack name is cdk-logs-destination-firehose-to-s3.
Apparently, there is a bug in CloudWatch Logs where destination ARNs that contain the string destination are rejected from creating a subscription.
Workaround is to remove the destination substring from the stack name.

Error: Error response from daemon: invalid volume specification - Windows 8.1 Docker Toolbox

Error: Error response from daemon: invalid volume specification: 'C:/Users/Anthony/magento2-devbox:/C:/Users/Anthony/magento2-devbox'
I have had a google around on this but I cant see how this path has been assembled. Most paths exclude the : and i am also not sure why it has assembled this :/C:/ or whether this is just produced for the error message.
This is the YAML for the replica set;
{
"kind": "ReplicaSet",
"apiVersion": "extensions/v1beta1",
"metadata": {
"name": "magento2-monolith-54cdd5b4b7",
"namespace": "default",
"selfLink": "/apis/extensions/v1beta1/namespaces/default/replicasets/magento2-monolith-54cdd5b4b7",
"uid": "e819bfbd-8820-11e9-a613-080027316036",
"resourceVersion": "22855",
"generation": 1,
"creationTimestamp": "2019-06-06T06:04:12Z",
"labels": {
"app.kubernetes.io/instance": "magento2",
"app.kubernetes.io/name": "monolith",
"pod-template-hash": "54cdd5b4b7"
},
"annotations": {
"deployment.kubernetes.io/desired-replicas": "1",
"deployment.kubernetes.io/max-replicas": "1",
"deployment.kubernetes.io/revision": "1"
},
"ownerReferences": [
{
"apiVersion": "apps/v1",
"kind": "Deployment",
"name": "magento2-monolith",
"uid": "9ec9d23e-8691-11e9-a3dd-080027316036",
"controller": true,
"blockOwnerDeletion": true
}
]
},
"spec": {
"replicas": 1,
"selector": {
"matchLabels": {
"app.kubernetes.io/instance": "magento2",
"app.kubernetes.io/name": "monolith",
"pod-template-hash": "54cdd5b4b7"
}
},
"template": {
"metadata": {
"creationTimestamp": null,
"labels": {
"app.kubernetes.io/instance": "magento2",
"app.kubernetes.io/name": "monolith",
"pod-template-hash": "54cdd5b4b7"
}
},
"spec": {
"volumes": [
{
"name": "nginx-config-volume",
"configMap": {
"name": "magento2-monolith-nginx-config",
"defaultMode": 420
}
},
{
"name": "varnish-config-volume",
"configMap": {
"name": "magento2-monolith-varnish-config",
"defaultMode": 420
}
},
{
"name": "code",
"hostPath": {
"path": "C:/Users/Anthony/magento2-devbox",
"type": ""
}
}
],
"containers": [
{
"name": "monolith",
"image": "magento2-monolith:dev",
"ports": [
{
"containerPort": 8050,
"protocol": "TCP"
}
],
"env": [
{
"name": "DEVBOX_ROOT",
"value": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "COMPOSER_HOME",
"value": "C:/Users/Anthony/magento2-devbox/.composer"
},
{
"name": "MAGENTO_ROOT",
"value": "C:/Users/Anthony/magento2-devbox/magento"
},
{
"name": "MAGENTO_ROOT_HOST",
"value": "C:/Users/Anthony/magento2-devbox/magento"
},
{
"name": "DEVBOX_ROOT_HOST",
"value": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "IS_WINDOWS_HOST",
"value": "0"
}
],
"resources": {},
"volumeMounts": [
{
"name": "code",
"mountPath": "C:/Users/Anthony/magento2-devbox"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Never",
"securityContext": {
"privileged": true,
"procMount": "Default"
}
},
{
"name": "monolith-xdebug",
"image": "magento2-monolith:dev-xdebug",
"ports": [
{
"containerPort": 8002,
"protocol": "TCP"
}
],
"env": [
{
"name": "DEVBOX_ROOT",
"value": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "COMPOSER_HOME",
"value": "C:/Users/Anthony/magento2-devbox/.composer"
},
{
"name": "MAGENTO_ROOT",
"value": "C:/Users/Anthony/magento2-devbox/magento"
},
{
"name": "MAGENTO_ROOT_HOST",
"value": "C:/Users/Anthony/magento2-devbox/magento"
},
{
"name": "DEVBOX_ROOT_HOST",
"value": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "IS_WINDOWS_HOST",
"value": "0"
}
],
"resources": {},
"volumeMounts": [
{
"name": "code",
"mountPath": "C:/Users/Anthony/magento2-devbox"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Never",
"securityContext": {
"privileged": true,
"procMount": "Default"
}
},
{
"name": "nginx",
"image": "nginx:1.9",
"resources": {},
"volumeMounts": [
{
"name": "code",
"mountPath": "C:/Users/Anthony/magento2-devbox"
},
{
"name": "nginx-config-volume",
"mountPath": "/etc/nginx/nginx.conf",
"subPath": "nginx.conf"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "IfNotPresent",
"securityContext": {
"privileged": true,
"procMount": "Default"
}
},
{
"name": "varnish",
"image": "million12/varnish",
"env": [
{
"name": "VCL_CONFIG",
"value": "/etc/varnish/magento.vcl"
},
{
"name": "VARNISHD_PARAMS",
"value": "-a 0.0.0.0:6081"
}
],
"resources": {},
"volumeMounts": [
{
"name": "varnish-config-volume",
"mountPath": "/etc/varnish/magento.vcl",
"subPath": "varnish.vcl"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Always"
}
],
"restartPolicy": "Always",
"terminationGracePeriodSeconds": 30,
"dnsPolicy": "ClusterFirst",
"securityContext": {},
"schedulerName": "default-scheduler"
}
}
},
"status": {
"replicas": 1,
"fullyLabeledReplicas": 1,
"observedGeneration": 1
}
}
I am new to Docker/Kubernetes after coming over from Vagrant so i do not know where to start. The information i have is from the web browser dashboard
Path is probably not converted to unix style. Since docker 1.9.0, Windows paths are not automatically converted (eg. C:\Users to /c/Users).
So your path should be like :
{
"name": "DEVBOX_ROOT",
"value": "/c/Users/Anthony/magento2-devbox"
}

Running two containers in a single pod via kubectl run does not terminate the pod with two containers

Trying to run a two docker containers in one kubernetes pod. The problem we face is that we have 2 images/containers in a Kubernetes pod whereas we used to have 1 image for this pod previously where it was working perfectly and shutting down the pod gracefully. Now the problem is that with 2 images or containers within the same pod both images are not shutting down at the same time gracefully.
We are doing integration testing of our container against the database which is MongoDB, our container terminates but MongoDB fails to terminate here and continues running
while getopts n:c:l:i:d:t: option
do
case "${option}"
in
n) Company_NAMESPACE=$OPTARG;;
c) Company_CONFIG=$OPTARG;;
l) Company_LICENSE=$OPTARG;;
i) Company_IMAGE=$OPTARG;;
d) Company_CONTAINER_NAME=$OPTARG;;
t) Company_TOKEN=$OPTARG;;
esac
done
kubectl run $Company_CONTAINER_NAME -n $Company_NAMESPACE --restart=Never --overrides='
{
"apiVersion": "v1",
"spec": {
"imagePullSecrets": [
{
"name": "Company-regsecret"
}
],
"initContainers": [
{
"name": "copy-configs",
"image": "busybox",
"command": ["sh", "-c", "cp /tmp/Company-config-volume/server/* /tmp/ng-rt/config/server/ 2>/dev/null || true; cp /tmp/Company-license-volume/licenses/* /tmp/ng-rt/config/licenses 2>/dev/null || true"],
"volumeMounts": [
{
"name": "Company-config-volume",
"mountPath": "mount_path"
},
{
"name": "'$Company_CONFIG'",
"mountPath": "mount_path"
},
{
"name": "Company-license-volume",
"mountPath": "mount_path"
},
{
"name": "'$Company_LICENSE'",
"mountPath": "mount_path"
}
]
}
],
"containers": [
{
"name": "mongodb-test",
"image": "mongo:3.6.8",
"command": [
"numactl",
"--interleave=all",
"mongod",
"--wiredTigerCacheSizeGB",
"0.1",
"--replSet",
"MainRepSet",
"--bind_ip_all"
],
"ports": [{
"containerPort": 27017
}],
"readinessProbe": {
"exec": {
"command": ["mongo", "--eval", "rs.initiate()"]
}
},
"terminationGracePeriodSeconds": 10
},
{
"env": [
{
"name": "AWS_ACCESS_KEY_ID",
"valueFrom": {
"secretKeyRef": {
"key": "AWS_ACCESS_KEY_ID",
"name": "aws-secrets"
}
}
},
{
"name": "AWS_SECRET_ACCESS_KEY",
"valueFrom": {
"secretKeyRef": {
"key": "AWS_SECRET_ACCESS_KEY",
"name": "aws-secrets"
}
}
},
{
"name": "AWS_REGION",
"valueFrom": {
"secretKeyRef": {
"key": "AWS_REGION",
"name": "aws-secrets"
}
}
},
{
"name": "BUILD_ID",
"valueFrom": {
"configMapKeyRef": {
"key": "BUILD_ID",
"name": "config"
}
}
}
],
"command": [
"sh",
"-c",
"mkdir -p mount_path 2\u003e/dev/null && npm test --skipConnectivityTestRethinkDB"
],
"name": "'$Company_CONTAINER_NAME'",
"image": "'$Company_IMAGE'",
"volumeMounts": [
{
"mountPath": "mount_path",
"name": "'$Company_CONFIG'"
},
{
"mountPath": "mount_path",
"name": "'$Company_LICENSE'"
}
]
}
],
"volumes": [
{
"name": "Company-config-volume",
"configMap": {
"name": "'$Company_CONFIG'"
}
},
{
"name": "'$Company_CONFIG'",
"emptyDir": {}
},
{
"name": "Company-license-volume",
"configMap": {
"name": "'$Company_LICENSE'"
}
},
{
"name": "'$Company_LICENSE'",
"emptyDir": {}
}
]
}
}
' --image=$Company_IMAGE -ti --rm --token=$Company_TOKEN
{
"kind": "Pod",
"apiVersion": "v1",
"metadata": {
"name": "test-dev-58663",
"namespace": "d-int-company-dev-v2",
"labels": {
"run": "test-dev-58663"
},
"annotations": {
"kubernetes.io/psp": "nfs-provisioner"
}
},
"spec": {
"volumes": [
{
"name": "company-config-volume",
"configMap": {
"name": "test-core",
"defaultMode": 420
}
},
{
"name": "test-core",
"emptyDir": {}
},
{
"name": "company-license-volume",
"configMap": {
"name": "company-license",
"defaultMode": 420
}
},
{
"name": "company-license",
"emptyDir": {}
},
{
"name": "default-token-wqp5x",
"secret": {
"secretName": "default-token-wqp5x",
"defaultMode": 420
}
}
],
"initContainers": [
{
"name": "copy-configs",
"image": "busybox",
"command": [
"sh",
"-c",
"cp mount_path* mount_path 2>/dev/null || true; cp mount_path* mount_path 2>mount_path|| true"
],
"resources": {},
"volumeMounts": [
{
"name": "company-config-volume",
"mountPath": "mount_path"
},
{
"name": "test-core",
"mountPath": "mount_path"
},
{
"name": "company-license-volume",
"mountPath": "mount_path"
},
{
"name": "company-license",
"mountPath": "mount_path"
},
{
"name": "default-token-wqp5x",
"readOnly": true,
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "Always"
}
],
"containers": [
{
"name": "mongodb-test",
"image": "mongo:3.6.8",
"ports": [
{
"containerPort": 27017,
"protocol": "TCP"
}
],
"resources": {},
"volumeMounts": [
{
"name": "default-token-wqp5x",
"readOnly": true,
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
}
],
"readinessProbe": {
"exec": {
"command": [
"mongo",
"--eval",
"rs.initiate()"
]
},
"timeoutSeconds": 1,
"periodSeconds": 10,
"successThreshold": 1,
"failureThreshold": 3
},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "IfNotPresent"
},
{
"name": "test-dev-58663",
"image": "image_path",
"command": [
"sh",
"-c",
"mkdir -p mount_path 2>/dev/null && npm test --skipConnectivityTestRethinkDB"
],
"env": [
{
"name": "AWS_ACCESS_KEY_ID",
"valueFrom": {
"secretKeyRef": {
"name": "aws-secrets",
"key": "AWS_ACCESS_KEY_ID"
}
}
},
{
"name": "AWS_SECRET_ACCESS_KEY",
"valueFrom": {
"secretKeyRef": {
"name": "aws-secrets",
"key": "AWS_SECRET_ACCESS_KEY"
}
}
},
{
"name": "AWS_REGION",
"valueFrom": {
"secretKeyRef": {
"name": "aws-secrets",
"key": "AWS_REGION"
}
}
},
{
"name": "BUILD_ID",
"valueFrom": {
"configMapKeyRef": {
"name": "tbsp-config",
"key": "BUILD_ID"
}
}
}
],
"resources": {},
"volumeMounts": [
{
"name": "test-core",
"mountPath": "mount_path"
},
{
"name": "company-license",
"mountPath": "mount_path"
},
{
"name": "default-token-wqp5x",
"readOnly": true,
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
}
],
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"imagePullPolicy": "IfNotPresent"
}
],
"restartPolicy": "Never",
"terminationGracePeriodSeconds": 10,
"dnsPolicy": "ClusterFirst",
"serviceAccountName": "default",
"serviceAccount": "default",
"nodeName": "pc212",
"securityContext": {},
"imagePullSecrets": [
{
"name": "company-regsecret"
}
],
"schedulerName": "default-scheduler",
"tolerations": [
{
"key": "node.kubernetes.io/not-ready",
"operator": "Exists",
"effect": "NoExecute",
"tolerationSeconds": 300
},
{
"key": "node.kubernetes.io/unreachable",
"operator": "Exists",
"effect": "NoExecute",
"tolerationSeconds": 300
}
]
},
"status": {
"phase": "Pending",
"conditions": [
{
"type": "Initialized",
"status": "True",
"lastProbeTime": null,
"lastTransitionTime": "2019-05-10T12:50:39Z"
},
{
"type": "Ready",
"status": "False",
"lastProbeTime": null,
"lastTransitionTime": "2019-05-10T12:49:54Z",
"reason": "ContainersNotReady",
"message": "containers with unready status: [mongodb-test test-dev-58663]"
},
{
"type": "PodScheduled",
"status": "True",
"lastProbeTime": null,
"lastTransitionTime": "2019-05-10T12:49:54Z"
}
],
"hostIP": "10.10.2.12",
"podIP": "172.16.4.22",
"startTime": "2019-05-10T12:49:54Z",
"initContainerStatuses": [
{
"name": "copy-configs",
"state": {
"terminated": {
"exitCode": 0,
"reason": "Completed",
"startedAt": "2019-05-10T12:50:39Z",
"finishedAt": "2019-05-10T12:50:39Z",
"containerID": "docker://1bcd12f5848e32e82f7dfde8e245223345e87f70061b789cbbabc0f798436b59"
}
},
"lastState": {},
"ready": true,
"restartCount": 0,
"image": "busybox:latest",
"imageID": "docker-pullable://busybox#sha256:0b184b74edc63924be0d7f67d16f5afbcdbe61caa1aca9312ed3b5c57792f6c1",
"containerID": "docker://1bcd12f5848e32e82f7dfde8e245223345e87f70061b789cbbabc0f798436b59"
}
],
"containerStatuses": [
{
"name": "mongodb-test",
"state": {
"waiting": {
"reason": "PodInitializing"
}
},
"lastState": {},
"ready": false,
"restartCount": 0,
"image": "mongo:3.6.8",
"imageID": ""
},
{
"name": "test-dev-58663",
"state": {
"waiting": {
"reason": "PodInitializing"
}
},
"lastState": {},
"ready": false,
"restartCount": 0,
"image": "image_path",
"imageID": ""
}
],
"qosClass": "BestEffort"
}
}
Both the containers and the hosting pod should terminate gracefully.
You have pod consisting of two containers, one of them is supposed to run indefinitely, another one runs to completion. This is bad practice. You should split your pod into two separate things: Pod with Mongo and Job with your integration script. You have to write logic that watches Job to finish and then terminates both Pod and Job. You can do it like this:
kubectl apply -f integration-test.yaml
kubectl wait --for=condition=Complete --timeout=5m job/test
kubectl delete -f integration-test.yaml

AWS CloudFormation/API Gateway gives 'Invalid Resource identifier specified'

I have been trying to use CloudFormation to deploy to API Gateway, however, I constantly run into the same issue with my method resources. The stack deployments keep failing with 'Invalid Resource identifier specified'.
Here is my method resource from my CloudFormation template:
"UsersPut": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"ResourceId": "UsersResource",
"RestApiId": "MyApi",
"ApiKeyRequired": true,
"AuthorizationType": "NONE",
"HttpMethod": "PUT",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["", ["arn:aws:apigateway:", {
"Ref": "AWS::Region"
}, ":lambda:path/2015-03-31/functions/", {
"Fn::GetAtt": ["MyLambdaFunc", "Arn"]
}, "/invocations"]]
}
},
"MethodResponses": [{
"StatusCode": 200
}]
}
}
Is anyone able to help me figure out why this keeps failing the stack deployment?
UPDATE: I forgot to mention that I had also tried using references to add the resource ID, that also gave me the same error:
"UsersPut": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"ResourceId": {
"Ref": "UsersResource"
},
"RestApiId": "MyApi",
"ApiKeyRequired": true,
"AuthorizationType": "NONE",
"HttpMethod": "PUT",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["", ["arn:aws:apigateway:", {
"Ref": "AWS::Region"
}, ":lambda:path/2015-03-31/functions/", {
"Fn::GetAtt": ["MyLambdaFunc", "Arn"]
}, "/invocations"]]
}
},
"MethodResponses": [{
"StatusCode": 200
}]
}
}
Here is the full CloudFormation template:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"LambdaDynamoDBRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}]
},
"Path": "/",
"Policies": [{
"PolicyName": "DynamoReadWritePolicy",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Sid": "1",
"Action": [
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:Query",
"dynamodb:Scan",
"dynamodb:UpdateItem"
],
"Effect": "Allow",
"Resource": "*"
}, {
"Sid": "2",
"Resource": "*",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Effect": "Allow"
}]
}
}]
}
},
"MyFirstLambdaFn": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": "myfirstlambdafn",
"S3Key": "lambda_handler.py.zip"
},
"Description": "",
"FunctionName": "MyFirstLambdaFn",
"Handler": "lambda_function.lambda_handler",
"MemorySize": 512,
"Role": {
"Fn::GetAtt": [
"LambdaDynamoDBRole",
"Arn"
]
},
"Runtime": "python2.7",
"Timeout": 3
},
"DependsOn": "LambdaDynamoDBRole"
},
"MySecondLambdaFn": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": "mysecondlambdafn",
"S3Key": "lambda_handler.py.zip"
},
"Description": "",
"FunctionName": "MySecondLambdaFn",
"Handler": "lambda_function.lambda_handler",
"MemorySize": 512,
"Role": {
"Fn::GetAtt": [
"LambdaDynamoDBRole",
"Arn"
]
},
"Runtime": "python2.7",
"Timeout": 3
},
"DependsOn": "LambdaDynamoDBRole"
},
"MyApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Name": "Project Test API",
"Description": "Project Test API",
"FailOnWarnings": true
}
},
"FirstUserPropertyModel": {
"Type": "AWS::ApiGateway::Model",
"Properties": {
"ContentType": "application/json",
"Name": "FirstUserPropertyModel",
"RestApiId": {
"Ref": "MyApi"
},
"Schema": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "FirstUserPropertyModel",
"type": "object",
"properties": {
"Email": {
"type": "string"
}
}
}
}
},
"SecondUserPropertyModel": {
"Type": "AWS::ApiGateway::Model",
"Properties": {
"ContentType": "application/json",
"Name": "SecondUserPropertyModel",
"RestApiId": {
"Ref": "MyApi"
},
"Schema": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "SecondUserPropertyModel",
"type": "object",
"properties": {
"Name": {
"type": "string"
}
}
}
}
},
"ErrorCfn": {
"Type": "AWS::ApiGateway::Model",
"Properties": {
"ContentType": "application/json",
"Name": "ErrorCfn",
"RestApiId": {
"Ref": "MyApi"
},
"Schema": {
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "Error Schema",
"type": "object",
"properties": {
"message": {
"type": "string"
}
}
}
}
},
"UsersResource": {
"Type": "AWS::ApiGateway::Resource",
"Properties": {
"RestApiId": {
"Ref": "MyApi"
},
"ParentId": {
"Fn::GetAtt": ["MyApi", "RootResourceId"]
},
"PathPart": "users"
}
},
"UsersPost": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"ResourceId": {
"Ref": "UsersResource"
},
"RestApiId": "MyApi",
"ApiKeyRequired": true,
"AuthorizationType": "NONE",
"HttpMethod": "POST",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["", ["arn:aws:apigateway:", {
"Ref": "AWS::Region"
}, ":lambda:path/2015-03-31/functions/", {
"Fn::GetAtt": ["MyFirstLambdaFn", "Arn"]
}, "/invocations"]]
}
},
"MethodResponses": [{
"ResponseModels": {
"application/json": {
"Ref": "FirstUserPropertyModel"
}
},
"StatusCode": 200
}, {
"ResponseModels": {
"application/json": {
"Ref": "ErrorCfn"
}
},
"StatusCode": 404
}, {
"ResponseModels": {
"application/json": {
"Ref": "ErrorCfn"
}
},
"StatusCode": 500
}]
}
},
"UsersPut": {
"Type": "AWS::ApiGateway::Method",
"Properties": {
"ResourceId": {
"Ref": "UsersResource"
},
"RestApiId": "MyApi",
"ApiKeyRequired": true,
"AuthorizationType": "NONE",
"HttpMethod": "PUT",
"Integration": {
"Type": "AWS_PROXY",
"IntegrationHttpMethod": "POST",
"Uri": {
"Fn::Join": ["", ["arn:aws:apigateway:", {
"Ref": "AWS::Region"
}, ":lambda:path/2015-03-31/functions/", {
"Fn::GetAtt": ["MySecondLambdaFn", "Arn"]
}, "/invocations"]]
}
},
"MethodResponses": [{
"ResponseModels": {
"application/json": {
"Ref": "SecondUserPropertyModel"
}
},
"StatusCode": 200
}, {
"ResponseModels": {
"application/json": {
"Ref": "ErrorCfn"
}
},
"StatusCode": 404
}, {
"ResponseModels": {
"application/json": {
"Ref": "ErrorCfn"
}
},
"StatusCode": 500
}]
}
},
"RestApiDeployment": {
"Type": "AWS::ApiGateway::Deployment",
"Properties": {
"RestApiId": {
"Ref": "MyApi"
},
"StageName": "Prod"
},
"DependsOn": ["UsersPost", "UsersPut"]
}
},
"Description": "Project description"
}
ResourceId must be a reference to a cloudformation resource, not a simple string.
e.g.
ResourceId:
Ref: UsersResource
I figured that actually it was the RestApiId which needed to be a reference too:
"RestApiId": {
"Ref": "MyApi"
},
When you create an API resource(1), a default root resource(2) for the API is created for path /. In order to get the id for the MyApi resource(1) root resource(2) use:
"ResourceId": {
"Fn::GetAtt": [
"MyApi",
"RootResourceId"
]
}
(1) The stack resource
(2) The API resource
Try
{
"$ref": "https://apigateway.amazonaws.com/restapis/YOUR_API_GATEWAY_IT/models/YOUR_MODEL_NAME"
}
In addition, you can also reference another model schema defined in an external model file by setting that model's URL as the value of the $ref property: "$ref": "https://apigateway.amazonaws.com/restapis/{restapi_id}/models/{model_name}".
See here for more details

Service fabric, AD and client certificate security

service fabric cluster communication fail when using my domainname.com certificate. I'm getting this error using the portal and/or using ARM template:
Failed to communicate with the cluster (get cluster health: Client certificate required).
Template (variables were removed):
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"resources": [
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('supportLogStorageAccountName')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"kind": "Storage",
"sku": {
"name": "[parameters('supportLogStorageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('applicationDiagnosticsStorageAccountName')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"kind": "Storage",
"sku": {
"name": "[parameters('applicationDiagnosticsStorageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('vNetApiVersion')]",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('virtualNetworkName')]",
"location": "[parameters('computeLocation')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('addressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnet0Name')]",
"properties": {
"addressPrefix": "[parameters('subnet0Prefix')]"
}
}
]
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('publicIPApiVersion')]",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[concat(parameters('lbIPName'),'-','0')]",
"location": "[parameters('computeLocation')]",
"properties": {
"dnsSettings": {
"domainNameLabel": "[parameters('dnsName')]"
},
"publicIPAllocationMethod": "Dynamic"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('lbApiVersion')]",
"type": "Microsoft.Network/loadBalancers",
"name": "[concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'))]",
"location": "[parameters('computeLocation')]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/',concat(parameters('lbIPName'),'-','0'))]"
],
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerIPConfig",
"properties": {
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]"
}
}
}
],
"backendAddressPools": [
{
"name": "LoadBalancerBEAddressPool",
"properties": {}
}
],
"loadBalancingRules": [
{
"name": "LBRule",
"properties": {
"backendAddressPool": {
"id": "[variables('lbPoolID0')]"
},
"backendPort": "[parameters('nt0fabricTcpGatewayPort')]",
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPort": "[parameters('nt0fabricTcpGatewayPort')]",
"idleTimeoutInMinutes": "5",
"probe": {
"id": "[variables('lbProbeID0')]"
},
"protocol": "tcp"
}
},
{
"name": "LBHttpRule",
"properties": {
"backendAddressPool": {
"id": "[variables('lbPoolID0')]"
},
"backendPort": "[parameters('nt0fabricHttpGatewayPort')]",
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPort": "[parameters('nt0fabricHttpGatewayPort')]",
"idleTimeoutInMinutes": "5",
"probe": {
"id": "[variables('lbHttpProbeID0')]"
},
"protocol": "tcp"
}
}
],
"probes": [
{
"name": "FabricGatewayProbe",
"properties": {
"intervalInSeconds": 5,
"numberOfProbes": 2,
"port": "[parameters('nt0fabricTcpGatewayPort')]",
"protocol": "tcp"
}
},
{
"name": "FabricHttpGatewayProbe",
"properties": {
"intervalInSeconds": 5,
"numberOfProbes": 2,
"port": "[parameters('nt0fabricHttpGatewayPort')]",
"protocol": "tcp"
}
}
],
"inboundNatPools": [
{
"name": "LoadBalancerBEAddressNatPool",
"properties": {
"backendPort": "3389",
"frontendIPConfiguration": {
"id": "[variables('lbIPConfig0')]"
},
"frontendPortRangeEnd": "4500",
"frontendPortRangeStart": "3389",
"protocol": "tcp"
}
}
]
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('storageApiVersion')]",
"type": "Microsoft.Storage/storageAccounts",
"name": "[variables('uniqueStringArray0')[copyIndex()]]",
"location": "[parameters('computeLocation')]",
"dependsOn": [],
"properties": {},
"copy": {
"name": "storageLoop",
"count": 5
},
"kind": "Storage",
"sku": {
"name": "[parameters('storageAccountType')]"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "[variables('vmssApiVersion')]",
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[parameters('vmNodeType0Name')]",
"location": "[parameters('computeLocation')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', parameters('virtualNetworkName'))]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[0])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[1])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[2])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[3])]",
"[concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[4])]",
"[concat('Microsoft.Network/loadBalancers/', concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name')))]",
"[concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName'))]",
"[concat('Microsoft.Storage/storageAccounts/', parameters('applicationDiagnosticsStorageAccountName'))]"
],
"properties": {
"overprovision": "[parameters('overProvision')]",
"upgradePolicy": {
"mode": "Automatic"
},
"virtualMachineProfile": {
"extensionProfile": {
"extensions": [
{
"name": "[concat(parameters('vmNodeType0Name'),'_ServiceFabricNode')]",
"properties": {
"type": "ServiceFabricNode",
"autoUpgradeMinorVersion": false,
"protectedSettings": {
"StorageAccountKey1": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('supportLogStorageAccountName')),'2015-05-01-preview').key1]",
"StorageAccountKey2": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('supportLogStorageAccountName')),'2015-05-01-preview').key2]"
},
"publisher": "Microsoft.Azure.ServiceFabric",
"settings": {
"clusterEndpoint": "[reference(parameters('clusterName')).clusterEndpoint]",
"nodeTypeRef": "[parameters('vmNodeType0Name')]",
"dataPath": "D:\\\\SvcFab",
"durabilityLevel": "Bronze",
"certificate": {
"thumbprint": "[parameters('certificateThumbprint')]",
"x509StoreName": "[parameters('certificateStoreValue')]"
}
},
"typeHandlerVersion": "1.0"
}
},
{
"name": "[concat('VMDiagnosticsVmExt','_vmNodeType0Name')]",
"properties": {
"type": "IaaSDiagnostics",
"autoUpgradeMinorVersion": true,
"protectedSettings": {
"storageAccountName": "[parameters('applicationDiagnosticsStorageAccountName')]",
"storageAccountKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts', parameters('applicationDiagnosticsStorageAccountName')),'2015-05-01-preview').key1]",
"storageAccountEndPoint": "https://core.windows.net/"
},
"publisher": "Microsoft.Azure.Diagnostics",
"settings": {
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": "50000",
"EtwProviders": {
"EtwEventSourceProviderConfiguration": [
{
"provider": "Microsoft-ServiceFabric-Actors",
"scheduledTransferKeywordFilter": "1",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableActorEventTable"
}
},
{
"provider": "Microsoft-ServiceFabric-Services",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableServiceEventTable"
}
}
],
"EtwManifestProviderConfiguration": [
{
"provider": "cbd93bc2-71e5-4566-b3a7-595d8eeca6e8",
"scheduledTransferLogLevelFilter": "Information",
"scheduledTransferKeywordFilter": "4611686018427387904",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricSystemEventTable"
}
}
]
}
}
},
"StorageAccount": "[parameters('applicationDiagnosticsStorageAccountName')]"
},
"typeHandlerVersion": "1.5"
}
}
]
},
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[concat(parameters('nicName'), '-0')]",
"properties": {
"ipConfigurations": [
{
"name": "[concat(parameters('nicName'),'-',0)]",
"properties": {
"loadBalancerBackendAddressPools": [
{
"id": "[variables('lbPoolID0')]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[variables('lbNatPoolID0')]"
}
],
"subnet": {
"id": "[variables('subnet0Ref')]"
}
}
}
],
"primary": true
}
}
]
},
"osProfile": {
"adminPassword": "[parameters('adminPassword')]",
"adminUsername": "[parameters('adminUsername')]",
"computernamePrefix": "[parameters('vmNodeType0Name')]",
"secrets": [
{
"sourceVault": {
"id": "[parameters('sourceVaultValue')]"
},
"vaultCertificates": [
{
"certificateStore": "[parameters('certificateStoreValue')]",
"certificateUrl": "[parameters('certificateUrlValue')]"
}
]
}
]
},
"storageProfile": {
"imageReference": {
"publisher": "[parameters('vmImagePublisher')]",
"offer": "[parameters('vmImageOffer')]",
"sku": "[parameters('vmImageSku')]",
"version": "[parameters('vmImageVersion')]"
},
"osDisk": {
"vhdContainers": [
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[0]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[1]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[2]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[3]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]",
"[concat(reference(concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[4]), variables('storageApiVersion')).primaryEndpoints.blob, parameters('vmStorageAccountContainerName'))]"
],
"name": "vmssosdisk",
"caching": "ReadOnly",
"createOption": "FromImage"
}
}
}
},
"sku": {
"name": "[parameters('vmNodeType0Size')]",
"capacity": "[parameters('nt0InstanceCount')]",
"tier": "Standard"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
},
{
"apiVersion": "2016-09-01",
"type": "Microsoft.ServiceFabric/clusters",
"name": "[parameters('clusterName')]",
"location": "[parameters('clusterLocation')]",
"dependsOn": [
"[concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName'))]"
],
"properties": {
"azureActiveDirectory": {
"clientApplication": "55dae335-8436-4900-8f33-abbff16e8036",
"clusterApplication": "d0392358-9fa2-4f9a-aa2a-101859e31c34",
"tenantId": "c8656f45-daf5-42c1-9b29-ac27d3e63bf3"
},
"certificate": {
"thumbprint": "[parameters('certificateThumbprint')]",
"x509StoreName": "[parameters('certificateStoreValue')]"
},
"clientCertificateCommonNames": [],
"clientCertificateThumbprints": [
{
"certificateThumbprint": "C5EFB021F5D8BA8966B43B523B2A6BF8EE8202C5",
"isAdmin": true
},
{
"certificateThumbprint": "C5EFB021F5D8BA8966B43B523B2A6BF8EE8202C5",
"isAdmin": false
}
],
"clusterState": "Default",
"diagnosticsStorageAccountConfig": {
"blobEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.blob]",
"protectedAccountKeyName": "StorageAccountKey1",
"queueEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.queue]",
"storageAccountName": "[parameters('supportLogStorageAccountName')]",
"tableEndpoint": "[reference(concat('Microsoft.Storage/storageAccounts/', parameters('supportLogStorageAccountName')), variables('storageApiVersion')).primaryEndpoints.table]"
},
"fabricSettings": [
{
"parameters": [
{
"name": "ClusterProtectionLevel",
"value": "[parameters('clusterProtectionLevel')]"
}
],
"name": "Security"
}
],
"managementEndpoint": "[concat('https://',reference(concat(parameters('lbIPName'),'-','0')).dnsSettings.fqdn,':',parameters('nt0fabricHttpGatewayPort'))]",
"nodeTypes": [
{
"name": "[parameters('vmNodeType0Name')]",
"applicationPorts": {
"endPort": "[parameters('nt0applicationEndPort')]",
"startPort": "[parameters('nt0applicationStartPort')]"
},
"clientConnectionEndpointPort": "[parameters('nt0fabricTcpGatewayPort')]",
"durabilityLevel": "Bronze",
"ephemeralPorts": {
"endPort": "[parameters('nt0ephemeralEndPort')]",
"startPort": "[parameters('nt0ephemeralStartPort')]"
},
"httpGatewayEndpointPort": "[parameters('nt0fabricHttpGatewayPort')]",
"isPrimary": true,
"vmInstanceCount": "[parameters('nt0InstanceCount')]"
}
],
"provisioningState": "Default",
"security": {
"metadata": "The Credential type X509 indicates this is cluster is secured using X509 Certificates.",
"ClusterCredentialType": "X509",
"ServerCredentialType": "X509",
"CertificateInformation": {
"ClusterCertificate": {
"Thumbprint": "[parameters('certificateThumbprint')]",
"X509StoreName": "My"
},
"ServerCertificate": {
"Thumbprint": "[parameters('certificateThumbprint')]",
"X509StoreName": "My"
},
"ClientCertificateThumbprints": [
{
"CertificateThumbprint": "[parameters('certificateThumbprint')]",
"IsAdmin": false
},
{
"CertificateThumbprint": "[parameters('certificateThumbprint')]",
"IsAdmin": true
}
]
}
},
"reliabilityLevel": "Bronze",
"upgradeMode": "Automatic",
"vmImage": "Windows"
},
"tags": {
"resourceType": "Service Fabric",
"clusterName": "[parameters('clusterName')]"
}
}
],
"outputs": {
"clusterProperties": {
"value": "[reference(parameters('clusterName'))]",
"type": "object"
}
}
}
hint?
It might be as simple as the client certificate is not imported (installed) in the machine from where you are trying to access the explorer. When you open the Service Fabric explorer, it might pop up to select one of the certificates installed in your machine. You have to choose the client certificate at this point.
I compared the ARM template with my (working) template and it looks OK. No obvious issues.
Your ARM template shows you are missing the client certificate within the VMSS OS Profile, which means it will not be installed to each of the nodes during deployment then it is missing when Service Fabric attempts to use it.
Here is an example from one of my ARM templates
"osProfile": {
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]",
"computernamePrefix": "[parameters('vmNodeType0Name')]",
"secrets": [
{
"sourceVault": {
"id": "[parameters('sourceVault')]"
},
"vaultCertificates": [
{
"certificateStore": "My",
"certificateUrl": "[parameters('clusterCertificateUrl')]"
},
{
"certificateStore": "My",
"certificateUrl": "[parameters('adminCertificateUrl')]"
}
]
}
]
},
EDIT:
Where you have two client certificates, I would expect your OS Profile to have three vault certificates for installation on deployment