I'm kinda stuck with this weird error.
I'm deploying an ECS Task (Fargate) using a CloudFormation script, it works and successfully deploy the Task. But when the HealthCheck property is set Cloudformation replies
Resource handler returned message: "Model validation failed (#: extraneous key [HealthCheck] is not permitted)" (RequestToken: xxxxxxx-xxx-xxxx-xxxx-xxxxxxxxxxxx, HandlerErrorCode: InvalidRequest)
and my task definition is :
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Sub ${ServiceName}-${EnvironmentName}
Cpu: !Ref 'ContainerCpu'
Memory: !Ref 'ContainerMemory'
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
ExecutionRoleArn: arn:aws:iam::xxxxxxxx:role/some-iam-role
HealthCheck:
Command:
- [ CMD-SHELL,./compose/healthcheck.sh || exit 1 ]
Interval: 10
Retries: 5
StartPeriod: 30
Timeout: 5
TaskRoleArn:
Fn::If:
- 'HasCustomRole'
- !Ref 'Role'
- !Ref "AWS::NoValue"
ContainerDefinitions:
- Name: !Ref 'ServiceName'
Cpu: !Ref 'ContainerCpu'
Memory: !Ref 'ContainerMemory'
Image: !Ref 'ImageUrl'
PortMappings:
- ContainerPort: !Ref 'ContainerPort'
LogConfiguration:
LogDriver: 'awslogs'
Options:
awslogs-group: !Sub ${ServiceName}-service-${EnvironmentName}
awslogs-region: !Ref 'AWS::Region'
awslogs-stream-prefix: !Ref 'ServiceName'
Environment:
- <some other environment that work ok>
HealthCheck is a sub-property of AWS::ECS::TaskDefinition.ContainerDefinition, not a top-level property of AWS::ECS::TaskDefinition
I am trying to create a cloudformation IaC for an app to do blue green deployment. It keep giving me The target group with targetGroupArn arn:aws:elasticloadbalancing:ap-xxx-9:000:targetgroup/master-tg-2 does not have an associated load balancer.
I wonder where did I go wrong. I add a DependsOn the masterLB listener just as stated in this question. I also link up both target groups in the MasterECSServices
The following is the cloudformation template
MasterLBSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Access to the public facing load balancer
VpcId:
Fn::ImportValue: # TAS-dev:VPCId
!Sub "${TasStackName}:VPCId"
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
IpProtocol: tcp
FromPort: 8000
ToPort: 8000
MasterLB:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Name: Master-Dev-LB
Scheme: internet-facing
LoadBalancerAttributes:
- Key: idle_timeout.timeout_seconds
Value: '30'
Subnets:
- !Sub "${StackName}:PublicSubnetOne"
- !Sub "${StackName}:PublicSubnetTwo"
SecurityGroups: [!Ref 'MasterLBSG']
MasterLBListener:
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn:
- MasterLB
Properties:
DefaultActions:
- TargetGroupArn: !Ref 'MasterTGOne'
Type: 'forward'
LoadBalancerArn: !Ref 'MasterLB'
Port: 8000
Protocol: HTTP
MasterTGOne: # Means MasterTargetGroupOne
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: master-tg-1
Port: 8000
Protocol: HTTP
VpcId:"${TasStackName}:VPCId"
TargetType: ip
## to be used as a spare TargetGroup for blue green deployment
MasterTGTwo: # Means MasterTargetGroupOne
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: master-tg-2
Port: 8000
Protocol: HTTP
VpcId:"${TasStackName}:VPCId"
TargetType: ip
MasterECSServices:
Type: AWS::ECS::Service
DependsOn:
- MasterLBListener
Properties:
Cluster:"${TasStackName}:ClusterName"
DeploymentController:
Type: CODE_DEPLOY
DesiredCount: 1
LaunchType: FARGATE
LoadBalancers:
- ContainerName: master-app
ContainerPort: '8000'
TargetGroupArn: !Ref 'MasterTGOne'
- ContainerName: master-app
ContainerPort: '8000'
TargetGroupArn: !Ref 'MasterTGTwo'
NetworkConfiguration:
AwsvpcConfiguration:
SecurityGroups:
- !Ref MasterAppSG
Subnets:
- "${TasStackName}:PrivateSubnetOne"
- "${TasStackName}:PrivateSubnetTwo"
Role:"${TasStackName}:ECSRole"
TaskDefinition: !Ref 'MasterTaskDef'
Update:
Since May 19, 2020 AWS CloudFormation now supports blue/green deployments for Amazon ECS
Before
An example of a custom resource in CloudFormation which makes blue/green deployment for ECS. It uses crhelper:
Lambda which creates blue/green deployment group for ECS (i.e. logic of your custom resource)
import logging
import json
import boto3
from time import sleep
from crhelper import CfnResource
logger = logging.getLogger(__name__)
# Initialise the helper, all inputs are optional,
# this example shows the defaults
helper = CfnResource(json_logging=False,
log_level='DEBUG',
boto_level='CRITICAL',
sleep_on_delete=120)
try:
## Init code goes here
cd = boto3.client('codedeploy')
pass
except Exception as e:
helper.init_failure(e)
#helper.create
def create(event, context):
logger.info("Got Create")
print(json.dumps(event))
application_name = event['ResourceProperties']['ApplicationName']
service_role_arn = event['ResourceProperties']['ServiceRoleArn']
cluster_name = event['ResourceProperties']['ClusterName']
service_name = event['ResourceProperties']['ServiceName']
elb_name = event['ResourceProperties']['ELBName']
tg1_name = event['ResourceProperties']['TG1Name']
tg2_name = event['ResourceProperties']['TG2Name']
listener_arn = event['ResourceProperties']['ListenerArn']
deployment_group_name = event['ResourceProperties']['GroupName']
deployment_style=event['ResourceProperties'].get(
'DeploymentStyle', 'BLUE_GREEN')
response = cd.create_deployment_group(
applicationName=application_name,
deploymentGroupName=deployment_group_name,
serviceRoleArn=service_role_arn,
autoRollbackConfiguration={
'enabled': True,
'events': ['DEPLOYMENT_FAILURE']
},
deploymentStyle={
'deploymentType': deployment_style,
'deploymentOption': 'WITH_TRAFFIC_CONTROL'
},
blueGreenDeploymentConfiguration={
"terminateBlueInstancesOnDeploymentSuccess": {
"action": "TERMINATE",
"terminationWaitTimeInMinutes": 0
},
"deploymentReadyOption": {
"actionOnTimeout": "CONTINUE_DEPLOYMENT",
"waitTimeInMinutes": 0
}
},
loadBalancerInfo={
"targetGroupPairInfoList": [
{
"targetGroups": [
{"name": tg1_name},
{"name": tg2_name}
],
"prodTrafficRoute": {
"listenerArns": [listener_arn]
}
}
]
},
ecsServices=[
{
"serviceName": service_name,
"clusterName": cluster_name
}
]
)
print(response)
helper.Data.update({"Name": deployment_group_name})
cd_group_id = response['deploymentGroupId']
return cd_group_id
#helper.delete
def delete(event, context):
# Delete never returns anything. Should not fail if the
# underlying resources are already deleted.
# Desired state.
logger.info("Got Delete")
print(json.dumps(event))
try:
application_name = event['ResourceProperties']['ApplicationName']
deployment_group_name = event['ResourceProperties']['GroupName']
response = cd.delete_deployment_group(
applicationName=application_name,
deploymentGroupName=deployment_group_name
)
print(response)
except Exception as e:
print(str(e))
def handler(event, context):
helper(event, context)
Execute the lambda from CloudFomration
Once you set up your lambda, then in CloudFormation you can use it as any other "normal" resource:
MyUseCustomLambda:
Type: Custom::CodeDeployCustomGroup
Version: "1.0"
Properties:
Name: UseCustomLambda
ServiceToken: !Ref CustomLambdaArn
ApplicationName: !Ref ApplicationName
ServiceRoleArn: !Ref ServiceRoleArn
ELBName: !Ref ELBName
TG1Name: !Ref TG1Name
TG2Name: !Ref TG2Name
GroupName: !Ref GroupName
ClusterName: !Ref ClusterName
ServiceName: !Ref ServiceName
ListenerArn: !Ref ListenerArn
DeploymentStyle: !Ref DeploymentStyle
I'm running two scripts, one that creates a cluster with the required resources, and then another script to create the tasks and service. The first script creates everything correctly, but when I run the second script to create the service and task, it gives the following error:
The target group with targetGroupArn arn:aws:elasticloadbalancing:ca-central-1:XXXXXXXXX:targetgroup/XXXXXXXXXXXXXXXXXXXXX does not have an associated load balancer. (Service: AmazonECS; Status Code: 400; Error Code: InvalidParameterException; Request ID: 3899fd23-3eee-473f-9914-453a8d669f14)
What am I doing wrong?
Script 1 to create cluster:
AWSTemplateFormatVersion: '2010-09-09'
Description: A stack for deploying containerized applications onto a cluster of EC2
hosts using Elastic Container Service. This stack runs containers on
hosts that are in a public VPC subnet, and includes a public facing load
balancer to register the services in.
Parameters:
DesiredCapacity:
Type: Number
Default: '3'
Description: Number of EC2 instances to launch in your ECS cluster.
MaxSize:
Type: Number
Default: '6'
Description: Maximum number of EC2 instances that can be launched in your ECS cluster.
ECSAMI:
Description: AMI ID
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id
InstanceType:
Description: EC2 instance type
Type: String
Default: t2.micro
AllowedValues: [t2.micro, t2.small, t2.medium, t2.large, m3.medium, m3.large,
m3.xlarge, m3.2xlarge, m4.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m4.10xlarge,
c4.large, c4.xlarge, c4.2xlarge, c4.4xlarge, c4.8xlarge, c3.large, c3.xlarge,
c3.2xlarge, c3.4xlarge, c3.8xlarge, r3.large, r3.xlarge, r3.2xlarge, r3.4xlarge,
r3.8xlarge, i2.xlarge, i2.2xlarge, i2.4xlarge, i2.8xlarge]
ConstraintDescription: Please choose a valid instance type.
Mappings:
# Hard values for the subnet masks. These masks define
# the range of internal IP addresses that can be assigned.
# The VPC can have all IP's from 10.0.0.0 to 10.0.255.255
# There are two subnets which cover the ranges:
#
# 10.0.0.0 - 10.0.0.255
# 10.0.1.0 - 10.0.1.255
#
# If you need more IP addresses (perhaps you have so many
# instances that you run out) then you can customize these
# ranges to add more
SubnetConfig:
VPC:
CIDR: '10.0.0.0/16'
PublicOne:
CIDR: '10.0.0.0/24'
PublicTwo:
CIDR: '10.0.1.0/24'
Resources:
# VPC in which containers will be networked.
# It has two public subnets
# We distribute the subnets across the first two available subnets
# for the region, for high availability.
VPC:
Type: AWS::EC2::VPC
Properties:
EnableDnsSupport: true
EnableDnsHostnames: true
CidrBlock: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']
# Two public subnets, where containers can have public IP addresses
PublicSubnetOne:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: {Ref: 'AWS::Region'}
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PublicOne', 'CIDR']
MapPublicIpOnLaunch: true
PublicSubnetTwo:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: {Ref: 'AWS::Region'}
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PublicTwo', 'CIDR']
MapPublicIpOnLaunch: true
# Setup networking resources for the public subnets. Containers
# in the public subnets have public IP addresses and the routing table
# sends network traffic via the internet gateway.
InternetGateway:
Type: AWS::EC2::InternetGateway
GatewayAttachement:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref 'VPC'
InternetGatewayId: !Ref 'InternetGateway'
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref 'VPC'
PublicRoute:
Type: AWS::EC2::Route
DependsOn: GatewayAttachement
Properties:
RouteTableId: !Ref 'PublicRouteTable'
DestinationCidrBlock: '0.0.0.0/0'
GatewayId: !Ref 'InternetGateway'
PublicSubnetOneRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnetOne
RouteTableId: !Ref PublicRouteTable
PublicSubnetTwoRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnetTwo
RouteTableId: !Ref PublicRouteTable
# ECS Resources
ECSCluster:
Type: AWS::ECS::Cluster
# A security group for the EC2 hosts that will run the containers.
# Two rules, allowing network traffic from a public facing load
# balancer and from other hosts in the security group.
#
# Remove any of the following ingress rules that are not needed.
# If you want to make direct requests to a container using its
# public IP address you'll need to add a security group rule
# to allow traffic from all IP addresses.
EcsHostSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Access to the ECS hosts that run containers
VpcId: !Ref 'VPC'
EcsSecurityGroupIngressFromPublicALB:
Type: AWS::EC2::SecurityGroupIngress
Properties:
Description: Ingress from the public ALB
GroupId: !Ref 'EcsHostSecurityGroup'
IpProtocol: -1
SourceSecurityGroupId: !Ref 'PublicLoadBalancerSG'
EcsSecurityGroupIngressFromSelf:
Type: AWS::EC2::SecurityGroupIngress
Properties:
Description: Ingress from other hosts in the same security group
GroupId: !Ref 'EcsHostSecurityGroup'
IpProtocol: -1
SourceSecurityGroupId: !Ref 'EcsHostSecurityGroup'
# Autoscaling group. This launches the actual EC2 instances that will register
# themselves as members of the cluster, and run the docker containers.
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
VPCZoneIdentifier:
- !Ref PublicSubnetOne
- !Ref PublicSubnetTwo
LaunchConfigurationName: !Ref 'ContainerInstances'
MinSize: '1'
MaxSize: !Ref 'MaxSize'
DesiredCapacity: !Ref 'DesiredCapacity'
CreationPolicy:
ResourceSignal:
Timeout: PT15M
UpdatePolicy:
AutoScalingReplacingUpdate:
WillReplace: 'true'
ContainerInstances:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !Ref 'ECSAMI'
SecurityGroups: [!Ref 'EcsHostSecurityGroup']
InstanceType: !Ref 'InstanceType'
IamInstanceProfile: !Ref 'EC2InstanceProfile'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
yum install -y aws-cfn-bootstrap
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
AutoscalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'application-autoscaling:*'
- 'cloudwatch:DescribeAlarms'
- 'cloudwatch:PutMetricAlarm'
- 'ecs:DescribeServices'
- 'ecs:UpdateService'
Resource: '*'
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
# Role for the EC2 hosts. This allows the ECS agent on the EC2 hosts
# to communciate with the ECS control plane, as well as download the docker
# images from ECR to run on your host.
EC2Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ec2.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'ecs:CreateCluster'
- 'ecs:DeregisterContainerInstance'
- 'ecs:DiscoverPollEndpoint'
- 'ecs:Poll'
- 'ecs:RegisterContainerInstance'
- 'ecs:StartTelemetrySession'
- 'ecs:Submit*'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
- 'ecr:GetAuthorizationToken'
- 'ecr:BatchGetImage'
- 'ecr:GetDownloadUrlForLayer'
Resource: '*'
# Load balancers for getting traffic to containers.
# This sample template creates one load balancer:
#
# - One public load balancer, hosted in public subnets that is accessible
# to the public, and is intended to route traffic to one or more public
# facing services.
# A public facing load balancer, this is used for accepting traffic from the public
# internet and directing it to public facing microservices
PublicLoadBalancerSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Access to the public facing load balancer
VpcId: !Ref 'VPC'
SecurityGroupIngress:
# Allow access to ALB from anywhere on the internet
- CidrIp: 0.0.0.0/0
IpProtocol: -1
PublicLoadBalancer:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Scheme: internet-facing
#LoadBalancerAttributes:
# - Key: idle_timeout.timeout_seconds
# Value: '30'
Subnets:
# The load balancer is placed into the public subnets, so that traffic
# from the internet can reach the load balancer directly via the internet gateway
- !Ref PublicSubnetOne
- !Ref PublicSubnetTwo
#SecurityGroups: [!Ref 'PublicLoadBalancerSG']
Type: network
# A dummy target group is used to setup the ALB to just drop traffic
# initially, before any real service target groups have been added.
DummyTargetGroupPublic:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
HealthCheckIntervalSeconds: 10
#HealthCheckPath: /
HealthCheckProtocol: TCP
#HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
Name: !Join ['-', [!Ref 'AWS::StackName', 'drop-1']]
Port: 80
Protocol: TCP
UnhealthyThresholdCount: 2
VpcId: !Ref 'VPC'
PublicLoadBalancerListener:
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn:
- PublicLoadBalancer
Properties:
DefaultActions:
- TargetGroupArn: !Ref 'DummyTargetGroupPublic'
Type: 'forward'
LoadBalancerArn: !Ref 'PublicLoadBalancer'
Port: 80
Protocol: TCP
# This is an IAM role which authorizes ECS to manage resources on your
# account on your behalf, such as updating your load balancer with the
# details of where your containers are, so that traffic can reach your
# containers.
ECSRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action:
# Rules which allow ECS to attach network interfaces to instances
# on your behalf in order for awsvpc networking mode to work right
- 'ec2:AttachNetworkInterface'
- 'ec2:CreateNetworkInterface'
- 'ec2:CreateNetworkInterfacePermission'
- 'ec2:DeleteNetworkInterface'
- 'ec2:DeleteNetworkInterfacePermission'
- 'ec2:Describe*'
- 'ec2:DetachNetworkInterface'
# Rules which allow ECS to update load balancers on your behalf
# with the information sabout how to send traffic to your containers
- 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer'
- 'elasticloadbalancing:DeregisterTargets'
- 'elasticloadbalancing:Describe*'
- 'elasticloadbalancing:RegisterInstancesWithLoadBalancer'
- 'elasticloadbalancing:RegisterTargets'
Resource: '*'
These are the values output by the CloudFormation template. Be careful
about changing any of them, because of them are exported with specific
names so that the other task related CF templates can use them.
Outputs:
ClusterName:
Description: The name of the ECS cluster
Value: !Ref 'ECSCluster'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ClusterName' ] ]
ExternalUrl:
Description: The url of the external load balancer
Value: !Join ['', ['http://', !GetAtt 'PublicLoadBalancer.DNSName']]
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ExternalUrl' ] ]
ECSRole:
Description: The ARN of the ECS role
Value: !GetAtt 'ECSRole.Arn'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ECSRole' ] ]
PublicListener:
Description: The ARN of the public load balancer's Listener
Value: !Ref PublicLoadBalancerListener
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicListener' ] ]
VPCId:
Description: The ID of the VPC that this stack is deployed in
Value: !Ref 'VPC'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'VPCId' ] ]
PublicSubnetOne:
Description: Public subnet one
Value: !Ref 'PublicSubnetOne'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicSubnetOne' ] ]
PublicSubnetTwo:
Description: Public subnet two
Value: !Ref 'PublicSubnetTwo'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicSubnetTwo' ] ]
EcsHostSecurityGroup:
Description: A security group used to allow containers to receive traffic
Value: !Ref 'EcsHostSecurityGroup'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'EcsHostSecurityGroup' ] ]
Script 2 to create service and task:
AWSTemplateFormatVersion: '2010-09-09'
Description: Deploy a service into an ECS cluster behind a public load balancer.
Parameters:
StackName:
Type: String
Default: PublicNLBCluster2
Description: The name of the parent cluster stack that you created. Necessary
to locate and reference resources created by that stack.
ServiceName:
Type: String
Default: nginx
Description: A name for the service
ImageUrl:
Type: String
Default: nginx
Description: The url of a docker image that contains the application process that
will handle the traffic for this service
ContainerPort:
Type: Number
Default: 80
Description: What port number the application inside the docker container is binding to
ContainerCpu:
Type: Number
Default: 256
Description: How much CPU to give the container. 1024 is 1 CPU
ContainerMemory:
Type: Number
Default: 512
Description: How much memory in megabytes to give the container
Path:
Type: String
Default: "*"
Description: A path on the public load balancer that this service
should be connected to. Use * to send all load balancer
traffic to this service.
Priority:
Type: Number
Default: 1
Description: The priority for the routing rule added to the load balancer.
This only applies if your have multiple services which have been
assigned to different paths on the load balancer.
DesiredCount:
Type: Number
Default: 2
Description: How many copies of the service task to run
Role:
Type: String
Default: ""
Description: (Optional) An IAM role to give the service's containers if the code within needs to
access other AWS resources like S3 buckets, DynamoDB tables, etc
Conditions:
HasCustomRole: !Not [ !Equals [!Ref 'Role', ''] ]
Resources:
# The task definition. This is a simple metadata description of what
# container to run, and what resource requirements it has.
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Ref 'ServiceName'
Cpu: !Ref 'ContainerCpu'
Memory: !Ref 'ContainerMemory'
TaskRoleArn:
Fn::If:
- 'HasCustomRole'
- !Ref 'Role'
- !Ref "AWS::NoValue"
ContainerDefinitions:
- Name: !Ref 'ServiceName'
Cpu: !Ref 'ContainerCpu'
Memory: !Ref 'ContainerMemory'
Image: !Ref 'ImageUrl'
PortMappings:
- ContainerPort: !Ref 'ContainerPort'
# The service. The service is a resource which allows you to run multiple
# copies of a type of task, and gather up their logs and metrics, as well
# as monitor the number of running tasks and replace any that have crashed
Service:
Type: AWS::ECS::Service
DependsOn: PublicLoadBalancerListener
Properties:
ServiceName: !Ref 'ServiceName'
Cluster:
Fn::ImportValue:
!Join [':', [!Ref 'StackName', 'ClusterName']]
DeploymentConfiguration:
MaximumPercent: 200
MinimumHealthyPercent: 75
DesiredCount: !Ref 'DesiredCount'
TaskDefinition: !Ref 'TaskDefinition'
LoadBalancers:
- ContainerName: !Ref 'ServiceName'
ContainerPort: !Ref 'ContainerPort'
TargetGroupArn: !Ref 'TargetGroup'
# A target group. This is used for keeping track of all the tasks, and
# what IP addresses / port numbers they have. You can query it yourself,
# to use the addresses yourself, but most often this target group is just
# connected to an application load balancer, or network load balancer, so
# it can automatically distribute traffic across all the targets.
TargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: !Ref 'ServiceName'
Port: 80
Protocol: TCP
VpcId:
Fn::ImportValue:
!Join [':', [!Ref 'StackName', 'VPCId']]
I am a similar question to some posts, but none of the specific issues relate as far as I can tell. I will post my stack later in this post.
I have:
ALB----->Listener->target group->Fargate service->task definition
80/http ->8080/http -> 8080/http
The problem is my health checks fail. When the Fargate task spins up an instance, I can go to that instance using the health check URL, and i get a 200 response. however, any attempt to go through the load balancer results in a gateway timeout.
$ curl -fv http://172.31.47.18:8080/healthz
* Trying 172.31.47.18...
* TCP_NODELAY set
* Connected to 172.31.47.18 (172.31.47.18) port 8080 (#0)
> GET /healthz HTTP/1.1
> Host: 172.31.47.18:8080
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
< Date: Sun, 24 Nov 2019 15:33:39 GMT
< Server: Warp/3.2.27
<
* Connection #0 to host 172.31.47.18 left intact
OK
However, the health check never passes on the LB.
the security group used for every thing right now is wide open. I wanted to eliminate that as an issue.
the fargate nodes are set up for public IPs.
This has been driving me crazy for the last couple of days. I stood up an EC2 backed ECS, and everything works on EC2. I should point out that the entire stack builds just fine in Fargate, except for not getting any traffic either from the load balancer or something.
The error in the service events says
service test-graph (port 8080) is unhealthy in target-group tg--test-graph due to (reason Request timed out).
Hopefully someone has an idea.
TaskDef0:
Type: AWS::ECS::TaskDefinition
DependsOn: Cluster0
Properties:
ExecutionRoleArn: arn:aws:iam::xxxxx:role/ECS_Hasura_Execution_Role
TaskRoleArn: arn:aws:iam::xxxxx:role/ecsTaskExecutionRole
Family: !Ref 'ServiceName'
Cpu: !FindInMap
- ContainerSizeMap
- !Ref ContainerSize
- Cpu
Memory: !FindInMap
- ContainerSizeMap
- !Ref ContainerSize
- Memory
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
ContainerDefinitions:
- Name: !Ref 'ServiceName'
Cpu: !FindInMap
- ContainerSizeMap
- !Ref ContainerSize
- Cpu
Memory: !FindInMap
- ContainerSizeMap
- !Ref ContainerSize
- Memory
Image: !FindInMap
- ServiceMap
- !Ref ServiceProvider
- ImageUrl
PortMappings:
-
ContainerPort: !Ref 'ContainerPort'
HostPort: !Ref ContainerPort
Protocol: tcp
ALB0:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
DependsOn: TaskDef0
Properties:
Name: !Join
- '-'
- - lb-
- !Ref ServiceName
Scheme: internet-facing
IpAddressType: ipv4
LoadBalancerAttributes:
- Key: deletion_protection.enabled
Value: false
- Key: idle_timeout.timeout_seconds
Value: 60
- Key: routing.http.drop_invalid_header_fields.enabled
Value: false
- Key: routing.http2.enabled
Value: true
SecurityGroups:
- sg-xxxxxx # allow HTTP/HTTPS to the load balancer
Subnets:
- subnet-111111
- subnet-222222
- subnet-333333
Type: application
targetGroup0:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
DependsOn: ALB0
Properties:
Name: !Join
- '-'
- - tg-
- !Ref ServiceName
Port: !Ref TargetGroupPort
Protocol: !Ref TargetGroupProtocol
TargetType: ip
VpcId: !FindInMap
- ServiceMap
- !Ref ServiceProvider
- VpcId
# all other paraneters can be changed without interruption
HealthCheckPort: traffic-port
HealthCheckEnabled: !FindInMap
- LBTGMap
- Parameters
- HealthCheckEnabled
HealthCheckIntervalSeconds: !FindInMap
- LBTGMap
- Parameters
- HealthCheckIntervalSeconds
HealthCheckPath: !FindInMap
- ServiceMap
- !Ref ServiceProvider
- HealthCheckPath
HealthCheckProtocol: !FindInMap
- ServiceMap
- !Ref ServiceProvider
- HealthCheckProtocol
HealthCheckTimeoutSeconds: !FindInMap
- LBTGMap
- Parameters
- HealthCheckTimeoutSeconds
HealthyThresholdCount: !FindInMap
- LBTGMap
- Parameters
- HealthyThresholdCount
UnhealthyThresholdCount: !FindInMap
- LBTGMap
- Parameters
- UnhealthyThresholdCount
Matcher:
HttpCode: !FindInMap
- ServiceMap
- !Ref ServiceProvider
- HealthCheckSuccessCode
TargetGroupAttributes:
- Key: deregistration_delay.timeout_seconds
Value: !FindInMap
- LBTGMap
- Parameters
- DeregistrationDelay
- Key: slow_start.duration_seconds
Value: !FindInMap
- LBTGMap
- Parameters
- SlowStart
- Key: stickiness.enabled
Value: !FindInMap
- LBTGMap
- Parameters
- Stickiness
Listener0:
# This is the fixed response test listener
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn: ALB0
Properties:
DefaultActions:
- Type: fixed-response
FixedResponseConfig:
ContentType: text/html
MessageBody: <h1>Working</h1><p>The load balancer test listener is operational</p>
StatusCode: 200
LoadBalancerArn: !Ref ALB0
Port: 9000
Protocol: HTTP
Listener1:
# This is the port 80 listener
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn: ALB0
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref targetGroup0
LoadBalancerArn: !Ref ALB0
Port: 80
Protocol: HTTP
Listener2:
# This is the port 8080 listener
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn: ALB0
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref targetGroup0
LoadBalancerArn: !Ref ALB0
Port: 8080
Protocol: HTTP
Listener3 :
# This is the port 443 listener
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn: ALB0
Properties:
Certificates:
- CertificateArn: !FindInMap
- CertificateMap
- !Ref AWS::Region
- CertifcateArn
DefaultActions:
- Type: forward
TargetGroupArn: !Ref targetGroup0
LoadBalancerArn: !Ref ALB0
Port: 443
Protocol: HTTPS
Service0:
Type: AWS::ECS::Service
DependsOn: Listener2
Properties:
ServiceName: !Ref 'ServiceName'
Cluster: !Ref Cluster0
LaunchType: FARGATE
DeploymentConfiguration:
MaximumPercent: !FindInMap
- ECSServiceMap
- Parameters
- MaximumPercent
MinimumHealthyPercent: !FindInMap
- ECSServiceMap
- Parameters
- MinimumHealthyPercent
DesiredCount: !Ref 'DesiredTaskCount'
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups: # this is allow all ports and IPs
- !FindInMap
- SecurityGroupMap
- !Ref AWS::Region
- sg0
Subnets:
- !FindInMap
- SubnetMap
- !Ref AWS::Region
- subnet0
- !FindInMap
- SubnetMap
- !Ref AWS::Region
- subnet1
- !FindInMap
- SubnetMap
- !Ref AWS::Region
- subnet2
TaskDefinition: !Ref 'TaskDef0'
LoadBalancers:
- ContainerName: !Ref 'ServiceName'
ContainerPort: !Ref 'ContainerPort'
TargetGroupArn: !Ref 'targetGroup0'
Tags:
- Key: Application
Value: !Ref "Application"
- Key: Customer
Value: !Ref "Customer"
- Key: Role
Value: !Ref "Role"
- Key: InternetAccessible
Value: !Ref "InternetAccessible"
- Key: CreationDate
Value: !Ref "CreationDate"
- Key: CreatedBy
Value: !Ref "CreatedBy"
Mappings:
ServiceMap:
GraphQL-Ohio:
ImageUrl: xxxxx.dkr.ecr.us-east-2.amazonaws.com/hasura/graphql-engine
HealthCheckPath: /healthz
HealthCheckSuccessCode: 200
HealthCheckProtocol: HTTP
VpcId: vpc-xxxxx
LBTGMap:
Parameters:
HealthCheckEnabled: True
HealthCheckIntervalSeconds: 30
HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 5
UnhealthyThresholdCount: 2
DeregistrationDelay: 300
SlowStart: 0
Stickiness: false
SubnetMap: # There is technical debt here to keep this up to date as subnets change
us-east-2:
subnet0: subnet-111111
subnet1: subnet-222222
subnet2: subnet-333333
SecurityGroupMap:
us-east-2:
sg0: sg-xxxxx
Ok - I figured this out. I had my HealthCheckPort set to traffic-port. The string literal "traffic-port", not the actual port number. Duh.
AWSTemplateFormatVersion: "2010-09-09"
Description: Not Have Any Idea
Parameters:
Service:
Description: Service Name
Type: String
Cluster:
Description: Cluster Name
Type: String
TaskDefinition:
Description: TaskDefinition Name
Type: String
securitygroup:
Description: securitygroup
Type: AWS::EC2::SecurityGroup::Id
SubnetId:
Type: List<AWS::EC2::Subnet::Id>
Description: Select at two subnets in your selected VPC.
Resources:
sernginx:
Type: "AWS::ECS::Service"
Properties:
ServiceName:
Ref: Service
LaunchType: "FARGATE"
DesiredCount: 1
Cluster:
Ref: Cluster
TaskDefinition:
Ref: TaskDefinition
DeploymentConfiguration:
MaximumPercent: 200
MinimumHealthyPercent: 70
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- Ref: securitygroup
Subnets:
Ref: SubnetId
I am using this code to create aws ecs service fargate
I am getting ROLLBACK_COMPLETE status with an error message saying
Service arn:aws:ecs:us-east-1:439138162442:service/yuiyiuyiu did not stabilize
I've used this kind of template for creating fargate task.
AWSTemplateFormatVersion: '2010-09-09'
Description: Docker Core App service
Parameters:
EnvironmentName:
Type: String
Default: coreapp
Description: A name for the environment that this cloudformation will be part of.
ServiceName:
Type: String
Default: coreapi
Description: A name for the service
ImageUrl:
Type: String
Default: your_image_URI
Description: The url of a docker image
ContainerPort:
Type: Number
Default: 5000
Description: What port number the application inside the docker container
ContainerCpu:
Type: Number
Default: 256
Description: How much CPU to give the container.
ContainerMemory:
Type: Number
Default: 512
Description: How much memory in megabytes to give the container
DesiredCount:
Type: Number
Default: 2
Description: How many copies of the service task to run
Role:
Type: String
Default: ""
Description: (Optional) An IAM role to give the service's containers
Resources:
Service:
Type: AWS::ECS::Service
Properties:
ServiceName: !Ref 'ServiceName'
Cluster:
- coreapp
LaunchType: FARGATE
DeploymentConfiguration:
MaximumPercent: 200
MinimumHealthyPercent: 75
DesiredCount: !Ref 'DesiredCount'
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups:
- sg-483aa03e
Subnets:
- subnet-2ce41866
- subnet-ceefe5aa
TaskDefinition: !Ref 'TaskDefinition'