The AWS docs for DeletionPolicy mention a default policy for DB Clusters:
For AWS::RDS::DBCluster resources, the default policy is Snapshot
But when I try to set it to Delete I get the following error:
#: extraneous key [DeletionPolicy] is not permitted
Is there a way to change the DeletionPolicy and/or the UpdateReplacePolicy for DB Clusters?
DBCluster CloudFormation template (part of a Serverless template):
auroraCluster:
Type: AWS::RDS::DBCluster
Properties:
AvailabilityZones:
- eu-north-1a
- eu-north-1b
- eu-north-1c
DatabaseName:
publisher
DeletionPolicy: Delete,
DBClusterIdentifier: ${self:service}-db-cluster-${sls:stage}
DBSubnetGroupName: !Ref subnetGroup
DeletionProtection: !If [isProd, 'true', 'false']
Engine: aurora-postgresql
Port: 5432
EngineVersion: 14.3
KmsKeyId: !Ref kmsKey
ManageMasterUserPassword: 'true'
MasterUsername: postgres
MasterUserSecret:
SecretArn: !Ref secretRds
ServerlessV2ScalingConfiguration:
MaxCapacity: 2
MinCapacity: 1
StorageEncrypted: 'true'
VpcSecurityGroupIds:
- !Ref securityGroupDb
DeletionPolicy goes directly under the resource name, on the same level as Type and the Properties-object, not inside the Properties-object. As such:
auroraCluster:
Type: AWS::RDS::DBCluster
DeletionPolicy: Delete
Properties:
...
Related
I have create a fargate task and trying to trigger it via s3 object creation event ( see sample below) via cloudformation.as it cannot trigger it directly, i have created a cloudwatchevent. I am trying to pass the bucket and obj name to my fargate task code . doing some research, i came across -> InputTransformer, but i'm not sure how to pass the value of my bucket and key name and how to read it in my python code. any help will be appreciated.
AWSTemplateFormatVersion: 2010-09-09
Description: An example CloudFormation template for Fargate.
Parameters:
VPC:
Type: AWS::EC2::VPC::Id
SubnetA:
Type: AWS::EC2::Subnet::Id
SubnetB:
Type: AWS::EC2::Subnet::Id
Image:
Type: String
Default: 123456789012.dkr.ecr.region.amazonaws.com/image:tag
Resources:
mybucket:
Properties:
BucketName: 'mytestbucket-us'
cloudwatchEvent:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.s3
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
- CompleteMultipartUpload
requestParameters:
bucketName:
- !Ref mybucket
Targets:
- Id: my-fargate-task
Arn: myclusterArn
RoleArn: myinvocationrolearn
Input:
'Fn::Sub':
- >-
{"containerOverrides": [{"name":"somecontainer"]}
EcsParameters:
TaskDefinition:
LaunchType: 'FARGATE'
...
NetworkConfiguration:
...
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Join ['', [!Ref ServiceName, Cluster]]
TaskDefinition:
Type: AWS::ECS::TaskDefinition
DependsOn: LogGroup
Properties:
Family: !Join ['', [!Ref ServiceName, TaskDefinition]]
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
Cpu: 256
Memory: 2GB
ExecutionRoleArn: !Ref ExecutionRole
TaskRoleArn: !Ref TaskRole
ContainerDefinitions:
- Name: !Ref ServiceName
Image: !Ref Image
# A role needed by ECS
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Join ['', [!Ref ServiceName, ExecutionRole]]
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: 'sts:AssumeRole'
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'
# A role for the containers
TaskRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Join ['', [!Ref ServiceName, TaskRole]]
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: 'sts:AssumeRole'
You would use a CloudWatch Event Input Transformer to extract the data you need from the event, and pass that data to the ECS task as environment variable(s) in the target's ContainerOverrides. I don't use CloudFormation, but here's an example using Terraform.
You can't. CloudWatch events do not pass data to ECS jobs. You need to develop your own mechanism for that. For example, trigger lambda first, store event in S3 Parameter Store or DynamoDB, and then invoke your ECS job which will get stored data.
I am trying to create postgres rds using cloudformation but it give me this error
RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t2.micro, Engine=postgres, EngineVersion=13.3, LicenseModel=postgresql-license. For supported combinations of instance class and database engine version, see the documentation. (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: cdb3ewcd-17ef-404c-adc5-fcd04a590553; Proxy: null).
I have tried changing the instance type and EngineVersion but same error. Any help would be appreciated.
myDBEC2SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Frontend Access
VpcId: !Ref Ec2Vpc
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 5432
ToPort: 5432
CidrIp: 10.0.0.0/16
myDBParamGroup:
Type: AWS::RDS::DBParameterGroup
Properties:
Description: Database Parameter Group + pg_stat_statements
Family: postgres13
Parameters:
shared_preload_libraries: pg_stat_statements
myDBSubnetGroup:
Type: AWS::RDS::DBSubnetGroup
Properties:
DBSubnetGroupDescription: DB Private Subnet
SubnetIds:
- !Ref Ec2SubnetPrivate1
- !Ref Ec2SubnetPrivate2
pgDB:
Type: AWS::RDS::DBInstance
Properties:
DBName: !Ref 'DBName'
AllocatedStorage: !Ref DBAllocatedStorage
DBInstanceClass: !Ref 'DBInstanceClass'
Engine: postgres
EngineVersion: '13.3'
MasterUsername: !Ref 'DBUser'
MasterUserPassword: !Ref 'DBPassword'
MultiAZ: !Ref 'MultiAZ'
Tags:
- Key: Name
Value: Master Database
DBSubnetGroupName: !Ref myDBSubnetGroup
DBParameterGroupName: !Ref myDBParamGroup
VPCSecurityGroups:
- Fn::GetAtt:
- myDBEC2SecurityGroup
- GroupId
You have to find the instance that supports your engine postgres and version 13.3 for your region. To do this you should use the following command:
aws rds describe-orderable-db-instance-options --engine postgres --engine-version 13.3 --query "*[].{DBInstanceClass:DBInstanceClass,StorageType:StorageType}|[?StorageType=='gp2']|[].{DBInstanceClass:DBInstanceClass}" --output text
which gives (t2 is not supported at all):
db.m5.12xlarge
db.m5.16xlarge
db.m5.24xlarge
db.m5.2xlarge
db.m5.4xlarge
db.m5.8xlarge
db.m5.large
db.m5.xlarge
db.m6g.12xlarge
db.m6g.16xlarge
db.m6g.2xlarge
db.m6g.4xlarge
db.m6g.8xlarge
db.m6g.large
db.m6g.xlarge
db.r5.12xlarge
db.r5.16xlarge
db.r5.24xlarge
db.r5.2xlarge
db.r5.4xlarge
db.r5.8xlarge
db.r5b.12xlarge
db.r5b.16xlarge
db.r5b.24xlarge
db.r5b.2xlarge
db.r5b.4xlarge
db.r5b.8xlarge
db.r5b.large
db.r5b.xlarge
db.r5.large
db.r5.xlarge
db.r6g.12xlarge
db.r6g.16xlarge
db.r6g.2xlarge
db.r6g.4xlarge
db.r6g.8xlarge
db.r6g.large
db.r6g.xlarge
db.t3.2xlarge
db.t3.large
db.t3.medium
db.t3.micro
db.t3.small
db.t3.xlarge
db.t4g.2xlarge
db.t4g.large
db.t4g.medium
db.t4g.micro
db.t4g.small
db.t4g.xlarge
db.x2g.12xlarge
db.x2g.16xlarge
db.x2g.2xlarge
db.x2g.4xlarge
db.x2g.8xlarge
db.x2g.large
db.x2g.xlarge
I'm running two scripts, one that creates a cluster with the required resources, and then another script to create the tasks and service. The first script creates everything correctly, but when I run the second script to create the service and task, it gives the following error:
The target group with targetGroupArn arn:aws:elasticloadbalancing:ca-central-1:XXXXXXXXX:targetgroup/XXXXXXXXXXXXXXXXXXXXX does not have an associated load balancer. (Service: AmazonECS; Status Code: 400; Error Code: InvalidParameterException; Request ID: 3899fd23-3eee-473f-9914-453a8d669f14)
What am I doing wrong?
Script 1 to create cluster:
AWSTemplateFormatVersion: '2010-09-09'
Description: A stack for deploying containerized applications onto a cluster of EC2
hosts using Elastic Container Service. This stack runs containers on
hosts that are in a public VPC subnet, and includes a public facing load
balancer to register the services in.
Parameters:
DesiredCapacity:
Type: Number
Default: '3'
Description: Number of EC2 instances to launch in your ECS cluster.
MaxSize:
Type: Number
Default: '6'
Description: Maximum number of EC2 instances that can be launched in your ECS cluster.
ECSAMI:
Description: AMI ID
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id
InstanceType:
Description: EC2 instance type
Type: String
Default: t2.micro
AllowedValues: [t2.micro, t2.small, t2.medium, t2.large, m3.medium, m3.large,
m3.xlarge, m3.2xlarge, m4.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m4.10xlarge,
c4.large, c4.xlarge, c4.2xlarge, c4.4xlarge, c4.8xlarge, c3.large, c3.xlarge,
c3.2xlarge, c3.4xlarge, c3.8xlarge, r3.large, r3.xlarge, r3.2xlarge, r3.4xlarge,
r3.8xlarge, i2.xlarge, i2.2xlarge, i2.4xlarge, i2.8xlarge]
ConstraintDescription: Please choose a valid instance type.
Mappings:
# Hard values for the subnet masks. These masks define
# the range of internal IP addresses that can be assigned.
# The VPC can have all IP's from 10.0.0.0 to 10.0.255.255
# There are two subnets which cover the ranges:
#
# 10.0.0.0 - 10.0.0.255
# 10.0.1.0 - 10.0.1.255
#
# If you need more IP addresses (perhaps you have so many
# instances that you run out) then you can customize these
# ranges to add more
SubnetConfig:
VPC:
CIDR: '10.0.0.0/16'
PublicOne:
CIDR: '10.0.0.0/24'
PublicTwo:
CIDR: '10.0.1.0/24'
Resources:
# VPC in which containers will be networked.
# It has two public subnets
# We distribute the subnets across the first two available subnets
# for the region, for high availability.
VPC:
Type: AWS::EC2::VPC
Properties:
EnableDnsSupport: true
EnableDnsHostnames: true
CidrBlock: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']
# Two public subnets, where containers can have public IP addresses
PublicSubnetOne:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: {Ref: 'AWS::Region'}
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PublicOne', 'CIDR']
MapPublicIpOnLaunch: true
PublicSubnetTwo:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: {Ref: 'AWS::Region'}
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PublicTwo', 'CIDR']
MapPublicIpOnLaunch: true
# Setup networking resources for the public subnets. Containers
# in the public subnets have public IP addresses and the routing table
# sends network traffic via the internet gateway.
InternetGateway:
Type: AWS::EC2::InternetGateway
GatewayAttachement:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId: !Ref 'VPC'
InternetGatewayId: !Ref 'InternetGateway'
PublicRouteTable:
Type: AWS::EC2::RouteTable
Properties:
VpcId: !Ref 'VPC'
PublicRoute:
Type: AWS::EC2::Route
DependsOn: GatewayAttachement
Properties:
RouteTableId: !Ref 'PublicRouteTable'
DestinationCidrBlock: '0.0.0.0/0'
GatewayId: !Ref 'InternetGateway'
PublicSubnetOneRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnetOne
RouteTableId: !Ref PublicRouteTable
PublicSubnetTwoRouteTableAssociation:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
SubnetId: !Ref PublicSubnetTwo
RouteTableId: !Ref PublicRouteTable
# ECS Resources
ECSCluster:
Type: AWS::ECS::Cluster
# A security group for the EC2 hosts that will run the containers.
# Two rules, allowing network traffic from a public facing load
# balancer and from other hosts in the security group.
#
# Remove any of the following ingress rules that are not needed.
# If you want to make direct requests to a container using its
# public IP address you'll need to add a security group rule
# to allow traffic from all IP addresses.
EcsHostSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Access to the ECS hosts that run containers
VpcId: !Ref 'VPC'
EcsSecurityGroupIngressFromPublicALB:
Type: AWS::EC2::SecurityGroupIngress
Properties:
Description: Ingress from the public ALB
GroupId: !Ref 'EcsHostSecurityGroup'
IpProtocol: -1
SourceSecurityGroupId: !Ref 'PublicLoadBalancerSG'
EcsSecurityGroupIngressFromSelf:
Type: AWS::EC2::SecurityGroupIngress
Properties:
Description: Ingress from other hosts in the same security group
GroupId: !Ref 'EcsHostSecurityGroup'
IpProtocol: -1
SourceSecurityGroupId: !Ref 'EcsHostSecurityGroup'
# Autoscaling group. This launches the actual EC2 instances that will register
# themselves as members of the cluster, and run the docker containers.
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
VPCZoneIdentifier:
- !Ref PublicSubnetOne
- !Ref PublicSubnetTwo
LaunchConfigurationName: !Ref 'ContainerInstances'
MinSize: '1'
MaxSize: !Ref 'MaxSize'
DesiredCapacity: !Ref 'DesiredCapacity'
CreationPolicy:
ResourceSignal:
Timeout: PT15M
UpdatePolicy:
AutoScalingReplacingUpdate:
WillReplace: 'true'
ContainerInstances:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !Ref 'ECSAMI'
SecurityGroups: [!Ref 'EcsHostSecurityGroup']
InstanceType: !Ref 'InstanceType'
IamInstanceProfile: !Ref 'EC2InstanceProfile'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config
yum install -y aws-cfn-bootstrap
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource ECSAutoScalingGroup --region ${AWS::Region}
AutoscalingRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [application-autoscaling.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: service-autoscaling
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'application-autoscaling:*'
- 'cloudwatch:DescribeAlarms'
- 'cloudwatch:PutMetricAlarm'
- 'ecs:DescribeServices'
- 'ecs:UpdateService'
Resource: '*'
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
# Role for the EC2 hosts. This allows the ECS agent on the EC2 hosts
# to communciate with the ECS control plane, as well as download the docker
# images from ECR to run on your host.
EC2Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ec2.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'ecs:CreateCluster'
- 'ecs:DeregisterContainerInstance'
- 'ecs:DiscoverPollEndpoint'
- 'ecs:Poll'
- 'ecs:RegisterContainerInstance'
- 'ecs:StartTelemetrySession'
- 'ecs:Submit*'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
- 'ecr:GetAuthorizationToken'
- 'ecr:BatchGetImage'
- 'ecr:GetDownloadUrlForLayer'
Resource: '*'
# Load balancers for getting traffic to containers.
# This sample template creates one load balancer:
#
# - One public load balancer, hosted in public subnets that is accessible
# to the public, and is intended to route traffic to one or more public
# facing services.
# A public facing load balancer, this is used for accepting traffic from the public
# internet and directing it to public facing microservices
PublicLoadBalancerSG:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Access to the public facing load balancer
VpcId: !Ref 'VPC'
SecurityGroupIngress:
# Allow access to ALB from anywhere on the internet
- CidrIp: 0.0.0.0/0
IpProtocol: -1
PublicLoadBalancer:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
Scheme: internet-facing
#LoadBalancerAttributes:
# - Key: idle_timeout.timeout_seconds
# Value: '30'
Subnets:
# The load balancer is placed into the public subnets, so that traffic
# from the internet can reach the load balancer directly via the internet gateway
- !Ref PublicSubnetOne
- !Ref PublicSubnetTwo
#SecurityGroups: [!Ref 'PublicLoadBalancerSG']
Type: network
# A dummy target group is used to setup the ALB to just drop traffic
# initially, before any real service target groups have been added.
DummyTargetGroupPublic:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
HealthCheckIntervalSeconds: 10
#HealthCheckPath: /
HealthCheckProtocol: TCP
#HealthCheckTimeoutSeconds: 5
HealthyThresholdCount: 2
Name: !Join ['-', [!Ref 'AWS::StackName', 'drop-1']]
Port: 80
Protocol: TCP
UnhealthyThresholdCount: 2
VpcId: !Ref 'VPC'
PublicLoadBalancerListener:
Type: AWS::ElasticLoadBalancingV2::Listener
DependsOn:
- PublicLoadBalancer
Properties:
DefaultActions:
- TargetGroupArn: !Ref 'DummyTargetGroupPublic'
Type: 'forward'
LoadBalancerArn: !Ref 'PublicLoadBalancer'
Port: 80
Protocol: TCP
# This is an IAM role which authorizes ECS to manage resources on your
# account on your behalf, such as updating your load balancer with the
# details of where your containers are, so that traffic can reach your
# containers.
ECSRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ecs.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
Policies:
- PolicyName: ecs-service
PolicyDocument:
Statement:
- Effect: Allow
Action:
# Rules which allow ECS to attach network interfaces to instances
# on your behalf in order for awsvpc networking mode to work right
- 'ec2:AttachNetworkInterface'
- 'ec2:CreateNetworkInterface'
- 'ec2:CreateNetworkInterfacePermission'
- 'ec2:DeleteNetworkInterface'
- 'ec2:DeleteNetworkInterfacePermission'
- 'ec2:Describe*'
- 'ec2:DetachNetworkInterface'
# Rules which allow ECS to update load balancers on your behalf
# with the information sabout how to send traffic to your containers
- 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer'
- 'elasticloadbalancing:DeregisterTargets'
- 'elasticloadbalancing:Describe*'
- 'elasticloadbalancing:RegisterInstancesWithLoadBalancer'
- 'elasticloadbalancing:RegisterTargets'
Resource: '*'
These are the values output by the CloudFormation template. Be careful
about changing any of them, because of them are exported with specific
names so that the other task related CF templates can use them.
Outputs:
ClusterName:
Description: The name of the ECS cluster
Value: !Ref 'ECSCluster'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ClusterName' ] ]
ExternalUrl:
Description: The url of the external load balancer
Value: !Join ['', ['http://', !GetAtt 'PublicLoadBalancer.DNSName']]
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ExternalUrl' ] ]
ECSRole:
Description: The ARN of the ECS role
Value: !GetAtt 'ECSRole.Arn'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ECSRole' ] ]
PublicListener:
Description: The ARN of the public load balancer's Listener
Value: !Ref PublicLoadBalancerListener
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicListener' ] ]
VPCId:
Description: The ID of the VPC that this stack is deployed in
Value: !Ref 'VPC'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'VPCId' ] ]
PublicSubnetOne:
Description: Public subnet one
Value: !Ref 'PublicSubnetOne'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicSubnetOne' ] ]
PublicSubnetTwo:
Description: Public subnet two
Value: !Ref 'PublicSubnetTwo'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicSubnetTwo' ] ]
EcsHostSecurityGroup:
Description: A security group used to allow containers to receive traffic
Value: !Ref 'EcsHostSecurityGroup'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'EcsHostSecurityGroup' ] ]
Script 2 to create service and task:
AWSTemplateFormatVersion: '2010-09-09'
Description: Deploy a service into an ECS cluster behind a public load balancer.
Parameters:
StackName:
Type: String
Default: PublicNLBCluster2
Description: The name of the parent cluster stack that you created. Necessary
to locate and reference resources created by that stack.
ServiceName:
Type: String
Default: nginx
Description: A name for the service
ImageUrl:
Type: String
Default: nginx
Description: The url of a docker image that contains the application process that
will handle the traffic for this service
ContainerPort:
Type: Number
Default: 80
Description: What port number the application inside the docker container is binding to
ContainerCpu:
Type: Number
Default: 256
Description: How much CPU to give the container. 1024 is 1 CPU
ContainerMemory:
Type: Number
Default: 512
Description: How much memory in megabytes to give the container
Path:
Type: String
Default: "*"
Description: A path on the public load balancer that this service
should be connected to. Use * to send all load balancer
traffic to this service.
Priority:
Type: Number
Default: 1
Description: The priority for the routing rule added to the load balancer.
This only applies if your have multiple services which have been
assigned to different paths on the load balancer.
DesiredCount:
Type: Number
Default: 2
Description: How many copies of the service task to run
Role:
Type: String
Default: ""
Description: (Optional) An IAM role to give the service's containers if the code within needs to
access other AWS resources like S3 buckets, DynamoDB tables, etc
Conditions:
HasCustomRole: !Not [ !Equals [!Ref 'Role', ''] ]
Resources:
# The task definition. This is a simple metadata description of what
# container to run, and what resource requirements it has.
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: !Ref 'ServiceName'
Cpu: !Ref 'ContainerCpu'
Memory: !Ref 'ContainerMemory'
TaskRoleArn:
Fn::If:
- 'HasCustomRole'
- !Ref 'Role'
- !Ref "AWS::NoValue"
ContainerDefinitions:
- Name: !Ref 'ServiceName'
Cpu: !Ref 'ContainerCpu'
Memory: !Ref 'ContainerMemory'
Image: !Ref 'ImageUrl'
PortMappings:
- ContainerPort: !Ref 'ContainerPort'
# The service. The service is a resource which allows you to run multiple
# copies of a type of task, and gather up their logs and metrics, as well
# as monitor the number of running tasks and replace any that have crashed
Service:
Type: AWS::ECS::Service
DependsOn: PublicLoadBalancerListener
Properties:
ServiceName: !Ref 'ServiceName'
Cluster:
Fn::ImportValue:
!Join [':', [!Ref 'StackName', 'ClusterName']]
DeploymentConfiguration:
MaximumPercent: 200
MinimumHealthyPercent: 75
DesiredCount: !Ref 'DesiredCount'
TaskDefinition: !Ref 'TaskDefinition'
LoadBalancers:
- ContainerName: !Ref 'ServiceName'
ContainerPort: !Ref 'ContainerPort'
TargetGroupArn: !Ref 'TargetGroup'
# A target group. This is used for keeping track of all the tasks, and
# what IP addresses / port numbers they have. You can query it yourself,
# to use the addresses yourself, but most often this target group is just
# connected to an application load balancer, or network load balancer, so
# it can automatically distribute traffic across all the targets.
TargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: !Ref 'ServiceName'
Port: 80
Protocol: TCP
VpcId:
Fn::ImportValue:
!Join [':', [!Ref 'StackName', 'VPCId']]
New the Yaml and Cloud Formation. Trying to utilize MarkLogics template for deploying a clustered MarkLogic DB utilizing our own VPC. We have the cluster working but have gotten to the point where we would like to mount an additional volume to save backups too.
Were added additional volumes:
MarklogicVolume1root:
Type: 'AWS::EC2::Volume'
Properties:
AvailabilityZone: !Select [0, !Ref AZ]
Size: !Ref VolumeSize
Tags:
- Key: Name
Value: MarkLogic-GroupA-Host1-Volume1Aroot
VolumeType: !Ref VolumeType
Encrypted: !If [UseVolumeEncryption, 'true', 'false']
KmsKeyId: !If [HasCustomEBSKey, !Ref VolumeEncryptionKey, !Ref 'AWS::NoValue']
Metadata:
'AWS::CloudFormation::Designer':
id: c81032f7-b0ec-47ca-a236-e24d57b49ae3
MarklogicVolume1data:
Type: 'AWS::EC2::Volume'
Properties:
AvailabilityZone: !Select [0, !Ref AZ]
Size: !Ref VolumeSizeData
Tags:
- Key: Name
Value: MarkLogic-GroupA-Host1-Volume1Adata
VolumeType: !Ref VolumeType
Encrypted: !If [UseVolumeEncryption, 'true', 'false']
KmsKeyId: !If [HasCustomEBSKey, !Ref VolumeEncryptionKey, !Ref 'AWS::NoValue']
MarklogicVolume1backup:
Type: 'AWS::EC2::Volume'
Properties:
AvailabilityZone: !Select [0, !Ref AZ]
Size: !Ref VolumeSizeBackup
Tags:
- Key: Name
Value: MarkLogic-GroupA-Host1-Volume1Abackup
VolumeType: !Ref VolumeType
Encrypted: !If [UseVolumeEncryption, 'true', 'false']
KmsKeyId: !If [HasCustomEBSKey, !Ref VolumeEncryptionKey, !Ref 'AWS::NoValue']
Updated the blockmapping within the Launch Configuration and the User Data script:
LaunchConfig1:
Type: 'AWS::AutoScaling::LaunchConfiguration'
DependsOn:
- InstanceSecurityGroup
Properties:
BlockDeviceMappings:
- DeviceName: !Ref MarklogicVolume1root
NoDevice: true
Ebs: {}
- DeviceName: !Ref MarklogicVolume1data
NoDevice: true
Ebs: {}
- DeviceName: !Ref MarklogicVolume1backup
NoDevice: true
Ebs: {}
KeyName: !Ref KeyName
ImageId: !If [EssentialEnterprise, !FindInMap [LicenseRegion2AMI,!Ref 'AWS::Region',"Enterprise"], !FindInMap [LicenseRegion2AMI, !Ref 'AWS::Region', "BYOL"]]
UserData: !Base64
'Fn::Join':
- ''
- - MARKLOGIC_CLUSTER_NAME=
- !Ref MarkLogicDDBTable
- |+
- MARKLOGIC_EBS_VOLUME1=
- !Ref MarklogicVolume1root
- ',:'
- !Ref VolumeSize
- '::'
- !Ref VolumeType
- |
::,*
- |
- MARKLOGIC_EBS_VOLUME2=
- !Ref MarklogicVolume1data
- ',:'
- !Ref VolumeSizeData
- '::'
- !Ref VolumeType
- |
::,*
- |
- MARKLOGIC_EBS_VOLUME3=
- !Ref MarklogicVolume1backup
- ',:'
- !Ref VolumeSizeBackup
- '::'
- !Ref VolumeType
- |
::,*
- |
MARKLOGIC_NODE_NAME=NodeA#
- MARKLOGIC_ADMIN_USERNAME=
- !Ref AdminUser
- |+
- MARKLOGIC_ADMIN_PASSWORD=
- !Ref AdminPass
- |+
- |
MARKLOGIC_CLUSTER_MASTER=1
- MARKLOGIC_LICENSEE=
- !Ref Licensee
- |+
- MARKLOGIC_LICENSE_KEY=
- !Ref LicenseKey
- |+
- MARKLOGIC_LOG_SNS=
- !Ref LogSNS
- |+
- !If
- UseVolumeEncryption
- !Join
- ''
- - 'MARKLOGIC_EBS_KEY='
- !If
- HasCustomEBSKey
- !Ref VolumeEncryptionKey
- 'default'
- ''
We are able to deploy the additional volumes but they are not mounting. This is also interrupting the final configuration of the Ec2 instances as well because they also fail their load balancer health checks. Any help or insight is greatly appreciated!
The documentation talks about the steps needed to add an EBS volume to an instance.
Creating an EBS Volume and Attaching it to an Instance
The brief over view is you need to:
Create the Volume
Attach the Volume (as /dev/sdf or higher)
Run init-volumes-from-system
Once that is done, you may also want to manually check the DynamoDB table associated with the CF Template, and ensure the entries have the updated volume information from the host.
Michael's answer is how to add a EBS volume after a stack is created, this is different from the question which is how to define a CF template which pre-defines volumes. If the volume is created but not mounted I recommend looking into the system logs, both marklogic logs and /var/log/messages for indications. If that does not provide sufficient information to resolve then you should open a support ticket for assistance.
I created a template to provision RDS using Cloudformation. While creating RDS we have two options io1,gp2 when we use gp2 we do not need to define iops but when using io1 we need to define iops.
I am able to use io1 but gp2 it show error:-
Encountered non numeric value for property Iops
Snippet of my template
StorageType:
Description: choose the storage type gp2 for 'General purpose SSD' & io1 for 'IOPS SSD'.
Default: "gp2"
Type: String
AllowedValues: ["gp2","io1"]
Conditions:
iops: !Equals [!Ref StorageType, "io1"]
gp2: !Equals [!Ref StorageType, "gp2"]
DB:
Type: AWS::RDS::DBInstance
Properties:
DBInstanceIdentifier: !Sub ${AWS::StackName}
DBName: !Ref 'DBName'
AllocatedStorage: !Ref 'Storage'
DBInstanceClass: !Ref 'InstanceType'
StorageType: !Ref StorageType
****Iops: !If [iops, !Ref iops, "AWS::NoValue"]****
StorageEncrypted: !Ref Encryption
Engine: postgres
EngineVersion: 11.2
Port: !Ref PortNo
MasterUsername: !Ref DBUser
MasterUserPassword: !Ref DBPassword
DBSubnetGroupName: !Ref RDSSubnetGroup
VPCSecurityGroups: [!Ref SecurityGroup]
DBParameterGroupName: !Ref RDSParamGroup
MultiAZ: !Ref MultiAz
PubliclyAccessible: !Ref PublicAccessibelity
Thank you in advance
Correct syntax would be:
Iops: !If [iops, !Ref iops, !Ref "AWS::NoValue"]
Documentation:
Condition Functions