I am trying to learn and practise AWS Cloudformation template.
While validating the template i am getting below error.
$ aws cloudformation validate-template --template-body file:///home/bhemanth/Downloads/ec2-templates/singe-instance-v2.yaml
An error occurred (ValidationError) when calling the ValidateTemplate operation: Invalid template resource property 'BlockDeviceMappings'
CloudFormation Template Code Error:
AWSTemplateFormatVersion: '2010-09-09'
Description: 'CentOS EC2 Instance template'
Parameters:
KeyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
Type: AWS::EC2::KeyPair::KeyName
Default: hemanth
AllowedValues:
- hemanth
- client
ConstraintDescription: must be the name of an existing EC2 KeyPair.
InstanceType:
Description: CentOS
Type: String
Default: t2.small
AllowedValues:
- t2.micro
- t2.small
- t2.medium
ConstraintDescription: must be a valid EC2 instance type.
SSHLocation:
Description: The IP address range that can be used to SSH to the EC2 instances
Type: String
MinLength: '9'
MaxLength: '18'
Default: 0.0.0.0/0
AllowedPattern: "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})"
ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x.
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType:
Ref: InstanceType
SecurityGroups:
- Ref: InstanceSecurityGroup
KeyName:
Ref: KeyName
ImageId: ami-01ed306a12b7d1c96
InstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: EnableAll
GroupDescription: Enable SSH access for all ports
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '0'
ToPort: '65535'
CidrIp:
Ref: SSHLocation
BlockDeviceMappings:
- DeviceName: /dev/sda1
Ebs:
DeleteOnTermination: true
Status: attached
Hypervisor: xen
RootDeviceName: /dev/sda1
RootDeviceType: ebs
Tags:
- Key: Name
Value: Docker
VirtualizationType: hvm
UserData:
Fn::Base64: !Sub |
#!/usr/bin/env bash
yum install -y wget
wget -O- https://get.docker.com/ | sh
systemctl status docker
systemctl start docker
systemctl enable docker
systemctl status docker
systemctl status -l docker
Volumes:
- Attachments:
Device: /dev/sda1
State: attached
DeleteOnTermination: true
AvailabilityZone: us-west-2a
Encrypted: false
Size: 30
State: in-use
Iops: 100
VolumeType: gp2
Outputs:
InstanceId:
Description: InstanceId of the newly created EC2 instance
Value:
Ref: EC2Instance
AZ:
Description: Availability Zone of the newly created EC2 instance
Value:
Fn::GetAtt:
- EC2Instance
- AvailabilityZone
PublicDNS:
Description: Public DNSName of the newly created EC2 instance
Value:
Fn::GetAtt:
- EC2Instance
- PublicDnsName
PublicIP:
Description: Public IP address of the newly created EC2 instance
Value:
Fn::GetAtt:
- EC2Instance
- PublicIp
I am trying to prepare aws cloudformation template which will install docker from userdata and delete the volume when instance is terminated.
Can you please advice what is wrong in my template.
If possible can you please good tips and cheats for creating aws cloudformation for beginners.
Thanks,
Hemanth.
The CloudFormation Linter catches this and more with:
E3001 Invalid resource attribute BlockDeviceMappings for resource InstanceSecurityGroup
singe-instance-v2.yaml:51:5
E3001 Invalid resource attribute Hypervisor for resource InstanceSecurityGroup
singe-instance-v2.yaml:56:5
E3001 Invalid resource attribute RootDeviceName for resource InstanceSecurityGroup
singe-instance-v2.yaml:57:5
E3001 Invalid resource attribute RootDeviceType for resource InstanceSecurityGroup
singe-instance-v2.yaml:58:5
E3001 Invalid resource attribute Tags for resource InstanceSecurityGroup
singe-instance-v2.yaml:59:5
E3001 Invalid resource attribute VirtualizationType for resource InstanceSecurityGroup
singe-instance-v2.yaml:62:5
E3001 Invalid resource attribute UserData for resource InstanceSecurityGroup
singe-instance-v2.yaml:63:5
E3001 Invalid resource attribute Volumes for resource InstanceSecurityGroup
singe-instance-v2.yaml:73:5
Property types like BlockDeviceMappings, Tags, UserData, Volumes, and AvailabilityZone should be indented a level further than Properties:
I also believe those properties should be underneath Properties: in the AWS::EC2::Instance resource, because most of them are not valid property types of AWS::EC2::SecurityGroup
I don't think Hypervisor is a valid property type of any resource type, so I'm not sure where that property type came from
I'd recommend referencing the documentation for the AWS::EC2::SecurityGroup and AWS::EC2::Instance resource types
Related
I am trying to create Amazon RDS instances in different environments using CloudFormation templates. There is a Multi-AZ requirement in Prod, but other environments do not need Multi-AZ. This calls for a condition function in CloudFormation.
Based on the RDS CloudFormation docs and using the if condition in CloudFormation, the following should work in the template:
Conditions:
IsProd: !Equals [ !Ref EnvironmentType, prod ]
...
Resources:
MyRDSInstance:
Properties:
...
AvailabilityZone:
!If [ IsProd, AWS::NoValue, af-south-1a ]
...
MultiAZ: !If [ IsProd, true, false ]
When IsProd evaluates to:
false, AvailabilityZone: af-south-1a and MultiAZ: false
true, AvailabilityZone is removed and MultiAZ: true, which meets the requirement specified in the docs:
You can't set the AvailabilityZone parameter if the MultiAZ parameter is set to true.
However, when trying to deploy the prod RDS instance, I still get the following error in CloudFormation when creating the stack, which then prevents the resources from being created at all:
Requesting a specific availability zone is not valid for Multi-AZ instances. (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: e6177fe4-4a4b-4db3-ba66-5f0e0f7218eb; Proxy: null)
I suspect this is a bug in AWS due to a recent change that was applied in the source code, even though it was related to the CDK and not CloudFormation:
Issue: Availability Zone parameter silently removed from stack when MultiAZ is true
Fix committed on May 25, 2021: fix(rds): Add exception throw when az is defined for multi-az db inst…. I am getting the error thrown in this exact fix.
Could it be that CloudFormation is now not making provision for the AWS::NoValue pseudo parameter? If this is a bug in the source code, is there any way to get around this so I can still achieve Multi-AZ in the prod environment only?
So I tried to replicate the same at my end but in my case I was able to successfully create the RDS resource. I am attaching the template which I used for your reference.
AWSTemplateFormatVersion: 2010-09-09
Description: >-
Description": "AWS CloudFormation Sample Template for creating an Amazon RDS DB instance:
Sample template showing how to create a DB instance with Enhanced Monitoring enabled.
**WARNING** This template creates an RDS DB instance. You will be billed for the AWS
resources used if you create a stack from this template.
Parameters:
IsMultiAZ:
Type: String
Default: false
AllowedValues: [true,false]
Description: Please enter either "true" or "false"
DBInstanceID:
Default: mydbinstance
Description: My database instance
Type: String
MinLength: '1'
MaxLength: '63'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: >-
Must begin with a letter and must not end with a hyphen or contain two
consecutive hyphens.
DBName:
Default: mydb
Description: My database
Type: String
MinLength: '1'
MaxLength: '64'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: Must begin with a letter and contain only alphanumeric characters.
DBInstanceClass:
Default: db.m5.large
Description: DB instance class
Type: String
ConstraintDescription: Must select a valid DB instance type.
DBAllocatedStorage:
Default: '50'
Description: The size of the database (GiB)
Type: Number
MinValue: '20'
MaxValue: '65536'
ConstraintDescription: must be between 20 and 65536 GiB.
DBUsername:
NoEcho: 'true'
Description: Username for MySQL database access
Type: String
MinLength: '1'
MaxLength: '16'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: must begin with a letter and contain only alphanumeric characters.
DBPassword:
NoEcho: 'true'
Description: Password MySQL database access
Type: String
MinLength: '8'
MaxLength: '41'
AllowedPattern: '[a-zA-Z0-9]*'
ConstraintDescription: must contain only alphanumeric characters.
Conditions:
CheckIsMultiZone:
!Equals [!Ref IsMultiAZ, true]
Resources:
MyDB:
Type: 'AWS::RDS::DBInstance'
Properties:
DBInstanceIdentifier: !Ref DBInstanceID
DBName: !Ref DBName
DBInstanceClass: !Ref DBInstanceClass
AllocatedStorage: !Ref DBAllocatedStorage
Engine: MySQL
EngineVersion: "8.0.16"
MasterUsername: !Ref DBUsername
MasterUserPassword: !Ref DBPassword
MultiAZ: !Ref IsMultiAZ
AvailabilityZone: !If [CheckIsMultiZone, !Ref AWS::NoValue, "us-east-1a"]
As you can see I have used the same concept you used. Can you test this template at your end to see if this is working or not. One issue I found in your template is that you are using AWS::NoValue while the correct format is !Ref AWS::NoValue as shown in my template. I believe this is the issue in your case. You can check the example here .
I have a cloudformation template that launches an ec2 instance, but I want to install packages as well. It is not doing that.
The commands in my template successfully install the packages when run manually on the instance. But I do not understand the correct syntax in order to have cloudformation do the installs for me.
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS CloudFormation Sample Template - spin up EC2 instance, install mule and jre
Parameters:
KeyName:
Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
Default: app-key
Type: AWS::EC2::KeyPair::KeyName
ConstraintDescription: must be the name of an existing EC2 KeyPair.
InstanceType:
Description: MuleSoft Enterprise Standalone EC2 instance
Type: String
Default: t2.small
AllowedValues:
- t2.small
- z1d.large
- r5d.large
- r5.large
- r5ad.large
- r5a.large
ConstraintDescription: must be a valid EC2 instance type.
SSHLocation:
Description: The IP address range that can be used to SSH to the EC2 instances
Type: String
MinLength: '9'
MaxLength: '18'
Default: 0.0.0.0/0
AllowedPattern: "(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})/(\\d{1,2})"
ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x.
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Metadata:
AWS::CloudFormation::Init:
config:
commands:
01_update_yum:
command: "sudo yum update -y"
02_rm_jre1_7:
command: "sudo sudo yum -y erase java-1.7.0"
03_install__jre1_8:
command: "sudo yum install -y java-1.8.0-openjdk"
04_change_into_opt:
command: "cd /opt"
05_download_mulesoft:
command: "sudo wget https://s3-us-west-1.amazonaws.com/mulesoft/mule-enterprise-standalone-4.1.3.2.zip"
06_install_mulesoft:
command: "sudo unzip mule-enterprise-standalone-4.1.3.2.zip"
07_add_mule_user:
command: "sudo useradd mule"
08_mule_ownership:
command: "sudo chown -R mule /opt/mule-enterprise-standalone-4.1.3.2"
09_run_mule:
command: "sudo -u mule bash -x /opt/mule-enterprise-standalone-4.1.3.2/bin/mule console"
Properties:
InstanceType:
Ref: InstanceType
SecurityGroups:
- Ref: WebSecurityGroup
KeyName:
Ref: KeyName
ImageId: ami-0080e4c5bc078760e
WebSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Enable SSH, HTTP, HTTPS, Custom port
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
FromPort: '80'
IpProtocol: tcp
ToPort: '80'
- CidrIp: 0.0.0.0/0
FromPort: '443'
IpProtocol: tcp
ToPort: '443'
- CidrIp: 0.0.0.0/0
FromPort: '8443'
IpProtocol: tcp
ToPort: '8443'
- CidrIp:
Ref: SSHLocation
FromPort: '22'
IpProtocol: tcp
ToPort: '22'
Outputs:
InstanceId:
Description: InstanceId of the newly created EC2 instance
Value:
Ref: EC2Instance
AZ:
Description: Availability Zone of the newly created EC2 instance
Value:
Fn::GetAtt:
- EC2Instance
- AvailabilityZone
PublicDNS:
Description: Public DNSName of the newly created EC2 instance
Value:
Fn::GetAtt:
- EC2Instance
- PublicDnsName
PublicIP:
Description: Public IP address of the newly created EC2 instance
Value:
Fn::GetAtt:
- EC2Instance
- PublicIp
I would like this template to launch the instance and perform the commands to install the packages.
You should use Userdata and cloud-init both for installation. You have commands, but these commands are not called.
Try this https://www.bogotobogo.com/DevOps/AWS/aws-CloudFormation-Bootstrap-UserData.php This page has proper example for installation of packages.
So the last item I added to this template was the attempt to have it use a particular SecurityGroup. I did not want it to create a new one. When I do the validate check that comes back ok but apparently my code is still not correct. Other that the template was working ok.
I have tried all I can think of. there is no error when i finally times out other than "internal error" so I am at a loss here.
Parameters:
VPC:
Description: Testing using this VPC
Type: String
Default: vpc-02765
SecGroup:
Description: Name of security group
Type: AWS::EC2::SecurityGroup
KeyName:
Description: Name of an existing EC2 key pair for SSH access to the EC2 instance.
Type: AWS::EC2::KeyPair::KeyName
InstanceType:
Description: EC2 instance type.
Type: String
Default: t2.micro
...
...
...
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref 'InstanceType'
SubnetId: subnet-08b
KeyName: !Ref 'KeyName'
SecurityGroupIds:
- !Ref SecGroup
ImageId: !FindInMap
- AWSRegionArch2AMI
- !Ref 'AWS::Region'
- HVM64
'''
all I am trying to do is use the items I listed in the template. the vpc,securitygroup. The last time this worked was when I had the code in the template that builds a new SG. I than changed my mind and want to use an existing SG. so somewhere I messed up
This works in my templates:
Parameters:
SecGroup:
Type: AWS::EC2::SecurityGroup::Id
...
Resources:
MyInstance:
Properties:
SecurityGroupIds:
- !Ref SecGroup
I get the error Value of property NetworkInterfaces must be a list of objects when referring to a NetworkInterface in a CloudFormation template.
Here is the relevant section:
MyAppNetworkInterface:
Type: AWS::EC2::NetworkInterface
Properties:
SubnetId: !Ref SubnetPrivate
MyApp:
Type: AWS::EC2::Instance
Properties:
InstanceType: t2.medium
NetworkInterfaces:
- !Ref MyAppNetworkInterface
You can actually refer the Network Interface directly from the EC2 Host. But the syntax is slightly different:
MyAppNetworkInterface:
Type: AWS::EC2::NetworkInterface
Properties:
SubnetId: !Ref SubnetPrivate
MyApp:
Type: AWS::EC2::Instance
Properties:
InstanceType: t2.medium
NetworkInterfaces:
- NetworkInterfaceId: !Ref MyAppNetworkInterface
DeviceIndex: 0
(see: http://docs.amazonaws.cn/en_us/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-network-interface.html#cfn-awsec2networkinterface-templateexamples)
You can't do it that way. Instead , create the two resources independently, then connect with a network interface attachment resource.
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-network-interface-attachment.html
I am using the following CloudFormation template to create ECS Cluster.
AWSTemplateFormatVersion: '2010-09-09'
Description: 'AWS Cloudformation Template to create the Infrastructure'
Resources:
ECSCluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: 'Blog-iac-test-1'
EC2InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles: [!Ref 'EC2Role']
ECSAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
VPCZoneIdentifier:
- subnet-****
LaunchConfigurationName: !Ref 'ECSAutoscalingLC'
MinSize: '1'
MaxSize: '2'
DesiredCapacity: '1'
ECSAutoscalingLC:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
AssociatePublicIpAddress: true
ImageId: 'ami-b743bed1'
SecurityGroups:
- sg-****
InstanceType: 't2.micro'
IamInstanceProfile: !Ref 'EC2InstanceProfile'
KeyName: 'test'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
echo ECS_CLUSTER=Blog-iac-test-1 >> /etc/ecs/ecs.config
EC2Role:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: [ec2.amazonaws.com]
Action: ['sts:AssumeRole']
Path: /
ECSServicePolicy:
Type: "AWS::IAM::Policy"
Properties:
PolicyName: "root"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action: ['ecs:*', 'logs:*', 'ecr:*', 's3:*']
Resource: '*'
Roles: [!Ref 'EC2Role']
The stack is created successfully, but while destroying, I am getting the following error:
The Cluster cannot be deleted while Container Instances are active or draining.
I was able to delete the stack earlier, this issue started to occur recently.
What could be a workaround to avoid this issue ? Should I need to add some dependencies ?
As mentioned in this AWS Documentation Link, have you tried deregistering the instances as well?:
Deregister Container Instances:
Before you can delete a cluster, you
must deregister the container instances inside that cluster. For each
container instance inside your cluster, follow the procedures in
Deregister a Container Instance to deregister it.
Alternatively, you can use the following AWS CLI command to deregister
your container instances. Be sure to substitute the Region, cluster
name, and container instance ID for each container instance that you
are deregistering.
aws ecs deregister-container-instance --cluster default
--container-instance container_instance_id --region us-west-2 --force