Is there a way to parse the EMR MasterPublicDNS in Amazon Cloudformation? - aws-cloudformation

Is there a way to parse the EMR MasterPublicDNS in Cloudformation? I don't see a replace function in Cloudformation.
ip-100-112-10-21.aws.internal
TO
100.112.10.21
Outputs:
IPAddress:
Description: IP address of the EMR clusters
Value: !GetAtt
- EMRCluster
- MasterPublicDNS
can I reference the output value in same script?
I need to use the formatted IP to set resourcerecords- or do I have to use
Type: AWS::Route53::RecordSetGroup
ResourceRecords: !Join [".",
[
!Select [1, !Split ['-', !GetAtt EMRCluster.MasterPublicDNS]],
!Select [2, !Split ['-', !GetAtt EMRCluster.MasterPublicDNS]],
!Select [3, !Split ['-', !GetAtt EMRCluster.MasterPublicDNS]],
!Select [0,
!Split ['.', !Select [4, !Split ['-', !GetAtt EMRCluster.MasterPublicDNS]]]]
]
]
gives error - Value of property ResourceRecords must be of type List of String
or
ResourceRecords: !ref IPAddress.value

If the format is always like that, you could combine Split, Select and Join Cloudformation intrinsic functions to achieve it:
Outputs:
IPAddress:
Description: IP address of the EMR clusters
Value: !Join ['.',
[
!Select: [1, !Split: ['-', !GetAtt EMRCluster.MasterPublicDNS]],
!Select: [2, !Split: ['-', !GetAtt EMRCluster.MasterPublicDNS]],
!Select: [3, !Split: ['-', !GetAtt EMRCluster.MasterPublicDNS]],
!Select: [0,
!Split: ['.', !Select: [4, !Split: ['-', !GetAtt EMRCluster.MasterPublicDNS]]]]
]
]
I know it makes you cringe, but that's the way to go in Cloudformation.
Alternatively, you could write a Cloudformation macro to do this for you.

Related

Unexpected value in YAML

trigger:
- develop
pool:
vmImage: windows-2019
steps:
- task: NuGetCommand#2
inputs:
command: "restore"
restoreSolution: "**/*.sln"
feedsToUse: "config"
- task: MSBuild#1
inputs:
solution: "**/*.sln"
- task: S3Upload#1
inputs:
regionName: "us-west-2"
bucketName: "ssw-nonprod-ado-artifacts"
globExpressions: "**"
targetFolder: "nsspreproc"
I added the missing "steps" keyword as suggested on my previous question.
Now the errors I'm getting are:
/azure-pipelines.yml (Line: 11, Col: 12): Unexpected value
/azure-pipelines.yml (Line: 12, Col: 5): Unexpected value
/azure-pipelines.yml (Line: 13, Col: 5): Unexpected value
/azure-pipelines.yml (Line: 14, Col: 5): Unexpected value
/azure-pipelines.yml (Line: 16, Col: 12): Unexpected value
/azure-pipelines.yml (Line: 17, Col: 5): Unexpected value
/azure-pipelines.yml (Line: 19, Col: 12): Unexpected value
/azure-pipelines.yml (Line: 20, Col: 5): Unexpected value
/azure-pipelines.yml (Line: 21, Col: 5): Unexpected value
/azure-pipelines.yml (Line: 22, Col: 5): Unexpected value
For example line 11 column 12 is right after the colon (:) on the first "inputs".
I've tried changing up the indentation and using single quotes vs double quotes, but none of that seems to help. What am I missing?
Dave Newton is absolutely right, your YAML is not valid.
This is the valid version of your YAML:
trigger:
- develop
pool:
vmImage: windows-2019
steps:
- task: NuGetCommand#2
inputs:
command: restore
restoreSolution: "**/*.sln"
feedsToUse: config
- task: MSBuild#1
inputs:
solution: "**/*.sln"
- task: S3Upload#1
inputs:
regionName: us-west-2
bucketName: ssw-nonprod-ado-artifacts
globExpressions: "**"
targetFolder: nsspreproc
Word of advice, please take a good look at your white spaces and indentation:

Cloudformation conditionals Yaml

I have to create an auto scaling group in two regions and the only difference between the two are the subnets. us-east-1 has 1 subnet whereas us-east-2 has two. How can I use the condition to call the subnet value from region map.
Mappings:
RegionMap:
us-east-1:
AMI: "ami-066f487d3b6819b0d"
Subnet1: "subnet-0e6f12f64042ea5b1"
us-east-2:
AMI: "ami-0aef5e0adcbc7cc0f"
Subnet1: "subnet-0e6f12f64042ea5b1"
Subnet2: "subnet-0bc661bb8d98f3f03"
Conditions:
region: !Equals [!Ref us-east-2, Subnet2]
autoscaling:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
AutoScalingGroupName: asg1
VPCZoneIdentifier:
- !FindInMap [RegionMap, !Ref "AWS::Region", Subnet1]
- If regions = us-east-2 then !FindInMap [RegionMap, !Ref "AWS::Region", Subnet2] # This is what I need to figure out
I couldn't find any examples of this. Has anyone used a region map and used conditionals with it?

Azure Devops - Matrix Strategy - Conditional Job Run

*please ignore formatting issue
trigger:
- none
parameters:
- name: stg
displayName: "Environment Names"
type: string
pool:
vmImage: ubuntu-latest
stages:
- stage: stage1
displayName: 'Stage C1'
jobs:
- job: jobabc
strategy:
maxParallel: 2
matrix:
env_1:
EnvironmentX: E1
Browser: IE
env_2:
EnvironmentX: E2
Browser: Firefox
env_3:
EnvironmentX: E3
Browser: Chrome
displayName: 'Job 1'
condition: contains('${{parameters.stg}}', $(EnvironmentX))}}
steps:
- task: CmdLine#2
displayName: 'My Task '
inputs:
script: 'echo Stage C1 $(Browser)'
I pass stg parameter as E1E3E4.
In Matrix I have defined EnvironmentX whose value can be E1, E2, or E3.
Below code runs Job 3 times with matrix values defined.
I do NOT want to run all 3 jobs but stg parameter value should decide JOb would run for which matrix values.
Job Condition is NOT working? How to fix this please
You could build the matrix conditionally. Not ideal but that will work:
trigger:
- none
parameters:
- name: stg
displayName: Environment Names
type: string
pool:
vmImage: ubuntu-latest
stages:
- stage: stage1
displayName: 'Stage C1'
jobs:
- job: jobabc
strategy:
maxParallel: 2
matrix:
${{ if contains(parameters.stg, 'E1') }}:
env_1:
EnvironmentX: E1
Browser: IE
${{ if contains(parameters.stg, 'E2') }}:
env_2:
EnvironmentX: E2
Browser: Firefox
${{ if contains(parameters.stg, 'E3') }}:
env_3:
EnvironmentX: E3
Browser: Chrome
displayName: 'Job 1'
steps:
- task: CmdLine#2
displayName: 'My Task '
inputs:
script: 'echo Stage C1 $(Browser)'

How to define a CloudWatch Alarm on the sum of two metrics with CloudFormation?

I need to trigger an alarm when the sum of the same metric (ApproximateNumberOfMessagesVisible) on two different queues exceed the value of 100
In September '17, this answer stated that the only way to do it was with a Lambda function getting the two values and summing them up via CloudWatch API.
At writing time, Feb. '19, it is possible to use "Metric Math", so there is no need to have a lambda function or an EC2 instance. Is it possible to use Metric Math to define an Alarm directly in CloudFormation ?
It is actually possible to implement the Alarm logic directly in CloudFormation.
Assuming to have two Scaling Policies ECSScaleUp and ECSScaleDown, the alarm definition will look like:
ECSWorkerSQSCumulativeAlarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: !Join ['-', [!Ref 'MyService', 'SQSCumulativeAlarm']]
AlarmDescription: "Trigger ECS Service Scaling based on TWO SQS queues"
Metrics:
- Id: e1
Expression: "fq + sq"
Label: "Sum of the two Metrics"
- Id: fq
MetricStat:
Metric:
MetricName: ApproximateNumberOfMessagesVisible
Namespace: AWS/SQS
Dimensions:
- Name: QueueName
Value: !GetAtt [ FirstQueue, QueueName]
Period: 60
Stat: Average
Unit: Count
ReturnData: false
- Id: sq
MetricStat:
Metric:
MetricName: ApproximateNumberOfMessagesVisible
Namespace: AWS/SQS
Dimensions:
- Name: QueueName
Value: !GetAtt [ SecondQueue, QueueName]
Period: 60
Stat: Average
Unit: Count
ReturnData: false
EvaluationPeriods: 2
Threshold: 100
ComparisonOperator: GreaterThanThreshold
AlarmActions:
- !Ref ECSScaleUp
- !Ref ECSScaleDown
OKActions:
- !Ref ECSScaleUp
- !Ref ECSScaleDown

How to determine ipv6 CIDR block prefix in AWS Cloudformation when creating subnets on a VPC

AWS generates the ipv6 CIDR block for VPCs so its not possible to determine ahead of time. The generated CIDR block looks something like: 2a05:d018:84c:c500::/56 and is always size 56.
When creating a subnet you have to specify a size 64 block using the full prefixed value. E.g. 2a05:d018:84c:c501::/64.
It's possible to look up the ipv6 CIDR blocks for a VPC in cloudformation, but this returns the full value, not just the prefix. To create a subnet we need to be able to append something 01::/64 to the prefix to create the 64 sized block for the subnet.
I've seen solutions that use a lambda function, but this greatly complicated the templates. I'd like to do this using just the built-in intrinsic functions available in the templates.
When deploying a VPC with ipv6 subnets in the same stack, how can you generate valid ipv6 CIDR blocks for the subnets?
Here is a one liner that does the same thing using the Fn::Cidr intrinsic function.
!Select [1, !Cidr [!Select [0, !GetAtt 'Vpc.Ipv6CidrBlocks'], 256, 64]]
For a given block 2a05:d018:84c:c500::/56 this will give you 2a05:d018:84c:c501::/64
Increment the first index to get the next block.
!Select [2, !Cidr [!Select [0, !GetAtt 'Vpc.Ipv6CidrBlocks'], 256, 64]]
will give you 2a05:d018:84c:c502::/64
Also here is a full minimal example including the crucial steps of using an AWS::EC2::VPCCidrBlock resource to attach the IPv6 block to the VPC and using the DependsOn property to make sure that the VPCCidrBlock is attached before the Subnet is created.
Resources:
Vpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Sub '10.255.0.0/16'
VpcCidrBlockIpv6:
Type: 'AWS::EC2::VPCCidrBlock'
Properties:
VpcId: !Ref 'Vpc'
AmazonProvidedIpv6CidrBlock: true
PrivateSubnet:
Type: AWS::EC2::Subnet
DependsOn: VpcCidrBlockIpv6 # Wait for IPv6 CIDR to be attached to VPC before creating subnet
Properties:
AvailabilityZone: !Select [ 0, !GetAZs '' ]
VpcId: !Ref 'Vpc'
AssignIpv6AddressOnCreation: true
CidrBlock: !Sub '10.255.0.0/20'
Ipv6CidrBlock: !Select [1, !Cidr [!Select [0, !GetAtt 'Vpc.Ipv6CidrBlocks'], 256, 64]]
AWS VPC Service allows us to create a VPC and associate an Amazon provided /56 IPv6 CIDR block [We cannot provide our own CIDR].
Each subnet can have an IP block of /64. Which means that there can be theoretically 2^(64-56)subnets which is 2^8 = 256 subnets.
For calculating the Subnets for the IPv6 CIDR block, Cloudformation provides an intrinsic function !Cidr for us to do just that.
Syntax of the CIDR Block goes as follows:
!Cidr [ ipBlock, count, cidrBits ]
where
count --> The number of CIDRs to generate. Valid range is between 1 and 256.
cidrBits -->The number of subnet bits for the CIDR. For example, specifying a value "8" for this parameter will create a CIDR with a mask of "/24".
So our statement Becomes:
!Select [0, !Cidr [!Select [0, !GetAtt 'Vpc.Ipv6CidrBlocks'], 256, 64]]
So a sample template to create a VPC with 2 subnets with IPv6 attached to them looks like this:
AWSTemplateFormatVersion: "2010-09-09"
Resources:
Vpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Sub '10.255.0.0/16'
VpcCidrBlockIpv6:
Type: 'AWS::EC2::VPCCidrBlock'
Properties:
VpcId: !Ref 'Vpc'
AmazonProvidedIpv6CidrBlock: true
Subnet1:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select [ 0, !GetAZs '' ]
VpcId: !Ref 'Vpc'
AssignIpv6AddressOnCreation: true
CidrBlock: !Sub '${PrivateSubnetCIDR1}'
Ipv6CidrBlock: !Select [0, !Cidr [!Select [0, !GetAtt 'Vpc.Ipv6CidrBlocks'], 256, 64]]
Subnet2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref 'Vpc'
AssignIpv6AddressOnCreation: true
CidrBlock: !Sub '${PrivateSubnetCIDR2}'
Ipv6CidrBlock: !Select [1, !Cidr [!Select [0, !GetAtt 'Vpc.Ipv6CidrBlocks'], 256, 64]]
Here's a way to calculate the first subnet in YAML:
Fn::Sub:
- "${VpcPart}${SubnetPart}"
- SubnetPart: 01::/64
VpcPart: !Select [0, !Split ['00::/56', !Select [0,!GetAtt YourVpc.Ipv6CidrBlocks]]]
You can determine the prefix by using a combination of Fn::Split (on 00::/56) and Fn::Select to get the prefix. Then you can append your own value to create the subnet CIDR blocks using Fn::Join. The following example assumes you have a VPC with one or more Ipv6 CIDR blocks associated with it.
Use this value for the Ipv6CidrBlock property on the subnet.
{
"Fn::Join": [
"",
[
{
"Fn::Select": [
0,
{
"Fn::Split": [
"00::/56",
{
"Fn::Select": [
0,
{
"Fn::GetAtt": [
"Vpc",
"Ipv6CidrBlocks"
]
}
]
}
]
}
]
},
"01::/64"
]
]
}