Grant access to RDS layer using Cloudformation for app layer - aws-cloudformation

I have an RDS database that I bring up using Cloudformation. Now I have a Cloudformation document that brings up my app server tier. How can I grant my app servers access to the RDS instance?
If the RDS instance was created by my Cloudformation document, I know I could do this:
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"EC2VpcId" : { "Ref" : "VpcId" },
"DBSecurityGroupIngress": { "EC2SecurityGroupId": { "Fn::GetAtt": [ "AppServerSecurityGroup", "GroupId" ]} },
"GroupDescription" : "Frontend Access"
}
}
But the DBSecurityGroup will already exist by the time I run my app cloudformation. How can I update it?
Update Following what huelbois pointed out to me below, I understood that I could just create an AWS::EC2::SecurityGroupIngress in my app Cloudformation. As I am using a VPC and the code huelbois posted is for classic, I can confirm that this works:
In RDS Cloudformation:
"DbVpcSecurityGroup" : {
"Type" : "AWS::EC2::SecurityGroup",
"Properties" : {
"GroupDescription" : "Enable JDBC access on the configured port",
"VpcId" : { "Ref" : "VpcId" },
"SecurityGroupIngress" : [ ]
}
}
And in app Cloudformation:
"specialRDSRule" : {
"Type": "AWS::EC2::SecurityGroupIngress",
"Properties" : {
"IpProtocol": "tcp",
"FromPort": 5432,
"ToPort": 5432,
"GroupId": {"Ref": "DbSecurityGroupId"},
"SourceSecurityGroupId": {"Ref": "InstanceSecurityGroup"}
}
}
where DbSecurityGroupId is the id of the group setup above (something like sg-27324c43) and is a parameter to the app Cloudformation document.

When you want to use already existing resources in a CloudFormation template, you can use the previously created ids, instead of Ref or GetAtt.
In your example, you can use:
{ "EC2SecurityGroupId": "sg-xxxNNN" }
where "sg-xxxNNN" is the id of your DB SecurityGroup (not sure of the DB SecurityGroup prefix, since we don't use EC2-classic but VPC).
I would recommend using a parameter for your SecurityGroup in your template.
*** update **
For your specific setup, I would use a "DBSecurityGroupIngress" resource, to add a new sg to your RDS instance.
In your first stack (RDS), you create an empty DBSecurityGroup like this:
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"EC2VpcId" : { "Ref" : "VpcId" },
"DBSecurityGroupIngress": [],
"GroupDescription" : "Frontend Access"
}
}
This DBSecurityGroup is refered to by the DBInstance. (I guess you have specific requisites for using DBSecurityGroup instead of VPCSecurityGroup).
In your App stack, you create a DBSecurityGroupIngress resource, which is a child of the DBSecurityGroup your created in the first stack:
"specialRDSRule" : {
"Type":"AWS::RDS::DBSecurityGroupIngress",
"Properties" : {
"DBSecurityGroupName": "<the arn of the DBSecurityGroup>",
"CIDRIP": String,
"EC2SecurityGroupId": String,
"EC2SecurityGroupName": String,
"EC2SecurityGroupOwnerId": String
}
}
You need the arn of the DBSecurityGroup, which is "arn:aws:rds:::secgrp:". The other parameters come from your App stack, not sure if you need everything (I don't do EC2-classic security groups, only VPC).
Reference : http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-security-group-ingress.html
We use the same mechanism with VPC SecurityGroups, with Ingress & Egress rules, so we can have two SG reference each-other.

Related

How to create a transi gateway attachement in a VPC to a transit gateway created in another account (with Cloudformation)

I've created a TransitGateway in an account of my organisation and I've sharing it using a resources manager with the other accounts, I am trying now to use the ID of this transit gateway to create an attachement using Cloudformation in another account using the "id" of the transit gateway but this is not working.
I've tried to get the id of the TGW in the account where it is created and pass it in the Cloudformation in the account where I want to create the attachement:
TransitGatewayAttachment":
{
"Type" : "AWS::EC2::TransitGatewayAttachment",
"Properties" : {
"SubnetIds": [
{
"Ref": "PrivateSubnet1A"
}
],
"TransitGatewayId" :"tgw-xxxexxxxxxxx",
"Ref": "VPC"
}
}
You are missing VpcId. It should be:
TransitGatewayAttachment":
{
"Type" : "AWS::EC2::TransitGatewayAttachment",
"Properties" : {
"SubnetIds": [
{
"Ref": "PrivateSubnet1A"
}
],
"TransitGatewayId" :"tgw-xxxexxxxxxxx",
"VpcId": {"Ref": "VPC"}
}
}

AWS RDS Stack update always replaces the DB Cluster

I first restored an Aurora RDS Cluster using a cluster snapshot with a cloud formation template. Then removed the snapshot identifier, updated the password and performed a stack update keeping everything else unchanged in the CFT. But stack always prints the
Requested update requires the creation of a new physical resource;
hence creating one.
message and start creating a new cluster. Here is my CFT for the cluster.
"DatabaseCluster": {
"Type": "AWS::RDS::DBCluster",
"DeletionPolicy": "Snapshot",
"Properties": {
"BackupRetentionPeriod": {
"Ref": "BackupRetentionPeriod"
},
"Engine": "aurora-postgresql",
"EngineVersion": {
"Ref": "EngineVersion"
},
"Port": {
"Ref": "Port"
},
"MasterUsername": {
"Fn::If" : [
"isUseDBSnapshot",
{"Ref" : "AWS::NoValue"},
{"Ref" : "MasterUsername"}
]
},
"MasterUserPassword": {
"Fn::If" : [
"isUseDBSnapshot",
{"Ref" : "AWS::NoValue"},
{"Ref" : "MasterPassword"}
]
},
"DatabaseName": {
"Fn::If" : [
"isUseDBSnapshot",
{"Ref" : "AWS::NoValue"},
{"Ref" : "DBName"}
]
},
"SnapshotIdentifier" : {
"Fn::If" : [
"isUseDBSnapshot",
{"Ref" : "SnapshotIdentifier"},
{"Ref" : "AWS::NoValue"}
]
},
"PreferredBackupWindow": "01:00-02:00",
"PreferredMaintenanceWindow": "mon:03:00-mon:04:00",
"DBSubnetGroupName": {"Ref":"rdsDbSubnetGroup"},
"StorageEncrypted":{"Ref" : "StorageEncrypted"},
"DBClusterParameterGroupName": {"Ref" : "RDSDBClusterParameterGroup"},
"VpcSecurityGroupIds": [{"Ref" : "CommonSGId"}]
}
}
According to the AWS RDS CFT doc MasterUserPassword update doesn't need a cluster replacement.
Is there anything wrong with my CFT or is this an issue with AWS?
If you just wish to update the password of the DB instance, you shouldn't remove the Snapshot identifier. I understand that you might be worried of losing data if the snapshot is being restored.
However, that is not the case with Cloudformation. Cloudformation precisely checks what changes you have made and performs a relevant operation. If you are changing just the password, then it will not tamper your data - whatever state it is in.
However, if you remove the snapshot identifier means you want to change the DB and remove the snapshot from it. So it will replace your DB instance.
Check the below link for more details on what happens on changing each parameter.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html#cfn-rds-dbcluster-snapshotidentifier
It clearly specifies that any chance in snapshot identifier will result in replacement

Cannot connect to EC2 PostgreSQL database from ASP.Net Core lambda function

I am new to AWS and my question is regarding connection to Postgresql database hosted as a EC2 instance in AWS.
I have an Asp.net core web api published as a AWS Serverless Application and an endpoint which connects to the mentioned DB. When running the api on localhost or connecting to DB from any DB client everything is ok but it does not work when testing on AWS. I assume it is connected to security configurations of EC2 but don't know how to figure it out.
Here's the serverless.template code
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Transform" : "AWS::Serverless-2016-10-31",
"Description" : "Starting template for an AWS Serverless Application.",
"Parameters" : {
},
"Resources" : {
"DefaultFunction" : {
"Type" : "AWS::Serverless::Function",
"Properties": {
"Handler": "AwsApp.WebApi::AwsApp.WebApi.LambdaEntryPoint::FunctionHandlerAsync",
"Runtime": "dotnetcore2.1",
"CodeUri": "",
"Description": "Default function",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [ "AWSLambdaFullAccess" ],
"Events": {
"ProxyResource": {
"Type": "Api",
"Properties": {
"Path": "/{proxy+}",
"Method": "ANY"
}
}
}
}
}
},
"Outputs" : {
"ApiURL" : {
"Description" : "API endpoint URL for Prod environment",
"Value" : { "Fn::Sub" : "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/" }
}
}
}
And this is the error message
An exception has been raised that is likely due to a transient failure.
at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
at lambda_method(Closure )
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ResultEnumerable`1.GetEnumerator()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider._TrackEntities[TOut,TIn](IEnumerable`1 results, QueryContext queryContext, IList`1 entityTrackingInfos, IList`1 entityAccessors)+MoveNext()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable`1 source, Boolean& found)
at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass15_1`1.<CompileQueryCore>b__0(QueryContext qc)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable`1 source, Expression`1 predicate)
at Microsoft.EntityFrameworkCore.Internal.EntityFinder`1.Find(Object[] keyValues)
at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.Find(Object[] keyValues)
at AwsApp.WebApi.Controllers.ValuesController.Get(Int32 id) in D:\TEST\.NET Core\AwsApp\AwsApp.WebApi\Controllers\ValuesController.cs:line 33
Also it is strange why there's a path to the file within the error message
Edit
PostgreSQL is running under VPC.
There's a dummy Get method in controller which returns a simple array:
[HttpGet]
public ActionResult<IEnumerable<string>> Get()
{
return new string[] { "value1", "value2" };
}
When I set my lambda function VPC then this function also fails with error 500.
Solution
I was able to fix it.
The role assigned to my lambda function didn't have required policies (see here).
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:CreateNetworkInterface",
"ec2:DescribeNetworkInterfaces",
"ec2:DeleteNetworkInterface"
],
"Resource": "*"
}
]
}
John,
There can be multiple issues....
If your PostgreSQL is running inside a VPC with private subnet configured, open the port in firewall (inbound traffic) so that your lambda can connect.
You need to create a new role with permission to PostgreSQL and assign it to the Lambda.
Please try and post your comments.

Cloudformation: Why does RDS instance creates default security group when i am already creating one while initating RDS?

This is my RDS instance, I am creating a security group which gives access to my Workbench and backend code. RDS creates default security group, which overlaps the security group i create and thus makes it not accessible. How can i stop RDS create default security group.
Here is my RDS template
"Resources": {
"epmoliteDB": {
"Type": "AWS::RDS::DBInstance",
"Properties": {
"DBName": {"Ref": "DBname"},
"DBSecurityGroups": [{"Ref": "DBSecurityGroup"}],
"AllocatedStorage": "5",
"DBInstanceClass": "db.t2.micro",
"Engine": "MySQL",
"MasterUsername": {"Ref": "DBuser"},
"MasterUserPassword": {"Ref": "DBpass"},
"DBParameterGroupName": {"Ref": "epmoliteDBParameterGroup"}
}
},
"DBSecurityGroup": {
"Type": "AWS::RDS::DBSecurityGroup",
"Properties": {
"DBSecurityGroupIngress": {
"EC2SecurityGroupName": {"Ref": "WebServerSecurityGroup"}
},
"GroupDescription": "Frontend Access"
}
},
"WebServerSecurityGroup": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription" : "Enable MYSQL access via port 3306",
"SecurityGroupIngress": [{
"IpProtocol": "tcp","FromPort": "3306","ToPort": "3306","CidrIp": "0.0.0.0/0"
}]
}
},
"epmoliteDBParameterGroup": {
"Type": "AWS::RDS::DBParameterGroup",
"Properties" : {
"Description" : "Parameter group to avoid schema import errors",
"Family" : "MySQL5.7",
"Parameters" : {
"log_bin_trust_function_creators": "1"
}
}
}
}
I can't exactly explain why a default security group is created and overlap with the one you specified. What I can tell you though is that you should really rely on VPCSecurityGroups which replaces the old DBSecurityGroups which was relevant in "EC2 Classic" (before the VPC era). Perhaps this will solve the issue.
There's an article in the doc to learn more about this: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.RDSSecurityGroups.html#Overview.RDSSecurityGroups.Compare.

How to set user name and group name in IAM using CloudFormation?

I created a CloudFormation template and I wanted to create IAM user, to do that I used this JSON string:
"CFNUser" : {
"Type" : "AWS::IAM::User",
"Properties" : {
"LoginProfile": {
"Password": { "Ref" : "AdminPassword" }
}
}
},
Then for group I used this:
"CFNUserGroup" : {
"Type" : "AWS::IAM::Group"
},
After creating the stack, I got the following:
user name - IAMUsers-CFNUser-E1BT342YK7G6
group name - IAMUsers-CFNUserGroup-1UBUBRYALTIMI
So my question is, how can I set the user name here? same goes for the group name?
After talking with one of the AWS support, at this time of writing, it is not possible to specify your own username and group name in IAM using CloudFormation template :-(
Maybe there's a reason why they do not allow user to do this...anyway it's good thing that I have answer to this question and I will be glad if someone find this useful.
Amazon has added support from 20th July 2016.
https://aws.amazon.com/about-aws/whats-new/2016/07/aws-cloudformation-adds-support-for-aws-iot-and-additional-updates/
{
"Type": "AWS::IAM::User",
"Properties": {
"Groups": [ String, ... ],
"LoginProfile": LoginProfile Type,
"ManagedPolicyArns": [ String, ... ],
"Path": String,
"Policies": [ Policies, ... ],
"UserName": String
}
}
For groups, it's a GroupName property:
"CFNUserGroup" : {
"Type" : "AWS::IAM::Group",
"Properties": {
"GroupName": "My_CFN_User_Group"
}
}