Nested Fn::ImportValue in Fn::Sub not working for SAM template - aws-api-gateway

Description:
I am trying to define Serverless API resource. But having trouble in defining location of swagger specification file using function ImportValue.
Steps to reproduce the issue:
I am not able to define AWS::Serverless::Api resource having nested function ImportValue in Location. I have tried following three ways, none of them work.
Note: Stack parameters are defined properly and export value from other stack exists. Not showing them here for brevity reason.
ApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
Name: !Sub ${AWS::StackName}-API
StageName: !Ref ApiGatewayStageName
DefinitionBody:
'Fn::Transform':
Name: 'AWS::Include'
Parameters:
Location:
Fn::Sub:
- s3://${BucketName}/${SwaggerSpecificationS3Key}
- BucketName:
Fn::ImportValue:
!Sub "${EnvironmentName}-dist-bucket-${AWS::Region}"
ApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
Name: !Sub ${AWS::StackName}-API
StageName: !Ref ApiGatewayStageName
DefinitionBody:
'Fn::Transform':
Name: 'AWS::Include'
Parameters:
Location:
Fn::Sub:
- s3://${BucketName}/${SwaggerSpecificationS3Key}
- BucketName:
!ImportValue 'dev-dist-bucket-us-east-1'
ApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
Name: !Sub ${AWS::StackName}-API
StageName: !Ref ApiGatewayStageName
DefinitionBody:
'Fn::Transform':
Name: 'AWS::Include'
Parameters:
Location:
Fn::Sub:
- s3://${BucketName}/${SwaggerSpecificationS3Key}
- BucketName:
Fn::ImportValue: 'dev-dist-bucket-us-east-1'
Cloudformation shows following error.
FAILED - The value of parameter Location under transform Include must
resolve to a string, number, boolean or a list of any of these.
However, if I do not use ImportValue it works with a nested Fn::Sub
ApiGatewayApi:
Type: AWS::Serverless::Api
Properties:
Name: !Sub ${AWS::StackName}-API
StageName: !Ref ApiGatewayStageName
DefinitionBody:
'Fn::Transform':
Name: 'AWS::Include'
Parameters:
Location:
Fn::Sub:
- s3://${BucketName}/${SwaggerSpecificationS3Key}
- BucketName:
Fn::Sub: dist-bucket-${EnvironmentName}-${AWS::Region}
Is it because of Fn::Transform or AWS::Include?

Related

AWS Cloudformation - security group ids list export and import - SecurityGroupIds not valid

Im working with 2 nested stacks. I need to use security group ids exported from NestedA in NestedB. The exported security group ids are to be used in a SecurityGroupIds property in NestedB based on conditions.
However cloudformation returns error: Property validation failure: [Value of property {/LaunchTemplateData/SecurityGroupIds/0} does not match type {String}]
The following are snippets of what I have tried:
NestedA export
Outputs:
SG1
Value: !Join
- ','
- - !Ref securitygroup1
- !Ref securitygroup2
Export:
Name: !Sub ${ExportVpcStackName}-SG1
SG2
Value: !Join
- ','
- - !Ref securitygroup3
- !Ref securitygroup4
Export:
Name: !Sub ${ExportVpcStackName}-SG2
ParentStack
Resources:
...
launchtemplate:
Type: AWS::Cloudformation::Stack
Properties:
TemplateURL: https://s3/nestedB.yaml
...
SG1:
Fn::ImportValue: !Sub ${ExportVpcStackName}-SG1
SG2:
Fn::ImportValue: !Sub ${ExportVpcStackName}-SG2
NestedB import
Parameters:
SG1
Type: List<AWS::EC2::SecurityGroup::Id>
SG2
Type: List<AWS::EC2::SecurityGroup::Id>
Resources:
launchtemplate:
Type: AWS::EC2::LaunchTemplate
Properties:
LaunchTemplateData:
...
SecurityGroupIds:
!If
- Condition1
-
- !Ref SG1
- !Ref SG2
- !If
- Condition2
-
- !Ref SG1
- !Ref AWS::NoValue
Ive also tried importing each of the security groups directly/individually into NestedB with no success ie:
NestedA export
Outputs:
securitygroup1:
Value: !Ref securitygroup1
Export:
Name: !Sub ${ExportVpcStackName}-securitygroup1
securitygroup2:
Value: !Ref securitygroup2
Export:
Name: !Sub ${ExportVpcStackName}-securitygroup2
securitygroup3:
Value: !Ref securitygroup3
Export:
Name: !Sub ${ExportVpcStackName}-securitygroup3
securitygroup4:
Value: !Ref securitygroup4
Export:
Name: !Sub ${ExportVpcStackName}-securitygroup4
NestedB import
Resources:
launchtemplate:
Type: AWS::EC2::LaunchTemplate
Properties:
LaunchTemplateData:
...
SecurityGroupIds:
!If
- Condition1
-
- Fn::ImportValue: !Sub ${ExportVpcStackName}-securitygroup1
- Fn::ImportValue: !Sub ${ExportVpcStackName}-securitygroup2
- Fn::ImportValue: !Sub ${ExportVpcStackName}-securitygroup3
- Fn::ImportValue: !Sub ${ExportVpcStackName}-securitygroup4
- !If
- Condition2
-
- Fn::ImportValue: !Sub ${ExportVpcStackName}-securitygroup1
- Fn::ImportValue: !Sub ${ExportVpcStackName}-securitygroup2
- !Ref AWS::NoValue
Whats the error Im making?
Edit: I have tried #marcin suggestion but still get the error:
Property validation failure: [Value of property {/LaunchTemplateData/SecurityGroupIds/0} does not match type {String}]
Instead of Type: List<AWS::EC2::SecurityGroup::Id>, please use CommaDelimitedList.
Also your SG1 is a list of SGs. You have to use Fn::Select to get individual SG values from the list.

how to pass bucket/key name to fargate job via a cloudwatch event trigger on s3 object creation event?

I have create a fargate task and trying to trigger it via s3 object creation event ( see sample below) via cloudformation.as it cannot trigger it directly, i have created a cloudwatchevent. I am trying to pass the bucket and obj name to my fargate task code . doing some research, i came across -> InputTransformer, but i'm not sure how to pass the value of my bucket and key name and how to read it in my python code. any help will be appreciated.
AWSTemplateFormatVersion: 2010-09-09
Description: An example CloudFormation template for Fargate.
Parameters:
VPC:
Type: AWS::EC2::VPC::Id
SubnetA:
Type: AWS::EC2::Subnet::Id
SubnetB:
Type: AWS::EC2::Subnet::Id
Image:
Type: String
Default: 123456789012.dkr.ecr.region.amazonaws.com/image:tag
Resources:
mybucket:
Properties:
BucketName: 'mytestbucket-us'
cloudwatchEvent:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.s3
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
- CompleteMultipartUpload
requestParameters:
bucketName:
- !Ref mybucket
Targets:
- Id: my-fargate-task
Arn: myclusterArn
RoleArn: myinvocationrolearn
Input:
'Fn::Sub':
- >-
{"containerOverrides": [{"name":"somecontainer"]}
EcsParameters:
TaskDefinition:
LaunchType: 'FARGATE'
...
NetworkConfiguration:
...
Cluster:
Type: AWS::ECS::Cluster
Properties:
ClusterName: !Join ['', [!Ref ServiceName, Cluster]]
TaskDefinition:
Type: AWS::ECS::TaskDefinition
DependsOn: LogGroup
Properties:
Family: !Join ['', [!Ref ServiceName, TaskDefinition]]
NetworkMode: awsvpc
RequiresCompatibilities:
- FARGATE
Cpu: 256
Memory: 2GB
ExecutionRoleArn: !Ref ExecutionRole
TaskRoleArn: !Ref TaskRole
ContainerDefinitions:
- Name: !Ref ServiceName
Image: !Ref Image
# A role needed by ECS
ExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Join ['', [!Ref ServiceName, ExecutionRole]]
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: 'sts:AssumeRole'
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'
# A role for the containers
TaskRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Join ['', [!Ref ServiceName, TaskRole]]
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: ecs-tasks.amazonaws.com
Action: 'sts:AssumeRole'
You would use a CloudWatch Event Input Transformer to extract the data you need from the event, and pass that data to the ECS task as environment variable(s) in the target's ContainerOverrides. I don't use CloudFormation, but here's an example using Terraform.
You can't. CloudWatch events do not pass data to ECS jobs. You need to develop your own mechanism for that. For example, trigger lambda first, store event in S3 Parameter Store or DynamoDB, and then invoke your ECS job which will get stored data.

Export and ImportValue in another stack under sub

I am trying to get a value from one stack to another using the below syntax.
stack one-
Outputs:
CompRestAPI:
Description: Rest Api Id
Value: !Ref CompRestAPI
Export:
Name: 'CompRestAPI'
Stack two -
CompRestApiWaf:
Type: AWS::WAFv2::WebACLAssociation
DependsOn: CompApiGatewayStage
Properties:
RestApiId: !ImportValue 'CompRestAPI'
ResourceArn: !Sub 'arn:aws:apigateway:${REGION}:/${RestApiId}/${STAGENAME}-apistage'
WebACLArn: !Ref WafId
I am able to get the values for other resources using 1st syntax, but I am not able to get the value for RestApiId under !Sub
RestApiId: !ImportValue 'CompRestAPI'
ResourceArn: !Sub 'arn:aws:apigateway:${REGION}:/${RestApiId}/apistage'
So is there any way to use !ImportValue under !Sub condition?
I tried it using below code, validation is pass but still showing me an error
Error reason: The ARN isn't valid. A valid ARN begins with arn: and includes other information separated by colons or slashes., field: RESOURCE_ARN, parameter:
CompRestApiWaf:
Type: AWS::WAFv2::WebACLAssociation
DependsOn: CompApiGatewayStage
Properties:
ResourceArn: !Sub 'arn:aws:apigateway:${REGION}:/{!ImportValue CompRestAPI}/stages/apistage'
WebACLArn: !Ref WafId
I am done with it using Fn::join:
SourceArn:
Fn::Join:
- ""
- - 'arn:aws:execute-api:'
- !Ref AWS::Region
- ':'
- !Ref AWS::AccountId
- ':'
- !Ref ApiGatewayRestApiResource
- '/*'
this should work
ResourceArn: !Sub
- 'arn:aws:apigateway:${REGION}:/${CompRestAPI}/stages/apistage'
- CompRestAPI: !ImportValue CompRestAPI
you can expand the second parameter to have multiple keys for multiple imports like so
SecretString: !Sub
- 'postgres://${username}:${password}#${dbhost}:${dbport}/${dbname}'
- username: !Ref 'DBUser'
password: !Ref 'DBPassword'
dbhost: !Ref DbMasterDnsEntry
dbport: !GetAtt AuroraPgCluster.Endpoint.Port
dbname: !Ref 'DBName'

deploy multiple services using serverless to Apigateway with shared path

The documentation addresses shared paths:
service: service-b
provider:
apiGateway:
restApiId: xxxxxxxxxx
restApiRootResourceId: xxxxxxxxxx
restApiResources:
/reports: xxxxxxxxxx
functions:
...
However, how can I reference the ID of the resource (the path, that is)? In the first service I have:
Outputs:
apiGatewayRestApiId:
Value:
Ref: ApiGatewayRestApi
Export:
Name: ${self:service}-${opt:stage, 'dev'}-apiGateway-restApiId
apiGatewayRestApiRootResourceId:
Value:
Fn::GetAtt:
- ApiGatewayRestApi
- RootResourceId
Export:
Name: ${self:service}-${opt:stage, 'dev'}-apiGateway-rootResourceId
apiGatewayResourceReports:
Value: !Ref ApiGatewayResource/reports
Export:
Name: ${self:service}-${opt:stage, 'dev'}-apiGateway-reportPath
The first two work, and can be referred to with FN::ImportValue in the 2nd service. However, the third doesn't work. I presume the problem is
that I have to explicitly create the resource ApiGatewayResource/reports
rather than have it created as a side effect of the function definitions in the first service. But how should I do that? And won't it conflict with the function definitions?
After some floundering, I hit upon the following: the first service should
define the resource path, but leave the rest of the gateway definition implicit. And it should output the relevant ids:
provider:
apiGateway:
restApiResources:
/reports: !Ref ReportPath
...
resources:
Resources:
ReportPath:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref ApiGatewayRestApi
ParentId:
Fn::GetAtt: [ ApiGatewayRestApi, RootResourceId ]
PathPart: 'reports'
Outputs:
apiGatewayRestApiId:
Value:
Ref: ApiGatewayRestApi
Export:
Name: ${self:service}-${opt:stage, 'dev'}-apiGateway-restApiId
apiGatewayRestApiRootResourceId:
Value:
Fn::GetAtt:
- ApiGatewayRestApi
- RootResourceId
Export:
Name: ${self:service}-${opt:stage, 'dev'}-apiGateway-rootResourceId
apiGatewayResourceReports:
Value: !Ref ReportPath
Export:
Name: ${self:service}-${opt:stage, 'dev'}-apiGateway-reportPath
The 2nd service can reference all three ids:
provider:
apiGateway:
restApiId:
'Fn::ImportValue': crane-mg-reports-${opt:stage, 'dev'}-apiGateway-restApiId
restApiRootResourceId:
'Fn::ImportValue': crane-mg-reports-${opt:stage, 'dev'}-apiGateway-rootResourceId
restApiResources:
/reports:
'Fn::ImportValue': crane-mg-reports-${opt:stage, 'dev'}-apiGateway-reportPath
In both cases, the function definition can define paths using /reports prefix without conflict.

Getting an error trying to create an AWS API Gateway via Cloudformation

I'm trying to make a simple Cloudformation to create a website hosted on S3 with an API Gateway backend. Everything seems OK as far as I can tell but I get errors when trying to create the API Gateway:
Errors found during import: Unable to put integration on 'ANY' for resource at path '/{proxy+}': AWS ARN for integration must contain path or action (Service: AmazonApiGateway; Status Code: 400; Error Code: BadRequestException; Request ID: b28983d9-687c-11e8-8692-27df1db97456)
The gateway should just be a single route that sends everything to a single lambda. Should be super simple.
---
AWSTemplateFormatVersion: '2010-09-09'
Description: Website S3 Hosted, API Gateway Backend
Parameters:
DomainName:
Type: String
Description: The DNS name of an Amazon Route 53 hosted zone e.g. server.com
AllowedPattern: '(?!-)[a-zA-Z0-9-.]{1,63}(?<!-)'
ConstraintDescription: must be a valid DNS zone name.
Mappings:
S3RegionMap:
us-east-1:
S3HostedZoneId: Z3AQBSTGFYJSTF
S3WebsiteEndpoint: s3-website-us-east-1.amazonaws.com
us-west-1:
S3HostedZoneId: Z2F56UZL2M1ACD
S3WebsiteEndpoint: s3-website-us-west-1.amazonaws.com
us-west-2:
S3HostedZoneId: Z3BJ6K6RIION7M
S3WebsiteEndpoint: s3-website-us-west-2.amazonaws.com
eu-west-1:
S3HostedZoneId: Z1BKCTXD74EZPE
S3WebsiteEndpoint: s3-website-eu-west-1.amazonaws.com
ap-southeast-1:
S3HostedZoneId: Z3O0J2DXBE1FTB
S3WebsiteEndpoint: s3-website-ap-southeast-1.amazonaws.com
ap-southeast-2:
S3HostedZoneId: Z1WCIGYICN2BYD
S3WebsiteEndpoint: s3-website-ap-southeast-2.amazonaws.com
ap-northeast-1:
S3HostedZoneId: Z2M4EHUR26P7ZW
S3WebsiteEndpoint: s3-website-ap-northeast-1.amazonaws.com
sa-east-1:
S3HostedZoneId: Z31GFT0UA1I2HV
S3WebsiteEndpoint: s3-website-sa-east-1.amazonaws.com
Resources:
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: '/'
Policies:
- PolicyName: execution
PolicyDocument:
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: '*'
- Effect: Allow
Action:
- s3:GetObject
- s3:PutObject
- s3:ListBucket
Resource: '*'
- Effect: Allow
Action:
- ec2:DescribeNetworkInterfaces
- ec2:CreateNetworkInterface
- ec2:DeleteNetworkInterface
Resource: '*'
- Effect: Allow
Action:
- cognito-idp:AdminGetUser
- cognito-idp:AdminUpdateUserAttributes
Resource: '*'
APIGatewayExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service: apigateway.amazonaws.com
Action:
- sts:AssumeRole
Path: '/'
Policies:
- PolicyName: execution
PolicyDocument:
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: '*'
- Effect: Allow
Action:
- lambda:InvokeFunction
Resource: '*'
LambdaFunctionAPI:
Type: AWS::Lambda::Function
Properties:
Code:
ZipFile: exports.handler = function (event, context, callback) { callback(null, event); };
Handler: index.handler
MemorySize: 128
Role: !GetAtt LambdaExecutionRole.Arn
Runtime: nodejs4.3
Timeout: 30
APIGateway:
Type: AWS::ApiGateway::RestApi
Properties:
FailOnWarnings: true
Name: !Join ['-', !Split ['.', !Join ['.', ['api', !Ref DomainName]]]]
Body:
swagger: '2.0'
info:
version: 0.0.1
title: !Join [' ', ['API route for', !Ref DomainName]]
basePath: '/api'
paths:
'/{proxy+}':
options:
summary: CORS support
description: |
Enable CORS by returning correct headers
consumes:
- application/json
produces:
- application/json
tags:
- CORS
x-amazon-apigateway-integration:
type: mock
requestTemplates:
application/json: |
{
"statusCode" : 200
}
responses:
"default":
statusCode: "200"
responseParameters:
method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key'"
method.response.header.Access-Control-Allow-Methods: "'DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT'"
method.response.header.Access-Control-Allow-Origin: "'*'"
responseTemplates:
application/json: |
{}
responses:
'200':
description: Default response for CORS method
headers:
Access-Control-Allow-Headers:
type: "string"
Access-Control-Allow-Methods:
type: "string"
Access-Control-Allow-Origin:
type: "string"
x-amazon-apigateway-any-method:
produces:
- "application/json"
responses:
'200':
description: "200 response"
schema:
$ref: "#/definitions/Empty"
x-swagger-router-controller: main
x-amazon-apigateway-integration:
type: aws_proxy
httpMethod: POST
uri: !GetAtt LambdaFunctionAPI.Arn
credentials: !Ref APIGatewayExecutionRole
definitions:
Empty:
type: "object"
title: "Empty Schema"
APIDeployment:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId: !Ref APIGateway
Description: Deploy for live
StageName: Live
WebsiteBucket:
Type: AWS::S3::Bucket
Properties:
BucketName:
Ref: DomainName
AccessControl: PublicRead
WebsiteConfiguration:
IndexDocument: index.html
ErrorDocument: 404.html
Tags:
- Key: Name
Value: !Join ['_', ['WebsiteBucket', !Ref 'AWS::StackName']]
- Key: Domain
Value: !Ref DomainName
DeletionPolicy: Retain
WWWBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Join ['.', ['www', !Ref DomainName]]
AccessControl: PublicRead
WebsiteConfiguration:
RedirectAllRequestsTo:
HostName: !Ref WebsiteBucket
Tags:
- Key: Name
Value: !Join ['_', ['WWWBucket', !Ref 'AWS::StackName']]
- Key: Domain
Value: !Ref DomainName
WebsiteBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref WebsiteBucket
PolicyDocument:
Statement:
- Action:
- s3:GetObject
Effect: Allow
Resource: !Join ['', ['arn:aws:s3:::', !Ref WebsiteBucket, '/*']]
Principal: '*'
WWWBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref WWWBucket
PolicyDocument:
Statement:
- Action:
- s3:GetObject
Effect: Allow
Resource: !Join ['', ['arn:aws:s3:::', !Ref WWWBucket, '/*']]
Principal: '*'
DNS:
Type: AWS::Route53::HostedZone
Properties:
HostedZoneConfig:
Comment: !Join [' ', ['Hosted zone for', !Ref DomainName]]
Name: !Ref DomainName
HostedZoneTags:
- Key: Application
Value: Blog
DNSRecord:
Type: AWS::Route53::RecordSetGroup
DependsOn: DNS
Properties:
HostedZoneName:
Fn::Join: ['', [!Ref DomainName, '.']]
Comment: Zone records.
RecordSets:
- Name: !Ref DomainName
Type: A
AliasTarget:
HostedZoneId: !FindInMap [S3RegionMap, !Ref 'AWS::Region', S3HostedZoneId]
DNSName: !FindInMap [S3RegionMap, !Ref 'AWS::Region', S3WebsiteEndpoint]
- Name: !Join ['.', ['www', !Ref DomainName]]
Type: A
AliasTarget:
HostedZoneId: !FindInMap [S3RegionMap, !Ref 'AWS::Region', S3HostedZoneId]
DNSName: !FindInMap [S3RegionMap, !Ref 'AWS::Region', S3WebsiteEndpoint]
Outputs:
S3WebsiteURL:
Value: !GetAtt WebsiteBucket.WebsiteURL
Description: URL for website hosted on S3
The URI you should use to connect to the Lambda is not the Arn of the Lambda, but an API gateway invocation URI. Additionally, you need to change the credential line from a ref to the Arn of the execution role.
Here a short excerpt of the changed section:
x-amazon-apigateway-integration:
type: aws_proxy
httpMethod: POST
uri: !Sub "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${LambdaFunctionAPI.Arn}/invocations"
credentials: !GetAtt APIGatewayExecutionRole.Arn
For those who are using terraform and getting this error, the following works (note that the uri parameter should not be the lambda arn, but the invoke arn which is different):
resource "aws_api_gateway_integration" "ordersIntegration" {
rest_api_id = aws_api_gateway_rest_api.msdApi.id
resource_id = aws_api_gateway_resource.orders.id
http_method = aws_api_gateway_method.ordersget.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = "arn:aws:apigateway:${var.AWS_REGION}:lambda:path/2015-03-31/functions/${var.LAMBDA_FN_ARN}/invocations"
}
Where the LAMBDA_FN_ARN is simply "arn:aws:lambda:${var.AWS_REGION}:${var.AWS_ACCOUNT_ID}:function:${var.LAMBDA_NAME}"
Also, 2015-03-31 is a constant value that needs to be used.