!ImportValue in Serverless Framework not working - aws-cloudformation

I'm attempting to export a DynamoDb StreamArn from a stack created in CloudFormation, then reference the export using !ImportValue in the serverless.yml.
But I'm getting this error message:
unknown tag !<!ImportValue> in "/codebuild/output/src/serverless.yml"
The cloudformation and serverless.yml are defined as below. Any help appreciated.
StackA.yml
AWSTemplateFormatVersion: 2010-09-09
Description: Resources for the registration site
Resources:
ClientTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: client
AttributeDefinitions:
- AttributeName: id
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 2
WriteCapacityUnits: 2
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
Outputs:
ClientTableStreamArn:
Description: The ARN for My ClientTable Stream
Value: !GetAtt ClientTable.StreamArn
Export:
Name: my-client-table-stream-arn
serverless.yml
service: my-service
frameworkVersion: ">=1.1.0 <2.0.0"
provider:
name: aws
runtime: nodejs6.10
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:DescribeStream
- dynamodb:GetRecords
- dynamodb:GetShardIterator
- dynamodb:ListStreams
- dynamodb:GetItem
- dynamodb:PutItem
Resource: arn:aws:dynamodb:*:*:table/client
functions:
foo:
handler: foo.main
events:
- stream:
type: dynamodb
arn: !ImportValue my-client-table-stream-arn
batchSize: 1

Solved by using ${cf:stackName.outputKey}

I struggled with this as well, and what did trick for me was:
functions:
foo:
handler: foo.main
events:
- stream:
type: dynamodb
arn:
!ImportValue my-client-table-stream-arn
batchSize: 1
Note, that intrinsic functions ImportValue is on a new line and indented, otherwise the whole event is ignored when cloudformation-template-update-stack.json is generated.

It appears that you're using the !ImportValue shorthand for CloudFormation YAML. My understanding is that when CloudFormation parses the YAML, and !ImportValue actually aliases Fn::ImportValue. According to the Serverless Function documentation, it appears that they should support the Fn::ImportValue form of imports.
Based on the documentation for Fn::ImportValue, you should be able to reference the your export like
- stream:
type: dynamodb
arn: {"Fn::ImportValue": "my-client-table-stream-arn"}
batchSize: 1
Hope that helps solve your issue.

I couldn't find it clearly documented anywhere but what seemed to resolve the issue for me is:
For the Variables which need to be exposed/exported in outputs, they must have an "Export" property with a "Name" sub-property:
In serverless.ts
resources: {
Resources: resources["Resources"],
Outputs: {
// For eventbus
EventBusName: {
Export: {
Name: "${self:service}-${self:provider.stage}-UNIQUE_EVENTBUS_NAME",
},
Value: {
Ref: "UNIQUE_EVENTBUS_NAME",
},
},
// For something like sqs, or anything else, would be the same
IDVerifyQueueARN: {
Export: {
Name: "${self:service}-${self:provider.stage}-UNIQUE_SQS_NAME",
},
Value: { "Fn::GetAtt": ["UNIQUE_SQS_NAME", "Arn"] },
}
},
}
Once this is deployed you can check if the exports are present by running in the terminal (using your associated aws credentials):
aws cloudformation list-exports
Then there should be a Name property in a list:
{
"ExportingStackId": "***",
"Name": "${self:service}-${self:provider.stage}-UNIQUE_EVENTBUS_NAME", <-- same as given above (but will be populated with your service and stage)
"Value": "***"
}
And then if the above is successful, you can reference it with "Fn::ImportValue" like so, e.g.:
"Resource": {
"Fn::ImportValue": "${self:service}-${self:provider.stage}-UNIQUE_EVENTBUS_NAME", <-- same as given above (but will be populated with your service and stage)
}

Related

How to access CloudWatch Event data from triggered Fargate task?

I read the docs on how to Run an Amazon ECS Task When a File is Uploaded to an Amazon S3 Bucket. However, this document stops short of explaining how to get the bucket/key values from the triggering event from within the Fargate task code itself. How can that be done?
I am not sure if you still need the answer for this one. But I did something similar to what Steven1978 mentioned but only using CloudFormation.
The config you're looking for is the InputTransformer. Check this example for a YAML CloudFormation template for an Event Rule:
rEventRuleForFileUpload:
Type: AWS::Events::Rule
Properties:
Description: "EventRule"
State: "ENABLED"
EventPattern:
source:
- "aws.s3"
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventSource:
- s3.amazonaws.com
eventName:
- "PutObject"
- "CompleteMultipartUpload"
requestParameters:
bucketName: "{YOUR_BUCKET_NAME}"
Targets:
- Id: '{YOUR_ECS_CLUSTER_ID}'
Arn: !Sub "arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:cluster/${NAME_OF_YOUR_CLUSTER_RESOURCE}"
RoleArn: !GetAtt {YOUR_ROLE}.Arn
EcsParameters:
TaskCount: 1
TaskDefinitionArn: !Ref {YOUR_TASK_DEFINITION}
LaunchType: FARGATE
{... WHATEVER CONFIG YOU MIGHT HAVE...}
InputTransformer:
InputPathsMap:
s3_bucket: "$.detail.requestParameters.bucketName"
s3_key: "$.detail.requestParameters.key"
InputTemplate: '{ "containerOverrides": [ { "name": "{THE_NAME_OF_YOUR_CONTAINER_DEFINITION}", "environment": [ { "name": "EVENT_BUCKET", "value": <s3_bucket> }, { "name": "EVENT_OBJECT_KEY", "value": <s3_key> }] } ] }'
With this approach, you'll be able to get the s3 bucket name (EVENT_BUCKET) and the s3 object key (EVENT_OBJECT_KEY) as environment variables inside your container.
The info isn't very clear, indeed, but here are some sources I used to finally get it working:
Container Override;
https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ContainerOverride.html
InputTransformer:
https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_InputTransformer.html#API_InputTransformer_Contents

Get Lambda Arn into Resources : Type: AWS::Lambda::Permission

I have the following in my serverless yml file :
lambdaQueueFirstInvokePermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: ServiceLambdaFunctionQualifiedArn
Action: ‘lambda:InvokeFunction’
Principal: sqs.amazonaws.com
and I have the following in the Outputs section :
Outputs:
ServiceLambdaFunctionQualifiedArn:
Value:
‘Fn::GetAtt’: [ lambdaQueueFirst, Arn ]
this comes back with a message:
Template error: instance of Fn::GetAtt references undefined resource lambdaQueueFirst
Am I missing something and if so, what? since it is very little in terms of help or examples…
Also, is there a better of getting the lambda arn into the permissions code? if so, what is it?
You can use the environment variables to construct the ARN value. In your case, you can define a variable in your provider section like below. You might need to modify a little bit according to your application.
service: serverless App2
provider:
name: aws
runtime: python3.6
region: ap-southeast-2
stage: dev
environment:
AWS_ACCOUNT: 1234567890 # use your own AWS ACCOUNT number here
# define the ARN of the function that you want to invoke
FUNCTION_ARN: "arn:aws:lambda:${self:provider.region}:${self:provider.environment.AWS_ACCOUNT}:function:${self:service}-${self:provider.stage}-lambdaQueueFirst"
Outputs:
ServiceLambdaFunctionQualifiedArn:
Value: "${self:provider.environment.FUNCTION_ARN}"
See this and serverless variables for aws for example.
you can do this:
resources:
Resources:
LoggingLambdaPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: { "Fn::GetAtt": ["LoghandlerLambdaFunction", "Arn" ] }
Action: lambda:InvokeFunction
Principal: { "Fn::Join" : ["", ["logs.", { "Ref" : "AWS::Region"}, ".amazonaws.com" ] ] }
reference:
https://github.com/andymac4182/serverless_example

Cloudformation LaunchTemplate referencing IamInstanceProfile fails to create

I am trying to create a LaunchTemplate, which references an IamInstanceProfile, in my Cloudformation stack. Here is the code- i have omitted the irrelevant parts:
...
Resources:
ServerLaunchTemplate:
Type: 'AWS::EC2::LaunchTemplate'
Properties:
LaunchTemplateData:
InstanceType: !Ref InstanceType
SecurityGroups:
- !Ref SecGroup
IamInstanceProfile: !Ref ServerProfile
UserData:
...
ServerProfile:
Type: 'AWS::IAM::InstanceProfile'
Properties:
Path: /
Roles:
- !Ref ServerRole
...
The ServerProfile gets created successfully. However when the stack creation process reaches the step of creating the ServerLaunchTemplate, it fails with the error:
Property validation failure: [Value of property {/LaunchTemplateData/IamInstanceProfile} does not match type {Object}]
If i omit the reference to the IamInstanceProfile, the LaunchTemplate get created successfully.
According to the documentation and some examples this should work... Based on the error i understand, that the InstanceType field of the LaunchTemplate needs to reference an object, but "!Ref InstanceType" returns the resource id.
How can i fix this? How could i retrieve the object, that is presumably required by the "/LaunchTemplateData/IamInstanceProfile" field?
Thank you
Easy to miss in the docs: IamInstanceProfile requires an IamInstanceProfile Cloudformation object with the Arn of the referenced IamInstanceProfile being a property of it.
See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-launchtemplate-launchtemplatedata.html#cfn-ec2-launchtemplate-launchtemplatedata-iaminstanceprofile and https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-launchtemplate-iaminstanceprofile.html.
This should work:
PortalLaunchTemplate:
Type: 'AWS::EC2::LaunchTemplate'
Properties:
LaunchTemplateName: !Sub ${InstanceName}-launch-template
LaunchTemplateData:
ImageId: !Ref AmiId
...
IamInstanceProfile:
Arn: !GetAtt InstanceProfile.Arn
What worked for me was:
"IamInstanceProfile":{ "Arn":
{
"Fn::Sub":"arn:aws:iam::${AWS::AccountId}:instanceprofile/${name_of_Instance_profile}"
}
Presumably because it requires the "Arn" value as a string.

Cannot deploy aws sam stack due to Handler not found error

I am having issues deploying a lambda with a handler in a nested directory using sam.
I perform the following steps:
package:
sam package --template template.yaml --output-template-file packaged.yaml --s3-bucket
Creates a packaged.yaml that I use in the next step.
deploy:
aws cloudformation deploy --template-file /Users/localuser/Do/learn-sam/dynamo-stream-lambda/packaged.yaml --stack-name barkingstack
ERROR
Failed to create the changeset: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state Status: FAILED. Reason: Transform AWS::Serverless-2016-10-31 failed with: Invalid Serverless Application Specification document. Number of errors found: 1. Resource with id [PublishNewBark] is invalid. Missing required property 'Handler'.
Cloudformation/SAM Template
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Globals:
Function:
Runtime: nodejs8.10
Timeout: 300
Resources:
PublishNewBark:
Type: AWS::Serverless::Function
FunctionName: publishNewBark
CodeUri: .
Handler: src/index.handler
Role: "<ROLE_ARN>"
Description: Reads from the DynamoDB Stream and publishes to an SNS topic
Events:
- ReceiveBark:
Type: DynamoDB
Stream: !GetAtt BarkTable.StreamArn
StartingPosition: TRIM_HORIZON
BatchSize: 1
BarkTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: BarkTable
KeySchema:
- KeyType: HASH
AttributeName: id
AttributeDefinitions:
- AttributeName: id
AttributeType: S
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
ProvisionedThroughput:
WriteCapacityUnits: 5
ReadCapacityUnits: 5
WooferTopic:
Type: AWS::SNS::Topic
Properties:
DisplayName: wooferTopic
TopicName: wooferTopic
Subscription:
- Endpoint: <my_email>
Protocol: email
DIRECTORY STRUCTURE
root_directory/
events/ (for sample events)
policies/ (for IAM Role to be created for the lambda using CLI)
src/index.js
package.json
node_modules
template.yaml
HANDLER CODE
async function handler (event, context) {
console.log(JSON.stringify(event, null, 2))
return {}
}
module.exports = {handler}
I believe you have to put everything except the resource type under "Properties".
Your function declaration should be:
PublishNewBark:
Type: AWS::Serverless::Function
Properties:
FunctionName: publishNewBark
CodeUri: .
Handler: src/index.handler
Role: "<ROLE_ARN>"
Description: Reads from the DynamoDB Stream and publishes to an SNS topic
Events:
- ReceiveBark:
Type: DynamoDB
Stream: !GetAtt BarkTable.StreamArn
StartingPosition: TRIM_HORIZON
BatchSize: 1

Mappings sections of the aws Cloudformation template and Serverless.yml

I have a little doubt about "Mappings section" of the aws cloudformation syntax:
Example:
...
Mappings:
accounts:
56565d644801:true
986958470041:true
090960219037:true
05166767667:false
functions:
MyFunction:
handler: src/MyFunction/func.lambda_handler
role: MyRole
events:
- schedule:
rate: rate(12 hours)
enabled: Fn::FindInMap
- accounts
- Ref "AWS::AccountId"
...
Could the Mappings section be included in a serverless.yml file ?
I meant, eventhough it is a valid cloudformation syntax, would it possible include it in the serverless.yml, so that later we can implement it (serverless | sls deploy ...)?
thanks,
You might be able to use:
functions:
# ...
resources:
Mappings:
accounts:
56565d644801:true
986958470041:true
090960219037:true
05166767667:false
Just another way to work with mapping is through stage params.
https://www.serverless.com/framework/docs/guides/parameters
params:
stage1:
schedule:true
stage2:
schedule:false
functions:
MyFunction:
handler: src/MyFunction/func.lambda_handler
role: MyRole
events:
- schedule:
rate: rate(12 hours)
enabled: ${param:schedule}
Then call adding the stage arg (default is dev)
serverless deploy --stage stage1