Adding lambda integration to HttpApi routes with SAM - aws-cloudformation

I am currently attempting to have a AWS::Serverless::HttpApi integrate with a group of AWS::Serverless::Function's. The goal is to define these resources within a SAM template, and define the actual API using a swagger file.
I have my SAM template defined as so:
Resources:
apiPing:
Type: AWS::Serverless::Function
Properties:
Description: 'Ping'
CodeUri: ../bin/cmd-api-ping.zip
Handler: cmd-api-ping
Runtime: go1.x
Role:
Fn::GetAtt: apiLambdaRole.Arn
Events:
PingEvent:
Type: HttpApi
Properties:
ApiId: !Ref api
Path: /ping
Method: post
api:
Type: AWS::Serverless::HttpApi
Properties:
StageName: prod
DefinitionBody:
Fn::Transform:
Name: AWS::Include
Parameters:
Location: swagger.yaml
AccessLogSettings:
DestinationArn: !GetAtt accessLogs.Arn
Format: $context.requestId
And my swagger file:
openapi: 3.0.1
info:
title: 'API'
version: 2019-10-13
paths:
/ping:
post:
summary: 'invoke ping'
operationId: 'apiPing'
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/PingRequest'
required: true
responses:
'200':
description: 'Successful'
content:
application/json:
schema:
$ref: '#/components/schemas/PongResponse'
x-amazon-apigateway-integration:
httpMethod: "POST"
type: aws_proxy
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${apiPing.Arn}/invocations
responses:
default:
statusCode: "200"
contentHandling: "CONVERT_TO_TEXT"
passthroughBehavior: "when_no_match"
components:
schemas:
PingRequest:
description: 'a ping request'
type: object
properties:
ping:
description: 'some text'
type: string
PongResponse:
description: 'a pong response'
type: object
properties:
pong:
description: 'some text'
type: string
This template deploys without any errors, however there is no integration attached to the /ping POST route.
The transformed template in CloudFormation does show a loaded swagger file:
"api": {
"Type": "AWS::ApiGatewayV2::Api",
"Properties": {
"Body": {
"info": {
"version": 1570924800000,
"title": "API"
},
"paths": {
"/ping": {
"post": {
"requestBody": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PingRequest"
}
}
},
"required": true
},
"x-amazon-apigateway-integration": {
"contentHandling": "CONVERT_TO_TEXT",
"responses": {
"default": {
"statusCode": "200"
}
},
"uri": {
"Fn::Sub": "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${apiPing.Arn}/invocations"
},
"httpMethod": "POST",
"passthroughBehavior": "when_no_match",
"type": "aws_proxy"
},
"summary": "invoke ping",
"responses": {
"200": {
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PongResponse"
}
}
},
"description": "Successful"
}
},
"operationId": "apiPing"
}
}
},
"openapi": "3.0.1",
"components": {
"schemas": {
"PingRequest": {
"type": "object",
"description": "a ping request",
"properties": {
"ping": {
"type": "string",
"description": "some text"
}
}
},
"PongResponse": {
"type": "object",
"description": "a pong response",
"properties": {
"pong": {
"type": "string",
"description": "some text"
}
}
}
}
},
"tags": [
{
"name": "httpapi:createdBy",
"x-amazon-apigateway-tag-value": "SAM"
}
]
}
}
}
I'm trying to understand what I may need to change or add to add the integration to the http api. I can't find any clear explanation in the aws documentation.

I have managed to resolve this. aws::serverless::httpapi creates a AWS::ApiGatewayV2::Api resource. This requires a different integration than the previous versioned ApiGateway.
x-amazon-apigateway-integration has a key defined, payloadFormatVersion. Despite documentation suggesting both 1.0 and 2.0 are supported, it seems 2.0 must be used. As such, my x-amazon-apigateway-integration has become the following (I did clean it up a bit):
x-amazon-apigateway-integration:
payloadFormatVersion: "2.0"
httpMethod: "POST"
type: "aws_proxy"
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${apiPing.Arn}/invocations
responses:
default:
statusCode: "200"
connectionType: "INTERNET"
And with this, integration is applied upon deployment.

Related

AWS Api Gateway VTL Merge Json Object

Trying to merge the principalId from Lambda Authorizer into the payload that will be sent to Event Bridge.
So far I've gotten close by transforming to map and re-creating the Json object. The problem is that this Json object is printed out as a key:value pair with no quotes, So there's an error thrown.
According to the docs, there's non $util.toJson($map) available, this seems to be only available for AppSync.
const eventsAPI = new RestApi(this, 'eventsAPI', apigwOps);
const LambdaAuth0Authorizer = new LambdaAuth0Authorizer(this, 'LambdaAuth0Authorizer', {
env: {
auth0Audience: '',
auth0Domain: '',
},
});
const eventTypeResource = eventsAPI.root.addResource('event');
const options: IntegrationOptions = {
credentialsRole: apigwRole,
passthroughBehavior: PassthroughBehavior.NEVER,
requestParameters: {
'integration.request.header.X-Amz-Target': "'AWSEvents.PutEvents'",
'integration.request.header.Content-Type': "'application/x-amz-json-1.1'",
},
requestTemplates: {
'application/json': `
#set ( $map = $util.parseJson($input.body) )
#set ( $j = {
"eventType": "$map['eventType']",
"action": "$map['action']",
"subject": "$map['subject']",
"eventTime": $map['eventTime'],
"actor": "$context.authorizer.principalId"
}
)
{"Entries":
[
{
"Source": "com.xyz",
"Detail": "$util.escapeJavascript($j)",
"Resources": [],
"DetailType": "event",
"EventBusName": "${eventBus.eventBusName}"
}
]
}
`,
},
integrationResponses: [
{
statusCode: '200',
responseTemplates: {
'application/json': '',
},
},
],
};
eventTypeResource.addMethod('POST', new Integration({
type: IntegrationType.AWS,
uri: `arn:aws:apigateway:${env.region}:events:path//`,
integrationHttpMethod: 'POST',
options: options,
}), {
authorizer: LambdaAuth0Authorizer.authorizer,
methodResponses: [{ statusCode: '200' }],
requestModels: { 'application/json': getEventModel(this, eventsAPI) },
requestValidator: new RequestValidator(this, 'eventValidator', {
restApi: eventsAPI,
validateRequestBody: true,
}),
});
This generates:
{"Entries":
[
{
"Source": "com.uproarapi",
"Detail": "{eventType=FOLLOW, action=CREATE, subject=USER_777POIUY, eventTime=51644444444, actor=}",
"Resources": [],
"DetailType": "UpRoarEvent",
"EventBusName": "UpRoarEventBus"
}
]
}
With an error:
{"Entries":[{"ErrorCode":"MalformedDetail","ErrorMessage":"Detail is malformed."}],"FailedEntryCount":1}

Swagger API specs Request object design

I have written an api specs following OpenAPI/Swagger Specification -
{
"post": {
"tags": [
"UserController"
],
"operationId": "getUsers",
"parameters": [
{
"name": "accountID",
"in": "path",
"required": true,
"schema": {
"type": "number"
}
},
{
"name": "sortKey",
"in": "query",
"required": false,
"schema": {
"type": "string"
}
},
{
"name": "sortOrder",
"in": "query",
"required": false,
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "default response",
"content": {
"*/*": {
"schema": {
"$ref": "#/components/schemas/UserResponse"
}
}
}
}
}
}
}
The API Request takes accountId, sortKey and sortOrder. Should they should be wrapped in a Top level request object (getUsersRequest) ? What is the best practice?
{
"GetUsersRequest": {
"accountID": "String",
"sortKey": "String",
"sortOrder": "String"
}
}
vs
{
"accountID": "String",
"sortKey": "String",
"sortOrder": "String"
}
Usually just use the properties. Using a "wrapper" object can be useful if the parameters belong to multiple groups.
For example if you have an api with paging:
/query?filter=findme&page=5&size=5
I see two groups of parameters.
the filter to limit the query result, that is the main purpose of the api.
the page & size parameters, which are more a technical help to limit the amount of results.
you can use an (wrapper) object to easily communicate that two of the three parameters belong together and are used for paging.
as yaml:
/query:
get:
description: ...
parameters:
- name: filter
description: filters the data by the given value
in: query
schema:
type: string
- name: paging
description: page selection
in: query
required: false
schema:
$ref: '#/components/schemas/Paging'
components:
schemas:
Paging:
type: object
properties:
page:
type: integer
size:
type: integer
So in your example you could group sortKey & sortOrder as a view group while accountId is the main parameter of the api.

Adding parameters to api using cloudformation

I tried the cloudformation template that I found here...
https://bl.ocks.org/magnetikonline/c314952045eee8e8375b82bc7ec68e88
It works as expected. But I will like to provide parameters to the post request. My Curl command should look something like this...
curl -d "mynumber=12345" -X POST https://tyin2sswj2.execute-api.us-east-1.amazonaws.com/mycall
How do I handle it at API gateway in the cloudformation template? I have already set the environment variable at lambda function level.
The template that does not work is this...
https://raw.githubusercontent.com/shantanuo/cloudformation/master/updated/lambda_api.tpl.txt
As it is clear that I am not able to pass the "mnumber" variable through the gateway.
I have updated my template and now it deploys function and gateway corretly. And still the URL generated does not work and shows "internal server error" message.
https://raw.githubusercontent.com/shantanuo/cloudformation/master/testapi.tpl.txt
You should change to using HTTP proxy integration.
Here is some info from AWS on proxy integration: https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-http-integrations.html
Try changing your RequestParameters from:
RequestParameters:
method.request.querystring.mnumber: false
to
RequestParameters:
method.request.path.proxy: true
and under integration from:
RequestParameters:
integration.request.querystring.mnumber: "method.request.querystring.mnumber"
to
RequestParameters:
integration.request.path.proxy: 'method.request.path.proxy'
This is a good tutorial on proxy integration with API Gateway:
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-http.html
There are two ways you can access mynumber
Method 1 which works best with the SAM serverless API template. Define "API Gateway" and "Lambda". In Lambda definitions, call Events of type API:
This makes it where query strings are automatically picked up due to the event property. The parameters can be found in the event response that is passed into all lambda functions. It can be accessed with multiValueQueryStringParameters or queryStringParameters from the event object.
exports.getByDateHandler = async (event) => {
console.info(event.queryStringParameters);
console.info(event.multiValueQueryStringParameters);
}
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Description",
"Transform": ["AWS::Serverless-2016-10-31"],
"Resources": {
"getByDateFunction": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "src/handlers/getByDate/get-by-date.getByIdHandler",
"Runtime": "nodejs14.x",
"Architectures": ["x86_64"],
"MemorySize": 128,
"Timeout": 100,
"Events": {
"Api": {
"Type": "Api",
"Properties": {
"Path": "/date",
"Method": "GET"
}
}
}
}
}
},
"Outputs": {
"WebEndpoint": {
"Description": "API Gateway endpoint URL for Prod stage",
"Value": {
"Fn::Sub": "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
}
}
}
}
Method 2 which I havent tested is by defining the "Lambda", "API Gateway", "API Resource" and "API Methods". Linking the Lambda using the URI statement under "API Method".
for this method I only have a yaml example
MyLambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Description: "Node.js Express REST API"
FunctionName: "get_list_function" (The name in AWS console)
Handler: lambda.handler
Runtime: nodejs12
MemorySize: 128
Role: <ROLE ARN>
Timeout: 60
apiGateway:
Type: "AWS::ApiGateway::RestApi"
Properties:
Name: "example-api-gw"
Description: "Example API"
ProxyResource:
Type: "AWS::ApiGateway::Resource"
Properties:
ParentId: !GetAtt apiGateway.RootResourceId
RestApiId: !Ref apiGateway
PathPart: '{proxy+}' OR "a simple string like "PetStore"
apiGatewayRootMethod:
Type: "AWS::ApiGateway::Method"
Properties:
AuthorizationType: NONE
HttpMethod: ANY
Integration:
IntegrationHttpMethod: POST
Type: AWS_PROXY
IntegrationResponses:
- StatusCode: 200
Uri: !Sub >-
arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyLambdaFunction.Arn}/invocations
ResourceId: !Ref ProxyResource
RestApiId: !Ref "apiGateway"

Using Exported values

I am able to export the keys using this cloudformation template...
https://github.com/shantanuo/cloudformation/blob/master/restricted.template.txt
But how do I import the saved keys directly into "UserData" section of another template? I tried this, but does not work...
aws-ec2-assign-elastic-ip --access-key !Ref {"Fn::ImportValue" : "accessKey" } --secret-key --valid-ips 35.174.198.170
The rest of the template (without access and secret key reference) is working as expected.
https://github.com/shantanuo/cloudformation/blob/master/security.template2.txt
So, if this is your script that does the export (sorry, this one is in yaml)
AWSTemplateFormatVersion: '2010-09-09'
Metadata:
License: Apache-2.0
Description: 'AWS CloudFormation Sample Template'
Parameters:
NewUsername:
NoEcho: 'false'
Type: String
Description: New account username
MinLength: '1'
MaxLength: '41'
ConstraintDescription: the username must be between 1 and 41 characters
Password:
NoEcho: 'true'
Type: String
Description: New account password
MinLength: '1'
MaxLength: '41'
ConstraintDescription: the password must be between 1 and 41 characters
Resources:
CFNUser:
Type: AWS::IAM::User
Properties:
LoginProfile:
Password: !Ref 'Password'
UserName : !Ref 'NewUsername'
CFNAdminGroup:
Type: AWS::IAM::Group
Admins:
Type: AWS::IAM::UserToGroupAddition
Properties:
GroupName: !Ref 'CFNAdminGroup'
Users: [!Ref 'CFNUser']
CFNAdminPolicies:
Type: AWS::IAM::Policy
Properties:
PolicyName: CFNAdmins
PolicyDocument:
Statement:
- Effect: Allow
Action: '*'
Resource: '*'
Condition:
StringEquals:
aws:RequestedRegion:
- ap-south-1
- us-east-1
Groups: [!Ref 'CFNAdminGroup']
CFNKeys:
Type: AWS::IAM::AccessKey
Properties:
UserName: !Ref 'CFNUser'
Outputs:
AccessKey:
Value: !Ref 'CFNKeys'
Description: AWSAccessKeyId of new user
Export:
Name: 'accessKey'
SecretKey:
Value: !GetAtt [CFNKeys, SecretAccessKey]
Description: AWSSecretAccessKey of new user
Export:
Name: 'secretKey'
Then here is an example of how you would import those values in userdata in the import cloudformation script:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Test instance stack",
"Parameters": {
"KeyName": {
"Description": "The EC2 Key Pair to allow SSH access to the instance",
"Type": "AWS::EC2::KeyPair::KeyName"
},
"BaseImage": {
"Description": "The AMI to use for machines.",
"Type": "String"
},
"VPCID": {
"Description": "ID of the VPC",
"Type": "String"
},
"SubnetID": {
"Description": "ID of the subnet",
"Type": "String"
}
},
"Resources": {
"InstanceSecGrp": {
"Type": "AWS::EC2::SecurityGroup",
"Properties": {
"GroupDescription": "Instance Security Group",
"SecurityGroupIngress": [{
"IpProtocol": "-1",
"CidrIp": "0.0.0.0/0"
}],
"SecurityGroupEgress": [{
"IpProtocol": "-1",
"CidrIp": "0.0.0.0/0"
}],
"VpcId": {
"Ref": "VPCID"
}
}
},
"SingleInstance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"KeyName": {
"Ref": "KeyName"
},
"ImageId": {
"Ref": "BaseImage"
},
"InstanceType": "t2.micro",
"Monitoring": "false",
"BlockDeviceMappings": [{
"DeviceName": "/dev/xvda",
"Ebs": {
"VolumeSize": "20",
"VolumeType": "gp2"
}
}],
"NetworkInterfaces": [{
"GroupSet": [{
"Ref": "InstanceSecGrp"
}],
"AssociatePublicIpAddress": "true",
"DeviceIndex": "0",
"DeleteOnTermination": "true",
"SubnetId": {
"Ref": "SubnetID"
}
}],
"UserData": {
"Fn::Base64": {
"Fn::Join": ["", [
"#!/bin/bash -xe\n",
"yum install httpd -y\n",
"sudo sh -c \"echo ",
{ "Fn::ImportValue" : "secretKey" },
" >> /home/ec2-user/mysecret.txt\" \n",
"sudo sh -c \"echo ",
{ "Fn::ImportValue" : "accessKey" },
" >> /home/ec2-user/myaccesskey.txt\" \n"
]]
}
}
}
}
}
}
In this example I am just echoing the value of the import into a file. If you ssh onto the SingleInstance and check the logs at /var/lib/cloud/instance/scripts/part-001 then you will see what the user data script looks like on the server itself. In my case the contents of that file is (values aren't real for the keys):
#!/bin/bash -xe
yum install httpd -y
sudo sh -c "echo hAc7/TJA123143235ASFFgKWkKSjIC4 >> /home/ec2-user/mysecret.txt"
sudo sh -c "echo AKIAQ123456789123D >> /home/ec2-user/myaccesskey.txt"
Using this as a starting point you can do whatever you need to with the import value.
I've tested all of this with the exact scripts above and it all works.
What is suggested in the comments seems to be correct. I can directly refer to the name (for e.g. 'accessKey' in this case) using ImportValue!
AWSTemplateFormatVersion: '2010-09-09'
Metadata:
License: Apache-2.0
Description: 'AWS CloudFormation Sample Template'
Resources:
CFNUser:
Type: AWS::IAM::User
Outputs:
AccessKey:
Value:
Fn::ImportValue: accessKey
Description: AWSAccessKeyId of new user
For e.g. the above template will return the value of accessKey if it is already exported by some other template.

How to document a response comprised of a list of resources using OpenAPI

I am trying to create OpenAPI yml documentation file (via swagger). One of my API calls returns a list of resources. Each resources has properties, a self link and a link to an additional link which will retrieve additional "stuff" which relate to the resource.
Please see the following example:
[
{
"name": "object-01",
"links": [
{
"rel": "self",
"href": "http://localhost:8800/foo/object-01"
},
{
"rel": "Supported stuff",
"href": "http://localhost:8800/foo/object-01/stuff"
}
]
}, {
"name": "object-02",
"links": [
{
"rel": "self",
"href": "http://localhost:8800/foo/object-02"
},
{
"rel": "Supported stuff",
"href": "http://localhost:8800/foo/object-02/stuff"
}
]
}, {
"name": "object-03",
"links": [
{
"rel": "self",
"href": "http://localhost:8800/foo/object-03"
},
{
"rel": "Supported stuff",
"href": "http://localhost:8800/foo/object-03/stuff"
}
]
}
]
I am not sure what is the right way to document this, this is what I have in place right now.
paths:
/foo/objects:
get:
operationId: getObject
responses:
'200':
description: Respresentation of objects
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/object'
links:
self:
$ref: '#/components/links/object'
components:
links:
object:
operationId: getSObject
stuff:
operationId: getStuff
schemas:
object:
type: object
properties:
name:
type: string
But I do not believe this is adequately represents my API.
Thanks for your help
Links that are included in the actual response need to be described as part of the response body schema:
paths:
/foo/objects:
get:
operationId: getObject
responses:
'200':
description: Respresentation of objects
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/object'
components:
schemas:
object:
type: object
properties:
name:
type: string
links: # <-------------
type: array
items:
$ref: '#/components/schemas/link'
link:
type: object
properties:
rel:
type: string
href:
type: string
format: uri
OpenAPI 3.0 links concept is similar to HATEOAS, but not really. These links are used to describe how the values returned from one operation can be used as input in other operations. For example, the create user operation returns the user ID, and this ID can be used to update or delete the user. This page has some more info about the links keyword: https://swagger.io/docs/specification/links