The below is the part of CloudForamtion file loaded by Serverless.
# resource.yml
.
.
.
{"Fn::Sub": "arn:aws:sqs:*:${AWS::AccountId}:sqs-spoon-*-${env:SERVICE}"}
# serverless.yml
.
.
resources:
- ${file:resource.yml}
${AWS::AccountId} is CloudFormation Pseudo Parameter and ${env:SERVICE} is Serverless variable.
When I run sls deploy, it returns the error.
Invalid variable reference syntax for variable AWS::AccountId. You can only reference env vars, options, & files. You can check our docs for more info.
It seems to say that Serverless recognize ${AWS::AccountId} as Serverless variable, not as CloudFormation Pseudo Parameter.
Right?
If so, how to have Serverless not to parse Pseudo Parameter so that it will be parsed by CloudFormation later?
I can solve it with the plugin.
With the plugin, It cloud be solved by replacing ${AWS::AccountId} with #{AWS::AccountId}.
{"Fn::Sub": "arn:aws:sqs:*:#{AWS::AccountId}:sqs-spoon-*-${env:SERVICE}"}
You can accomplish support for the native AWS syntax with a single config line in serverless.yml to define the variableSyntax. Details can be found here https://github.com/serverless/serverless/pull/3694.
provider:
name: aws
runtime: nodejs8.10
variableSyntax: "\${((env|self|opt|file|cf|s3)[:\(][ :a-zA-Z0-9._,\-\/\(\)]*?)}"
Related
I've got release pipelines defined that have worked. I've got a config transform that will write a API url to a config file (currently with a hardcoded api url).
What I'd like to do is be able to have the config be re-written based on the agent its being deployed on.
eg. if the machine being deployed to is TEST-1, I'd like to write https://TEST-1.somedomain.com/api into a config using that transform step.
The .somedomain.com/api can be static.
I've tried modifying the pipeline variable's value to be https://${{Environment.Name}}.somedomain.com/api, but it just replaces the API_URL in the config with that literal string (does not populate machine name in that variable).
Being that variables are the source of value that is being written to configs during the transform, I'm struggling to see another way to do this.
some gotchas
Using non yaml pipeline definitions (I know I saw people put logic in variable definitions within yaml pipelines)
Can't just use localhost, as the configuration is being read into a javascript rich app that would have js trying to connect to localhost vs trying to connect to the server.
I'm interested in any ways I could solve this problem
${{Environment.Name}} is not valid syntax for either YAML or classic pipelines.
In classic pipelines it would be $(Environment.Name).
In YAML, $(Environment.Name) or ${{ variables['Environment.Name'] }} would work.
Question
What is the correct way to get the output of a cloudformation stack in a serverless.yml file without hardcoding the stack name?
Steps
I have a serverless.yml file where I import a cloudformation template to create an ElastiCache cluster.
When I try to do so, I get this error:
Serverless Error ---------------------------------------
Invalid variable reference syntax for variable AWS::StackName. You can only reference env vars, options, & files. You can check our docs for more info.
In my file I'd like to expose as an environment variable the ElastiCacheAddress output from the cloudformation stack. I am using the serverless pseudo-parameters plugin to get the output:
# Here is where I try to reference the CF output value
service: hello-world
provider:
name: aws
# ...
environment:
cacheUrl: ${cf:#{AWS::StackName}.ElastiCacheAddress}
# Reference to the CF template
resources:
- '${file(./cf/cf-elasticache.yml)}'
The CF template is the one from the AWS Samples GitHub repository.
The snippet with the output is here:
ElastiCacheAddress:
Description: ElastiCache endpoint address
Value: !If [ IsRedis, !GetAtt ElastiCacheCluster.RedisEndpoint.Address, !GetAtt ElastiCacheCluster.ConfigurationEndpoint.Address]
Export:
Name: !Sub ${AWS::StackName}-ElastiCacheAddress
You can use a workaround to get your way through these syntax caveats.
In this case, I would suggest you to create a custom node to set variables you would want to reuse. You can then reference these variables using Serverless Framework syntax only, to avoid that error, like so:
# Here is where I try to reference the CF output value
service: hello-world
custom:
stackName:'#{AWS::StackName}'
provider:
name: aws
# ...
environment:
cacheUrl: ${cf:${self:custom.stackName}.ElastiCacheAddress}
# Reference to the CF template
resources:
- '${file(./cf/cf-elasticache.yml)}'
We are using AWS SAM to build and manage an AWS Layer. The same SAM template can easily associate the latest version to lambda functions which are also managed by this template. However, we have other Lambda functions that are managed by other CloudFormation/SAM templates and I don't know latest version (ARN)
This is what we use in the SAM template to associate the layer
Globals:
Function:
Layers:
- !Ref ToolkitLayer
How do I determine the latest version programmatically from a completely different CloudFormation/SAM template? I thought about using an SSM Parameter since it appears CloudFormation can dynamically pull a value. The issue here is that the SSM Parameter also has a version, same issue.
Have you thought about using a Cloudformation macro/transform (both refer to the same thing)?
Using a Cloudformation macro, CF can call a Lambda function with the snippet from your template, and your Lambda function returns the transformed snippet back to CF. In your Lambda function, you would query the latest version of your layer and return that result back to CF.
More details at:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-macros.html
I am trying to add a MongoDB cluster as part of a Serverless deployment, but I can't set the environment variable.
Here is part of the serverless.yml file:
service: serverless-test
plugins:
- serverless-offline
provider:
name: aws
runtime: nodejs4.3
environment:
MONGO_URI: "mongodb://mongo-6:27000,mongo-7:27000,mongo-8:27000/db-dev?replicaSet=mongo"
How do I pass the MONGO_URI to contain the cluster as a comma separated value?
Any advise is much appreciated.
Unfortunately, you can't use commas in Lambda environment variables. This is an AWS limitation and not a Serverless issue.
For example, browse the AWS console and try to add a environment variable that contains a comma:
When you save, you will get the following error:
1 validation error detected: Value at 'environment.variables' failed to satisfy constraint: Map value must satisfy constraint: [Member must satisfy regular expression pattern: [^,]*]
The error message says that the regex [^,]* must be satisfied and what this small regex explicitly says is to not (^) accept the comma (,). Any other char is acceptable.
I don't know why they don't accept the comma and this is not explained in their documentation, but at least their error message shows that it is intentional.
As a workaround, you can replace your commas by another symbol (like #) to create the env var and replace it back to comma after reading the variable, or you will need to create multiple env vars to store the endpoints.
The AWS CLI for Dynamodb create-table is a little bit confusion when it comes to create global secondary index. In the CLI document, it says global secondary index could be expressed with the following expression (shorthand):
IndexName=string,KeySchema=[{AttributeName=string,KeyType=string},{AttributeName=string,KeyType=string}],Projection={ProjectionType=string,NonKeyAttributes=[string,string]},ProvisionedThroughput={ReadCapacityUnits=long,WriteCapacityUnits=long} ...
My interpretation is, I should do
--global-secondary-indexes IndexName=requesterIndex,Projection={ProjectionType=ALL},ProvisionedThroughput={ReadCapacityUnits=1,WriteCapacityUnits=1}
Note that I am not including KeySchema here to deduce complexity. The console gives me the following error:
Parameter validation failed:
Missing required parameter in GlobalSecondaryIndexes[0]: "KeySchema"
Unknown parameter in GlobalSecondaryIndexes[0]: "WriteCapacityUnits", must be one of: IndexName, KeySchema, Projection, ProvisionedThroughput
Invalid type for parameter GlobalSecondaryIndexes[0].ProvisionedThroughput, value: ReadCapacityUnits=1, type: <class 'str'>, valid types: <class 'dict'>
So somehow AWS CLI does not recognize the map expression for ProvisionedThroughput. I tried several ways to express it and could not make it work. I also failed to find any web page in Google describing how to do it.
This is the cli call I used to create the Reply sample in the aws documentation from the command line. The $EP i used at the end can be set in the environment to EP="--endpoint-url http://localhost:8000" to create the table on your local dynamodb instead of aws.
aws dynamodb create-table --table-name Reply --attribute-definitions \
AttributeName=Id,AttributeType=S AttributeName=ReplyDateTime,AttributeType=S \
AttributeName=PostedBy,AttributeType=S AttributeName=Message,AttributeType=S \
--key-schema AttributeName=Id,KeyType=HASH \
AttributeName=ReplyDateTime,KeyType=RANGE --global-secondary-indexes \
IndexName=PostedBy-Message-Index,KeySchema=["\
{AttributeName=PostedBy,KeyType=HASH}","\
{AttributeName=Message,KeyType=RANGE}"],Projection="{ProjectionType=INCLUDE \
,NonKeyAttributes=["ReplyDateTime"]}",ProvisionedThroughput="\
{ReadCapacityUnits=10,WriteCapacityUnits=10}" --provisioned-throughput \
ReadCapacityUnits=5,WriteCapacityUnits=4 $EP
Read through AWS CLI source code on Github, it could parse double quote content. So adding double quote in the script solved the issue. There is the new code -
--global-secondary-indexes IndexName=requesterIndex,Projection={ProjectionType=ALL},ProvisionedThroughput="{ReadCapacityUnits=${CURRENT_READUNIT},WriteCapacityUnits=${CURRENT_WRITEUNIT}}"
Define the table structure in a JSON file, including the index structures. Use following to create a template structure.
aws dynamodb create-table --generate-cli-skeleton
Run the cli command with the table definition input json
aws dynamodb create-table --cli-input-json file://path-to-yourtable-definition.json