I am trying to add a MongoDB cluster as part of a Serverless deployment, but I can't set the environment variable.
Here is part of the serverless.yml file:
service: serverless-test
plugins:
- serverless-offline
provider:
name: aws
runtime: nodejs4.3
environment:
MONGO_URI: "mongodb://mongo-6:27000,mongo-7:27000,mongo-8:27000/db-dev?replicaSet=mongo"
How do I pass the MONGO_URI to contain the cluster as a comma separated value?
Any advise is much appreciated.
Unfortunately, you can't use commas in Lambda environment variables. This is an AWS limitation and not a Serverless issue.
For example, browse the AWS console and try to add a environment variable that contains a comma:
When you save, you will get the following error:
1 validation error detected: Value at 'environment.variables' failed to satisfy constraint: Map value must satisfy constraint: [Member must satisfy regular expression pattern: [^,]*]
The error message says that the regex [^,]* must be satisfied and what this small regex explicitly says is to not (^) accept the comma (,). Any other char is acceptable.
I don't know why they don't accept the comma and this is not explained in their documentation, but at least their error message shows that it is intentional.
As a workaround, you can replace your commas by another symbol (like #) to create the env var and replace it back to comma after reading the variable, or you will need to create multiple env vars to store the endpoints.
Related
I've got release pipelines defined that have worked. I've got a config transform that will write a API url to a config file (currently with a hardcoded api url).
What I'd like to do is be able to have the config be re-written based on the agent its being deployed on.
eg. if the machine being deployed to is TEST-1, I'd like to write https://TEST-1.somedomain.com/api into a config using that transform step.
The .somedomain.com/api can be static.
I've tried modifying the pipeline variable's value to be https://${{Environment.Name}}.somedomain.com/api, but it just replaces the API_URL in the config with that literal string (does not populate machine name in that variable).
Being that variables are the source of value that is being written to configs during the transform, I'm struggling to see another way to do this.
some gotchas
Using non yaml pipeline definitions (I know I saw people put logic in variable definitions within yaml pipelines)
Can't just use localhost, as the configuration is being read into a javascript rich app that would have js trying to connect to localhost vs trying to connect to the server.
I'm interested in any ways I could solve this problem
${{Environment.Name}} is not valid syntax for either YAML or classic pipelines.
In classic pipelines it would be $(Environment.Name).
In YAML, $(Environment.Name) or ${{ variables['Environment.Name'] }} would work.
I need to issue a command "python manage.py rebuild_index" for rebuilding index of apache solr in my app deployment. But i get the following error:
The Deployment "test-app" is invalid: spec.template.spec.initContainers[5].name: Invalid value: "rebuild_index": a DNS-1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name', or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?')
As per your error message I don't think the error is due to '_' in command, I guess you have named initContainers name as well "rebuild_index" that is where the error has been generated.
Object Name in k8s should be an alphanumeric character where only "." and "-" special characters are allowed. You can read more on this documentation link. Please change the initContainer name to "rebuild-index" which will resolve your issue.
Thanks,
The below is the part of CloudForamtion file loaded by Serverless.
# resource.yml
.
.
.
{"Fn::Sub": "arn:aws:sqs:*:${AWS::AccountId}:sqs-spoon-*-${env:SERVICE}"}
# serverless.yml
.
.
resources:
- ${file:resource.yml}
${AWS::AccountId} is CloudFormation Pseudo Parameter and ${env:SERVICE} is Serverless variable.
When I run sls deploy, it returns the error.
Invalid variable reference syntax for variable AWS::AccountId. You can only reference env vars, options, & files. You can check our docs for more info.
It seems to say that Serverless recognize ${AWS::AccountId} as Serverless variable, not as CloudFormation Pseudo Parameter.
Right?
If so, how to have Serverless not to parse Pseudo Parameter so that it will be parsed by CloudFormation later?
I can solve it with the plugin.
With the plugin, It cloud be solved by replacing ${AWS::AccountId} with #{AWS::AccountId}.
{"Fn::Sub": "arn:aws:sqs:*:#{AWS::AccountId}:sqs-spoon-*-${env:SERVICE}"}
You can accomplish support for the native AWS syntax with a single config line in serverless.yml to define the variableSyntax. Details can be found here https://github.com/serverless/serverless/pull/3694.
provider:
name: aws
runtime: nodejs8.10
variableSyntax: "\${((env|self|opt|file|cf|s3)[:\(][ :a-zA-Z0-9._,\-\/\(\)]*?)}"
The AWS CLI for Dynamodb create-table is a little bit confusion when it comes to create global secondary index. In the CLI document, it says global secondary index could be expressed with the following expression (shorthand):
IndexName=string,KeySchema=[{AttributeName=string,KeyType=string},{AttributeName=string,KeyType=string}],Projection={ProjectionType=string,NonKeyAttributes=[string,string]},ProvisionedThroughput={ReadCapacityUnits=long,WriteCapacityUnits=long} ...
My interpretation is, I should do
--global-secondary-indexes IndexName=requesterIndex,Projection={ProjectionType=ALL},ProvisionedThroughput={ReadCapacityUnits=1,WriteCapacityUnits=1}
Note that I am not including KeySchema here to deduce complexity. The console gives me the following error:
Parameter validation failed:
Missing required parameter in GlobalSecondaryIndexes[0]: "KeySchema"
Unknown parameter in GlobalSecondaryIndexes[0]: "WriteCapacityUnits", must be one of: IndexName, KeySchema, Projection, ProvisionedThroughput
Invalid type for parameter GlobalSecondaryIndexes[0].ProvisionedThroughput, value: ReadCapacityUnits=1, type: <class 'str'>, valid types: <class 'dict'>
So somehow AWS CLI does not recognize the map expression for ProvisionedThroughput. I tried several ways to express it and could not make it work. I also failed to find any web page in Google describing how to do it.
This is the cli call I used to create the Reply sample in the aws documentation from the command line. The $EP i used at the end can be set in the environment to EP="--endpoint-url http://localhost:8000" to create the table on your local dynamodb instead of aws.
aws dynamodb create-table --table-name Reply --attribute-definitions \
AttributeName=Id,AttributeType=S AttributeName=ReplyDateTime,AttributeType=S \
AttributeName=PostedBy,AttributeType=S AttributeName=Message,AttributeType=S \
--key-schema AttributeName=Id,KeyType=HASH \
AttributeName=ReplyDateTime,KeyType=RANGE --global-secondary-indexes \
IndexName=PostedBy-Message-Index,KeySchema=["\
{AttributeName=PostedBy,KeyType=HASH}","\
{AttributeName=Message,KeyType=RANGE}"],Projection="{ProjectionType=INCLUDE \
,NonKeyAttributes=["ReplyDateTime"]}",ProvisionedThroughput="\
{ReadCapacityUnits=10,WriteCapacityUnits=10}" --provisioned-throughput \
ReadCapacityUnits=5,WriteCapacityUnits=4 $EP
Read through AWS CLI source code on Github, it could parse double quote content. So adding double quote in the script solved the issue. There is the new code -
--global-secondary-indexes IndexName=requesterIndex,Projection={ProjectionType=ALL},ProvisionedThroughput="{ReadCapacityUnits=${CURRENT_READUNIT},WriteCapacityUnits=${CURRENT_WRITEUNIT}}"
Define the table structure in a JSON file, including the index structures. Use following to create a template structure.
aws dynamodb create-table --generate-cli-skeleton
Run the cli command with the table definition input json
aws dynamodb create-table --cli-input-json file://path-to-yourtable-definition.json
I'm trying to modify the config of a running Wildfly (inside Docker) using the JBoss CLI. One thing I need to do is
data-source add --connection-url=jdbc:mysql://${DB_PORT_3306_TCP_ADDR}:3306/xplore (...)
I need the resulting config in standalone.xml to literally contain
<connection-url>jdbc:mysql://${DB_PORT_3306_TCP_ADDR}:3306/xplore</connection-url>
DB_PORT_3306_TCP_ADDR is a variable that will be resolved by Wildfly when it's rebooted. It does not exist when I'm CLI-ing.
The problem is that CLI attempts to resolve it (and fails) during my CLI session (instead of injecting it as a plain string), resulting in output like errors and even
<connection-url>$</connection-url>
I've tried to work around this by adding "..." and '...', escaping the $ to \$, removing the brackets, and defining a variable containing my string in the script and inserting that variable (still gets resolved). I've looked up the docs on this, and while there is a command resolve-expression(), there is no inverse, no setting for it, nothing. It doesn't seem possible to prevent the resolution from happening.
How can I insert an unresolved expression into my config via the CLI?
The answer was to issue this command:
data-source add --connection-url=jdbc:mysql://${DB_PORT_3306_TCP_ADDR:}:3306/xplore (...)
With a : after ADDR. This obviously solves the resolution error since it will now default to an empty value, and somehow it actually causes the string value itself to be written. It also turns out it wasn't the CLI that was resolving my variable, but the server itself.
The real solution though was to stop the session, modify the server settings and disable variable resolution entirely, then start a new session.