We use serverless-plugin-split-stacks to break resources into nested stacks and have set it up in serverless.yml as follows.
custom:
splitStacks:
perFunction: false
perType: true
perGroupFunction: false
webpack:
webpackConfig: ./webpack.config.js
includeModules: true
Everything was going well until we were greeted with the following error
Error: The CloudFormation template is invalid: Template format error: Number of resources, 206, is > greater than maximum allowed, 200
When this error happens, the condition of the nested stacks is as follows.
Serverless: [serverless-plugin-split-stacks]: Resources per stack:
Serverless: [serverless-plugin-split-stacks]: - (root): 206
Serverless: [serverless-plugin-split-stacks]: - APINestedStack: 55
Serverless: [serverless-plugin-split-stacks]: - PermissionsNestedStack: 49
My problem is that even though we have set up split-stacks properly, why doesn't it split the resources in the root stack into new stacks?
If there's anything I have missed here, please educate me on this. Thanks for all helpful suggestions.
This npm package serverless-plugin-split-stacks not working. This package has been Deprecated - FAILED - BUG
For more information: https://www.npmjs.com/package/serverless-plugin-split-stacks
To resolve your problem (Serverless Workarounds for CloudFormation's 200 Resource Limit) read some tips:
https://www.serverless.com/blog/serverless-workaround-cloudformation-200-resource-limit
Related
I have a CF template (serverless Typescript file to be precise) which creates a container definition in the following way.
Type: "AWS::ECS::TaskDefinition",
Properties: {
Family: "client",
RequiresCompatibilities: ["FARGATE"],
NetworkMode: "awsvpc",
Memory: 1024,
CPU: 512,
TaskRoleArn: { Ref: ECS_TASK_ROLE },
ExecutionRoleArn: { Ref: ECS_TASK_ROLE },
ContainerDefinitions: [
{
Name: application,
Essential: true,
Image: `${ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/repository:client-{{resolve:ssm:param-name}}`,
.
.
.
},
],
}
Please note the usage of AWS SSM Dynamic Reference.
When I deploy it I get the following error
Could not create Change Set "change-set-name" due to: Parameters: "[ssm:param-name:167:167710252826] cannot be found.
It used to work fine and all of a sudden started breaking. Same code still works in a different env/AWS account.
Seems like it can't find a specific version of param-name for some reason. The latest param-name verion is 400. AWS SSM only keeps last 100 params. This parameter is updated quite often and it's been a long time since I deployed CF template. From the error message it seems like it is looking for version 167 (I could be wrong because I am not sure what is the number trailing 167 i.e. 167:167710252826)
What I tried:
I hardcoded the version number I pass as below and removed all the references to resolve:ssm:param-name.
Image: `${ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/repository:client-1.6.3`,
It still throws the same error. Which hints that the problem is not related to code. CF is trying to run an old change set from history to create a diff and can't find an old version of the same param.
What could be the solution to this?
I can't delete the old change set. It throws an error on deleting.
I can't go back and create a parameter in SSM with version 167 if it's a version issue.
So I have a stack exceeding 500 resources and found out this serverless plugin which splits stack according to the several configurations.
Below is my configuration for splitting the stack. Upon using the below configuration I was able to split the stacks in 2. with that I also got the warning Serverless: Recoverable error occurred (TooManyRequestsException: Rate exceeded
custom:
splitStacks:
nestedStackCount: 2 # Controls the number of created nested stacks
perFunction: false
perType: false
perGroupFunction: true
To resolve the API rate limit I used resourceConcurrency property as below
custom:
splitStacks:
nestedStackCount: 2 # Controls the number of created nested stacks
perFunction: false
perType: false
perGroupFunction: true
resourceConcurrency: 20 # Controls how much resources are deployed in parallel. Disabled if absent.
Upon deployment, I received following error
ServerlessError: The CloudFormation template is invalid: ValidationError: Circular dependency between resources: [GetAllUsersLambdaFunction,.....
is there any workaround to resolve this issue? Is resourceConcurrency even in a working state?
I m trying to remove some fields, I use filebeat 7.14 on Kubernetes
I tried as described in the doc
processors:
- drop_fields:
when:
contains
fields: ["host.os.name", "host.os.codename", "host.os.family"]
ignore_missing: false
container failed "ERROR instance/beat.go:989
Exiting: Failed to start crawler:
starting input failed: Error while initializing input:
missing or invalid condition
failed to initialize condition"
ignore_missing still messing
- drop_fields:
fields: ["host.os.name", "host.os.codename", "host.os.family"]
fields are still present
you don't seem to have a condition set under the when. take a look at https://www.elastic.co/guide/en/beats/filebeat/7.14/defining-processors.html#conditions and make sure you've got something for it to match
I am trying to apply my cloudformation template and I am getting the following cryptic error:
botocore.exceptions.ClientError: An error occurred (ValidationError)
when calling the CreateStack operation: Template error: Condition
token can only be used in Conditions block
The stack trace is
File "/Users/user/.env/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/Users/user/.env/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
raise error_class(parsed_response, operation_name)
The code looks like
cf_client = session.client('cloudformation')
cf_client.create_stack(
StackName=stack_name,
TemplateBody=template_body,
Parameters=aws_parameters,
TimeoutInMinutes=10,
OnFailure='DELETE',
Tags=aws_tags,
Capabilities=['CAPABILITY_IAM'],
)
The cloudformation template is massive and not appropriate to paste here. It stands up an application with service discovery, app mesh, fargate, etc.
What is this Condition they're referring to and what is wrong?
The error is rather cryptic and unhelpful but in my case, it was a typo in my ECS task definition.
My container has a depends on relationship and I had misconstructed the
DependsOn:
- ContainerName: envoy
- Condition: HEALTHY
Depends on is a list of maps so there should not be a - in front of Condition.
This corrects my problem:
DependsOn:
- ContainerName: envoy
Condition: HEALTHY
We are using some community custom resource types (https://github.com/ljfranklin/terraform-resource and https://github.com/cloudfoundry/bosh-deployment-resource). After upgrading to concourse 3.3.0, we've begun consistently seeing the following error on a few of our jobs at the same step: json: unsupported type: map[interface {}]interface {}.
This is fairly hard to debug as there is no other log output other than that. We are unsure what is incompatible between those resources and Concourse.
Notes about our pipeline:
We originally had substituted all of our usages of {{}} to (()), but reverting that did not lead to the error going away.
We upgraded concourse from v3.0.1.
The failing step can be found here: https://github.com/cloudfoundry/capi-ci/blob/6a73764d09f544820ce39f16dca166d6d6861996/ci/pipeline.yml#L731-L739
We are using a resource called elsa-aws-storage-terraform, found here: https://github.com/cloudfoundry/capi-ci/blob/6a73764d09f544820ce39f16dca166d6d6861996/ci/pipeline.yml#L731-L739
That resource is of a custom resource-type terraform found here: https://github.com/cloudfoundry/capi-ci/blob/6a73764d09f544820ce39f16dca166d6d6861996/ci/pipeline.yml#L45-L48
A similar failing step can be found here: https://github.com/cloudfoundry/capi-ci/blob/6a73764d09f544820ce39f16dca166d6d6861996/ci/pipeline.yml#L871-L886
This is related to issue of not being able to define nested maps in resource configuration https://github.com/concourse/concourse/issues/1345