I believe this requirement seems pretty straight forward for anyone trying to host their Tier3 i.e. database in a container.
I have MVP 3x Tier MERN app using -
1x Container instance
3x ECS services (Frontend, Backend and Database)
3x Tasks (1x running task per service)
The Database Task (mongodb) has its task definition updated to use EFS and have tested stopping the task and re-starting a new one for data persistence.
Question - How to ensure auto mount of EFS volume on the ECS container host (SPOT instance). If ECS leverages cloud formation template under the covers, do I need to update or modify this template to gain this persistent efs volume auto mounted on all container ec2 instances? I have come across various articles talking about a script in the ec2 launch config but I don't see any launch config created by ECS / cloud formation.
What is the easiest and simplest way to achieve something as trivial as persistent efs volume across my container host instances. Am guessing task definition alone doesn't solve this problem?
Thanks
Actually, I think below steps achieved persistence for the db task using efs -
Updated task definition for the database container to use EFS.
Mounted the EFS vol on container instance
sudo mount -t efs -o tls fs-:/ /database/data
The above mount command did not add any entries within the /etc/fstab but still seems to be persistent on the new ECS SPOT instance.
Related
I have an ECS service with a task definition that has three containers:
Frontend container
Backend container
MongoDB container
I would like to update the frontend container and/or the backend container without loosing the mongoDB data.
what would be the best way to do it?
Is it possible to update the service without redeploy the entire task (and loosing the db data) but only the container(s) I need?
Should I use a bind volume for the mongo db so I can save the data there and when the service is redeployed the new mongo db container will retrieve the data from there?
Hope that make sense.
Thanks in advance.
what would be the best way to do it?
Ideally you wouldn't be running MongoDB in the same Task. You would be running it in a separate ECS task (or just running it on an EC2 instance directly). Right now you can't scale-out your frontend/backend services because doing so would spin up the new service instances with a new, empty database.
Is it possible to update the service without redeploy the entire task
(and loosing the db data) but only the container(s) I need?
No that's not possible
Should I use a bind volume for the mongo db so I can save the data there and when the service is redeployed the new mongo db container will retrieve the data from there?
If you are deploying your ECS tasks to EC2 targets, then you could use a bind mount that would store the DynamoDB data on one of the EC2 instances in your cluster. However if that EC2 instance ever went down you would still lose all your data. The more fault-tolerant and highly-available method would be to bind an EFS volume to your MongoDB container to store the data.
If you are deploying your ECS tasks to Fargate, then bind mounts would not be an option, EFS would be the only option.
I want to deploy Mongo image on a container service like amazon Fargate.
can i write data to that container, if its possible to write data to the container where the data will be stored and they will charge it as a task?
Each Fargate task (PV 1.4) comes with 20GB of ephemeral storage included in the price. You can extend it up to 200GB (for an additional fee). See here. Again this space is ephemeral, if the task shuts down your disk is wiped.
One other option (in this case persistent) would be to mount an EFS volume to the Fargate task but probably not a great fit for a database workload.
I am using Auto Scaling with a Load Balancer and have attached 2 EBS volumes.
Now whenever an instance is terminated it stores the snapshot of the EBS volumes.
I have gone through several links but cannot find how to retrieve/mount the EBS volume when a Launch Configuration launches a new instance.
Can I can get any reference or PowerShell script to identify a volume via tag name from the volume list and mount it when the instance is initiating?
There is no automatic facility to mount an existing EBS snapshot or volume when Auto Scaling launches an instance.
Best practice for Auto Scaling is to store data off-instance, such as in Amazon S3 or Amazon EFS. This way, the data is accessible to all instances simultaneously and can be used by new instances that are launched.
There is also no automatic facility to create an EBS snapshot when an Auto Scaling instance is terminated. Rather, there is the option to Delete on Termination, which controls whether the EBS volume should be deleted when the instance is terminated. If this option is off, then the EBS volumes will remain after an instance is terminated. You could write some code (eg in a User Data script) that re-attached an EBS volume to a new instance launched by Auto Scaling but this can get messy. (For example: Which instance to attach? What happens if more instances are launched?)
Bottom line: Yes, you could write a script to do this, but it is a poor architectural design.
Yes, you can attach (mount) an EBS volume to an EC2 instance using the AWS CLI command line tool. You run this command in the EC2 User Data at instance launch.
Running Commands on Your Linux Instance at Launch
AWS CLI attach-volume
Note: There is a problem with this strategy. The ASG Launch Configuration is for creating new EC2 instances that are identical. This would mean that you would be attempting to attach the same EBS volume to each instance which will fail. You may want to consider using EFS instead.
Amazon Elastic File System
Mount EFS on EC2 using the AWS CLI
Note: Use IAM roles to provide your instances with credentials instead of storing credentials on the EC2 instance.
Once you have configured your "master" EC2 instance create a new AMI for your ASG launch configuration.
When mounted on Amazon EC2 instances, an Amazon EFS file system provides a standard file system interface and file system access semantics, allowing you to seamlessly integrate Amazon EFS with your existing applications and tools. Multiple Amazon EC2 instances can access an Amazon EFS file system at the same time, allowing Amazon EFS to provide a common data source for workloads and applications running on more than one Amazon EC2 instance.
We currently have a 2-node Kubernetes environment running on bare-metal machines (no GCE) and now we wish to set up a PostgreSQL instance on top of this.
Our plan was to map a data volume for the PostgreSQL Data Directory to the node using the volumeMounts option in Kubernetes. However this would be a problem because if the Pod ever gets stopped, Kubernetes will re-launch it at random on one of the other nodes. Thus we have no guarantee that it will use the correct data directory on re-launch...
So what is the best approach for maintaining a consistent and persistent PostgreSQL Data Directory across a Kubernetes cluster?
one solution is to deploy HA postgresql, for example https://github.com/sorintlab/stolon
another is to have some network storage attached to all nodes(NFS, glusterFS) and use volumeMounts in the pods
I started to use Docker and I'm trying out Google's Kubernetes project for my container orchestration. It looks really good!
The only thing I'm curious of is how I would handle the volume storage.
I'm using EC2 instances and the containers do volume from the EC2 filesystem.
The only thing left is the way I have to deploy my application code into all those EC2 instances, right? How can I handle this?
It's somewhat unclear what you're asking, but a good place to start would be reading about your options for volumes in Kubernetes.
The options include using local EC2 disk with a lifetime tied to the lifetime of your pod (emptyDir), local EC2 disk with lifetime tied to the lifetime of the node VM (hostDir), and an Elastic Block Store volume (awsElasticBlockStore).
The Kubernetes Container Storage Interface (CSI) project is reaching maturity and includes a volume driver for AWS EBS that allows you to attach EBS volumes to your containers.
The setup is relatively advanced, but does work smoothly once implemented. The advantage of using EBS rather than local storage is that the EBS storage is persistent and independent of the lifetime of the EC2 instance.
In addition, the CSI plugin takes care of the disk creation -> mounting -> unmounting -> deletion lifecycle for you.
The EBS CSI driver has a simple example that could get you started quickly