Elastic Beanstalk Deployment with MongoDB - mongodb

Question

Would really appreciate some suggestions for resources on how to properly deploy with Elastic Beanstalk with the following stack:
MongoDB
Rails (Puma)
Sidekiq/Redis
Elasticsearch
Do I need to get all these things setup in ebextension files? Or is it a matter of settings things up manually in AWS and then routing them together properly somewhere?

Answer 1

You definitely don't want to run all those on your Elastic Beanstalk servers. Elastic Beanstalk will automatically add or remove servers based on your traffic/server load. You don't want your database to be on one of those servers when it gets deleted.
Elastic Beanstalk is a Platform as a Service that is great for running web servers. There are other services on AWS such as ElastiCache (Redis/Memcached as a service) and Elasticsearch as a service. There are also third parties that provide services that run on AWS such as RedisLabs (Redis as a service) and MongoLab (MongoDB as a service).
You can decide to use any of these services to reduce the amount of system administration work you have to do yourself. Or you can manually setup EC2 Linux servers (outside of Elastic Beanstalk) and install things like Rails and MongoDB and ElasticSearch on them and manage them yourself.
For your case I would recommend something like the following:
Rails: ElasticBeanstalk
MongoDB: MongoLab
Redis: RedisLabs
Elasticsearch: AWS Elasticsearch Service
You would want to setup each of those services and then simply add the connection information for each of them to your Elastic Beanstalk environment so Rails can use them.
Edit:
Here are the best instructions on setting up MongoDB on EC2 manually: https://docs.mongodb.org/ecosystem/platforms/amazon-ec2/
For ElastiCache and Elasticsearch, you just click around in the AWS console to provision a Redis server and get the URLs to connect to. Once you have set all these things up, you just need to put the connection parameters in your ElasticBeanstalk environments as custom environment variables, something like:
MONGO_DB_URL="Your MongoDB EC2 internal IP address"
REDIS_URL="the url ElastiCache provided you"
Then read those environment variables in your application when creating connections to those services.
Also, you are going to have to learn about setting up your VPN and security groups to enable everything to connect. For example you will want your Elastic Beanstalk servers in one security group, and MongoDB server(s) in another group. Then you will have to configure the MongoDB security group to allow access from the beanstalk group on the MongoDB port. It's similar for ElastiCache. I think for Elasticsearch you will have to create an IAM role with access to the Elasticsearch API, and then assign that role to your Beanstalk servers.
Of course there is also the administrative tasks of setting up Linux servers for your MongoDB cluster, configuring clustering, fail-over, automated backups, log archives, periodic security updates, etc. I know you have all this AWS credit, but you should weigh moving everything over to AWS versus the cost of all the administrative tasks you will be spending time on. Elastic Beanstalk, Elasticsearch and ElasticCache are a no-brainer if you are getting them for free, but my MongoLab bill would have to be fairly high to justify setting all that up and managing it myself.