I have been reading these few posts but they are a bit outdated so I just want to confirm if my understanding is correct up to date.
I am currently playing around with mongodb atlas and using the free tier only.
I am thinking of using aws lambda to periodically access mongodb to make a backup to s3.
To my understanding, I have to turn on 0.0.0.0 which I do not want to do at all because it'll be a bad idea if it's a production db though.
Through the readings and up to today. If I don't want to add 0.0.0.0 into mongodb atlas whitelist I have to create a VPC in aws just for the lambda function right?
Also, free tiers at mongodb atlas does not support the connection?
Thanks in advance for any help and suggestions to my understanding.
These are the posts I read
https://www.mongodb.com/blog/post/introducing-vpc-peering-for-mongodb-atlas
https://www.mongodb.com/blog/post/serverless-development-with-nodejs-aws-lambda-mongodb-atlas
AWS Lambda To Atlas
Related
Hi I'd like to move some data from an AWS DocumentDB to a PostgreSQL, I've found on the web ToroDB that would be the best tool for my task. It's implemented for MongoDB I'd like to know if I can use it also in AWS DocumentDB (with MongoDB compatibility)
Thank you in advice!
From what I can see in ToroDB documentation , is using the oplog to tail the changes in MongoDB, not change streams. Amazon DocumentDB doesn't have the oplog collection, but does supports change streams. Unless ToroDB gets updated to support change streams (which I doubt it will, documentation says only MongoDB 3.2 and 3.4 are supported and last Github update is from 5 years ago), is not going to be able to live replicate from DocumentDB.
However, if the destination is AWS RDS PostgreSQL or AWS Aurora PostgreSQL, then AWS DMS can be used to replicate from DocumentDB.
I've tried multiple things to try to connect AWS glue to MongoDB atlas. Has someone been successful in doing so and if so, please can someone help me with the steps.
The AWS documentation claims that it should work with any compatible MongoDB link but it doesn't.
I am facing a similar issue. I checked with the AWS support team and it seems like they have a huge backlog of similar issues where customers have requested the ability to connect to MongoDB Atlas. Unfortunately, they don't have an ETA for this.
Either you can opt to migrate to AWS Document DB and then use Glue to crawl your data store or you probably have to think of some other way to get your data from atlas to a layer that is supported properly by Glue.
I have a project which has the following characteristics:
Local MongoDB replica set on an on-premise database
Cloud MongoDB instance in MongoDB atlas
On-premise MongoDB should keep in sync with MongoDB atlas
Local MongoDB instance may be offline several days
Once its online, it should start synchronizing with MongoDB atlas
Basically, I'm looking for something similar to Realm, except that this solution runs on an actual local server and not a mobile device.
I have looked into live migrations, see here. But this doesn't seem to fit this use-case entirely, as its intended for an eventual cutover, which I don't want.
Therefore, how can I achieve the following with MongoDB atlas? What am I missing?
Can I treat MongoDB atlas, as if its a part of my local replica set, and use the standard replication capability of MongoDB? I.e. Atlas will always be a secondary.
This functionality with native MongoDB Atlas not possible. You need to look for customised solution.
i am working on my next project currently which works 100% on mongo,
my past projects worked on SQL + Mongo on which i used AWS RDS + AWS EC2 and could connect them both in AWS internal IP which result me with much faster connection.
Now in mongo there is alot of fancy cloud servers like MLab and MongoDB Atlas which is actually cheaper then AWS.
My concern is that moving back to external DB connection will be slower and more network consuming then the internal connection in RDS
Have anyone experienced in such issue? maybe the different isn't that big as i make it but i need it to be optimized
This depends on your setup. Many of the "fancy" services also host stuff on AWS, so latency is minimal. Some even offer "private environments" or such, so you can hide your databases from public view.
The only thing left to care about is the amount of network traffic. But this will be your problem regardless of your database host. You can test this relatively easily (e.g. get a trial from one of the providers and test for throughput, or raise your own MongoDB docker cluster to use as a test etc) just to get an idea of the performance range you'll be in.
My Mongo database is hosted at MongoLab. I'd like to use ElasticSearch as a full text search engine on top of my DB.
As I understand MongoDB needs to run as a replica-set, but I don't have any control on how the database run. I'm currently using the 500mb free plan.
On the top of that, I'm using the scala playframework.
Was anyone successful with those technologies and services?
Update:
Finally I'm not using MongoDB anymore, and went straight for a ElasticSearch solution.
I found this nice cloud host providing a 500MB free plan http://facetflow.com/
It was very useful for my development.
I didn't find any satisfying Scala library for ES, therefore I'm using Dispatch and make direct http requests to the ES instance.
I hope that someone will find this useful.
Just a quick note ... MongoHQ has oplog support with their MongoDB Elastic Deployments ... those could help you with using Elastic Search and River.
http://blog.mongohq.com/elastic-deployments-now-with-oplog-access/
I haven't looked into this too deeply, but you might want to check out Searchly http://www.searchly.com/features/ . The features mention
Built-in crawler for crawling web pages and databases. (Currently MongoDB)
If you try this out, please let me know how it goes. I will do the same.
Update:
I haven't tried searchly, but I was able to start a MongoDB instance in replica mode on OpenShift.
I have also an Elastic Search server running on the same OpenShift "gear".
Now I need time to try connecting those two together, and then the fun will start :-)