Is there any way to automate backups from my MongoDB Atlas cluster to a local machine using mongoose and node-cron? I've looked around but it would seem I need a paid cluster to have automatic updates enabled and even then they also can't be stored.
Related
I have a project which has the following characteristics:
Local MongoDB replica set on an on-premise database
Cloud MongoDB instance in MongoDB atlas
On-premise MongoDB should keep in sync with MongoDB atlas
Local MongoDB instance may be offline several days
Once its online, it should start synchronizing with MongoDB atlas
Basically, I'm looking for something similar to Realm, except that this solution runs on an actual local server and not a mobile device.
I have looked into live migrations, see here. But this doesn't seem to fit this use-case entirely, as its intended for an eventual cutover, which I don't want.
Therefore, how can I achieve the following with MongoDB atlas? What am I missing?
Can I treat MongoDB atlas, as if its a part of my local replica set, and use the standard replication capability of MongoDB? I.e. Atlas will always be a secondary.
This functionality with native MongoDB Atlas not possible. You need to look for customised solution.
The idea here is, I have mongo cluster deployed in managed cloud service atlas. I have enabled Continuous Backup.
Now what I want to do is :
1) I want to use existing backup.
2) Using this existing backup I want to create similar cluster
(having same data form backup)
3) Automate this process so that every day my new cluster gets upto date from original cluster.
Note: The idea here for cloning cluster is, The original cluster is production data. I want to create a db which has similar data on which I can plug and play using any analytic tools and perform diffrent operations without affecting production data and load.
So far what I have found is to use mongorestore and mongodump.But here mongodump is putting load on production db even though my backup is enabled. I want to use same backup to clone this to another db cluster.
Deployed on Atlas, your server must have replica set.
Here are 2 solutions :
You need only reading data : connect your tools to a secondary server (ideally dedicated with priority 0 for becoming primary)
You need to read/write data : on the same server than above, play your mongodump command with --oplog option. By this way, you're dumping your data from a read-only server, preventing slowing performances of your main servers.
In this last case, what you need will find its solution in backup strategies, take a look at the doc to know more.
There's an offering for this purpose in ATLAS called analytic node.Link.
Analytic node is read replica of your database. Plus it will not interfere with your production traffic which makes it safer.
Also, you can connect BI connectors to this node and create your analytic platform.
We used redash.
I have an express app using MongoDB up and running locally. I am looking at options to deploy and wasn't clear on how MongoDB atlas fit in. I planning on just deploying the express app and database to an ec2 instance. Is that alright? Or do I need a separate instance for mongo to run on? MongoDB Atlas offers M2, M5, M10 etc. as options for nodes. I am very new at backend and want to know if those would be separate from my EC2 instance or if those would be my EC2 instance running my express app for clients to connect to as well.
Mongo Atlas is a standalone hosted MongoDB instance. It's a separate server, or typically a cluster of several servers, that only runs MongoDB. You'd run your Express app on an EC2 instance and have it talk over the network to the Mongo Atlas instance on another server.
The advantage is that you don't have to worry about installing or handholding Mongo, about configuring a redundant Mongo cluster, about upgrades or backups. Generally, separating the database server from the application server also means easier longterm maintenance of both. If your Express server doesn't store any data itself, then it is entirely disposable in case of emergencies, while you can be assured* that the critical data stored in your database is well cared for.
* As far as your contract with Atlas stipulates that the data is being cared for…
We have our own mongo replica set setup in aws ec2, it is running all good. Now, what we want is to get MMS setup to monitor and backup the replica set. It seems that MMS will only monitor and backup the mongo instance that was deployed by the MMS.
Is there a work around for this?
As of the mid-October 2014 release of MMS, monitoring & backup has been integrated with deployment via the automation agent (http://blog.mongodb.org/post/99991498968/mongodb-management-service-re-imagined-the-easiest-way). There currently isn't an option to attach MMS to existing MongoDB deployments, however you can watch/upvote MMS-1991 in the MongoDB issue tracker for updates.
Is there a work around for this?
One option would be to deploy a new replica set using MMS and migrate your data to the managed deployment.
If you have an MMS group created with the pre-automation version, the "classic" MMS groups do not have support for automation and the agents can be installed independently. Note that the classic version of MMS is closed to new users.
One of our clients have a server running a MongoDB instance and we have to build an analytical application using the data stored in their MongoDB database which changes frequently.
Clients requirements are:
That we do not connect to their MongoDB instance directly or run another instance of MongoDB on their server but just somehow run our own MongoDB instance on our machine in our office using their MongoDB database directory with read only access remotely.
We've suggested deploying a REST application, getting a copy of their database dump but they did not want that. They just want us to run our own MongoDB intance which is hooked up with the MongoDB instance directory. Is this even possible ?
I've been searching for a solution for the past two days and we have to submit a solution by Monday. I really need some help.
I think this is normal request because analytical queries could cause too much load on the production server. It is pretty normal to separate production and analytical databases.
The easiest option is to use MongoDB replication. Set up MongoDB replica set with production database instance as primary and analytical database instance as secondary, also configure the analytical instance to never become primary.
If it is not possible to use replication - for example client doesn't want this, the servers could not connect directly to each other... - there is another option. You can read oplog from remote database and apply operations to your database instance. This is exactly the low level mechanism how replica set works, but you can do it manually too. For example MMS (Mongo Monitoring Sevice) Backup uses reading oplog for online backups of MongoDB.
Update: mongooplog could be the right tool for real-time application of replication oplog pulled from remote server on local server.
I don't think that running two databases that points to the same database files is possible or even recommended.
You could use mongorestore to restore from their data files directly, but this will only work if their mongod instance is not running (because mongorestore will need to lock the directory).
Another solution will be to do file system snapshots and then restore to your local database.
The downside to this backup/restore solutions is that your data will not be synced all the time.
Probably the best solution will be to use replica sets with hidden members.
You can create a replica set with just two members:
Primary - this will be the client server.
Secondary - hidden, with votes and priority set to 0. This will be your local instance.
Their server will always be primary (because hidden members cannot become primaries). Clients cannot see hidden members so for all intents and purposes your server will be read only.
Another upside to this is that the MongoDB replication will do all the "heavy" work of syncing the data between servers and your instance will always have the latest data.