I have a project which has the following characteristics:
Local MongoDB replica set on an on-premise database
Cloud MongoDB instance in MongoDB atlas
On-premise MongoDB should keep in sync with MongoDB atlas
Local MongoDB instance may be offline several days
Once its online, it should start synchronizing with MongoDB atlas
Basically, I'm looking for something similar to Realm, except that this solution runs on an actual local server and not a mobile device.
I have looked into live migrations, see here. But this doesn't seem to fit this use-case entirely, as its intended for an eventual cutover, which I don't want.
Therefore, how can I achieve the following with MongoDB atlas? What am I missing?
Can I treat MongoDB atlas, as if its a part of my local replica set, and use the standard replication capability of MongoDB? I.e. Atlas will always be a secondary.
This functionality with native MongoDB Atlas not possible. You need to look for customised solution.
Related
Is there any way to automate backups from my MongoDB Atlas cluster to a local machine using mongoose and node-cron? I've looked around but it would seem I need a paid cluster to have automatic updates enabled and even then they also can't be stored.
I have an express app using MongoDB up and running locally. I am looking at options to deploy and wasn't clear on how MongoDB atlas fit in. I planning on just deploying the express app and database to an ec2 instance. Is that alright? Or do I need a separate instance for mongo to run on? MongoDB Atlas offers M2, M5, M10 etc. as options for nodes. I am very new at backend and want to know if those would be separate from my EC2 instance or if those would be my EC2 instance running my express app for clients to connect to as well.
Mongo Atlas is a standalone hosted MongoDB instance. It's a separate server, or typically a cluster of several servers, that only runs MongoDB. You'd run your Express app on an EC2 instance and have it talk over the network to the Mongo Atlas instance on another server.
The advantage is that you don't have to worry about installing or handholding Mongo, about configuring a redundant Mongo cluster, about upgrades or backups. Generally, separating the database server from the application server also means easier longterm maintenance of both. If your Express server doesn't store any data itself, then it is entirely disposable in case of emergencies, while you can be assured* that the critical data stored in your database is well cared for.
* As far as your contract with Atlas stipulates that the data is being cared for…
Is it possible, and if so how, to use an Azure MongoDB as the backend for my Meteor Application.
I have added the connection string from my database into the MONGO_URL variable with no success. I have found some previous threads over Stack Overflow and on here about incompatibility related to oplog errors, but they seem to be using DocumentDB instead of Azure's MongoDB (which I think is newer than a few years ago).
In your example, you're actually using DocumentDB with MongoDB compatibility. You're not using native MongoDB (nor is this native MongoDB as-a-service).
DocumentDB (even with MongoDB compat) does not provide an oplog. And since Meteor has a dependency on reading the oplog, you wouldn't be able to point Meteor at DocumentDB.
In your case, you'd need to either run native MongoDB on your own (e.g. in VMs) or take advantage of a 3rd-party MongoDB hosting solution which provides MongoDB support within the same region as your app. (ok, yes, you can run your app in a different region, but you'd see latency along with data egress charges).
I have a MongoDB replica set that recently got hacked and hackers deleted my database. I don't want this to happen again. What I would like is, only a handful of programs like my MongoDB replica set members, node.js program, and my terminal should be the only one that can communicate with the database and no other program. How should I go about it?
For starters, I have set bind_ip property in my mongod.conf to be [127.0.0.1,ip_1,ip_2,ip_3], one of the IP is it's own IP. Is this enough?
First I identified it is possible to securely connect all my servers. I found the answer with this article on setting up ufw on my ubuntu 16.04. Then I established mongo replica set connection between them. Now the challenge came in establishing Keyfile access control in existing replica set
which I found to be difficult Because of this issue on Github in mongoose.
Using SSL/TLS is easy on the native driver than it is on the mongoose. Plus native driver is much faster than any ORM. So in a phase wise manner I replaced mongoose with mongodb native driver and introduced key file access. Maybe in later versions of mongoose, they might introduce this access.
One of our clients have a server running a MongoDB instance and we have to build an analytical application using the data stored in their MongoDB database which changes frequently.
Clients requirements are:
That we do not connect to their MongoDB instance directly or run another instance of MongoDB on their server but just somehow run our own MongoDB instance on our machine in our office using their MongoDB database directory with read only access remotely.
We've suggested deploying a REST application, getting a copy of their database dump but they did not want that. They just want us to run our own MongoDB intance which is hooked up with the MongoDB instance directory. Is this even possible ?
I've been searching for a solution for the past two days and we have to submit a solution by Monday. I really need some help.
I think this is normal request because analytical queries could cause too much load on the production server. It is pretty normal to separate production and analytical databases.
The easiest option is to use MongoDB replication. Set up MongoDB replica set with production database instance as primary and analytical database instance as secondary, also configure the analytical instance to never become primary.
If it is not possible to use replication - for example client doesn't want this, the servers could not connect directly to each other... - there is another option. You can read oplog from remote database and apply operations to your database instance. This is exactly the low level mechanism how replica set works, but you can do it manually too. For example MMS (Mongo Monitoring Sevice) Backup uses reading oplog for online backups of MongoDB.
Update: mongooplog could be the right tool for real-time application of replication oplog pulled from remote server on local server.
I don't think that running two databases that points to the same database files is possible or even recommended.
You could use mongorestore to restore from their data files directly, but this will only work if their mongod instance is not running (because mongorestore will need to lock the directory).
Another solution will be to do file system snapshots and then restore to your local database.
The downside to this backup/restore solutions is that your data will not be synced all the time.
Probably the best solution will be to use replica sets with hidden members.
You can create a replica set with just two members:
Primary - this will be the client server.
Secondary - hidden, with votes and priority set to 0. This will be your local instance.
Their server will always be primary (because hidden members cannot become primaries). Clients cannot see hidden members so for all intents and purposes your server will be read only.
Another upside to this is that the MongoDB replication will do all the "heavy" work of syncing the data between servers and your instance will always have the latest data.