Does anyone know if its more feasible to install mongo-connector on its own server or to just have it running on one of the mongo database servers? I currently have it running on a secondary mongo server and want to move it off the secondary server but I am not sure if it really matters.
Mongo Connector is just another client application connecting to your MongoDB server. You can install it wherever you like, as long as it is able to connect to your MongoDB server and whatever external system you're trying to feed data to.
Note: If Mongo Connector is matching a comparatively small selection of data in the replication oplog to insert into your external system, it would make sense to keep this on the same secondary or node whose oplog is being monitored. Otherwise the Connector will be pulling the full oplog over the network, which may consume unnecessary bandwidth.
Related
We are using mongodb cluster with set of 3 members , and all consumers are able to read/write data.
Use mongo.conf file to create cluster && configure replica set, which are executed in ansible script as part of system restart.
Our system does heavy writes/read operation on mongo clusters ,and sometime replica get broken.
Then it is no longer feasible to restore replica without deleting data from one of the corrupted server.
Here are my queries:
what is better way to notify when replica is broken using script/some code?We can not use Mongo atlas
etc.
What is efficient way to restore data on corrupted member.
What is better way to notify when replica is broken using script/some code?
Use the replSetGetStatus database command.
You can run this command from any driver or the shell. Among other things, it returns the state (primary/secondary/startup/recovery/unreachable) and last applied operation time for each member.
What is efficient way to restore data on corrupted member.
Check out the docs for a few options: https://docs.mongodb.com/manual/tutorial/resync-replica-set-member/#resync-a-member-of-a-replica-set
I need access to the OpLog of the primary of my replica set in MongoDB. I am developing an OpLog driven ETL job as it is far more efficient than simply querying and transferring data from the entire table.
I can access the OpLog easily with the following steps in the terminal:
mongo mymongoserver.com:10733/my-db -u oplogUser -pxxxxxxxx
Then I run:
use local
and then I can query the OpLog using:
db.oplog.rs.find()
My question is: Are there any settings I can pass to the mongo connect command to get me straight into the local database for the primary node?
I am using Talend Open Studio for my ETL needs. Am I approaching this the wrong way?
I have come across this from the guys at Stripe, which means that this is definitely possible! https://www.mongodb.com/presentations/building-real-time-systems-mongodb-using-oplog-stripe
I have a python app running on Heroku using a PostgreSQL database. If I create a database follower, will that follower be used to balance the read database load automatically? I know this provides me a failover copy of sorts, but will it relieve my database load?
No -- you'll need to configure your Python software to send SQL queries to both the follower AND the master database in order to actually 'relieve' your database load.
If you're using Django, you'll want to read this: https://docs.djangoproject.com/en/1.8/topics/db/multi-db/
If you're using SQLAlchemy, you'll want to read this: read slave , read-write master setup
It's possible to connect meteor manually to 2 or more databases in order to have a normal mongo that saves to the disk and a memory one like redis?
I'm asking because mongo has already full support from meteor and using it would be a lot easier than redis or other database
Right now, a Meteor server can only connect to one (and exactly one) Mongo database.
Redis support is on the roadmap, as is SQL support. Once Meteor supports multiple databases, you will have more options for how to set up your databases as well as dividing up your data between them. The only way to do what you are saying right now is to have your Meteor client connect to two different Meteor servers, and have one of them clear/dump the database regularly.
Source: discussions at Meteor's offices.
For development reasons, I need to backup a production replica set mongodb and restore it on a stand alone, different machine test instance.
Some docs are talking about the opposite ( standalone 2 replica-set ), but I cannot find his downgrade/rollback way.
What's the way to go, in this case ?
No matter how many nodes you have in a replica set, each of them holds the same data.
So getting the data is easy - just use mongodump (preferably against the secondary, for performance reasons) and then mongorestore into a new mongod for your development stand-alone system.
mongodump does not pick up any replication related collections (they live in database called local). If you end up taking a file system snapshot of a replica node rather than using mongodump, be sure to drop the local database when you restore the snapshot into your production stand-alone server and then restart mongod so that it will properly detect that it is not part of a replica set.