Does oplog play a role in replication ,recovery or both.
If we are looking to only recover certain 'non-temporal' collections in our database - in case of a node recover - is it possible to do so selectively.
ie For a particular collection or data operation on a collection,is it possible to flag to omit the operation/collection from oplog.
Related
How should i trigger mongodb event for instance trigger on Insert, Update or Delete to synchronize mysql DB. My requirement is to synchronize Mongodb to mysql using mongodb trigger, for instance if there is any insert, it should be replicate at mysql
MongoDB does not support trigeers AFAIK.
However in case of replication they do maintain an oplog (short for operation log), it is used to sync operation from master to slave. In case if replication is enabled you can leverage oplog and query on that, it works like any other collection.
One more option is to use tailable cursor in capped collection. A tailable cursor is open even after it returns the result, so if a new result is returned, it sends the result to cursor. Though I think it won't work for update or delete cases. (https://docs.mongodb.com/manual/core/tailable-cursors/)
One custo approach is to do it on application level by scheduler which periodically do syncing or if you might use AOP.
Is it possible to create a capped collection in a read only mode at a single replica in MongoDB? There is a main database with replica set and I need to use a capped collection with a AWS queue to listen for new insertions. In order to avoid possible listening overloads in the main database, I was wonder whether is possible to create a capped in one of the replicas.
This is not possible as at now. Also keep in mind that replication's main purpose it to have similar data through out the replica set.
Is the data for same timestamps ( i.e. pairs: [timestamp:number_of_operation] -- [operation] ) between oplogs:
on primary server;
on secondary server;
in oplog, formed by mongodump (mongodump --oplog ...);
are identical?
Some my checks displays that are the same, but i can`t find this fact in official documentation.
Thanks in advance!
The entries in the oplogs for a MongoDB replicaset are identical everywhere. The entries are created on the primary and then copied everywhere else - either via replication by reading from the local.oplog.rs collection or by being copied via mongodump with the --oplog option.
From the documentation:
Replica Set Oplog
The oplog (operations log) is a special capped collection that keeps a
rolling record of all operations that modify the data stored in your
databases. MongoDB applies database operations on the primary and then
records the operations on the primary’s oplog. The secondary members
then copy and apply these operations in an asynchronous process. All
replica set members contain a copy of the oplog, in the local.oplog.rs
collection, which allows them to maintain the current state of the
database.
Keep in mind that each entry may be copied a few times, if your replicaset is chaining it's replication, from primary to secondary to secondary for example. In each case however it's an identical copy of what came from the primary. The entries are created there and not updated.
http://docs.mongodb.org/manual/core/replica-set-oplog/
I have a MongoDB running with 3 member replica set. All the members are up & running.
I have one database with 4-5 collections and I want to delete that database.
What is the best way to do it. Can I just use db.dropDatabase() on primary?
Do I need to stop secondary before I drop the database? After I drop the database will secondary members sync automatically to primary? What about the memory, will it get free after I delete the database?
Yes. Just drop the database using the command db.dropDatabase() in the primary and the changes will be propagated to secondaries as well. You don't need to shutdown the secondaries.
as Anand Jayabalan said, you just need to drop the database on primary node. The replication do the rest reading the oplog for the secondaries.
From Reference links and official documentation.
The primary is the only member in the replica set that receives write operations. MongoDB applies write operations on the primary and then records the operations on the primary’s oplog. Secondary members replicate this log and apply the operations to their data sets.
Reference:
https://docs.mongodb.org/manual/core/replication-introduction/
https://docs.mongodb.org/manual/core/replica-set-primary/
I'm developing an application with a Elastich Search and MongoDB. The elastic search is using the MongoDB oplog to index the content via a component called a river.
Is it possible to reset the MongoDB oplog so that all previous entries dissapear?
The oplog is for replication and shouldn't be tampered with.
The oplog is a capped collection:
You cannot delete documents from a capped collection. To remove all
records from a capped collection, use the ‘emptycapped’ command. To
remove the collection entirely, use the drop() method.
http://docs.mongodb.org/manual/core/capped-collections/
You might want to use a tailable cursor and tail the oplog in your river.
If your app is going to read the oplog continuously, it would need the ability to start at a particular timestamp (ts) value. Without that ability, if the app (or mongod) had to be restarted for any reason, it would have to re-process all the oplog entries that it had already processed but were still in the oplog. If the app does have the ability to start at a ts value, then just query the oplog for the max value of ts, and use that as the starting point.