How should i trigger mongodb event for instance trigger on Insert, Update or Delete to synchronize mysql DB. My requirement is to synchronize Mongodb to mysql using mongodb trigger, for instance if there is any insert, it should be replicate at mysql
MongoDB does not support trigeers AFAIK.
However in case of replication they do maintain an oplog (short for operation log), it is used to sync operation from master to slave. In case if replication is enabled you can leverage oplog and query on that, it works like any other collection.
One more option is to use tailable cursor in capped collection. A tailable cursor is open even after it returns the result, so if a new result is returned, it sends the result to cursor. Though I think it won't work for update or delete cases. (https://docs.mongodb.com/manual/core/tailable-cursors/)
One custo approach is to do it on application level by scheduler which periodically do syncing or if you might use AOP.
Related
A MongoDB collection is slow to provide data as it has grown huge overtime.
I need to add an index on a few fields and to reflect it immediately in search. So I seek for clarification on followings things:
Is it mandatory to restart MongoDB after indexing?
If yes, then is there any way to add index without restarting the server? I don't want any downtime...
MongoDB does not need to be restarted after indexing.
However, by default, the createIndex operation blocks read/write on the affected database (note that it is not only the collection but the db). You may change the behaviour using background mode like this:
db.collectionName.createIndex( { collectionKey: 1 }, { background: true } )
It might seem that your client is blocked when creating the index. The mongo shell session or connection where you are creating the index will block, but if there are more connections to the database, these will still be able to query and operate on the database.
Docs: https://docs.mongodb.com/manual/core/index-creation/
There is no need to restart MongoDB after you add an index!
However,an index could be created in the foreground which is the default.
What does it mean? MongoDB documentation states: ‘By default, creating an index on a populated collection blocks all other operations on a database. When building an index on a populated collection, the database that holds the collection is unavailable for reading or write operations until the index build completes. Any operation that requires a read or writes lock on all databases will wait for the foreground index build to complete’.
For potentially long-running index building operations on standalone deployments, the background option should be used. In that case, the MongoDB database remains available during the index building operation.
To create an index in the background, the following snippet should be used, see the image below.
I have 100+ worker threads, which are going to poll database, looking for a new job.
To take a job, a thread need to change status of the bunch of documents from NEW to IN_PROGRESS, so no other threads can peek the same job.
This can be solved perfectly fine in PostgreSQL with SELECT FOR UPDATE SKIP LOCKED WHERE status = "NEW" statement.
Is there a way to do such atomic update in MongoDB for a single document? For a batch?
There's a findAndModify method, which works exactly as you've described for a single document.
For a batch, it's not possible right now, as
In MongoDB, write operations, e.g. db.collection.update(), db.collection.findAndModify(), db.collection.remove(), are atomic on the level of a single document.
It will be possible in MongoDB 4.0 though, with transactions.
I have a MongoDB running with 3 member replica set. All the members are up & running.
I have one database with 4-5 collections and I want to delete that database.
What is the best way to do it. Can I just use db.dropDatabase() on primary?
Do I need to stop secondary before I drop the database? After I drop the database will secondary members sync automatically to primary? What about the memory, will it get free after I delete the database?
Yes. Just drop the database using the command db.dropDatabase() in the primary and the changes will be propagated to secondaries as well. You don't need to shutdown the secondaries.
as Anand Jayabalan said, you just need to drop the database on primary node. The replication do the rest reading the oplog for the secondaries.
From Reference links and official documentation.
The primary is the only member in the replica set that receives write operations. MongoDB applies write operations on the primary and then records the operations on the primary’s oplog. Secondary members replicate this log and apply the operations to their data sets.
Reference:
https://docs.mongodb.org/manual/core/replication-introduction/
https://docs.mongodb.org/manual/core/replica-set-primary/
I'm developing an application with a Elastich Search and MongoDB. The elastic search is using the MongoDB oplog to index the content via a component called a river.
Is it possible to reset the MongoDB oplog so that all previous entries dissapear?
The oplog is for replication and shouldn't be tampered with.
The oplog is a capped collection:
You cannot delete documents from a capped collection. To remove all
records from a capped collection, use the ‘emptycapped’ command. To
remove the collection entirely, use the drop() method.
http://docs.mongodb.org/manual/core/capped-collections/
You might want to use a tailable cursor and tail the oplog in your river.
If your app is going to read the oplog continuously, it would need the ability to start at a particular timestamp (ts) value. Without that ability, if the app (or mongod) had to be restarted for any reason, it would have to re-process all the oplog entries that it had already processed but were still in the oplog. If the app does have the ability to start at a ts value, then just query the oplog for the max value of ts, and use that as the starting point.
What type of locking does findAndModify() offer? Is is a write lock only, or read/write? Does it prevent simultaneous updates on the same record?
MongoDB has a global (per-instance) write lock, which serializes all updates across all data in the server (though different servers in a sharded cluster will each have their own independent locks). This means that at any given instant in time, only one update is taking place on any document, and therefore only one update for any given document.
findAndModify doesn't do anything different in this regard than an ordinary update -- it just returns the document to you.
According to the MongoDB docs for MongoDB: findAndModify() for under MongoDB: Atomic Operations it should be.