Is there a way for Node.js to listen to a change in a particular data in a collection of MongoDB, and fire an event if a change happens?
Well, this is an old question, but I was struggling with the same thing. I found a number of tidbits that helped me put together a solution, and I've published it as a library:
https://github.com/TorchlightSoftware/mongo-watch
The library is written in coffeescript. Here's an example in javascript, for those that prefer.
var MongoWatch = require('mongo-watch'),
watcher = new MongoWatch({parser: 'pretty'});
watcher.watch('test.users', function(event) {
return console.log('something changed:', event);
});
MongoDB apparently now supports triggers and watch, this answer is outdated.
[Original] I believe you are looking for a database trigger.
Unfortunately, MongoDB has no support for them yet, so I don't think you can listen for changes directly from the database. You'll need to setup some sort of notification system (e.g. pub/sub) that alerts interested parties when a collection has changed.
MongoDB 3.6 introduced change streams, which are designed to solve precisely this problem.
Here's an example: https://github.com/thakkaryash94/mongodb-change-streams-nodejs-example
const pipeline = [
{
$project: { documentKey: false }
}
];
const db = client.db("superheroesdb");
const collection = db.collection("superheroes");
const changeStream = collection.watch(pipeline);
// start listen to changes
changeStream.on("change", function (change) {
console.log(change);
});
Well, kind of late.
But still, if someone looking to notify the MongoDB changes to the node.js then they can use mubsub library.
This is active library and very easy to integrate with nodejs.
It uses Mongo's tailable cursors and capped collections.
It works in pub/sub fashion to notify the subscriber when the document inserted into the MongoDB.
Check on Github for details.
Related
I am a mongodb driver developer.
Is any way to get a stream for the changes? Like websocket/sse, keep sending data without close it.
Below is the $cmd which sent to mongodb to get a new changes from the server (I am using mongodb-core#3.0.2)
{
"getMore":"5293718446697444994",
"collection":"event",
"batchSize":1
}
Is any way to get a stream for the changes?
According to the official MongoDB driver specifications for ChangeStream, it is an abstraction of a TAILABLE_AWAIT cursor. You may choose to implement it as an extension of an existing tailable cursor implementation.
Extending an existing cursor implementation would provide benefits as you don't have to implement other behaviour/features that comes automatically with a cursor.
I've created a JSON API with Express.js, Mongoose and MongoDB. Currently, there's no way for the clients of the API to check if the data in a collection has changed - they would need to download the whole collection periodically.
How could I allow the clients of the API to check for changes to a collections (inserts, updates, deletions) without downloading the collection itself?
Is there a way of getting the version number of the collection, the last change timestamp or a hash of the collection with Mongoose? What is the best practice solution to this problem?
In current and earlier versions of MongoDB, you have to do it on the application side, maybe using polling.
MongoDB 3.6 has a brand new feature called change stream that allows you to listen changes happening on your collections in real time.
The sample code to listen selected changes happening on your collection is below:
var MongoClient = require('mongodb').MongoClient
, assert = require('assert');
MongoClient.connect("mongodb://172.16.0.110:27017/myproject?readConcern=majority").then(function(client){
var db = client.db('myproject')
var changeStreams = db.collection('documents').watch()
changeStreams.on('change', function(change){
console.log(change)
})
})
If you are using node.js, you need to use following node module to get it working:
"dependencies": {
"mongodb": "mongodb/node-mongodb-native#3.0.0"
}
i try to build a homeautomation system with meteor. therefore i would like to do the following thing.
i have a collection with all my different liveValues i'm reading from any kind of source. each document is a value of a for example sensor with the actual value.
now i want to create a second collection called thing. in this collection i'd like to add all my "Things" for example "Roomtemperature living" with the data for this thing. one attribute should be the connection to one of liveValues.
Now i want to publish and subscribe with Meteor the Thing collection, because on the webinterface it doesn't matter what liveValue is behind the Thing.
Here, the in my optionen, complicated part starts.
How can i publish the data to the client and i will have a reactive update when the LiveValue has changend for the thing? because it's an differnt collection than "Thing" collection.
In my idea i would like to do this via one subscrition to one "thing" document and i will get back with this subscription the update of the liveValue of the liveValue collection.
Is this workable?
has somebody an idea how i can handle this?
i've heard something about meteor-reactive-publish but i'not sure if this is the solution. also i've heard that this needs a lots of power for the server.
thanks for your help.
So basically you want to merge the documents on server side to one reactive collection on client-side.
You should use observeChanges provided by Meteor Collections as described in the docs.
By this you can observe the changes on your server side collections and publish to your client-side aggregated collection, like this:
// Get the data from a kind of sensor
var cursor = SomeSensor.find({/* your query */});
var self = this;
// Observe the changes in the cursor and publish
// it to the 'things' collection in client
var observer = cursor.observeChanges({
added: function (document) {
self.added('things', document._id, document);
},
removed: function (document) {
self.removed('things', document._id, document);
},
changed: function (document) {
self.changed('things', document._id, document);
}
});
// Make the publication ready
self.ready();
// Stop the observer on subscription stop
self.onStop(function () {
observer.stop();
});
With this the things collection will have the data from all the sensors reactively.
Hope it helps you.
How do I check if mongodb is still up from within Meteor? I tried (on the server) to stop mongodb and then write to a collection and read the record again, that works even if mongodb is not connected.
Problem is (on a self-hosted app) if the mongodb crashes the app does not throw an error, but there is no data. I would like to display an error message in that case.
The simplest way would be to create a document at startup and check whether it exists.
Meteor will poll all collections on a regular basis. If the database dies, the control document will be empty and this should trigger a ´removed´ event on the collection.
var control = Meteor.Collection("control-doc");
Meteor.startup(function () {
control.remove({});
control.insert({text: "just to fill"});
control.find().observe({
removed: function(old) {
//send an alert
console.log("alert");
}
})
});
it is not the nicest of tricks but it should work.
Have you tried Meteor.status()? Returns a reactive object with connection status stuff.
I can't find anything in the docs, but in case I'm missing something, is there any way in Meteor to enrich documents inserted from the client on the server before they're sent to Mongo.
The use-case for this is to handle things like setting up timestamps and other (for server-side use only) fields, without having to set them as part of the document on the client.
The obvious method would be to use Meteor.call('addMyNewRecord', publicFields) then insert solely from the server-side, but I quite like having the minimongo api on the client-side and I'm hoping to avoid the call requirement. I know in CouchDB you can do some basic modifications in their on-update handler but can't seem to find anything similar for Meteor/Mongo.
I finally found a nice approach to doing this whilst still using the minimongo interface on the client-side.
It seems you can intercept the insert and enrich the documents as part of the Deny policy, like this:
myCollection.deny({
insert: function(userId, doc) {
doc.created = new Date().valueOf();
doc.creator = userId;
//other default fields
return false;
}
});
I've tried to do similar with the update Deny policy to add a modified field but that doesn't seem to work the same way. As noted in the post linked above, this won't work for the Allow policy.