I would like to create an endless processing loop for capped collection directly in MongoDB, but I can't find how I can get tailabale cursor in MongoDB shell. It's possible in Python with tailable option in Collection.find() though..
You can add the option after find() using addOption():
db.coll.find().addOption(2) // probably want to use 2(tailable) + 32(await_data)
See all the options here: http://www.mongodb.org/display/DOCS/Mongo+Wire+Protocol#MongoWireProtocol-OPQUERY
You will want to put this in a loop as even tailable cursors (w/await_data) return no results sometimes.
Related
I want to download ~100K records from mongodb.
Using pymongo.
find() returns cursor, which, i guess, go to the db (cloud) for each single doc. This takes ~0.003 per document, not so good.
Is there some way to perform "fetch all" or something like that, to reduce run time?
I have Documents which have a reference key to another Document.
Is it possible to remove every Document which has a reference to that field in one go?
Right now, I would use db.collection.remove(<query for the reference>) and run it through every single collection, but this does not seem efficient at all.
What I want to do is to use one single remove query which would run through all documents. I am using the command prompt in RoboMongo.
The answer to your first question is yes you can do it.
How you can do it:
If you want to perform actions like that, you can write a small mongoDB script. learn more about mongo shell scripts
A solution to your post could be a script like that:
cursor = db.collection.find();
while ( cursor.hasNext() ) {
// Compare the two keys and remove the appropriate document
}
MongoDB Shell Scripting is a good solution if you have to automate actions.
What is
Meteor.Collection
and
Meteor.Collection.Cursor
?
How does these two related to each other? Did:
new Meteor.Collection("name")
create a MONGODB collection with the parameter name?
Did new Meteor.Collection("name") create a MONGODB collection with the parameter name?
Not exactly. A Meteor.Collection represents a MongoDB collection that may or may not exist yet, but the actual MongoDB collection isn't actually created until you insert a document.
A Meteor.Collection.Cursor is a reactive data source that represents a changing subset of documents that exist within a MongoDB collection. This subset of documents is specified by the selector and options arguments you pass to the Meteor.Collection.find(selector, options) method. This find() method returns the cursor object. I think the Meteor Docs explain cursors well:
find returns a cursor. It does not immediately access the database or return documents. Cursors provide fetch to return all matching documents, map and forEach to iterate over all matching documents, and observe and observeChanges to register callbacks when the set of matching documents changes.
Collection cursors are not query snapshots. If the database changes between calling Collection.find and fetching the results of the cursor, or while fetching results from the cursor, those changes may or may not appear in the result set.
Cursors are a reactive data source. The first time you retrieve a cursor's documents with fetch, map, or forEach inside a reactive computation (eg, a template or autorun), Meteor will register a dependency on the underlying data. Any change to the collection that changes the documents in a cursor will trigger a recomputation. To disable this behavior, pass {reactive: false} as an option to find.
The reactivity of cursors is important. If I have a cursor object, I can retrieve the current set of documents it represents by calling fetch() on it. If the data changes in between calls, the fetch() method will actually return a different array of documents. Many things in Meteor natively understand the reactivity of cursors. This is why we can return a cursor object from a template helper function:
Template.foo.documents = function() {
return MyCollection.find(); // returns a cursor object, rather than an array of documents
};
Behind the scenes, Meteor's templating system knows to call fetch() on this cursor object. When the server sends the client updates telling it that the collection has changed, the cursor is informed of this change, which causes the template helper to be recomputed, which causes the template to be rerendered.
A Meteor.Collection is an object that you would define like this:
var collection = new Meteor.Collection("collection");
This object then lets you store data in your mongo database. Note just defining a collection this way does not create a collection in your mongo database. The colleciton would be created after you insert a document in.
So in this way you would not have a collection called name until you insert a document to it.
A cursor is the result of a .find() operation:
var cursor = collection.find()
You may have 1000s of documents, the cursor lets you go through them, one by one, without having to load all of them into your server's RAM.
You can then loop through using forEach, or use some of the other operations as specified in the docs : http://docs.meteor.com/#meteor_collection_cursor
A Cursor is also a reactive data source on the client, so if data changes, you can use the same query to update your DOM.
As Neil mentions its also worthwhile knowing Mongo is a NoSQL database. This means you don't have to really create tables/collections. You would just define a collecction like above, then insert a document to it. This way the collection would be created if it didn't exist. If it already existed, it would be inserted into that collection instead.
Browsing your local database
You don't really need to concern yourself with MongoDB until you are publishing your app, you can just interact with it using Meteor alone. In case you want to have a look at what it looks like:
If you want to have a look at your Mongo database. While meteor is running, in the same directory use meteor mongo to bring up a mongo shell, or use a tool like robomongo (Gui tool) to connect to localhost on port 3002 to have a peek at what your mongo database looks like.
I have a python script which needs to do some actions whenever a new object is added in a collection
is there any efficient method to poll for addition of new object in mongodb collection?
Have a look at mongodb 'tailable' cursor.
http://www.mongodb.org/display/DOCS/Tailable+Cursors
Use "find" method of your python driver with "tailable" = true,
it will keep realtime track of additions in the database just like "tail -f" of a file in linux .
Tailable is FALSE by default.
http://api.mongodb.org/python/current/api/pymongo/collection.html
find([spec=None[, fields=None[, skip=0[, limit=0[, timeout=True[, snapshot=False[, tailable=False[, sort=None[, max_scan=None[, as_class=None[, slave_okay=False[, **kwargs]]]]]]]]]]]])
tailable (optional): the result of this find call will be a tailable cursor - tailable cursors aren’t closed when the last data is retrieved but are kept open and the cursors location marks the final document’s position. if more data is received iteration of the cursor will continue from the last document received. For details, see the tailable cursor documentation.
use a separate thread to poll the data. It is less efficient but works..
the alternative solution is to use twisted and its async driver but you still need to poll
the data.
How to create a tailable cursor on a mongodb capped collection with the C# driver?
I don't believe this feature is currently implemented in the C# driver.
https://jira.mongodb.org/browse/CSHARP-219
You should be able to add a watch to that item which will let you know when it's done.