I am using MongoClient on a NodeJS service.
As part of improving my service's performance I want to determine if I should act differently when handling with single id vs with multiple ids in the terms of querying Mongo.
My current code looks like this:
collection.find({ id: {$in: ids } })
I wanted to know if I would split the code to handle single id separately would make the performance any better, somewhat like this:
if ids.count == 1
collection.findOne({ id: ids.first })
else
collection.find({ id: {$in: ids } })
end
According with MongoDB Driver for NodeJS, findOne is deprecated in favour of find().
Deprecated, use find().limit(1).next(function(err, doc){}) instead
http://mongodb.github.io/node-mongodb-native/2.0/api/Collection.html#findOne
Even more, findOne() is syntax sugar of find().limit(1).toArray()
I strongly suggest you to take a look if something is deprecated before use it in production at that link
Related
I am looking for a way to perform MongoDB's upsert operation in waterline. One way is to use the native method.
Model.native(function (err, Collection){
Collection.update({"key": "val"}, {"$set": {"x": "val"}}, {"upsert": true}, function (err, updated){
// do something
})
})
But if I don't want to use native, I could see on similar function findOrCreate, which creates a new entry if the item does not exist. However, I could not find a way to update the item if it exists. Is that possible or one has to use the native method only for that ?
Note: I am using Sails 0.11
As of now (v0.10.26) there is no way of doing a Mongo-style upsert besides the ones you've mentioned. This keeps coming up every once in a while but I couldn't find an issue addressing it directly, so maybe you could raise one.
Update
This issue talks about updateOrCreate and the linked threads suggest this might be added soon. I was looking into the Sails issues initially so no wonder I found nothing.
I have a collection containing around 100k documents. I want to add an auto incrementing "custom_id" field to my documents, and keep adding my documents by incrementing that field from now on.
What's the best approach for this? I've seen some examples in the official document (http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/) however they're only for adding new documents, not for updating an existing collection.
Example code I created based on the link above to increment my counter:
function incrementAndGetNext(counter, callback) {
counters.findAndModify({
name: counter
}, [["_id", 1]], {
$inc: {
"count": 1
}
}, {
"new": true
}, function (err, doc) {
if (err) return console.log(err);
callback(doc.value);
})
}
On the above code counters is db.counters collection and I have this document there:
{_id:"...",name:"post",count:"0"}
Would love to know.
Thank you.
P.S. I'm using native mongojs driver for js
Well, using the link you mentionned, I'd rather use the counters collection approach.
The counters collections approach has some drawbacks including :
It always generates multiples request (two): one to get the sequence number, another to do the insertion using the id you got via the sequence,
If you are using sharding features of mongodb, a document responsible for storing a counter state may be used a lot, and each time it will reach the same server.
However it should be appropriate for most uses.
The approach you mentionned ("the optimistic loop") should not break IMO, and I don't guess why you have a problem with it. However I'd not recommend it. What happens if you execute the code on multiple mongo clients, if one has a lot of latency and others keep taking IDs? I'd not like to encounter this kind of problem... Furthermore, there are at least two request per successful operation, but no maximum of retries before a success...
I am using spring data to talk to mongo (although I know there are few downsides to it over the mongo driver for advanced use cases). My application has a mongodb document with one of the elements being a list. Now I want to add/delete specific elements from the list in the document.
Can someone please tell me if and how spring data supports that (considering its a basic operation)?
I did quite some web search but didn't seem to land on anything concrete.
Any help would be appreciable.
UPDATE 1:
Sample document:
{ id: 123, arr: [ 1,2,3,4]}
and after update
{ id: 123, arr: [ 1,2,3]} or { id: 123, arr: [ 1,2,3,4,5]}
I'm looking at the unit tests and it looks like you can use the mongo $pull and $push methods using the Update class.
https://github.com/spring-projects/spring-data-mongodb/blob/master/spring-data-mongodb/src/test/java/org/springframework/data/mongodb/core/query/UpdateTests.java
You can see a tutorial here:
http://www.dineshonjava.com/2013/01/spring-data-mongodb-update-document.html#.VVexP3UVhBc
So you'd be doing something like:
Update().pull("arr", 4);
Update().push("arr", 5);
You would have to create a service which would handle this operation.
Assuming you are using one of Springs Repository interfaces your service could pull out the objects you need to update from the DB and then you would simply call .save() on each object.
Object o = repository.findById(someId);
o.setList(new ArrayList());
repository.save(o);
Another option would be to use the mongoOperation exposed by Spring data:
http://www.mkyong.com/mongodb/spring-data-mongodb-update-document/
This could be helpful if you are trying to update multiple documents.
I have trouble understanding when exactly the database is hit when using node-mongodb-native. Couldn't find any reference on that. As everything is callback based, it gave me the feeling that every single call hits the database ... For example, are those two snippets any different in terms of how many times the database is hit :
// ---- 1
db.collection('bla', function(err, coll) {
coll.findOne({'blo': 'bli'}, function(err, doc) {
coll.count(function(err, count) {
console.log(doc, count)
})
})
})
// ---- 2
db.collection('bla', function(err, coll) {
coll.findOne({'blo': 'bli'}, function(err, doc) {
db.collection('bla', function(err, coll) {
coll.count(function(err, count) {
console.log(doc, count)
})
})
})
})
I was basically wondering whether I can cache instances of collections and cursors. For example, why not fetch the collections I need only once, at server start, and reuse the same instances indefinitely ?
I'd really like to understand how the whole thing work, so I'd really appreciate a good link explaining stuff in details.
Looking at the source code for the node.js driver for collection it seems it will not ping MongoDB upon creation of the collection unless you have strict mode on: https://github.com/mongodb/node-mongodb-native/blob/master/Readme.md#strict-mode
The source code I looked at ( https://github.com/mongodb/node-mongodb-native/blob/master/lib/mongodb/db.js#L446 ) reinforced the idea that if strict was not on then it would just try and create a new node.js collection object and run the callback.
However findOne and count will break the "lazy" querying of node.js and will force it to query the database in order to get results.
Note: The count being on the collection won't enforce a "true" count of all items in the collection. Instead it will garnish this information from the collection meta.
So for the first snippet you should see two queries run. One for the findOne and one for the count and two for the second snippet as well since creating the collection after the findOne should not enforce a query to MongoDB.
After some googling, I have find this link about best practices for node-mongodb-native. It is answered by Christian Kvalheim who seem to be the maintainer of the library. He says :
"You can safely store the collection objects if you wish and reuse them"
So even if the call to collection might hit the database in case it is made in strict mode, the actual client-side collection instance can be reused.
I am wondering if it is possible to use a mongodb geospatial index with Meteor architecture.
Minimongo does not implement geospatial indices, but does this mean that we cannot use this mongo feature on the server side?
For example, with the todos app, if we use location on the todo, will it be possible to do:
// Publish complete set of lists to all clients.
Meteor.publish('todos', function (lon,lat) {
return Todos.find({loc: {$near:[lon,lat]}}).limit(2);
});
And on the client side :
Meteor.subscribe('todos', lon, lat );
Yes, you can use the MongoDB geospatial index within Meteor, and you can create that index from within your Meteor app too.
- Geospatial Search
I'm using the $within operator below, as opposed to the $near operator mentioned above, but this still applies:
Meteor.publish('places', function(box) {
return Places.find({ loc : { $within : { $box : box }}});
});
Reminder: These kinds of geo queries are only available on the server (currently).
- Creating a Geospatial Index from within Meteor (rather than in a MongoDB shell)
Places._ensureIndex({ loc : "2d" });
e.g. You could use the above in your bootstrap.js.
Also, you'll probably want to put your ensureIndex in Meteor.startup, or perhaps when you're inserting some initial data.
Warning: As mentioned here, the above method of calling ensureIndex is a work around for want of an official way to call it, so please expect that this might change.
Update: now reflects changes in Meteor 0.5.0, see #Dror's comment below.
Yes, I think you can.
On the server side, Meteor delegates find/update/.. into node-mongo-native call. You can take a look the code in packages/mongo-livedata/mongo_driver.js. And as I know, node-mongo-native supports geospatial index.