in my application I insert/update some documents.
I would need to act somehow depending on the result of the operation, but I do not understand how to use the WriteResult object.
This is the toString() of an update succesfully terminated:
Update write result: { "serverUsed" : "xxx.xxx.xxx.xxx:27017" , "ok" : 1 , "n" : 1 , "updatedExisting" : true}
Now, from the documentation I read that getLastError methods are deprecated.
I've getN that just tell me how many record has been updated (meaningless with inserts).
I've no methods to retrieve the OK value.
Do you have any suggestion on how manage the WriteResult object to understand the result of an operation?
Thanks in advance,
Samuel
I was using another framework that was hiding this deprecation, but since having removed that framework, I also came upon this problem.
I was also puzzled as you were on what the documentation was trying to say. Having gone through the MongoDB source code it looks like what was previously this:
WriteResult result = collection.update(query, update, true, false);
if (!result.getLastError().ok()) {
// handle error here
}
now looks like this:
try {
collection.update(query, update, true, false);
}
catch (CommandFailureException e) {
CommandResult result = e.getCommandResult();
// you can use result.ok() here, but it should almost always return false.
}
MongoException is a runtime exception that has several subclasses including CommandFailureException, so you may want to take a look at the javadocs here
Related
I think it should be fairly obvious what I'm trying to do here - query a collection m_coll, and get all unique values of Density from that collection. However, the thing it's returning is an element, not a full document, so you can't seem to key it, and it screams at you, namely C++ exception with description "unset document::element" thrown in the test body.. What modification needs to be made to make this work?
std::vector<int> MongoReader::getLvlOne()
{
std::vector<int> ret;
bsoncxx::builder::stream::document empty;
mongocxx::cursor cursor = m_coll.distinct("Density",empty.view());
for (bsoncxx::document::view doc : cursor)
{
ret.push_back(doc["Density"].get_int32());
}
return ret;
}
This is really obscure and poorly documented, for which I apologize. I've opened a Jira ticket, CXX-1406, about improving docs and providing an example.
The distinct method returns a cursor, but it only ever provides a single document that looks like this:
{
"values" : [ "A", "B" ],
"ok" : 1
}
That's just exactly what the distinct database command returns.
You can see an example of usage in the tests for distinct.
There's a ticket, CXX-1126, for a better API, but it would be a breaking change, so we're not sure when we'll address it.
In order to iterate over all the documents in a MongoDB (2.6.9) collection using Grails (2.5.0) and the MongoDB Plugin (3.0.2) I created a forEach like this:
class MyObjectService {
def forEach(Closure func) {
def criteria = MyObject.createCriteria()
def ids = criteria.list { projections { id() } }
ids.each { func(MyObject.get(it)) }
}
}
Then I do this:
class AnalysisService{
def myObjectService
#Async
def analyze(){
MyObject.withStatelessSession {
myObjectService.forEach { myObject ->
doSomethingAwesome(myObject)
}
}
}
}
This works great...until I hit a collection that is large (>500K documents) at which point a CommandFailureException is thrown because the size of the aggregation result is greater than 16MB.
Caused by CommandFailureException: { "serverUsed" : "foo.bar.com:27017" , "errmsg" : "exception: aggregation result exceeds maximum document size (16MB)" , "code" : 16389 , "ok" : 0.0}
In reading about this, I think that one way to handle this situation is to use the option allowDiskUse in the aggregation function that runs on the MongoDB side so that the 16MB memory limit won't apply and I can get a larger aggregation result.
How can I pass this option to my criteria query? I've been reading the docs and the Javadoc for the Grails MongoDB plugin, but I can't seem to find it. Is there is another way to approach the generic problem (iterate over all members of a large collection of domain objects)?
This is not possible with the current implementation of MongoDB Grails plugin. https://github.com/grails/grails-data-mapping/blob/master/grails-datastore-gorm-mongodb/src/main/groovy/org/grails/datastore/mapping/mongo/query/MongoQuery.java#L957
If you look at the above line, then you will see that the default options are being used for building AggregationOptions instance so there is no method to provide an option.
But there is another hackish way to do it using the Groovy's metaclass. Let's do it..:-)
Store the original method reference of builder() method before writing criteria in your service:
MetaMethod originalMethod = AggregationOptions.metaClass.static.getMetaMethod("builder", [] as Class[])
Then, replace the builder method to provide your implementation.
AggregationOptions.metaClass.static.builder = { ->
def builderInstance = new AggregationOptions.Builder()
builderInstance.allowDiskUse(true) // solution to your problem
return builderInstance
}
Now, your service method will be called with criteria query and should not results in the aggregation error you are getting since we have not set the allowDiskUse property to true.
Now, reset the original method back so that it should not affect any other call (optional).
AggregationOptions.metaClass.static.addMetaMethod(originalMethod)
Hope this helps!
Apart from this, why do you pulling all IDs in forEach method and then re getting the instance using get() method? You are wasting the database queries which will impact the performance. Also, if you follow this, you don't have to do the above changes.
An example with the same: (UPDATED)
class MyObjectService {
void forEach(Closure func) {
List<MyObject> instanceList = MyObject.createCriteria().list {
// Your criteria code
eq("status", "ACTIVE") // an example
}
// Don't do any of this
// println(instanceList)
// println(instanceList.size())
// *** explained below
instanceList.each { myObjectInstance ->
func(myObjectInstance)
}
}
}
(I'm not adding the code of AnalysisService since there is no change)
*** The main point is here at this point. So whenever you write any criteria in domain class (without projection and in mongo), after executing the criteria code, Grails/gmongo will not immediately fetch the records from the database unless you call some methods like toString(), 'size()ordump()` on them.
Now when you apply each on that instance list, you will not actually loading all instances into memory but instead you are iterating over Mongo Cursor behind the scene and in MongoDB, cursor uses batches to pull record from database which is extremely memory safe. So you are safe to directly call each on your criteria result which will not blow up the JVM unless you called any of the methods which triggers loading all records from the database.
You can confirm this behaviour even in the code: https://github.com/grails/grails-data-mapping/blob/master/grails-datastore-gorm-mongodb/src/main/groovy/org/grails/datastore/mapping/mongo/query/MongoQuery.java#L1775
Whenever you write any criteria without projection, you will get an instance of MongoResultList and there is a method named initializeFully() which is being called on toString() and other methods. But, you can see the MongoResultList is implementing iterator which is in turn calling MongoDB cursor method for iterating over the large collection which is again, memory safe.
Hope this helps!
I'm using MongoDB and Mongoose with Express to store tweets that I retrieve via the Twitter API.
I want to avoid saving duplicate tweets. I am doing something like that :
TweetsModel.find({tweet_id: tweet.tweet_id}, function (err, tweets) {
if(tweets.length > 0){
cb('Tweet already exists',null);
} else{
tweet.save(function(err){
cb(err,user);
});
}
});
My question is : for performance reason, is there a way using Mongoose to avoid doing two requests ? One find and one save ?
Knowing that I don't want to update the tweet if it already exists either.
Thank you
You can use an update call with the upsert option to do this:
TweetsModel.update(
{tweet_id: tweet.tweet_id},
{$setOnInsert: tweet},
{upsert: true},
function(err, numAffected) { .. }
);
If a doc already exists with that tweet id, then this is a no-op. Otherwise it will add the doc.
$setOnInsert requires v2.4+ of MongoDB. If your version is less than 2.4, things get more complicated.
Is in Mongo any way to do check and setting like atomic operation ? I am making booking for hotels and if there is free room you can reserve, but what if two or more people want to reserve in same time. Is there anything similar to transaction in Mongo or any way to solve this problem ?
Yes, that's the classic use case for MongoDB's findAndModify command.
Specifically for pymongo: find_and_modify.
All updates are atomic operations over a document. Now find_and_modify locks that document and returns it back in the same operation.
This allows you to combine a lock over the document during find and then applies the update operation.
You can find more about atomic operations:
http://www.mongodb.org/display/DOCS/Atomic+Operations
Best,
Norberto
The answers reference findAndModify documentation. But a practical example given the OP's requirements will do justice:
const current = new ISODate();
const timeAgoBy30Minutes = new Date(current.getTime() - 1000 * 30 ).toISOString();
db.runCommand(
{
findAndModify: "rooms",
query: {
"availability" : true,
"lastChecked" : {
"$lt": timeAgoBy30Minutes
}
},
update: { $set: { availability: false, lastChecked: current.toISOString() } }
}
)
In the above example, my decision to use db.runCommand verses db.rooms.findAndModify was strategic. db.runCommand will return a status code as to whether the document was updated, which allows me to perform additional work if the return value was true. findAndModify simply returns the old document, unless the new flag is passed to the argument list by which it will return the updated document.
Is there a function in mongo to return the number of documents that were updated in an update statement?
I have tried to use count() but apparently the update statement only returns true or false so i think I'm getting the count of a string.
Thanks
Use the getLastError command to get information about the result of your operation.
I don't know the ruby driver, but most drivers automatically do this in 'safe mode'. In safe mode, each write will examine the result of getLastError to make sure the write was successful. The update operation should return an object that looks like the JSON object further down, and it includes the number of updated documents (n). You can fine-tune the safe mode settings, but be warned that the default mode is "fire and forget", so safe mode is a good idea for many use cases.
In the shell,
> db.customers.update({}, {$set : {"Test" : "13232"}}, true, true);
> db.runCommand( "getlasterror" )
{
"updatedExisting" : true,
"n" : 3,
"connectionId" : 18,
"err" : null,
"ok" : 1
}
Here, I updated n = 3 documents. Note that by default, update operations in mongodb only apply to the first matched document. In the shell, the fourth parameter is used to indicate we want to update multiple documents.
For future reference as of the latest version of mongoDB, you can save the result of the update command and look for the property modifiedCount. Note that if the update operation results in no change to the document, such as setting the value of the field to its current value, it may be less than the number of documents that were matched in the query.
For example (in JS):
var updateResult = await collection.updateOne( .... )
if(updateResult.modifiedCount < 1) throw new Error("No docs updated");
There are also two fields result.n and result.nModified in the response object that tell you the number of documents matched, and the number updated, respectively. Here's a portion of the response object that mongoDB returns
result: {
n: 1,
nModified: 1,
...
},
...
modifiedCount: 1
More info here: https://docs.mongodb.com/manual/reference/command/update/