Here is my statement in mongodb:
var target = db.test.find({},{_id:1}).sort({_id:-1}).limit(1);
which can give me the largest _id. When I type target in shell, the output is like this:
> target
{ "_id" : ObjectId("51e062189c6ae665454e301d") }
However, when I type target again, nothing returns. Also, when I use target in other queries, those queries don't work either. Can anyone help me out?
This is most probably because target becomes a cursor since it is a result of .find(). However you can still obtain the resulting document by using toArray() method of MongoDB cursor like the following:
> var target = db.test.find({},{_id:1}).sort({_id:-1}).limit(1).toArray()
> if (target.length > 0) { target = target[0]; } else { target = null; }
You may find some information on toArray here.
Related
I have a document like below:
"application" : "test",
"QA1" : {
"url" : "https://google.co.in",
"db" : {
"userName" : "user",
"password" : "pswd"
}
}
I want to retrieve the value "https://google.co.in" by calling the key "url".
While using the below shell command, I am able to retrieve the value as expected.
db.getCollection('Application_Data').findOne({"application":"test"}).QA1.url
But when I convert it to java code, it is throwing me null pointer exception:
cursor = collection.find(Filters.eq("application","test")).iterator();
while (cursor.hasNext()) {
value=cursor.next().get("QA1.url").toString();
break;
}
I also tried with projection like below to get the required values:
cursor = collection.find(Filters.and(Filters.eq("application","test"))).projection(Projections.fields(Projections.include("QA1.url"),Projections.excludeId())).iterator();
while (cursor.hasNext()) {
System.out.println(cursor.next().get("QA1").toString());
}
This is giving me the output as below:
Document{{url=https://google.co.in}}
It is still not giving me the exact value. Appreciate your help on this.
Mongodb Documents are nested. So say, in your example, when you do cursor.next().get("QA1"), you get the Document under "QA1" property.
So the way to achieve what you want is
cursor.next().get("QA1").get("url"). That, with proper error handling (think exceptions).
A better way to do what you want is to use a mapper like spring-data-mongodb that would map the JSON response of mongodb into a java object for you automatically.
You need to use the Document class's getEmbedded method to extract the nested field's value.
while (cursor.hasNext()) {
Document doc = cursor.next();
System.out.println(doc.toJson());
String urlValue = doc.getEmbedded(Arrays.asList("QA1","url"), String.class);
System.out.println(urlValue); // output below
}
The result is as expected - a string: https://google.co.in
I need to search for the existence of a property that is within another object.
the collection contains documents that look like:
"properties": {
"source": {
"a/name": 12837,
"a/different/name": 76129
}
}
As you can see below, part of the query string is from a variable.
With some help from JohnnyHK (see mongo query - does property exist? for more info), I've got a query that works by doing the following:
var name = 'a/name';
var query = {};
query['properties.source.' + name] = {$exists: true};
collection.find(query).toArray(function...
Now I need to see if I can index the collection to improve the performance of this query.
I don't have a clue how to do this or if it is even possible to index for this.
Suggestions?
2 things happening in here.
First probably you are looking for sparse indexes.
http://docs.mongodb.org/manual/core/index-sparse/
In your case it could be a sparse index on "properties.source.a/name" field. Making indexes on field will dramatically improve your query lookup time.
db.yourCollectionName.createIndex( { "properties.source.a/name": 1 }, { sparse: true } )
Second thing. Always when you want to know whether your query is fast/slow, use mongo console, run your query and on its result call explain method.
db.yourCollectionName.find(query).explain();
Thanks to it you will know whether your query uses indexes or not, how many documents it had to check in order to complete query and some others useful information.
In order to iterate over all the documents in a MongoDB (2.6.9) collection using Grails (2.5.0) and the MongoDB Plugin (3.0.2) I created a forEach like this:
class MyObjectService {
def forEach(Closure func) {
def criteria = MyObject.createCriteria()
def ids = criteria.list { projections { id() } }
ids.each { func(MyObject.get(it)) }
}
}
Then I do this:
class AnalysisService{
def myObjectService
#Async
def analyze(){
MyObject.withStatelessSession {
myObjectService.forEach { myObject ->
doSomethingAwesome(myObject)
}
}
}
}
This works great...until I hit a collection that is large (>500K documents) at which point a CommandFailureException is thrown because the size of the aggregation result is greater than 16MB.
Caused by CommandFailureException: { "serverUsed" : "foo.bar.com:27017" , "errmsg" : "exception: aggregation result exceeds maximum document size (16MB)" , "code" : 16389 , "ok" : 0.0}
In reading about this, I think that one way to handle this situation is to use the option allowDiskUse in the aggregation function that runs on the MongoDB side so that the 16MB memory limit won't apply and I can get a larger aggregation result.
How can I pass this option to my criteria query? I've been reading the docs and the Javadoc for the Grails MongoDB plugin, but I can't seem to find it. Is there is another way to approach the generic problem (iterate over all members of a large collection of domain objects)?
This is not possible with the current implementation of MongoDB Grails plugin. https://github.com/grails/grails-data-mapping/blob/master/grails-datastore-gorm-mongodb/src/main/groovy/org/grails/datastore/mapping/mongo/query/MongoQuery.java#L957
If you look at the above line, then you will see that the default options are being used for building AggregationOptions instance so there is no method to provide an option.
But there is another hackish way to do it using the Groovy's metaclass. Let's do it..:-)
Store the original method reference of builder() method before writing criteria in your service:
MetaMethod originalMethod = AggregationOptions.metaClass.static.getMetaMethod("builder", [] as Class[])
Then, replace the builder method to provide your implementation.
AggregationOptions.metaClass.static.builder = { ->
def builderInstance = new AggregationOptions.Builder()
builderInstance.allowDiskUse(true) // solution to your problem
return builderInstance
}
Now, your service method will be called with criteria query and should not results in the aggregation error you are getting since we have not set the allowDiskUse property to true.
Now, reset the original method back so that it should not affect any other call (optional).
AggregationOptions.metaClass.static.addMetaMethod(originalMethod)
Hope this helps!
Apart from this, why do you pulling all IDs in forEach method and then re getting the instance using get() method? You are wasting the database queries which will impact the performance. Also, if you follow this, you don't have to do the above changes.
An example with the same: (UPDATED)
class MyObjectService {
void forEach(Closure func) {
List<MyObject> instanceList = MyObject.createCriteria().list {
// Your criteria code
eq("status", "ACTIVE") // an example
}
// Don't do any of this
// println(instanceList)
// println(instanceList.size())
// *** explained below
instanceList.each { myObjectInstance ->
func(myObjectInstance)
}
}
}
(I'm not adding the code of AnalysisService since there is no change)
*** The main point is here at this point. So whenever you write any criteria in domain class (without projection and in mongo), after executing the criteria code, Grails/gmongo will not immediately fetch the records from the database unless you call some methods like toString(), 'size()ordump()` on them.
Now when you apply each on that instance list, you will not actually loading all instances into memory but instead you are iterating over Mongo Cursor behind the scene and in MongoDB, cursor uses batches to pull record from database which is extremely memory safe. So you are safe to directly call each on your criteria result which will not blow up the JVM unless you called any of the methods which triggers loading all records from the database.
You can confirm this behaviour even in the code: https://github.com/grails/grails-data-mapping/blob/master/grails-datastore-gorm-mongodb/src/main/groovy/org/grails/datastore/mapping/mongo/query/MongoQuery.java#L1775
Whenever you write any criteria without projection, you will get an instance of MongoResultList and there is a method named initializeFully() which is being called on toString() and other methods. But, you can see the MongoResultList is implementing iterator which is in turn calling MongoDB cursor method for iterating over the large collection which is again, memory safe.
Hope this helps!
I'm new to Mongo, and I'm trying to retrieve the first document from a find() query:
> db.scores.save({a: 99});
> var collection = db.scores.find();
[
{ "a" : 99, "_id" : { "$oid" : "51a91ff3cc93742c1607ce28" } }
]
> var document = collection[0];
JS Error: result is undefined
This is a little weird, since a collection looks a lot like an array. I'm aware of retrieving a single document using findOne(), but is it possible to pull one out of a collection?
The find method returns a cursor. This works like an iterator in the result set. If you have too many results and try to display them all in the screen, the shell will display only the first 20 and the cursor will now point to the 20th result of the result set. If you type it the next 20 results will be displayed and so on.
In your example I think that you have hidden from us one line in the shell.
This command
> var collection = db.scores.find();
will just assign the result to the collection variable and will not print anything in the screen. So, that makes me believe that you have also run:
> collection
Now, what is really happening. If you indeed have used the above command to display the content of the collection, then the cursor will have reached the end of the result set (since you have only one document in your collection) and it will automatically close. That's why you get back the error.
There is nothing wrong with your syntax. You can use it any time you want. Just make sure that your cursor is still open and has results. You can use the collection.hasNext() method for that.
Is that the Mongo shell? What version? When I try the commands you type, I don't get any extra output:
MongoDB shell version: 2.4.3
connecting to: test
> db.scores.save({a: 99});
> var collection = db.scores.find();
> var document = collection[0];
In the Mongo shell, find() returns a cursor, not an array. In the docs you can see the methods you can call on a cursor.
findOne() returns a single document and should work for what you're trying to accomplish.
So you can have several options.
Using Java as the language, but one option is to get a db cursor and iterate over the elements that are returned. Or just simply grab the first one and run.
DBCursor cursor = db.getCollection(COLLECTION_NAME).find();
List<DOCUMENT_TYPE> retVal = new ArrayList<DOCUMENT_TYPE>(cursor.count());
while (cursor.hasNext()) {
retVal.add(cursor.next());
}
return retVal;
If you're looking for a particular object within the document, you can write a query and search all the documents for it. You can use the findOne method or simply find and get a list of objects matching your query. See below:
DBObject query = new BasicDBObject();
query.put(SOME_ID, ID);
DBObject result = db.getCollection(COLLECTION_NAME).findOne(query) // for a single object
DBCursor cursor = db.getCollection(COLLECTION_NAME).find(query) // for a cursor of multiple objects
I like Mongo for simple things so I was hoping to use it for something more advanced. And that worked fine until I needed this:
UPDATE tbl SET a = b WHERE c <> 0
The a = b part is what I can't figure out. I tried mongodb.org, but I can't find it there. I also looked for WHERE a = b but I can't find that either.
An alternative is so fetch all rows and than update them individually, but I don't like that. It has to be simpler.
Thanks.
You want to check the documentation for updating.
http://www.mongodb.org/display/DOCS/Updating
Your code might look like:
db.tbl.update( { c:{$ne:0}}, { $set: { a : b } } );
If you need to brush up on advanced queries (e.g. using $ne), then check here:
http://www.mongodb.org/display/DOCS/Advanced+Queries
EDIT:
Apparently you can't update with data from the same document.
MongoDB: Updating documents using data from the same document
EDIT 2 (solution with map reduce):
var c = new Mongo();
var db = c.getDB('db')
var s = db.getCollection('s')
s.drop();
s.save({z:1,q:5});
s.save({z:11,q:55});
db.runCommand({
mapreduce:'s',
map:function(){
var i = this._id; //we will emit with a unique key. _id in this case
this._id=undefined; //strange things happen with merge if you leave the id in
//update your document with access to all fields!
this.z=this.q;
emit(i,this);
},
query:{z:1}, //apply to only certain documents
out:{merge:'s'} //results get merged (overwrite themselves in collection)
});
//now take a look
s.find();