Basically the question is simple:
How can I issue a query on a collection, which starts with _?
For example if I have 2 collections test and _test, and I am trying db.test.findOne() and db._test.findOne() in mongoshell the first one is working as intended, whereas the second tells me TypeError: db._testhas no properties (shell):1
Place it in quotes and use the getCollection method. See this article
Example To create a collection _foo and insert the { a : 1 } document, use the following operation:
db.getCollection("_foo").insert( { a : 1 } )
To perform a query, use the find() method, in as the following:
db.getCollection("_foo").find()
Related
I am new to MongoDB. I have a collection called person. I'm trying to get all the records without an _id field with this query:
db.person.find({}{_id:0})
but the error is
syntax error: unexpected {
but if i write
db.person.find()
it works perfectly.
Consider following documents inserted in person collection as
db.person.insert({"name":"abc"})
db.person.insert({"name":"xyz"}
If you want to find exact matching then use query as
db.person.find({"name":"abc"})
this return only matched name documents
If you want all names without _id then use projeciton id query as
db.person.find({},{"_id":0})
which return
{ "name" : "abc" }
{ "name" : "xyz" }
According to Mongodb manual you have little wrong syntax, you forgot to give comma after {}
Try this :
db.person.find({}, { _id: 0 } )
In order to iterate over all the documents in a MongoDB (2.6.9) collection using Grails (2.5.0) and the MongoDB Plugin (3.0.2) I created a forEach like this:
class MyObjectService {
def forEach(Closure func) {
def criteria = MyObject.createCriteria()
def ids = criteria.list { projections { id() } }
ids.each { func(MyObject.get(it)) }
}
}
Then I do this:
class AnalysisService{
def myObjectService
#Async
def analyze(){
MyObject.withStatelessSession {
myObjectService.forEach { myObject ->
doSomethingAwesome(myObject)
}
}
}
}
This works great...until I hit a collection that is large (>500K documents) at which point a CommandFailureException is thrown because the size of the aggregation result is greater than 16MB.
Caused by CommandFailureException: { "serverUsed" : "foo.bar.com:27017" , "errmsg" : "exception: aggregation result exceeds maximum document size (16MB)" , "code" : 16389 , "ok" : 0.0}
In reading about this, I think that one way to handle this situation is to use the option allowDiskUse in the aggregation function that runs on the MongoDB side so that the 16MB memory limit won't apply and I can get a larger aggregation result.
How can I pass this option to my criteria query? I've been reading the docs and the Javadoc for the Grails MongoDB plugin, but I can't seem to find it. Is there is another way to approach the generic problem (iterate over all members of a large collection of domain objects)?
This is not possible with the current implementation of MongoDB Grails plugin. https://github.com/grails/grails-data-mapping/blob/master/grails-datastore-gorm-mongodb/src/main/groovy/org/grails/datastore/mapping/mongo/query/MongoQuery.java#L957
If you look at the above line, then you will see that the default options are being used for building AggregationOptions instance so there is no method to provide an option.
But there is another hackish way to do it using the Groovy's metaclass. Let's do it..:-)
Store the original method reference of builder() method before writing criteria in your service:
MetaMethod originalMethod = AggregationOptions.metaClass.static.getMetaMethod("builder", [] as Class[])
Then, replace the builder method to provide your implementation.
AggregationOptions.metaClass.static.builder = { ->
def builderInstance = new AggregationOptions.Builder()
builderInstance.allowDiskUse(true) // solution to your problem
return builderInstance
}
Now, your service method will be called with criteria query and should not results in the aggregation error you are getting since we have not set the allowDiskUse property to true.
Now, reset the original method back so that it should not affect any other call (optional).
AggregationOptions.metaClass.static.addMetaMethod(originalMethod)
Hope this helps!
Apart from this, why do you pulling all IDs in forEach method and then re getting the instance using get() method? You are wasting the database queries which will impact the performance. Also, if you follow this, you don't have to do the above changes.
An example with the same: (UPDATED)
class MyObjectService {
void forEach(Closure func) {
List<MyObject> instanceList = MyObject.createCriteria().list {
// Your criteria code
eq("status", "ACTIVE") // an example
}
// Don't do any of this
// println(instanceList)
// println(instanceList.size())
// *** explained below
instanceList.each { myObjectInstance ->
func(myObjectInstance)
}
}
}
(I'm not adding the code of AnalysisService since there is no change)
*** The main point is here at this point. So whenever you write any criteria in domain class (without projection and in mongo), after executing the criteria code, Grails/gmongo will not immediately fetch the records from the database unless you call some methods like toString(), 'size()ordump()` on them.
Now when you apply each on that instance list, you will not actually loading all instances into memory but instead you are iterating over Mongo Cursor behind the scene and in MongoDB, cursor uses batches to pull record from database which is extremely memory safe. So you are safe to directly call each on your criteria result which will not blow up the JVM unless you called any of the methods which triggers loading all records from the database.
You can confirm this behaviour even in the code: https://github.com/grails/grails-data-mapping/blob/master/grails-datastore-gorm-mongodb/src/main/groovy/org/grails/datastore/mapping/mongo/query/MongoQuery.java#L1775
Whenever you write any criteria without projection, you will get an instance of MongoResultList and there is a method named initializeFully() which is being called on toString() and other methods. But, you can see the MongoResultList is implementing iterator which is in turn calling MongoDB cursor method for iterating over the large collection which is again, memory safe.
Hope this helps!
I want to print the value for this JSON document in the mongo shell. Like a simple console output, without creating a new collection or document.
Thanks in advance
I found a solution, by using .forEach() to apply a JavaScript method:
db.widget.find(
{ id : "4" },
{quality_level: 1, _id:0}
).forEach(function(x) {
print(x.quality_level);
});
db.collection.find() returns a cursor.
There are a bunch of methods for the cursor object which can be used to get the cursor information, iterate, get and work with the individual documents from the query result (in the Mongo Shell). For example,
let result = db.widget.find( { id : "4" }, { quality_level: 1, _id: 0 } );
The methods print, printjson and tojson are useful for printing the query results assigned to a cursor and iterated over it, in the Mongo Shell.
In this particular example, each of the following statements will print the expected output:
result.forEach( doc => print( doc.quality_level ) );
result.forEach( doc => print( tojson(doc)) );
result.forEach( printjson );
Note the query must be run each time the cursor is to be iterated, as the cursor gets closed after iterating thru it. But, one can use the cursor.toArray() method to get a JavaScript array, and iterate the array and work with the query result documents without running the query again.
NOTES:
The db.collection.findOne() method returns a document (not a cursor).
Iterate a Cursor in the mongo Shell
Mongo Shell cursor methods
To print each document when returning a cursor:
db.widget.find().forEach(printjson);
db.widget.findOne({"id":4},{quality_level:1,_id:0}).quality_level
The projection is not necessary for this to work, but it reduces the data which is to be transferred in case of a sharded collection.
I have a collection which have a document like this:
collection 1
{
_id: ObjectID(),
name: foo
}
I would get ObjectID of the above collection and copy into a document of another collection in order to reference correctly. Should I do simply:
db.collection1.find({name:"foo"},{_id:1})
EDIT
A call to find will return a cursor. Cursors works like an iterator in other languages. You can either attempt to find the first element in the cursor using the next() function and then get it's _id property or simplify your statement using findOne:
var x = db.collection1.findOne({name:"foo"}, {_id:1});
var id = x._id;
This is making an assumption that you are getting a document back from that query. You'll probably want to add a null check on x before grabbing the _id property.
I'm new to Mongo, and I'm trying to retrieve the first document from a find() query:
> db.scores.save({a: 99});
> var collection = db.scores.find();
[
{ "a" : 99, "_id" : { "$oid" : "51a91ff3cc93742c1607ce28" } }
]
> var document = collection[0];
JS Error: result is undefined
This is a little weird, since a collection looks a lot like an array. I'm aware of retrieving a single document using findOne(), but is it possible to pull one out of a collection?
The find method returns a cursor. This works like an iterator in the result set. If you have too many results and try to display them all in the screen, the shell will display only the first 20 and the cursor will now point to the 20th result of the result set. If you type it the next 20 results will be displayed and so on.
In your example I think that you have hidden from us one line in the shell.
This command
> var collection = db.scores.find();
will just assign the result to the collection variable and will not print anything in the screen. So, that makes me believe that you have also run:
> collection
Now, what is really happening. If you indeed have used the above command to display the content of the collection, then the cursor will have reached the end of the result set (since you have only one document in your collection) and it will automatically close. That's why you get back the error.
There is nothing wrong with your syntax. You can use it any time you want. Just make sure that your cursor is still open and has results. You can use the collection.hasNext() method for that.
Is that the Mongo shell? What version? When I try the commands you type, I don't get any extra output:
MongoDB shell version: 2.4.3
connecting to: test
> db.scores.save({a: 99});
> var collection = db.scores.find();
> var document = collection[0];
In the Mongo shell, find() returns a cursor, not an array. In the docs you can see the methods you can call on a cursor.
findOne() returns a single document and should work for what you're trying to accomplish.
So you can have several options.
Using Java as the language, but one option is to get a db cursor and iterate over the elements that are returned. Or just simply grab the first one and run.
DBCursor cursor = db.getCollection(COLLECTION_NAME).find();
List<DOCUMENT_TYPE> retVal = new ArrayList<DOCUMENT_TYPE>(cursor.count());
while (cursor.hasNext()) {
retVal.add(cursor.next());
}
return retVal;
If you're looking for a particular object within the document, you can write a query and search all the documents for it. You can use the findOne method or simply find and get a list of objects matching your query. See below:
DBObject query = new BasicDBObject();
query.put(SOME_ID, ID);
DBObject result = db.getCollection(COLLECTION_NAME).findOne(query) // for a single object
DBCursor cursor = db.getCollection(COLLECTION_NAME).find(query) // for a cursor of multiple objects