I need to query documents with mongoDb that contain nested arrays. I see a lot of examples using the simple $in operator. The only problem is that I strictly need to check for proper subsets.
Consider the following document.
{data: [[1,2,3], [4,5,6]]}
The query needs to be able to get documents with all of [1,2,3] where 1,2,3 can be in any order, which rules out the following query, because it will only match in the correct order.
{data:{$elemMatch:{$all:[[1,2,3]]}}}
I've also tried nested $elemMatch operators with no success, because the $in operator will return the document even if only one element matches such as the following.
{data:{$elemMatch:{$elemMatch:{$in:[1,4]}}}}
Not sure what your actual query looks like, but this should do what you need:
db.documentDto.find({"some_field":{"$elemMatch":{"$in":[1,2,3]}} })
I haven't got a complete answer (and not much time as its late here) but I would consider
Using aggregation pipeline instead of a query if your not already
Use $unwind operator to deconstruct your nested arrays
Use $sort to sort the contents of the arrays - so you can now compare
Use $match to filter out the arrays which don't fit the array subset values as you can now check based on order.
Use $group to group the result back together based on the _id value
Ref:
http://docs.mongodb.org/manual/reference/operator/aggregation-pipeline/ will give you info on each of the above.
From a quick search I came up with a similar question/example that might be helpful: Mongodb sort inner array
Related
I have an existing collection where I can do some queries on. For further data processing, it would be handy to create some subset collections via query.
I understood that I can use the aggregate function with $match and $expr to e.g. $group some values and at the end use $out to get a new collection with the results.
The thing I am hanging on is not to $group anything, but just put the objects that $match finds into a new collection only. So not the complete objects with all their values. Just the one I am matching. Like when you db[collection].find({$match: {...}}, {"key1": 1, "key2": 0})
Where I get the new matching objects just containing key1: value1 but not key2: value2, which is also in the original collection.
How do I achieve that using aggregate without grouping anything? I read through the documentation and couldn't find any other stage operation that looks good.
As i mentioned in the comments $project is the right operator to use to achieve this.
In this case How do I keep documents in aggregation with $unwind aggregation can be ommited but what if you have a more complex case where you need aggregation framework? How can you keep these documents and still unwind those who have non-empty lists?
My case involves aggregation pipeline with multiple stages (match, unwind, project, match, group,..), i simply cannot fallback to plain "match" because those nested arrays need to be presented in a form of report (JasperReports). I first thought that this $unwind empty array will solve my problem but it won't because this array needs to be empty since I'm later grouping these values in JasperReport and showing them as they are.
I hope I was clear enough.
Any ideas?
Starting from MongoDb 3.2, $unwind operator supports preserveNullAndEmptyArrays:<boolean>.
So when preserveNullAndEmptyArrays:true, it will also include value which doesn't have any data or empty data.
for more info, visit - https://docs.mongodb.com/manual/reference/operator/aggregation/unwind/#document-operand-with-options
I'm querying a mongo collection that has a field that is an array of arrays. I want to find a record with a projection of one deep value out of the array of arrays. Conceptually, this is a $slice of a $slice. Is there a way to do this in Mongo?
For example - I have a record:
{
name: "foo",
text: [["part 1:1", "part 1:2"],["part 2:1","part 2:2"]]
}
and want to select the record with projection "part 2:2".
db.collection.find({"name":"foo"},{text: {$slice: [1,1]}}
gives me the array with both "part 2:1" and "part 2:2". How do I get just "part 2:2"?
You need to use the aggregation pipeline to achieve a $slice chain, due to the limitations in the project statement being part of the find query.
The below query is invalid because the first $slice would return an array, instead of an index, and the execution of the outer scoped $slice fails.
db.collection.find({"name":"foo"},{text: {$slice:[{$slice: [1,1]}]}})
Moreover there is no way to work on an projected field in the same project statement, if possible, we could have modified the text further by applying a $slice to it.
The way to go would be:
Match the record with the name as foo.
Unwind the text array to get to the first level.
Unwind again to get to the level that we want.
Group the records together by name.
Project the last record in the group which is also the last element
of the last nested array.
The Code:
db.collection.aggregate([
{$match:{"name":"foo"}},
{$unwind:"$text"},
{$unwind:"$text"},
{$group:{"_id":"$name","text":{$last:"$text"}}},
{$project:{"name":"$_id","text":1}}
])
or if you would want to project an element appearing in a particular order, then you could use the $skip and $limit operations to achieve this.
var orderOfElement = 2;
db.collection.aggregate([
{$match:{"name":"foo"}},
{$unwind:"$text"},
{$unwind:"$text"},
{$skip:orderOfElement -1},
{$limit:1}
])
Which projects the second element in order in the nested arrays.
If you want the specific part 2:2 the below query will help.
db.user.find({"name":"foo"},{_id:0,name:0})[0].text[1][1];
part 2:2
What is the difference between the $match operator used inside the aggregate function and the regular find in Mongodb?
Why doesn't the find function allow renaming the field names like the aggregate function?
e.g. In aggregate we can pass the following string:
{ "$project" : { "OrderNumber" : "$PurchaseOrder.OrderNumber" , "ShipDate" : "$PurchaseOrder.ShipDate"}}
Whereas, find does not allow this.
Why does not the aggregate output return as a DBCursor or a List? and also why can't we get a count of the documents that are returned?
Thank you.
Why does not the aggregate output return as a DBCursor or a List?
The aggregation framework was created to solve easy problems that otherwise would require map-reduce.
This framework is commonly used to compute data that requires the full db as input and few document as output.
What is the difference between the $match operator used inside the aggregate function and the regular find in Mongodb?
One of differences, like you stated, is the return type. Find operations output return as a DBCursor.
Other differences:
Aggregation result must be under 16MB. If you are using shards, the full data must be collected in a single point after the first $group or $sort.
$match only purpose is to improve aggregation's power, but it has some other uses, like improve the aggregation performance.
and also why can't we get a count of the documents that are returned?
You can. Just count the number of elements in the resulting array or add the following command to the end of the pipe:
{$group: {_id: null, count: {$sum: 1}}}
Why doesn't the find function allow renaming the field names like the aggregate function?
MongoDB is young and features are still coming. Maybe in a future version we'll be able to do that. Renaming fields is more critical in aggregation than in find.
EDIT (2014/02/26):
MongoDB 2.6 aggregation operations will return a cursor.
EDIT (2014/04/09):
MongoDB 2.6 was released with the predicted aggregation changes.
I investigated a few things about the aggregation and find call:
I did this with a descending sort in a table of 160k documents and limited my output to a few documents.
The Aggregation command is slower than the find command.
If you access to the data like ToList() the aggregation command is faster than the find.
if you watch at the total times (point 1 + 2) the commands seem to be equal
Maybe the aggregation automatically calls the ToList() and does not have to call it again. If you dont call ToList() afterwards the find() call will be much faster.
7 [ms] vs 50 [ms] (5 documents)
What I'm trying to do:
Filter a field of a collection that matches a given condition. Instead of returning every item in the field (which is an array of items), I only want to see matched items.
Similar to
select items from test where items.histPrices=[10,12]
It is also similar to what's found on the mongodb website here: http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields
Here's what I have been trying:
db.test.save({"name":"record", "items":[{"histPrices":[10,12],"name":"stuff"}]})
db.test.save({"name":"record", "items":[{"histPrices":[10,12],"name":"stuff"},
{"histPrices":[12,13],"name":"stuff"},{"histPrices":[11,14],"name":"stuff"}]})
db.test.find({},{"name":1,"items.histPrices":[10, 12]})
It will return all the objects that have a match for items.histPrices:[10,12], including ALL of the items in items[]. But I don't want the ones that don't match the condition.
From the comments left on Mongodb two years ago, the solution to get only the items with that histPrices[10,12] is to do it with javascript code, namely, loop through the result set and filter out the other items.
I wonder if there's a way to do that with just the query.
Your find query is wrong
db.test.find({},{"name":1,"items.histPrices":[10, 12]})
Your condition statement should be in the first part of the find statement.In your query {} means fetch all documents similar to this sql
select items from test (no where clause)
you have to change your mongodb find to
db.test.find({"items.histPrices":[10, 12]},{"name":1})
make it work
since your items is an array and if you wanted to return only the matching sub item, you have to use positional operator
db.test.find({"items.histPrices":[10, 12]},{"name":1,'items.$':1})
When working with arrays Embedded to the Document, the best approach is the one suggested by Chien-Wei Huang.
I would just add another aggregation, with the $group (in cases the document is very long, you may not want to retrieve all its content, only the array elements) Operator.
Now the command would look like:
db.test.aggregate({$match:{name:"record"}},
{$unwind:"$items"},
{$match {"items.histPrices":[10, 12]}},
{$group: {_id: "$_id",items: {$push: "$items"}}});)
If you are interested to return only one element from the array in each collection, then you should use projection instead
The same kind of issue solved here:
MongoDB Retrieve a subset of an array in a collection by specifying two fields which should match
db.test.aggregate({$unwind:"$items"}, {$match:{"items.histPrices":[10, 12]}})
But I don't know whether the performance would be OK. You have to verify it with your data.
The usage of $unwind
If you want add some filter condition like name="record", just add another $march at first, ex:
db.test.aggregate({$match:{name:"record"}}, {$unwind:"$items"}, {$match:{"items.histPrices":[10, 12]}})
https://jira.mongodb.org/browse/SERVER-828
Get particular element from mongoDB array
MongoDB query to retrieve one array value by a value in the array