Mongodb retrieving subarray, aggregation performance [duplicate] - mongodb

This question already has answers here:
Retrieve only the queried element in an object array in MongoDB collection
(18 answers)
Closed 5 years ago.
The community reviewed whether to reopen this question 5 months ago and left it closed:
Original close reason(s) were not resolved
I have array in subdocument like this
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 1
},
{
"a" : 2
},
{
"a" : 3
},
{
"a" : 4
},
{
"a" : 5
}
]
}
Can I filter subdocument for a > 3
My expect result below
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 4
},
{
"a" : 5
}
]
}
I try to use $elemMatch but returns the first matching element in the array
My query:
db.test.find( { _id" : ObjectId("512e28984815cbfcb21646a7") }, {
list: {
$elemMatch:
{ a: { $gt:3 }
}
}
} )
The result return one element in array
{ "_id" : ObjectId("512e28984815cbfcb21646a7"), "list" : [ { "a" : 4 } ] }
and I try to use aggregate with $match but not work
db.test.aggregate({$match:{_id:ObjectId("512e28984815cbfcb21646a7"), 'list.a':{$gte:5} }})
It's return all element in array
{
"_id" : ObjectId("512e28984815cbfcb21646a7"),
"list" : [
{
"a" : 1
},
{
"a" : 2
},
{
"a" : 3
},
{
"a" : 4
},
{
"a" : 5
}
]
}
Can I filter element in array to get result as expect result?

Using aggregate is the right approach, but you need to $unwind the list array before applying the $match so that you can filter individual elements and then use $group to put it back together:
db.test.aggregate([
{ $match: {_id: ObjectId("512e28984815cbfcb21646a7")}},
{ $unwind: '$list'},
{ $match: {'list.a': {$gt: 3}}},
{ $group: {_id: '$_id', list: {$push: '$list.a'}}}
])
outputs:
{
"result": [
{
"_id": ObjectId("512e28984815cbfcb21646a7"),
"list": [
4,
5
]
}
],
"ok": 1
}
MongoDB 3.2 Update
Starting with the 3.2 release, you can use the new $filter aggregation operator to do this more efficiently by only including the list elements you want during a $project:
db.test.aggregate([
{ $match: {_id: ObjectId("512e28984815cbfcb21646a7")}},
{ $project: {
list: {$filter: {
input: '$list',
as: 'item',
cond: {$gt: ['$$item.a', 3]}
}}
}}
])
$and:
get data between 0-5:
cond: {
$and: [
{ $gt: [ "$$item.a", 0 ] },
{ $lt: [ "$$item.a", 5) ] }
]}

Above solution works best if multiple matching sub documents are required.
$elemMatch also comes in very use if single matching sub document is required as output
db.test.find({list: {$elemMatch: {a: 1}}}, {'list.$': 1})
Result:
{
"_id": ObjectId("..."),
"list": [{a: 1}]
}

Use $filter aggregation
Selects a subset of the array to return based on the specified
condition. Returns an array with only those elements that match the
condition. The returned elements are in the original order.
db.test.aggregate([
{$match: {"list.a": {$gt:3}}}, // <-- match only the document which have a matching element
{$project: {
list: {$filter: {
input: "$list",
as: "list",
cond: {$gt: ["$$list.a", 3]} //<-- filter sub-array based on condition
}}
}}
]);

Related

How to make a sum only query in a whole collection?

I have this collection
{
"_id" : BinData(0, "JUw6VoBVdtqAQ2g7sn0sog=="),
"os_name" : "android"
}
I want to retrieve in one aggregate query to count of documents with
os_android => count of documents where "os_name" = "android"
os_ios => count of documents where "os_name" = "ios"
total => the total count of document
How can i do ?
I try this :
db.getCollection("myCollection").aggregate(
[
{$group:{
_id:{},
by_os_ios:{$sum:{$cond:[{os_name:"ios"},NumberInt(1),NumberInt(0)]}},
by_os_android:{$sum:{$cond:[{os_name:"android"},NumberInt(1),NumberInt(0)]}},
total:{$sum:1}}
},
{$addFields:{
"by_os.ios":"$by_os_ios",
"by_os.android":"$by_os_android"}},
{$project:{
_id:0,
by_os_ios:0,
by_os_android:0}}
]
);
but it's not work :( what did I miss ?
You missed $eq in $cond, you can check condition using { $eq:["$field_name", "value to matched"] }
db.getCollection("myCollection").aggregate([
{
$group: {
_id: {},
by_os_ios: { $sum: { $cond: [{ $eq: ["$os_name", "ios"] }, 1, 0] } },
by_os_android: { $sum: { $cond: [{ $eq: ["$os_name", "android"] }, 1, 0] } },
total: { $sum: 1 }
}
},
... // your other pipeline stages
])
Playground
Second possible way,
$group to create array called root
$project to get count of total documents using $size
$reduce to iterate loop through root array and check condition if its ios then sum and if its android then sum using $add and merge with current object of value by $mergeObjects
Playground

how to query records which field A before field B in the doc

Does MongoDB support query like this?
for example I have data like this
> db.foo.find()
{ "_id" : 1, "x" : 1, "y" : 2, "z" : 3 }
{ "_id" : 2, "y" : 2, "x" : 1, "z" : 3 }
{ "_id" : 3, "z" : 3, "y" : 2, "x" : 1 }
now I want to query the records which field y before field x, that is the last two records.
Does MongoDB support it?
You can use following aggregation:
db.foo.aggregate([
{
$addFields: {
keys: {
$map: {
input: { $objectToArray: "$$ROOT" },
as: "item",
in: "$$item.k"
}
}
}
},
{
$match: {
$expr: { $lt: [ { $indexOfArray: [ "$keys", "y" ] } , { $indexOfArray: [ "$keys", "x" ] } ] }
}
},
{
$project: {
keys: 0
}
}
])
$objectToArray can transform your root document to an array of key-value pairs. Then you can use $indexOfArray to get the position of x and y keys and compare them using $expr.
Two things you need to be aware of (based on this page):
Updates that include renaming of field names may result in the reordering of fields in the document.
Starting in version 2.6, MongoDB actively attempts to preserve the field order in a document. Before version 2.6, MongoDB did not actively preserve the order of the fields in a document.

mongodb aggregate nested nested array

how i can find the document with $match on position 3 (only last item in array "ndr"). It is necessary that the aggreation search only in the last array-item of ndr.
{
"_id" : ObjectId("58bd5c63a3d24b4a2e4cde03"),
"name" : "great document",
"country" : "us_us",
"cdate" : ISODate("2017-03-06T12:56:03.405Z"),
"nodes" : [
{
"node" : 3244343,
"name" : "Best Node ever",
"ndr" : [
{
"position" : 7,
"cdate" : ISODate("2017-03-06T10:55:20.000Z")
},
{
"position" : 3,
"cdate" : ISODate("2017-03-06T10:55:20.000Z")
}
]
}
],
}
I need this result after aggregation
{
"name" : "great document",
"country" : "us_us",
"cdate" : ISODate("2017-03-06T12:56:03.405Z"),
"nodes" : [
{
"node" : 3244343,
"name" : "Best Node ever",
"ndr" : [
{
"position" : 3,
"cdate" : ISODate("2017-03-06T10:55:20.000Z")
}
]
}
]
}
I hope anyone can help me.
You can try below aggregation with Mongo 3.4 version.
The below query finds the last item (-1) using $arrayElemAt operator in the ndr array and stores the variable in last using $let operator for each nodes and compare the last variable position value using $$ notation to 3 and wraps the nbr element within array [] if entry found and else returns empty array.
$map operator to reach nbr array inside the nodes array and project the updated nbr array while mapping the rest of nodes fields.
$addFields stage will overwrite the existing nodes with new nodes while keeping the all the other fields.
db.collection.aggregate([{
$addFields: {
nodes: {
$map: {
input: "$nodes",
as: "value",
in: {
node: "$$value.node",
name: "$$value.name",
ndr: {
$let: {
vars: {
last: {
$arrayElemAt: ["$$value.ndr", -1]
}
},
in: {
$cond: [{
$eq: ["$$last.position", 3]
},
["$$last"],
[]
]
}
}
}
}
}
}
}
}]);
Update:
You can try $redact which will keep the whole document if it finds the matching position from with the given filter.
$map to project the true, false values based on the filter for each of the nodes nbr position value and $anyElementTrue will inspect the previous boolean values for each doc and return a true or false value and $redact will use the booelan value from above comparison; true value to keep and false value to remove the document.
db.collection.aggregate([{
$redact: {
$cond: [{
$anyElementTrue: {
$map: {
input: "$nodes",
as: "value",
in: {
$let: {
vars: {
last: {
$arrayElemAt: ["$$value.ndr", -1]
}
},
in: {
$cond: [{
$eq: ["$$last.position", 3]
},
true,
false
]
}
}
}
}
}
}, "$$KEEP", "$$PRUNE"]
}
}]);
you will need to unwind both nested arrays.
db.<collection>.aggregate([
{ $unwind: '$nodes' },
{ $unwind: '$nodes.ndr'},
{ $group: {'_id':{'_id':'$_id', 'nodeID', '$nodes.node' },
'name':{'$last':'$name'},
'country':{'$last':'$country'},
'cdate':{'$last':'$cdate'},
'nodes':{'$last':'$nodes'}
}
},
{ $match : { nodes.ndr.position: 3 } }
]);
From here you can reassemble the aggregate results with a $group on the and do a projection. I'm not sure what your ultimate end result should be.

Comparing two object arrays and check if they have common elements

How can I execute a query in MongoDB that returns _id if FirstArray and SecondArray has elements in common in "Name" field?
This is the collection structure:
{
"_id" : ObjectId("58b8d9e3b2b4e07bff8feed5"),
"FirstArray" : [
{
"Name" : "A",
"Something" : "200 ",
},
{
"Name" : "GF",
"Something" : "100 ",
}
],
"SecondArray" : [
{
"Name" : "BC",
"Something" : "200 ",
},
{
"Name" : "A",
"Something" : "100 ",
}
]
}
3.6 Update:
Use $match with $expr. $expr allows use of aggregation expressions inside $match stage.
db.collection.aggregate([
{"$match":{
"$expr":{
"$eq":[
{"$size":{"$setIntersection":["$FirstArray.Name","$SecondArray.Name"]}},
0
]
}
}},
{"$project":{"_id":1}}
])
Old version:
You can try $redact with $setIntersection for your query.
$setIntersection to compare the FirstArrays Names with SecondArrays Names and return array of common names documents followed by $size and $redact and compare result with 0 to keep and else remove the document.
db.collection.aggregate(
[{
$redact: {
$cond: {
if: {
$eq: [{
$size: {
$setIntersection: ["$FirstArray.Name", "$SecondArray.Name"]
}
}, 0]
},
then: "$$KEEP",
else: "$$PRUNE"
}
}
}, {
$project: {
_id: 1
}
}]
)

Can the MongoDB aggregation framework $group return an array of values?

How flexible is the aggregate function for output formatting in MongoDB?
Data format:
{
"_id" : ObjectId("506ddd1900a47d802702a904"),
"port_name" : "CL1-A",
"metric" : "772.0",
"port_number" : "0",
"datetime" : ISODate("2012-10-03T14:03:00Z"),
"array_serial" : "12345"
}
Right now I'm using this aggregate function to return an array of DateTime, an array of metrics, and a count:
{$match : { 'array_serial' : array,
'port_name' : { $in : ports},
'datetime' : { $gte : from, $lte : to}
}
},
{$project : { port_name : 1, metric : 1, datetime: 1}},
{$group : { _id : "$port_name",
datetime : { $push : "$datetime"},
metric : { $push : "$metric"},
count : { $sum : 1}}}
Which is nice, and very fast, but is there a way to format the output so there's one array per datetime/metric? Like this:
[
{
"_id" : "portname",
"data" : [
["2012-10-01T00:00:00.000Z", 1421.01],
["2012-10-01T00:01:00.000Z", 1361.01],
["2012-10-01T00:02:00.000Z", 1221.01]
]
}
]
This would greatly simplify the front-end as that's the format the chart code expects.
Combining two fields into an array of values with the Aggregation Framework is possible, but definitely isn't as straightforward as it could be (at least as at MongoDB 2.2.0).
Here is an example:
db.metrics.aggregate(
// Find matching documents first (can take advantage of index)
{ $match : {
'array_serial' : array,
'port_name' : { $in : ports},
'datetime' : { $gte : from, $lte : to}
}},
// Project desired fields and add an extra $index for # of array elements
{ $project: {
port_name: 1,
datetime: 1,
metric: 1,
index: { $const:[0,1] }
}},
// Split into document stream based on $index
{ $unwind: '$index' },
// Re-group data using conditional to create array [$datetime, $metric]
{ $group: {
_id: { id: '$_id', port_name: '$port_name' },
data: {
$push: { $cond:[ {$eq:['$index', 0]}, '$datetime', '$metric'] }
},
}},
// Sort results
{ $sort: { _id:1 } },
// Final group by port_name with data array and count
{ $group: {
_id: '$_id.port_name',
data: { $push: '$data' },
count: { $sum: 1 }
}}
)
MongoDB 2.6 made this a lot easier by introducing $map, which allows a simplier form of array transposition:
db.metrics.aggregate([
{ "$match": {
"array_serial": array,
"port_name": { "$in": ports},
"datetime": { "$gte": from, "$lte": to }
}},
{ "$group": {
"_id": "$port_name",
"data": {
"$push": {
"$map": {
"input": [0,1],
"as": "index",
"in": {
"$cond": [
{ "$eq": [ "$$index", 0 ] },
"$datetime",
"$metric"
]
}
}
}
},
"count": { "$sum": 1 }
}}
])
Where much like the approach with $unwind, you supply an array as "input" to the map operation consisting of two values and then essentially replace those values with the field values you want via the $cond operation.
This actually removes all the pipeline juggling required to transform the document as was required in previous releases and just leaves the actual aggregation to the job at hand, which is basically accumulating per "port_name" value, and the transformation to array is no longer a problem area.
Building arrays in the aggregation framework without $push and $addToSet is something that seems to be lacking. I've tried to get this to work before, and failed. It would be awesome if you could just do:
data : {$push: [$datetime, $metric]}
in the $group, but that doesn't work.
Also, building "literal" objects like this doesn't work:
data : {$push: {literal:[$datetime, $metric]}}
or even data : {$push: {literal:$datetime}}
I hope they eventually come up with some better ways of massaging this sort of data.