The below document has the dob of student and its parent's dob.
{
"_id" : ObjectId("56a31573a3b1f89cb895abd3"),
"dob" : {
"isodate" : ISODate("1996-01-21T18:30:00.000+0000")
},
"parent" : [
{
"dob" : {
"isodate" : ISODate("1956-07-21T18:30:00.000+0000")
},
"type" : "father"
},
{
"dob" : {
"isodate" : ISODate("1958-11-01T18:30:00.000+0000")
},
"type" : "mother"
}
]
}
In one of the application use case, it is better to receive output in the below format
{
"_id" : ObjectId("56a31573a3b1f89cb895abd3"),
"dob" : {
"isodate" : ISODate("1996-01-21T18:30:00.000+0000")
},
"type" : "student"
},
{
"_id" : ObjectId("56a31573a3b1f89cb895abd3"),
"dob" : {
"isodate" : ISODate("1956-07-21T18:30:00.000+0000")
},
"type" : "father"
},
{
"_id" : ObjectId("56a31573a3b1f89cb895abd3"),
"dob" : {
"isodate" : ISODate("1958-11-01T18:30:00.000+0000")
},
"type" : "mother"
}
The approach is to $project the fields into array and then $unwind that array. However, projection doesn't allow me to create array.
I believe $group and its associated aggregation cannot be used as my operations are on the same document in the pipeline.
Is this possible?
Note - i have the flexibility to change the document design as well.
For Mongo 3.0
Here I have included a [null] array which gives me the option to insert array in projection using a combination of $setDiffernce and $cond. The output of this is given to $setUnion with $parent array.
db.p1.aggregate(
{ "$project": {
"allVal": {
'$setUnion': [
{"$setDifference": [
{ "$map": {
"input": [null],
"as": "type",
"in": { "$cond": [
{"$eq": ["$$type", null]},
{dob:"$dob", type:{$literal:'student'}},
null
]}
}},
[null]
]}
,
'$parent'
]
}
}},
{$unwind : '$allVal'}
)
For mongo 3.2
Feels heaven as I have avoided $setDifference and $literal hack adjustments.
db.p1.aggregate([
{
$project:{
parent : 1,
type: {$literal : 'student'},
'dob.isodate' : 1
}
},
{
$project:{
allValues: { $setUnion: [ [{dob:"$dob", type:'$type'}], "$parent" ] }
}
},
{
$unwind : '$allValues'
}
])
In the first projection, I am adding a new field called type
In the 2nd projection, I am creating a new array with 2 different nodes of the same document.
Currently this solution works for Mongo 3.2
Related
I am trying to retrieve elements in an array in mongo db. I would like to retrieve the 15 first elements which do not match a pattern
So let's imagine I have
{
"_id" : ObjectId("s4dcsd5s4d6c54s6d"),
"items" : [
{
type : "TYPE_1",
text : "blablabla"
},
{
type : "TYPE_2",
text : "blablabla"
},
{
type : "TYPE_3",
text : "blablabla"
},
{
type : "TYPE_1",
text : "blablabla"
},
{
type : "TYPE_2",
text : "blablabla"
},
{
type : "TYPE_1",
text : "blablabla"
}
]
}
So currently I have more element to match compared to the element to not match that's why I use nin. but it is to simplifiy
If I use
db.history.find({ "_id" : ObjectId("s4dcsd5s4d6c54s6d")}, { "items" : { "$elemMatch" : { "type" : { "$nin" : [ "TYPE_2" , "TYPE_3"]}}}, "items" : { $slice : [0, 2]}}).pretty()
It seems that the element match is not taken into account (inverse as well if i put element match after slice)
Then if I do:
db.history.find({ "_id" : ObjectId("s4dcsd5s4d6c54s6d")}, { "items" : { "$elemMatch" : { "type" : { "$nin" : [ "TYPE_2" , "TYPE_3"]}}, $slice : [0, 2]}}).pretty()
An error is thrown by mongo
Do you know how I can do?
Thanks a lot
You can't use $elemMatch for your case since it will only return the first element. From documentation :
$elemMatch The $elemMatch operator limits the contents of an
field from the query results to contain only the first element
matching the $elemMatch condition.
You can do an aggregation query which will do the following:
match your _id
unwind your items array to have one record per items in the array
match the types $nin your array [ "TYPE_2" , "TYPE_3"]
limit the number of result
The query is :
db.history.aggregate([{
$match: {
_id: ObjectId("s4dcsd5s4d6c54s6d")
}
}, {
$unwind: '$items'
}, {
$match: {
'items.type': { '$nin': ["TYPE_2", "TYPE_3"] }
}
},
{ $limit: 2 }
])
It gives :
{ "_id" : "s4dcsd5s4d6c54s6d", "items" : { "type" : "TYPE_1", "text" : "blablabla" } }
{ "_id" : "s4dcsd5s4d6c54s6d", "items" : { "type" : "TYPE_1", "text" : "blablabla" } }
You will need to use aggregation for restricting the array in the form you have. Use $filter to apply the condition and $slice to limit the array elements.
db.history.aggregate([{
$match: {
_id: ObjectId("586309d6772c68234445f2a5")
}
}, {
"$project": {
"items": {
"$slice": [{
"$filter": {
"input": "$items",
"as": "item",
"cond": {
"$and": [{
$ne: ["$$item.type", "TYPE_2"]
}, {
$ne: ["$$item.type", "TYPE_3"]
}]
}
}
},
2
]
}
}
}])
Sample Output:
{ "_id" : ObjectId("586309d6772c68234445f2a5"), "items" : [ { "type" : "TYPE_1", "text" : "blablabla" }, { "type" : "TYPE_1", "text" : "blablabla" } ] }
I want to filter according the sub documents, but actually I am repeating the document for each sub document. I want one document and a list of sub documents if that is the case.
My data looks like:
{
"_id" : ObjectId("582eeb5f75f58055246bd22d"),
"filename" : "file1",
"cod" : NumberLong(90),
"subdocs" : [
{
"length" : NumberLong(10),
"desc" : "000"
},
{
"length" : NumberLong(15),
"desc" : "011"
},
{
"length" : NumberLong(30),
"desc" : "038"
}
]
}
{
"_id" : ObjectId("582eeb5f75f58055246bd22e"),
"filename" : "file2",
"cod" : NumberLong(95),
"subdocs" : [
{
"length" : NumberLong(11),
"desc" : "000"
},
{
"length" : NumberLong(21),
"desc" : "018"
},
{
"length" : NumberLong(41),
"desc" : "008"
}
]
}
I am using this query to filter for the desc (000, 011) on the subdocs
db.ftmp.aggregate(
{ $match:
{ "subdocs.desc":
{ $in: ["000", "011"] }
}
},
{ $unwind : "$subdocs" },
{ $match :
{ "subdocs.desc" :
{ $in:["000", "011"] }
}
}
)
But the result shows 3 documents, 1 document for each sub-document that matches with that query.
{
"_id" : ObjectId("582eeb5f75f58055246bd22d"),
"filename" : "file1",
"cod" : NumberLong(90),
"subdocs" : {
"length" : NumberLong(10),
"desc" : "000"
}
}
{
"_id" : ObjectId("582eeb5f75f58055246bd22d"),
"filename" : "file1",
"cod" : NumberLong(90),
"subdocs" : {
"length" : NumberLong(15),
"desc" : "011"
}
}
{
"_id" : ObjectId("582eeb5f75f58055246bd22e"),
"filename" : "file2",
"cod" : NumberLong(95),
"subdocs" : {
"length" : NumberLong(11),
"desc" : "000"
}
}
However I want to get: file1 with the subdocuments with desc 000 and 011, and file2 with the subdocumnt 000
{
"_id" : ObjectId("582eeb5f75f58055246bd22d"),
"filename" : "file1",
"cod" : NumberLong(90),
"subdocs" : [
{
"length" : NumberLong(10),
"desc" : "000"
},
{
"length" : NumberLong(15),
"desc" : "011"
}
]
}
{
"_id" : ObjectId("582eeb5f75f58055246bd22e"),
"filename" : "file2",
"cod" : NumberLong(95),
"subdocs" : {
"length" : NumberLong(11),
"desc" : "000"
}
}
What is the correct way to do that? Any idea?
First of all using the $unwind operator as mentioned in this answer will cause a drop of performance in your application because unwinding your array result in more documents to process down in the pipeline. There is a better way to achieve this since MongoDB 2.6.
That being said, this is a perfect job for the $filter operator new in MongoDB 3.2.
The most efficient way to do this is in MongoDB 3.4. MongoDB 3.4 introduced the $in array operator for the aggregation framework which can be used in the $filter conditional expression which, when evaluates to true include the sub-document in the resulting array.
let values = [ '000', '011' ];
db.collection.aggregate([
{ "$project": {
"filename": 1,
"cod": 1,
"subdocs": {
"$filter": {
"input": "$subdocs",
"as": "s",
"cond": { "$in": [ "$$s.desc", values ] }
}
}
}}
])
In MongoDB 3.2 we need a slightly different approach because we can use the $in operator there. But luckily we have the $setIsSubset operator and as you might have guess performs a set operation on two array and return true if the first array is a subset of the second array. Because $setIsSubset first expression must be an array, need to make the desc field an array in our pipeline. To do this, we simply use the [] bracket the create that array field which is new MongoDB 3.2
db.collection.aggregate([
{ "$project": {
"filename": 1,
"cod": 1,
"subdocs": {
"$filter": {
"input": "$subdocs",
"as": "s",
"cond": { "$setIsSubset": [ [ "$$s.desc" ], values ] }
}
}
}}
])
MongoDB 3.0 is dead to me but if for some reasons you are running that version, you can use the $literal operator to return the one element array you need for the set operation and the $setDifference operator. This is left as exercise to the reader.
You just need to add $group & $push. First you $unwind the subdocs to apply the $match followed by $group on id and $push the grouped subdocs.
db.ftmp.aggregate({
$unwind: "$subdocs"
}, {
$match: {
"subdocs.desc": {
$in: ["000", "011"]
}
}
}, {
$group: {
_id: "$_id",
subdocs: {
$push: "$subdocs"
},
filename: {
$first: "$filename"
},
cod: {
$first: "$cod"
}
}
})
I want to filter according the sub documents, but actually I am repeating the document for each sub document. I want one document and a list of sub documents if that is the case.
My data looks like:
{
"_id" : ObjectId("582eeb5f75f58055246bd22d"),
"filename" : "file1",
"cod" : NumberLong(90),
"subdocs" : [
{
"length" : NumberLong(10),
"desc" : "000"
},
{
"length" : NumberLong(15),
"desc" : "011"
},
{
"length" : NumberLong(30),
"desc" : "038"
}
]
}
{
"_id" : ObjectId("582eeb5f75f58055246bd22e"),
"filename" : "file2",
"cod" : NumberLong(95),
"subdocs" : [
{
"length" : NumberLong(11),
"desc" : "000"
},
{
"length" : NumberLong(21),
"desc" : "018"
},
{
"length" : NumberLong(41),
"desc" : "008"
}
]
}
I am using this query to filter for the desc (000, 011) on the subdocs
db.ftmp.aggregate(
{ $match:
{ "subdocs.desc":
{ $in: ["000", "011"] }
}
},
{ $unwind : "$subdocs" },
{ $match :
{ "subdocs.desc" :
{ $in:["000", "011"] }
}
}
)
But the result shows 3 documents, 1 document for each sub-document that matches with that query.
{
"_id" : ObjectId("582eeb5f75f58055246bd22d"),
"filename" : "file1",
"cod" : NumberLong(90),
"subdocs" : {
"length" : NumberLong(10),
"desc" : "000"
}
}
{
"_id" : ObjectId("582eeb5f75f58055246bd22d"),
"filename" : "file1",
"cod" : NumberLong(90),
"subdocs" : {
"length" : NumberLong(15),
"desc" : "011"
}
}
{
"_id" : ObjectId("582eeb5f75f58055246bd22e"),
"filename" : "file2",
"cod" : NumberLong(95),
"subdocs" : {
"length" : NumberLong(11),
"desc" : "000"
}
}
However I want to get: file1 with the subdocuments with desc 000 and 011, and file2 with the subdocumnt 000
{
"_id" : ObjectId("582eeb5f75f58055246bd22d"),
"filename" : "file1",
"cod" : NumberLong(90),
"subdocs" : [
{
"length" : NumberLong(10),
"desc" : "000"
},
{
"length" : NumberLong(15),
"desc" : "011"
}
]
}
{
"_id" : ObjectId("582eeb5f75f58055246bd22e"),
"filename" : "file2",
"cod" : NumberLong(95),
"subdocs" : {
"length" : NumberLong(11),
"desc" : "000"
}
}
What is the correct way to do that? Any idea?
First of all using the $unwind operator as mentioned in this answer will cause a drop of performance in your application because unwinding your array result in more documents to process down in the pipeline. There is a better way to achieve this since MongoDB 2.6.
That being said, this is a perfect job for the $filter operator new in MongoDB 3.2.
The most efficient way to do this is in MongoDB 3.4. MongoDB 3.4 introduced the $in array operator for the aggregation framework which can be used in the $filter conditional expression which, when evaluates to true include the sub-document in the resulting array.
let values = [ '000', '011' ];
db.collection.aggregate([
{ "$project": {
"filename": 1,
"cod": 1,
"subdocs": {
"$filter": {
"input": "$subdocs",
"as": "s",
"cond": { "$in": [ "$$s.desc", values ] }
}
}
}}
])
In MongoDB 3.2 we need a slightly different approach because we can use the $in operator there. But luckily we have the $setIsSubset operator and as you might have guess performs a set operation on two array and return true if the first array is a subset of the second array. Because $setIsSubset first expression must be an array, need to make the desc field an array in our pipeline. To do this, we simply use the [] bracket the create that array field which is new MongoDB 3.2
db.collection.aggregate([
{ "$project": {
"filename": 1,
"cod": 1,
"subdocs": {
"$filter": {
"input": "$subdocs",
"as": "s",
"cond": { "$setIsSubset": [ [ "$$s.desc" ], values ] }
}
}
}}
])
MongoDB 3.0 is dead to me but if for some reasons you are running that version, you can use the $literal operator to return the one element array you need for the set operation and the $setDifference operator. This is left as exercise to the reader.
You just need to add $group & $push. First you $unwind the subdocs to apply the $match followed by $group on id and $push the grouped subdocs.
db.ftmp.aggregate({
$unwind: "$subdocs"
}, {
$match: {
"subdocs.desc": {
$in: ["000", "011"]
}
}
}, {
$group: {
_id: "$_id",
subdocs: {
$push: "$subdocs"
},
filename: {
$first: "$filename"
},
cod: {
$first: "$cod"
}
}
})
I have one collection named 'ctrlcharts'.
e.g.
{
"_id" : ObjectId("57fc695492af567031246736"),
"deviceId" : "A001",
"sensorId" : "S003",
"time" : "2016/10/11 12:23:50",
"charts" : [
{
"sensor" : "ch_11",
"value" : 120
},
{
"sensor" : "ch_12",
"value" : 150
}
]
}
How to filter "sensor" : "ch_11" and aggregate data from one collection into another format using MongoDB
e.g.
{
"time" : "2016/10/11 12:23:50",
"sensor" : "ch_12",
"value" : 150
}
I tried below code
db.ctrlcharts.aggregate([
{ $match: {"deviceId" : "A001", "sensorId" : "S003", "time" : "2016/10/11 12:23:50"}},
{ $project: {
_id: 0,
time : 1 ,
sensor : "$charts.sensor"
value : "$charts.value"
}
}
])
But I got the result as
{
"time" : "2016/10/11 12:23:50",
"sensor" : ["ch_11","ch_12"],
"value" : [120,150]
}
Thanks
You tried best....just use $unwind
db.ctrlcharts.aggregate(
{$unwind:"$charts"},
{$match: {"deviceId" :"A001", "charts.sensor":"ch_12", "time" : "2016/10/11 12:23:50"}},
{$project:{_id:0,time:1, sensor : "$charts.sensor", value :"$charts.value"}}).pretty()
You can use $unwind (aggregation) to separate charts array.
db.ctrlcharts.aggregate( [ { $unwind : "$charts" } ] )
This will produce result like -
{ "_id" : ObjectId("57ff397a007c43ecacf10512"), "deviceId" : "A001", "sensorId" : "S003", "time" : "2016/10/11 12:23:50", "charts" : { "sensor" : "ch_11", "value" : 120 } }
{ "_id" : ObjectId("57ff397a007c43ecacf10512"), "deviceId" : "A001", "sensorId" : "S003", "time" : "2016/10/11 12:23:50", "charts" : { "sensor" : "ch_12", "value" : 150 } }
and then use your match query
Use the $arrayElemAt and $filter operators to query the array more efficiently without the need to $unwind. The reason why $unwind is not as efficient is that it produces a cartesian product of the documents i.e. a copy of each document per array entry, which uses more memory (possible memory cap on aggregation pipelines of 10% total memory) and therefore takes time to produce as well processing the documents during the flattening process.
The $filter will return a subset of the array that only contains the elements that match the filter condition. The $arrayElemAt operator then returns the element from the filtered array at the specified array index to give you the subdocument you need.
A further $project is necessary to flatten the fields to give you the desired result:
db.ctrlcharts.aggregate([
{ "$match": {
"deviceId": "A001",
"sensorId": "S003",
"time": "2016/10/11 12:23:50",
"charts.sensor": "ch_11"
} },
{
"$project": {
"time": 1,
"chart": {
"$arrayElemAt": [
"$filter": {
"input": "$charts",
"as": "item",
"cond": { "$eq": ["$$item.sensor", "ch_11"] }
}, 0
]
}
}
},
{
"$project": {
"_id": 0,
"time": 1,
"sensor": "$chart.sensor"
"value": "$chart.value"
}
}
])
I have a collection of documents, each has a field which is an array of subdocuments, and all subdocuments have a common field 'status'. I want to find all documents that have the same status for all subdocuments.
collection:
{
"name" : "John",
"wives" : [
{
"name" : "Mary",
"status" : "dead"
},
{
"name" : "Anne",
"status" : "alive"
}
]
},
{
"name" : "Bill",
"wives" : [
{
"name" : "Mary",
"status" : "dead"
},
{
"name" : "Anne",
"status" : "dead"
}
]
},
{
"name" : "Mohammed",
"wives" : [
{
"name" : "Jane",
"status" : "dead"
},
{
"name" : "Sarah",
"status" : "dying"
}
]
}
I want to check if all wives are dead and find only Bill.
You can use the following aggregation query to get records of person whose wives are all dead:
db.collection.aggregate(
{$project: {name:1, wives:1, size:{$size:'$wives'}}},
{$unwind:'$wives'},
{$match:{'wives.status':'dead'}},
{$group:{_id:'$_id',name:{$first:'$name'}, wives:{$push: '$wives'},size:{$first:'$size'},count:{$sum:1}}},
{$project:{_id:1, wives:1, name:1, cmp_value:{$cmp:['$size','$count']}}},
{$match:{cmp_value:0}}
)
Output:
{ "_id" : ObjectId("56d401de8b953f35aa92bfb8"), "name" : "Bill", "wives" : [ { "name" : "Mary", "status" : "dead" }, { "name" : "Anne", "status" : "dead" } ], "cmp_value" : 0 }
If you need to find records of users who has same status, then you may remove the initial match stage.
The most efficient way to handle this is always going to be to "match" on the status of "dead" as the opening query, otherwise you are processing items that cannot possibly match, and the logic really quite simply followed with $map and $allElementsTrue:
db.collection.aggregate([
{ "$match": { "wives.status": "dead" } },
{ "$redact": {
"$cond": {
"if": {
"$allElementsTrue": {
"$map": {
"input": "$wives",
"as": "wife",
"in": { "$eq": [ "$$wife.status", "dead" ] }
}
}
},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
Or the same thing with $where:
db.collection.find({
"wives.status": "dead",
"$where": function() {
return this.wives.length
== this.wives.filter(function(el) {
el.status == "dead";
}).length;
}
})
Both essentially test the "status" value of all elements to make sure they match in the fastest possible way. But the aggregate pipeline with just $match and $redact should be faster. And "less" pipeline stages ( essentially each a pass through the data ) means faster as well.
Of course keeping a property on the document is always fastest, but it would involve logic to set that only where "all elements" are the same property. Which of course would typically mean inspecting the document by loading it from the server prior to each update.