I have nested arrays and objects. I want to filter out each medicine separately. Such as document:
{
person: "X",
treatment: [
{
id: 1,
medication: [
{
name: "A"
},
{
name: "B"
}
]
},
{
id: 2,
medication: [
{
name: "A"
},
{
name: "C"
}
]
}
]
},
{
person: "Y",
treatment: [
{
id: 1,
medication: [
{
name: "B"
},
{
name: "C"
}
]
}
]
}
How could I get a distinct list of top medication? Like this: A:2, B:2, C:2
You need to use $unwind which outputs one document for each element in the array. As there are nested array's you'll need to apply $unwind twice.
E.g. for a collection named medicine:
db.medicine.aggregate([
{
$unwind : "$treatment"
},
{
$unwind : "$treatment.medication"
},
{
$group : { _id : "$treatment.medication.name", total: { $sum: 1 } }
},
{
$replaceRoot: {
newRoot: {
$arrayToObject: {
$let: {
vars: { data: [ { k: "$_id", v: "$total" } ] },
in: "$$data"
}
}
}
}
}
])
Outputs:
{ "C" : 2 }
{ "B" : 2 }
{ "A" : 2 }
Note that when unwinding nested arrays, many documents can be generated if there are large arrays, so you might want to check this question Consequences of using $unwind on nested arrays?
Related
New to Mongo, have found lots of examples of removing dupes from arrays of strings using the aggregation framework, but am wondering if possible to remove dupes from array of objects based on a field in the object. Eg
{
"_id" : ObjectId("5e82661d164941779c2380ca"),
"name" : "something",
"values" : [
{
"id" : 1,
"val" : "x"
},
{
"id" : 1,
"val" : "x"
},
{
"id" : 2,
"val" : "y"
},
{
"id" : 1,
"val" : "xxxxxx"
}
]
}
Here I'd like to remove dupes based on the id field. So would end up with
{
"_id" : ObjectId("5e82661d164941779c2380ca"),
"name" : "something",
"values" : [
{
"id" : 1,
"val" : "x"
},
{
"id" : 2,
"val" : "y"
}
]
}
Picking the first/any object with given id works. Just want to end up with one per id. Is this doable in aggregation framework? Or even outside aggregation framework, just looking for a clean way to do this. Need to do this type of thing across many documents in collection, which seems like a good use case for aggregation framework, but as I mentioned, newbie here...thanks.
Well, you may get desired result 2 ways.
Classic
Flatten - Remove duplicates (pick first occurrence) - Group by
db.collection.aggregate([
{
$unwind: "$values"
},
{
$group: {
_id: "$values.id",
values: {
$first: "$values"
},
id: {
$first: "$_id"
},
name: {
$first: "$name"
}
}
},
{
$group: {
_id: "$id",
name: {
$first: "$name"
},
values: {
$push: "$values"
}
}
}
])
MongoPlayground
Modern
We need to use $reduce operator.
Pseudocode:
values : {
var tmp = [];
for (var value in values) {
if !(value.id in tmp)
tmp.push(value);
}
return tmp;
}
db.collection.aggregate([
{
$addFields: {
values: {
$reduce: {
input: "$values",
initialValue: [],
in: {
$concatArrays: [
"$$value",
{
$cond: [
{
$in: [
"$$this.id",
"$$value.id"
]
},
[],
[
"$$this"
]
]
}
]
}
}
}
}
}
])
MongoPlayground
You can use $reduce, Try below query :
db.collection.aggregate([
{
$addFields: {
values: {
$reduce: {
input: "$values",
initialValue: [],
in: {
$cond: [
{ $in: ["$$this.id", "$$value.id"] }, /** Check if 'id' exists in holding array if yes push same array or concat holding array with & array of new object */
"$$value",
{ $concatArrays: ["$$value", ["$$this"]] }
]
}
}
}
}
}
]);
Test : MongoDB-Playground
I have 2 collections in MongoDB :
Collection1 :
{
_id:1,
Field1: "Some info",
Field2: "Some other info",
Elements: [
{
id: 0,
Enabled: false
},
{
id: 1,
Enabled: false
},
{
id: 2,
Enabled: false
}
]
}
Collection2 :
{
Identifier: "identifier",
ElementsOverride: [
{
id: 0,
Enabled: true
},
{
id: 1,
Enabled: false
},
{
id: 2,
Enabled: true
}
]
}
What I would like to do is perform an operation which flattens "Element" collection and returns Collection1 with the flattened Element collection (basically the Enabled field from collection 2 overwrites the enabled field of Collection 1.
Is there a way to achieve this in Mongodb?
Adding more clarification for what the output should be like:
Essentially what I'm trying to do is merge the document identified by _id:1 in collection 1 (document1), with the document identified by Identifier: "identifier" in collection 2 (document 2) such that:
All the properties in document1 and document2 are available in the output.
The ElementsOverride from document2 with the same ID's as document1 (ex; id: 0) will overwrite the values in document1
Required Output :
{
_id:1,
Identifier: "identifier",
Field1: "Some info",
Field2: "Some other info",
Elements: [
{
id: 0,
Enabled: true
},
{
id: 1,
Enabled: false
},
{
id: 2,
Enabled: true
}
]
}
You can try below query :
db.Collection1.aggregate([
/** get only one required doc from Collection1 */
{ $match: { _id: 1 } },
/** Join relative doc from Collection2 */
{
$lookup:
{
from: "Collection2",
pipeline: [
{
$match:
{
$expr:
{ $eq: ["$Identifier", "identifier"] }
}
}
],
as: "data"
}
},
/** As lookup will default to an array of objects getting an object out of array */
{ $unwind: '$data' },
/** Replacing existing elements field of Collection1 & adding Identifier field to existing doc */
{
$addFields: {
Identifier: '$data.Identifier', Elements:
{
$reduce: {
input: { $reverseArray: { $setUnion: ["$Elements", "$data.ElementsOverride"] } },
initialValue: [],
in: { $concatArrays: ["$$value", { $cond: [{ $in: ['$$this.id', '$$value.id'] }, [], ['$$this']] }] }
}
}
}
},
/** removing unnecessary field created at lookup stage */
{ $project: { data: 0 } }
])
Test : MongoDB-Playground
I am not sure how you want the output.
flattens "Element" collection
generally means the array Element is unwound. Please correct my interpretation, in case I have misunderstood.
But, the following steps in Mongo Shell will get the result:
arr1 = db.c1.aggregate( [ { $unwind: "$Elements" }, { $sort: { "Elements.id": 1 } ] ).toArray()
arr2 = db.c2.aggregate( [ { $unwind: "$ElementsOverride" }, { $sort: { "ElementsOverride.id": 1 } ] ).toArray()
for (let i=0; i < arr1.length; i++) {
updated = Object.assign(arr1[i].Elements, arr2[i].ElementsOverride);
arr1[i].Elements = updated
}
The variable arr1 will have:
[
{
"_id" : 1,
"Field1" : "Some info",
"Field2" : "Some other info",
"Elements" : {
"id" : 0,
"Enabled" : true
}
},
{
"_id" : 1,
"Field1" : "Some info",
"Field2" : "Some other info",
"Elements" : {
"id" : 1,
"Enabled" : false
}
},
{
"_id" : 1,
"Field1" : "Some info",
"Field2" : "Some other info",
"Elements" : {
"id" : 2,
"Enabled" : true
}
}
]
[ EDIT ADD ]
Updated to reflect the required output:
arr2 = db.c2.aggregate( [
{ $unwind: "$ElementsOverride" },
{ $replaceRoot: { newRoot: "$ElementsOverride" } }
] ).toArray()
db.c1.aggregate( [
{ $unwind: "$Elements" },
{ $addFields: {
"Elements.Enabled": {
$filter: {
input: arr2,
cond: { $eq: [ "$$this.ElementsOverride.id", "$Elements.id" ] }
}
}
} },
{ $group: {
_id: "$_id",
doc: { $first: "$$ROOT"},
Identifier: { $first: "$Elements.Enabled.Identifier"},
Elements: { $push: { $arrayElemAt: [ "$Elements.Enabled", 0 ] } }
} },
{ $addFields: {
"doc.Elements": "$Elements.ElementsOverride",
"doc.Identifier": { $arrayElemAt: [ "$Identifier", 0 ] }
} },
{ $replaceRoot: { newRoot: "$doc" } }
] )
[ EDIT ADD 2 ]
Here is another way of merging the documents:
doc1 = db.c1.findOne()
arr2 = db.c2.aggregate( [ { $unwind: "$ElementsOverride" } ] ).toArray()
for (let e2 of arr2) {
for (i = 0; i < doc1.Elements.length; i++) {
if (doc1.Elements[i].id == e2.ElementsOverride.id) {
doc1.Elements[i].Enabled = e2.ElementsOverride.Enabled
doc1.Identifier = e2.Identifier
}
}
}
The output is the doc1 document.
{
_id: ObjectId("5dbdacc28cffef0b94580dbd"),
"comments" : [
{
"_id" : ObjectId("5dbdacc78cffef0b94580dbf"),
"replies" : [
{
"_id" : ObjectId("5dbdacd78cffef0b94580dc0")
},
]
},
]
}
How to count the number of element in comments and sum with number of relies
My approach is do 2 query like this:
1. total elements of replies
db.posts.aggregate([
{$match: {_id:ObjectId("5dbdacc28cffef0b94580dbd")}},
{ $unwind: "$comments",},
{$project:{total:{$size:"$comments.replies"} , _id: 0} }
])
2. count total elements of comments
db.posts.aggregate([
{$match: {_id:ObjectId("5dbdacc28cffef0b94580dbd")}},
{$project:{total:{$size:"$comments.replies"} , _id: 0} }
])
Then sum up both, do we have any better solution to write the query like return the sum of of total element comments + replies
You can use $reduce and $concatArrays to "merge" an inner "array of arrays" into a single list and measure the $size of that. Then simply $add the two results together:
db.posts.aggregate([
{ "$match": { _id:ObjectId("5dbdacc28cffef0b94580dbd") } },
{ "$addFields": {
"totalBoth": {
"$add": [
{ "$size": "$comments" },
{ "$size": {
"$reduce": {
"input": "$comments.replies",
"initialValue": [],
"in": {
"$concatArrays": [ "$$value", "$$this" ]
}
}
}}
]
}
}}
])
Noting that an "array of arrays" is the effect of an expression like $comments.replies, so hence the operation to make these into a single array where you can measure all elements.
Try using the $unwind to flatten the list you get from the $project before using $count.
This is another way of getting the result.
Input documents:
{ "_id" : 1, "array1" : [ { "array2" : [ { id: "This is a test!"}, { id: "test1" } ] }, { "array2" : [ { id: "This is 2222!"}, { id: "test 222" }, { id: "222222" } ] } ] }
{ "_id" : 2, "array1" : [ { "array2" : [ { id: "aaaa" }, { id: "bbbb" } ] } ] }
The query:
db.arrsizes2.aggregate( [
{ $facet: {
array1Sizes: [
{ $project: { array1Size: { $size: "$array1" } } }
],
array2Sizes: [
{ $unwind: "$array1" },
{ $project: { array2Size: { $size: "$array1.array2" } } },
],
} },
{ $project: { result: { $concatArrays: [ "$array1Sizes", "$array2Sizes" ] } } },
{ $unwind: "$result" },
{ $group: { _id: "$result._id", total1: { $sum: "$result.array1Size" }, total2: { $sum: "$result.array2Size" } } },
{ $addFields: { total: { $add: [ "$total1", "$total2" ] } } },
] )
The output:
{ "_id" : 2, "total1" : 1, "total2" : 2, "total" : 3 }
{ "_id" : 1, "total1" : 2, "total2" : 5, "total" : 7 }
I am trying to fetch all records (and count of all records) for a structure like the following,
{
id: 1,
level1: {
level2:
[
{
field1:value1;
},
{
field1:value1;
},
]
}
},
{
id: 2,
level1: {
level2:
[
{
field1:null;
},
{
field1:value1;
},
]
}
}
My requirement is to fetch the number of records that have field1 populated (atleast one in level2). I need to say fetch all the ids or the number of such ids.
The query I am using is,
db.table.find({},
{
_id = id,
value: {
$elemMatch: {'level1.level2.field1':{$exists: true}}
}
}
})
Please suggest.
EDIT1:
This is the question I was trying to ask in the comment. I was unable to elucidate in the comment properly. Hence, editing the question.
{
id: 1,
level1: {
level2:
[
{
field1:value1;
},
{
field1:value1;
},
]
}
},
{
id: 2,
level1: {
level2:
[
{
field1:value2;
},
{
field1:value2;
},
{
field1:value2;
}
]
}
}
{
id: 3,
level1: {
level2:
[
{
field1:value1;
},
{
field1:value1;
},
]
}
}
The query we used results in
value1: 4
value2: 3
I want something like
value1: 2 // Once each for documents 1 & 3
value2: 1 // Once for document 2
You can do that with the following find query:
db.table.find({ "level1.level2" : { $elemMatch: { field1 : {$exists: true} } } }, {})
This will return all documents that have a field1 in the "level1.level2" structure.
For your question in the comment, you can use the following aggregation to "I had to return a grouping (and the corresponding count) for the values in field1":
db.table.aggregate(
[
{
$unwind: "$level1.level2"
},
{
$match: { "level1.level2.field1" : { $exists: true } }
},
{
$group: {
_id : "$level1.level2.field1",
count : {$sum : 1}
}
}
]
UPDATE: For your question "'value1 - 2` At level2, for a document, assume all values will be the same for field1.".
I hope i understand your question correctly, instead of grouping only on the value of field1, i added the document _id as an xtra grouping:
db.table.aggregate(
[
{
$unwind: "$level1.level2"
},
{
$match: {
"level1.level2.field1" : { $exists: true }
}
},
{
$group: {
_id : { id : "$_id", field1: "$level1.level2.field1" },
count : {$sum : 1}
}
}
]
);
UPDATE2:
I altered the aggregation and added a extra grouping, the aggregation below gives you the results you want.
db.table.aggregate(
[
{
$unwind: "$level1.level2"
},
{
$match: {
"level1.level2.field1" : { $exists: true }
}
},
{
$group: {
_id : { id : "$_id", field1: "$level1.level2.field1" }
}
},
{
$group: {
_id : { id : "$_id.field1"},
count : { $sum : 1}
}
}
]
);
I have a collection with documents like this
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"bsk" : {
"bskItems" : [
{
"id" : 4,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "reblochon"
}
},
{
"id" : 5,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "Pinot Noir"
}
},
{
"id" : 13,
"bskItemLineType" : "PromotionItem",
"promotionApplied" : {
"bskIds" : [
4,
5
]
}
},
{
"id" : 8,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "Food"
}
},
{
"id" : 10,
"bskItemLineType" : "SubTotalItem"
},
{
"id" : 12,
"bskItemLineType" : "TenderItem"
},
{
"id" : 14,
"bskItemLineType" : "ChangeDue"
}
]
}
}
I want an output where I can see the "promotionsApplied" and the descriptions of the items they applied to. For the document above the "promotionsApplied" were to "bsk.BskItems.id" 4 and 5 so I would like the output to be:
{
"_id": xxxxx,
"promotionAppliedto : "reblochon"
},
{
"_id": xxxxx,
"promotionAppliedto : "Pinot Noir"
}
the query below:
db.getCollection('DSTest').aggregate([
{$project:{"bsk.bskItems.product.description":1,"bsk.bskItems.id":1}},
{$unwind: "$bsk.bskItems"},
])
gets me the descriptions
db.getCollection('DSTest').aggregate([
{$project:{"bsk.bskItems.promotionApplied.bskIds":1}},
{$unwind: "$bsk.bskItems"},
{$unwind:"$bsk.bskItems.promotionApplied.bskIds"},
])
gets me the promotions applied. I was hoping to be able to use $lookup to join the two based on _id and bsk.bskItems.promotionApplied.bskIds and _id and bsk.bskItems.id, but I can't figure out how.
I don't know if you solved your problem or if this is relevant anymore but I figured out your question:
db.DSTest.aggregate([
{
$unwind: "$bsk.bskItems"
},
{
$project: {
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] },
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
},
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] },
}
},
{
$unwind: "$bsk.bskItems.promotionApplied.bskIds"
},
{
$project: {
baItId: 1,
proAppliedId:
{
$cond: { if: { $eq: [ "$bsk.bskItems.promotionApplied.bskIds", 0 ] }, then: "$baItId", else: "$bsk.bskItems.promotionApplied.bskIds" }
},
product: 1
}
},
{
$group: {
_id: { proAppliedId: "$proAppliedId", docId: "$_id"},
product: { $push: { "p": "$product" } },
groupCount: { $sum: 1 }
}
},
{
$unwind: "$product"
},
{
$match: {
"product.p": {$ne: ""}, "groupCount": { $gt: 1}
}
},
{
$project: {
_id: "$_id.docId",
"promotionAppliedto": "$product.p"
}
}
])
With the dummy document you gave this is the result I get:
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"promotionAppliedto" : "reblochon"
}
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"promotionAppliedto" : "Pinot Noir"
}
But my advise is to put some thought in your database structure next time. You had apples and pears, so we had to make an Asian pear in order to get to this result. Also from the aggregation levels you see it was not an easy job. That could have been much easier if you had separated the arrays that contained the field product from the ones that contained the field promotionApplied.
To break it down and explain what is happening step by step:
{
$unwind: "$bsk.bskItems"
}
By unwinding we are flattening our array. We need this in order to access the fields inside the array and do operations on them . More about $unwind
{
$project: {
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] },
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
},
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] },
}
}
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] }
With this line we just make sure that every document gets an basket item id. In your case they all do, I just added it to make sure. And if some document didn't have a value for that field we set it to 0 (you can set it to -1 or whatever you want)
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
}
Here we are creating an array for the field "$bsk.bskItems.promotionApplied.bskIds". Since not all documents have this field we have to add to them all, otherwise we are comparing oranges with apples.
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] }
As said before, we have to make our documents look all alike so we also add $bsk.bskItems.product.description to the ones that don't have this field. Those who don't have the field we set it to an empty string
Now all our documents have the same structure and we can start with the actual sorting out.
{
$unwind: "$bsk.bskItems.promotionApplied.bskIds"
}
Since we want to access the ids inside $bsk.bskItems.promotionApplied.bskIds we have to unwind this array as well.
{
$project: {
baItId: 1,
proAppliedId:
{
$cond: { if: { $eq: [ "$bsk.bskItems.promotionApplied.bskIds", 0 ] }, then: "$baItId", else: "$bsk.bskItems.promotionApplied.bskIds" }
},
product: 1
}
}
baItId: 1 and product: 1, are just being passed on. The proAppliedId will contain our bsk.bskItems.promotionApplied.bskIds. If they are 0 then the get the same id as the field $baItId, otherwise they keep their id.
{
$group: {
_id: { proAppliedId: "$proAppliedId", docId: "$_id"},
product: { $push: { "p": "$product" } },
groupCount: { $sum: 1 }
}
}
Now finally we can group our documents by $proAppliedId that we created in the previous aggregation pipeline.
We also push the product values in an array. So there will be now arrays that contain two entries.
One with the value that we look for and one with an empty string because we did that in a previous aggregation pipeline "product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] }
We also create a new field called groupCount to count the documents that were grouped together.
{ $project: {
_id: "$_id.docId",
"promotionAppliedto": "$product.p" } }
In the final project we just build the final document by how we want it to look like.
Hope you understand now why thinking, were and how we save things, matter.
Using document type database - it will be better to store promotion metadtaa instead of only id.
Please see attached example
"promotionApplied" : [{
bskId : 4,
name : "name",
otherData : "otherData"
}, {
bskId : 5,
name : "name5",
otherData : "otherData5"
}
]