Mongo DB - Second Level Search - elemMatch - mongodb

I am trying to fetch all records (and count of all records) for a structure like the following,
{
id: 1,
level1: {
level2:
[
{
field1:value1;
},
{
field1:value1;
},
]
}
},
{
id: 2,
level1: {
level2:
[
{
field1:null;
},
{
field1:value1;
},
]
}
}
My requirement is to fetch the number of records that have field1 populated (atleast one in level2). I need to say fetch all the ids or the number of such ids.
The query I am using is,
db.table.find({},
{
_id = id,
value: {
$elemMatch: {'level1.level2.field1':{$exists: true}}
}
}
})
Please suggest.
EDIT1:
This is the question I was trying to ask in the comment. I was unable to elucidate in the comment properly. Hence, editing the question.
{
id: 1,
level1: {
level2:
[
{
field1:value1;
},
{
field1:value1;
},
]
}
},
{
id: 2,
level1: {
level2:
[
{
field1:value2;
},
{
field1:value2;
},
{
field1:value2;
}
]
}
}
{
id: 3,
level1: {
level2:
[
{
field1:value1;
},
{
field1:value1;
},
]
}
}
The query we used results in
value1: 4
value2: 3
I want something like
value1: 2 // Once each for documents 1 & 3
value2: 1 // Once for document 2

You can do that with the following find query:
db.table.find({ "level1.level2" : { $elemMatch: { field1 : {$exists: true} } } }, {})
This will return all documents that have a field1 in the "level1.level2" structure.
For your question in the comment, you can use the following aggregation to "I had to return a grouping (and the corresponding count) for the values in field1":
db.table.aggregate(
[
{
$unwind: "$level1.level2"
},
{
$match: { "level1.level2.field1" : { $exists: true } }
},
{
$group: {
_id : "$level1.level2.field1",
count : {$sum : 1}
}
}
]
UPDATE: For your question "'value1 - 2` At level2, for a document, assume all values will be the same for field1.".
I hope i understand your question correctly, instead of grouping only on the value of field1, i added the document _id as an xtra grouping:
db.table.aggregate(
[
{
$unwind: "$level1.level2"
},
{
$match: {
"level1.level2.field1" : { $exists: true }
}
},
{
$group: {
_id : { id : "$_id", field1: "$level1.level2.field1" },
count : {$sum : 1}
}
}
]
);
UPDATE2:
I altered the aggregation and added a extra grouping, the aggregation below gives you the results you want.
db.table.aggregate(
[
{
$unwind: "$level1.level2"
},
{
$match: {
"level1.level2.field1" : { $exists: true }
}
},
{
$group: {
_id : { id : "$_id", field1: "$level1.level2.field1" }
}
},
{
$group: {
_id : { id : "$_id.field1"},
count : { $sum : 1}
}
}
]
);

Related

Mongoldb aggregation average of a number in an array?

I have a collection of documents that look like this
{
_id : 21353456,
product : "xy",
text : "asdf",
reviews : [
{
username : "User1",
userID: 12
text : "hi"
rate: 4,
},
{
username : "User2",
userID: 123
text : "hi1"
rate:2,
}
]
}
I want to retrieve the average rating for user1 on all the product they have rated.
db.collection.aggregate([{$unwind: "$reviews"},{$match: {"$review.userID": "12"}},{$group: { _id: "$reviews.userName",
{avgRate: {$avg: "$reviews.rate"}}})]
)
I tried this but I keep getting unexpected token errors for "," where the last ")" is.
You did this:
db.collection.aggregate([
{ $unwind: "$reviews" },
{ $match: { "$review.userID": "12" } },
{ $group: { _id: "$reviews.userName",
{ avgRate: { $avg: "$reviews.rate" } }})]
)
Must be this:
db.collection.aggregate([
{ $unwind: "$reviews" },
{ $match: { "$review.userID": "12" } },
{
$group: {
_id: "$reviews.userName",
avgRate: { $avg: "$reviews.rate" }
}
}
])

MongoDB Aggregation to get count and Y sample entries

MongoDB version:4.2.17.
Trying out aggregation on data in a collection.
Example data:
{
"_id" : "244",
"pubName" : "p1",
"serviceIdRef" : "36e9c779-7865-4b74-a30b-e4d6a0cc5295",
"serviceName" : "my-service",
"subName" : "c1",
"pubState" : "INVITED"
}
I would like to:
Do a match by something (let’s say subName) and group by serviceIdRef and then limit to return X entries
Also return for each of the serviceIdRefs, the count of the documents in each of ACTIVE or INVITED states. And Y (for this example, say Y=3) documents that are in this state.
For example, the output would appear as (in brief):
[
{
serviceIdRef: "36e9c779-7865-4b74-a30b-e4d6a0cc5295",
serviceName:
state:[
{
pubState: "INVITED"
count: 200
sample: [ // Get those Y entries (here Y=3)
{
// sample1 like:
"_id" : "244",
"pubName" : "p1",
"serviceIdRef" : "36e9c779-7865-4b74-a30b-e4d6a0cc5295",
"serviceName" : "my-service",
"subName" : "c1",
"pubState" : "INVITED"
},
{
sample2
},
{
sample3
}
]
},
{
pubState: "ACTIVE", // For this state, repeat as we did for "INVITED" state above.
......
}
]
}
{
repeat for another service
}
]
So far I have written this but am not able to get those Y entries. Is there a (better) way?
This is what I have so far (not complete and not exactly outputs in the format above):
db.sub.aggregate(
[{
$match:
{
"subName": {
$in: ["c1", "c2"]
},
"$or": [
{
"pubState": "INVITED",
},
{
"pubState": "ACTIVE",
}
]
}
},
{
$group: {
_id: "$serviceIdRef",
subs: {
$push: "$$ROOT",
}
}
},
{
$sort: {
_id: -1,
}
},
{
$limit: 22
},
{
$facet:
{
facet1: [
{
$unwind: "$subs",
},
{
$group:
{
_id: {
"serviceName" : "$_id",
"pubState": "$subs.pubState",
"subState": "$subs.subsState"
},
count: {
$sum: 1
}
}
}
]
}
}
])
You have to do the second $group stage to manage nested structure,
$match your conditions
$sort by _id in descending order
$group by serviceIdRef and pubState, get first required fields and prepare the array for sample, and get count of documents
$group by only serviceIdRef and construct the state array
$slice for limit the document in sample
db.collection.aggregate([
{
$match: {
subName: { $in: ["c1", "c2"] },
pubState: { $in: ["INVITED", "ACTIVE"] }
}
},
{ $sort: { _id: -1 } },
{
$group: {
_id: {
serviceIdRef: "$serviceIdRef",
pubState: "$pubState"
},
serviceName: { $first: "$serviceName" },
sample: { $push: "$$ROOT" },
count: { $sum: 1 }
}
},
{
$group: {
_id: "$_id.serviceIdRef",
serviceName: { $first: "$serviceName" },
state: {
$push: {
pubState: "$_id.pubState",
count: "$count",
sample: { $slice: ["$sample", 22] }
}
}
}
}
])
Playground

MongoDB returns only the specified query

db.customerOrder.insert({
firstName: "Andrew",
lastName: "Lee",
DOB: ISODate("1974-10-28T00:00:00Z"),
phone: "+1 (959) 567-3312",
email: "mark#gmail.com",
address: {
street: "Cornish Street, Victoria",
houseNumber: "68",
postalCode: "3024",
country: "Australia",
},
language: ["English", "Mandarin"],
balance: 0,
orders: [
{
orderNumber: "ord003",
orderDate: ISODate("2020-01-10T00:00:00Z"),
staffNumber: "stf789"
}
]
});
Given the document above, and other documents which contain other orders and order number, how do i specify an aggregation so that it will only list all orderNumbers that's handled by a staffNumber x?
Example, orderNumber ord004 and ord005 is handled by staffNumber stf890
I tried doing
db.customerOrder.aggregate([ {"$match":{"orders.staffNumber":"stf890"}}, {"$project":{"orders.orderNumber":1, "_id":0}} ])
but the result was
{
"orders" : [
{
"orderNumber" : "ord003"
},
{
"orderNumber" : "ord003"
},
{
"orderNumber" : "ord005"
}
]
}
{
"orders" : [
{
"orderNumber" : "ord001"
},
{
"orderNumber" : "ord005"
}
]
}
{
"orders" : [
{
"orderNumber" : "ord003"
},
{
"orderNumber" : "ord004"
}
]
}
I expect the result to output only ord004 and ord005
How do i achieve this?
Thank you for your help
Try this! your query is almost correct but you're missing the case of matching orderNumber.
db.customerOrder.aggregate([
{
"$match":{
"orders.staffNumber":"stf890"
}
},
{
$unwind:{
"path":"$orders"
}
},
{
"$match":{
"orders.orderNumber":{$in:["ord004","ord005"]}
}
},
{
"$project":{
"orders.orderNumber":1,
"_id":0
}
}
])
If you don't care about the structure you can just $unwind and then match. otherwise you need to use something like $filter
Option 1:
db.customerOrder.aggregate([
{
"$match": {
"orders.staffNumber": "stf890"
}
},
{
"$unwind": "$orders"
},
{
"$match": {
"orders.staffNumber": "stf890"
}
},
{
"$project": {"orders.orderNumber": 1, "_id": 0}
}])
Option 2:
db.customerOrder.aggregate([
{
"$match": {
"orders.staffNumber": "stf890"
}
},
{
$project: {
orders: {
$filter: {
input: "$orders",
as: "order",
cond: {
$eq: ["$$order.staffNumber", "stf890"]
}
}
}
}
},
{
"$project": {
"orders.orderNumber": 1,
"_id": 0
}
}
])

Merge documents from 2 collections in MongoDB & overwrite properties on a field

I have 2 collections in MongoDB :
Collection1 :
{
_id:1,
Field1: "Some info",
Field2: "Some other info",
Elements: [
{
id: 0,
Enabled: false
},
{
id: 1,
Enabled: false
},
{
id: 2,
Enabled: false
}
]
}
Collection2 :
{
Identifier: "identifier",
ElementsOverride: [
{
id: 0,
Enabled: true
},
{
id: 1,
Enabled: false
},
{
id: 2,
Enabled: true
}
]
}
What I would like to do is perform an operation which flattens "Element" collection and returns Collection1 with the flattened Element collection (basically the Enabled field from collection 2 overwrites the enabled field of Collection 1.
Is there a way to achieve this in Mongodb?
Adding more clarification for what the output should be like:
Essentially what I'm trying to do is merge the document identified by _id:1 in collection 1 (document1), with the document identified by Identifier: "identifier" in collection 2 (document 2) such that:
All the properties in document1 and document2 are available in the output.
The ElementsOverride from document2 with the same ID's as document1 (ex; id: 0) will overwrite the values in document1
Required Output :
{
_id:1,
Identifier: "identifier",
Field1: "Some info",
Field2: "Some other info",
Elements: [
{
id: 0,
Enabled: true
},
{
id: 1,
Enabled: false
},
{
id: 2,
Enabled: true
}
]
}
You can try below query :
db.Collection1.aggregate([
/** get only one required doc from Collection1 */
{ $match: { _id: 1 } },
/** Join relative doc from Collection2 */
{
$lookup:
{
from: "Collection2",
pipeline: [
{
$match:
{
$expr:
{ $eq: ["$Identifier", "identifier"] }
}
}
],
as: "data"
}
},
/** As lookup will default to an array of objects getting an object out of array */
{ $unwind: '$data' },
/** Replacing existing elements field of Collection1 & adding Identifier field to existing doc */
{
$addFields: {
Identifier: '$data.Identifier', Elements:
{
$reduce: {
input: { $reverseArray: { $setUnion: ["$Elements", "$data.ElementsOverride"] } },
initialValue: [],
in: { $concatArrays: ["$$value", { $cond: [{ $in: ['$$this.id', '$$value.id'] }, [], ['$$this']] }] }
}
}
}
},
/** removing unnecessary field created at lookup stage */
{ $project: { data: 0 } }
])
Test : MongoDB-Playground
I am not sure how you want the output.
flattens "Element" collection
generally means the array Element is unwound. Please correct my interpretation, in case I have misunderstood.
But, the following steps in Mongo Shell will get the result:
arr1 = db.c1.aggregate( [ { $unwind: "$Elements" }, { $sort: { "Elements.id": 1 } ] ).toArray()
arr2 = db.c2.aggregate( [ { $unwind: "$ElementsOverride" }, { $sort: { "ElementsOverride.id": 1 } ] ).toArray()
for (let i=0; i < arr1.length; i++) {
updated = Object.assign(arr1[i].Elements, arr2[i].ElementsOverride);
arr1[i].Elements = updated
}
The variable arr1 will have:
[
{
"_id" : 1,
"Field1" : "Some info",
"Field2" : "Some other info",
"Elements" : {
"id" : 0,
"Enabled" : true
}
},
{
"_id" : 1,
"Field1" : "Some info",
"Field2" : "Some other info",
"Elements" : {
"id" : 1,
"Enabled" : false
}
},
{
"_id" : 1,
"Field1" : "Some info",
"Field2" : "Some other info",
"Elements" : {
"id" : 2,
"Enabled" : true
}
}
]
[ EDIT ADD ]
Updated to reflect the required output:
arr2 = db.c2.aggregate( [
{ $unwind: "$ElementsOverride" },
{ $replaceRoot: { newRoot: "$ElementsOverride" } }
] ).toArray()
db.c1.aggregate( [
{ $unwind: "$Elements" },
{ $addFields: {
"Elements.Enabled": {
$filter: {
input: arr2,
cond: { $eq: [ "$$this.ElementsOverride.id", "$Elements.id" ] }
}
}
} },
{ $group: {
_id: "$_id",
doc: { $first: "$$ROOT"},
Identifier: { $first: "$Elements.Enabled.Identifier"},
Elements: { $push: { $arrayElemAt: [ "$Elements.Enabled", 0 ] } }
} },
{ $addFields: {
"doc.Elements": "$Elements.ElementsOverride",
"doc.Identifier": { $arrayElemAt: [ "$Identifier", 0 ] }
} },
{ $replaceRoot: { newRoot: "$doc" } }
] )
[ EDIT ADD 2 ]
Here is another way of merging the documents:
doc1 = db.c1.findOne()
arr2 = db.c2.aggregate( [ { $unwind: "$ElementsOverride" } ] ).toArray()
for (let e2 of arr2) {
for (i = 0; i < doc1.Elements.length; i++) {
if (doc1.Elements[i].id == e2.ElementsOverride.id) {
doc1.Elements[i].Enabled = e2.ElementsOverride.Enabled
doc1.Identifier = e2.Identifier
}
}
}
The output is the doc1 document.

Using mongodb $lookup on a single collection

I have a collection with documents like this
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"bsk" : {
"bskItems" : [
{
"id" : 4,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "reblochon"
}
},
{
"id" : 5,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "Pinot Noir"
}
},
{
"id" : 13,
"bskItemLineType" : "PromotionItem",
"promotionApplied" : {
"bskIds" : [
4,
5
]
}
},
{
"id" : 8,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "Food"
}
},
{
"id" : 10,
"bskItemLineType" : "SubTotalItem"
},
{
"id" : 12,
"bskItemLineType" : "TenderItem"
},
{
"id" : 14,
"bskItemLineType" : "ChangeDue"
}
]
}
}
I want an output where I can see the "promotionsApplied" and the descriptions of the items they applied to. For the document above the "promotionsApplied" were to "bsk.BskItems.id" 4 and 5 so I would like the output to be:
{
"_id": xxxxx,
"promotionAppliedto : "reblochon"
},
{
"_id": xxxxx,
"promotionAppliedto : "Pinot Noir"
}
the query below:
db.getCollection('DSTest').aggregate([
{$project:{"bsk.bskItems.product.description":1,"bsk.bskItems.id":1}},
{$unwind: "$bsk.bskItems"},
])
gets me the descriptions
db.getCollection('DSTest').aggregate([
{$project:{"bsk.bskItems.promotionApplied.bskIds":1}},
{$unwind: "$bsk.bskItems"},
{$unwind:"$bsk.bskItems.promotionApplied.bskIds"},
])
gets me the promotions applied. I was hoping to be able to use $lookup to join the two based on _id and bsk.bskItems.promotionApplied.bskIds and _id and bsk.bskItems.id, but I can't figure out how.
I don't know if you solved your problem or if this is relevant anymore but I figured out your question:
db.DSTest.aggregate([
{
$unwind: "$bsk.bskItems"
},
{
$project: {
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] },
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
},
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] },
}
},
{
$unwind: "$bsk.bskItems.promotionApplied.bskIds"
},
{
$project: {
baItId: 1,
proAppliedId:
{
$cond: { if: { $eq: [ "$bsk.bskItems.promotionApplied.bskIds", 0 ] }, then: "$baItId", else: "$bsk.bskItems.promotionApplied.bskIds" }
},
product: 1
}
},
{
$group: {
_id: { proAppliedId: "$proAppliedId", docId: "$_id"},
product: { $push: { "p": "$product" } },
groupCount: { $sum: 1 }
}
},
{
$unwind: "$product"
},
{
$match: {
"product.p": {$ne: ""}, "groupCount": { $gt: 1}
}
},
{
$project: {
_id: "$_id.docId",
"promotionAppliedto": "$product.p"
}
}
])
With the dummy document you gave this is the result I get:
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"promotionAppliedto" : "reblochon"
}
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"promotionAppliedto" : "Pinot Noir"
}
But my advise is to put some thought in your database structure next time. You had apples and pears, so we had to make an Asian pear in order to get to this result. Also from the aggregation levels you see it was not an easy job. That could have been much easier if you had separated the arrays that contained the field product from the ones that contained the field promotionApplied.
To break it down and explain what is happening step by step:
{
$unwind: "$bsk.bskItems"
}
By unwinding we are flattening our array. We need this in order to access the fields inside the array and do operations on them . More about $unwind
{
$project: {
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] },
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
},
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] },
}
}
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] }
With this line we just make sure that every document gets an basket item id. In your case they all do, I just added it to make sure. And if some document didn't have a value for that field we set it to 0 (you can set it to -1 or whatever you want)
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
}
Here we are creating an array for the field "$bsk.bskItems.promotionApplied.bskIds". Since not all documents have this field we have to add to them all, otherwise we are comparing oranges with apples.
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] }
As said before, we have to make our documents look all alike so we also add $bsk.bskItems.product.description to the ones that don't have this field. Those who don't have the field we set it to an empty string
Now all our documents have the same structure and we can start with the actual sorting out.
{
$unwind: "$bsk.bskItems.promotionApplied.bskIds"
}
Since we want to access the ids inside $bsk.bskItems.promotionApplied.bskIds we have to unwind this array as well.
{
$project: {
baItId: 1,
proAppliedId:
{
$cond: { if: { $eq: [ "$bsk.bskItems.promotionApplied.bskIds", 0 ] }, then: "$baItId", else: "$bsk.bskItems.promotionApplied.bskIds" }
},
product: 1
}
}
baItId: 1 and product: 1, are just being passed on. The proAppliedId will contain our bsk.bskItems.promotionApplied.bskIds. If they are 0 then the get the same id as the field $baItId, otherwise they keep their id.
{
$group: {
_id: { proAppliedId: "$proAppliedId", docId: "$_id"},
product: { $push: { "p": "$product" } },
groupCount: { $sum: 1 }
}
}
Now finally we can group our documents by $proAppliedId that we created in the previous aggregation pipeline.
We also push the product values in an array. So there will be now arrays that contain two entries.
One with the value that we look for and one with an empty string because we did that in a previous aggregation pipeline "product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] }
We also create a new field called groupCount to count the documents that were grouped together.
{ $project: {
_id: "$_id.docId",
"promotionAppliedto": "$product.p" } }
In the final project we just build the final document by how we want it to look like.
Hope you understand now why thinking, were and how we save things, matter.
Using document type database - it will be better to store promotion metadtaa instead of only id.
Please see attached example
"promotionApplied" : [{
bskId : 4,
name : "name",
otherData : "otherData"
}, {
bskId : 5,
name : "name5",
otherData : "otherData5"
}
]