How to retrieve all matching elements present inside array in Mongo DB? - mongodb

I have document shown below:
{
name: "testing",
place:"London",
documents: [
{
x:1,
y:2,
},
{
x:1,
y:3,
},
{
x:4,
y:3,
}
]
}
I want to retrieve all matching documents i.e. I want o/p in below format:
{
name: "testing",
place:"London",
documents: [
{
x:1,
y:2,
},
{
x:1,
y:3,
}
]
}
What I have tried is :
db.test.find({"documents.x": 1},{_id: 0, documents: {$elemMatch: {x: 1}}});
But, it gives first entry only.

As JohnnyHK said, the answer in MongoDB: select matched elements of subcollection explains it well.
In your case, the aggregate would look like this:
(note: the first match is not strictly necessary, but it helps in regards of performance (can use index) and memory usage ($unwind on a limited set)
> db.xx.aggregate([
... // find the relevant documents in the collection
... // uses index, if defined on documents.x
... { $match: { documents: { $elemMatch: { "x": 1 } } } },
... // flatten array documennts
... { $unwind : "$documents" },
... // match for elements, "documents" is no longer an array
... { $match: { "documents.x" : 1 } },
... // re-create documents array
... { $group : { _id : "$_id", documents : { $addToSet : "$documents" } }}
... ]);
{
"result" : [
{
"_id" : ObjectId("515e2e6657a0887a97cc8d1a"),
"documents" : [
{
"x" : 1,
"y" : 3
},
{
"x" : 1,
"y" : 2
}
]
}
],
"ok" : 1
}
For more information about aggregate(), see http://docs.mongodb.org/manual/applications/aggregation/

Related

Count if a value exists inside the array of objects [ MongoDB ]

I am developing a simple chat application with MongoDB and got stuck into a situation.
My document in database is as
{
"_id" : ObjectId("605a217ed8168f4c262f4782"),
"message" : "Hi, This is a test message",
"created" : ISODate("2021-03-23T17:12:30.000Z"),
"user" : {
"_id" : ObjectId("5977af7df1d8cc4623283b14"),
"name" : "Sender Of Message"
},
"recipients" : [
{
"_id" : ObjectId("5977af7df1d8cc4623283b14"),
"time" : ISODate("2021-03-23T17:12:30.000Z")
},
{
"_id" : ObjectId("5df50a5eaa0e3c3104006101"),
"time" : ISODate("2021-03-23T17:12:35.000Z")
}
],
"target" : {
"_id" : ObjectId("5df50a5eaa0e3c3104006101"),
"name" : "Target Person"
},
"status" : 1
}
When I try to get the last message with the unread count of the user I am always getting 1
Here is the query that I tried on.
db.collection.aggregate([
{ $match: { 'target._id': ObjectId('5df50a5eaa0e3c3104006101'), status: 1 } },
{ $sort: { _id: -1 } },
{
$group: {
_id: '$user._id',
doc: { $first: '$$ROOT' },
unread: {
$sum: {
$cond: {
if: { $ne: [ ObjectId('5df50a5eaa0e3c3104006101'), '$recipients._id' ] },
then: 1,
else: 0
}
}
}
}
}
])
If the collection contains even just the one document above, it is supposed to give 0 as the object inside the recipients array already contains the _id as ObjectId('5df50a5eaa0e3c3104006101'), but I'm getting 1 for the unread count. Any help?
Here is the output that I get from the query
{
"_id" : ObjectId("5977af7df1d8cc4623283b14"),
"doc" : {
"_id" : ObjectId("605a217ed8168f4c262f4782"),
"message" : "Hi, This is a test message",
"created" : ISODate("2021-03-23T17:12:30.000Z"),
"user" : {
"_id" : ObjectId("5977af7df1d8cc4623283b14"),
"name" : "Sender Of Message"
},
"recipients" : [
{
"_id" : ObjectId("5977af7df1d8cc4623283b14"),
"time" : ISODate("2021-03-23T17:12:30.000Z")
},
{
"_id" : ObjectId("5df50a5eaa0e3c3104006101"),
"time" : ISODate("2021-03-23T17:12:35.000Z")
}
],
"target" : {
"_id" : ObjectId("5df50a5eaa0e3c3104006101"),
"name" : "Target Person"
},
"status" : 1
},
"unread" : 1.0
}
I know why its showing with the count as 1
The array recipients contains an object with _id as ObjectId("5977af7df1d8cc4623283b14") inside it, so its a non matching condition. Which is causing the if condition to be satisfied and produce a value 1.
But I need to figure out how to query it to get the actual value.
Please note that I cant use $push operator on recipients array as it might have greater amount of object ( maybe in future )
Thanks for the support, but I have found the answer by myself.
Here is my approch to get the data as per the requirement.
Instead of searching for the records within the array what I did is
Filtered the data array to the _id that I don't need, so the array will have exactly one document or else it will be empty.
When taking the negation of the condition. ie, when there is one value in the array I need the counter to be 0 or else it should be 1
So I used the $size to check the array's size and $filter to filter out the other _ids and then used $sum to increment the counter as required.
db.collection.aggregate([
{ $match: { 'target._id': ObjectId('5df50a5eaa0e3c3104006101'), status: 1 } },
{ $sort: { _id: -1 } },
{
$group: {
_id: '$user._id',
doc: { $first: '$$ROOT' },
unread: {
$sum: {
$cond:{
if: {
$size: {
$filter: {
input: '$recipients',
as: 'item',
cond: { $eq: [ ObjectId('5df50a5eaa0e3c3104006101'), '$$item._id' ] }
}
}
},
then: 0,
else: 1
}
}
}
}
}
])
Try to Use like this:
db.getCollection('test').aggregate([
{ $match: { 'target._id': ObjectId('5df50a5eaa0e3c3104006101'), status: 1
} },
{ $sort: { _id: -1 } },
{ $unwind: { path: "$recipients", preserveNullAndEmptyArrays: true } },
{
$group: {
_id: '$user._id',
doc: { $first: '$$ROOT' },
unread: {
$sum: {
$cond: {
if: { $ne: ['$recipients._id',
ObjectId('5df50a5eaa0e3c3104006101') ] },
then: 1,
else: 0
}
}
}
}
}
])

MongoDB - Find documents matching certain condition for unknown field keys

How can I query a MongoDB collection to find documents with a structure as below? The documents have a field called thing which is a subdocument, and the keys for this field are a form of ID number which will generally not be known by the person writing the query (making dot notation difficult and I assume impossible).
{
"_id" : 3,
"_id2" : 234,
"thing":
{
"2340945683":
{"attribute1": "typeA",
"attribute2": "typeB",
"attribute3": "typeA"
},
"349687346":
{"attribute1": "typeC",
"attribute2": "typeB",
"attribute3": "typeA"
}
},
"username": "user1"
}
Say I want to set a filter which will return the document only if some one or more of the fields within thing have the condition "attribute1" : "typeC"?
I need something like
db.collection.find( {thing.ANY_FIELD: $elemMatch:{"attribute1":"typeC"}})
You need to start with $objectToArray to read your keys dynamically. Then you can $map properties along with $anyElementTrue to detect if there's any nested field in thing containing {"attribute1":"typeC"}:
db.collection.aggregate([
{
$match: {
$expr: {
$anyElementTrue: {
$map: {
input: { $objectToArray: "$thing" },
in: { $eq: [ "$$this.v.attribute1", "typeC" ] }
}
}
}
}
}
])
Mongo Playground
My solution to this was to use two aggregate operations, the first one is called objectToArray and it's purpose is to convert a object into a list of objects with keys and values (see the documentation examples), and the reduce to search in this array of key-values, at the end we end up with a boolean "hasAttribute" indicating that the one field matched the value wee are looking for.
Here is the solution:
db.getCollection("thing").aggregate([
{
$addFields: {
hasAttribute: {
$reduce: {
input: {
$objectToArray: "$thing"
},
initialValue: false,
in: {$or: ["$$value", {$eq: ["typeC", "$$this.v.attribute1"]}]}
}
}
}
},
{
$match: {
hasAttribute: true
}
}
])
Here is the sample output and how the boolean value behaves:
{
"_id" : ObjectId("5ddd63c02e5c579c5076c76f"),
"thing" : {
"349687346" : {
"attribute1" : "typeC",
"attribute2" : "typeB",
"attribute3" : "typeA"
},
"2340945683" : {
"attribute1" : "typeA",
"attribute2" : "typeB",
"attribute3" : "typeA"
}
},
"hasAttribute" : true
}
// ----------------------------------------------
{
"_id" : ObjectId("5ddd63d12e5c579c5076c770"),
"thing" : {
"2340945683" : {
"attribute1" : "typeA",
"attribute2" : "typeB",
"attribute3" : "typeA"
}
},
"hasAttribute" : false
}
// ----------------------------------------------
{
"_id" : ObjectId("5ddd63d12e5c579c5076c771"),
"thing" : {
"349687346" : {
"attribute1" : "typeC",
"attribute2" : "typeB",
"attribute3" : "typeA"
}
},
"hasAttribute" : true
}
Ask for clarifications if you need!

How can I make $lookup embed the document directly instead of wrapping it into array?

I have a document like this:
{
"_id": ObjectId("5d779541bd4e75c58d598212")
"client": ObjectId("5d779558bd4e75c58d598213")
}
When I do $lookup like this:
{
from: 'client',
localField: 'client',
foreignField: 'id',
as: 'client',
}
I get:
{
"_id": ObjectId("5d779541bd4e75c58d598212")
"client":[
{
... client info wrapped in array
}
]
}
This forces me to add $unwind after the lookup stage.
This would work fine in this example because I know that it is a regular field (not array). But on other collections I have arrays of ObjectId's and I don't want to unwind them.
How should I tell mongo to unwind only if it's not an array?
Add $project stage with $arrayElemAt
{ $lookup ..... },
{ $project: { client: { $arrayElemAt: [ "$client" , 0 ]}} // Add other filed
The lookup always returns an array as it doesn't know if its a one-to-one or one-to-many mapping. But we can ensure that the lookup returns a single document and that document would hold all documents which were supposed to come as an array in the general lookup.
Following is the way:
db.collection.aggregate([
{
$lookup:{
"from":"client",
"let":{
"client":"$client"
},
"pipeline":[
{
$match:{
$expr:{
$eq:["$id","$$client"]
}
}
},
{
$group:{
"_id":null,
"data":{
$push:"$$ROOT"
}
}
},
{
$project:{
"_id":0
}
}
],
"as":"clientLookup"
}
},
{
$unwind:"$clientLookup"
}
]).pretty()
Query analysis: We are looking up into client collection and executing a pipeline inside that. The output of that pipeline would hold every matched document inside data field.
Data set:
Collection: collection
{
"client":1
}
{
"client":2
}
Collection: client
{
"id":1,
"name":"Tony"
}
{
"id":1,
"name":"Thor"
}
{
"id":1,
"name":"Natasha"
}
{
"id":2,
"name":"Banner"
}
Output:
{
"_id" : ObjectId("5d7792c6bd4e75c58d59820c"),
"client" : 1,
"clientLookup" : {
"data" : [
{
"_id" : ObjectId("5d779322bd4e75c58d59820e"),
"id" : 1,
"name" : "Tony"
},
{
"_id" : ObjectId("5d779322bd4e75c58d59820f"),
"id" : 1,
"name" : "Thor"
},
{
"_id" : ObjectId("5d779322bd4e75c58d598210"),
"id" : 1,
"name" : "Natasha"
}
]
}
}
{
"_id" : ObjectId("5d7792c6bd4e75c58d59820d"),
"client" : 2,
"clientLookup" : {
"data" : [
{
"_id" : ObjectId("5d779322bd4e75c58d598211"),
"id" : 2,
"name" : "Banner"
}
]
}
}

Using mongodb $lookup on a single collection

I have a collection with documents like this
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"bsk" : {
"bskItems" : [
{
"id" : 4,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "reblochon"
}
},
{
"id" : 5,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "Pinot Noir"
}
},
{
"id" : 13,
"bskItemLineType" : "PromotionItem",
"promotionApplied" : {
"bskIds" : [
4,
5
]
}
},
{
"id" : 8,
"bskItemLineType" : "SaleItem",
"product" : {
"description" : "Food"
}
},
{
"id" : 10,
"bskItemLineType" : "SubTotalItem"
},
{
"id" : 12,
"bskItemLineType" : "TenderItem"
},
{
"id" : 14,
"bskItemLineType" : "ChangeDue"
}
]
}
}
I want an output where I can see the "promotionsApplied" and the descriptions of the items they applied to. For the document above the "promotionsApplied" were to "bsk.BskItems.id" 4 and 5 so I would like the output to be:
{
"_id": xxxxx,
"promotionAppliedto : "reblochon"
},
{
"_id": xxxxx,
"promotionAppliedto : "Pinot Noir"
}
the query below:
db.getCollection('DSTest').aggregate([
{$project:{"bsk.bskItems.product.description":1,"bsk.bskItems.id":1}},
{$unwind: "$bsk.bskItems"},
])
gets me the descriptions
db.getCollection('DSTest').aggregate([
{$project:{"bsk.bskItems.promotionApplied.bskIds":1}},
{$unwind: "$bsk.bskItems"},
{$unwind:"$bsk.bskItems.promotionApplied.bskIds"},
])
gets me the promotions applied. I was hoping to be able to use $lookup to join the two based on _id and bsk.bskItems.promotionApplied.bskIds and _id and bsk.bskItems.id, but I can't figure out how.
I don't know if you solved your problem or if this is relevant anymore but I figured out your question:
db.DSTest.aggregate([
{
$unwind: "$bsk.bskItems"
},
{
$project: {
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] },
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
},
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] },
}
},
{
$unwind: "$bsk.bskItems.promotionApplied.bskIds"
},
{
$project: {
baItId: 1,
proAppliedId:
{
$cond: { if: { $eq: [ "$bsk.bskItems.promotionApplied.bskIds", 0 ] }, then: "$baItId", else: "$bsk.bskItems.promotionApplied.bskIds" }
},
product: 1
}
},
{
$group: {
_id: { proAppliedId: "$proAppliedId", docId: "$_id"},
product: { $push: { "p": "$product" } },
groupCount: { $sum: 1 }
}
},
{
$unwind: "$product"
},
{
$match: {
"product.p": {$ne: ""}, "groupCount": { $gt: 1}
}
},
{
$project: {
_id: "$_id.docId",
"promotionAppliedto": "$product.p"
}
}
])
With the dummy document you gave this is the result I get:
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"promotionAppliedto" : "reblochon"
}
{
"_id" : ObjectId("5773ac6a486f811694711875"),
"promotionAppliedto" : "Pinot Noir"
}
But my advise is to put some thought in your database structure next time. You had apples and pears, so we had to make an Asian pear in order to get to this result. Also from the aggregation levels you see it was not an easy job. That could have been much easier if you had separated the arrays that contained the field product from the ones that contained the field promotionApplied.
To break it down and explain what is happening step by step:
{
$unwind: "$bsk.bskItems"
}
By unwinding we are flattening our array. We need this in order to access the fields inside the array and do operations on them . More about $unwind
{
$project: {
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] },
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
},
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] },
}
}
baItId: { $ifNull: [ "$bsk.bskItems.id", 0 ] }
With this line we just make sure that every document gets an basket item id. In your case they all do, I just added it to make sure. And if some document didn't have a value for that field we set it to 0 (you can set it to -1 or whatever you want)
"bsk": {
"bskItems": {
"promotionApplied": {
"bskIds": { $ifNull: [ "$bsk.bskItems.promotionApplied.bskIds", [0] ] }
}
}
}
Here we are creating an array for the field "$bsk.bskItems.promotionApplied.bskIds". Since not all documents have this field we have to add to them all, otherwise we are comparing oranges with apples.
"product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] }
As said before, we have to make our documents look all alike so we also add $bsk.bskItems.product.description to the ones that don't have this field. Those who don't have the field we set it to an empty string
Now all our documents have the same structure and we can start with the actual sorting out.
{
$unwind: "$bsk.bskItems.promotionApplied.bskIds"
}
Since we want to access the ids inside $bsk.bskItems.promotionApplied.bskIds we have to unwind this array as well.
{
$project: {
baItId: 1,
proAppliedId:
{
$cond: { if: { $eq: [ "$bsk.bskItems.promotionApplied.bskIds", 0 ] }, then: "$baItId", else: "$bsk.bskItems.promotionApplied.bskIds" }
},
product: 1
}
}
baItId: 1 and product: 1, are just being passed on. The proAppliedId will contain our bsk.bskItems.promotionApplied.bskIds. If they are 0 then the get the same id as the field $baItId, otherwise they keep their id.
{
$group: {
_id: { proAppliedId: "$proAppliedId", docId: "$_id"},
product: { $push: { "p": "$product" } },
groupCount: { $sum: 1 }
}
}
Now finally we can group our documents by $proAppliedId that we created in the previous aggregation pipeline.
We also push the product values in an array. So there will be now arrays that contain two entries.
One with the value that we look for and one with an empty string because we did that in a previous aggregation pipeline "product": { $ifNull: [ "$bsk.bskItems.product.description", "" ] }
We also create a new field called groupCount to count the documents that were grouped together.
{ $project: {
_id: "$_id.docId",
"promotionAppliedto": "$product.p" } }
In the final project we just build the final document by how we want it to look like.
Hope you understand now why thinking, were and how we save things, matter.
Using document type database - it will be better to store promotion metadtaa instead of only id.
Please see attached example
"promotionApplied" : [{
bskId : 4,
name : "name",
otherData : "otherData"
}, {
bskId : 5,
name : "name5",
otherData : "otherData5"
}
]

Returning only subdocuments in MongoDB

I have the following documents in my MongoDB collection:
{
"name": "name",
"items": [
{
"raw": { ... }
"processed": { ... }
},
{
"raw": { ... }
"processed": { ... }
}
]
}
And I'm trying to aggregate / query the database such that I get these items:
[
{"raw": { ... }},
{"raw": { ... }}
]
I'm using the aggregation framework now, but I'm stuck at the part where I want to exclude fields of the outer document.
My current query is:
db.mycollections.aggregate([
{ $unwind: "$items" },
{ $project: { "items.raw": 1 } }
])
And it returns:
[
{"items: {"raw": { ... }}},
{"items: {"raw": { ... }}}
]
Is there a way to only return the subdocuments from the query above?
If you write aggregation with unwind as :
db.mycollections.aggregate({"$unwind":"$items"})
then output looks like :
{ "_id" : ObjectId(), "name" : "name", "items" : { "raw" : {... }, "processed" : { ... } } }
{ "_id" : ObjectId() , "name" : "name", "items" : { "raw" : { ...}, "processed" : { ...} } }
$project passes along the documents with only the specified fields to the next stage in the pipeline. The specified fields can be existing fields from the input documents or newly computed fields.
and you pass $project as your existing fields with items.raw so instead of passing this existing field to project use expression with new field name as raw and changed your aggregation as
db.mycollections.aggregate({"$unwind":"$items"},{"$project":{"raw":"$items.raw"}})
For more details check mongo aggregation pipeline