I have written an aggregation pipeline that up to now returns 2 documents (after matching, projecting and grouping) that look something like this:
{"_id": "group1",
"items": [
{
...item1 fields...
},
{
...item2 fields...
},
....
]
},
{"_id": "group2",
"items": [
{
...item1 fields...
},
{
...item3 fields...
},
....
]
}
The results always have these 2 groups and both groups have an array with items. I want to extend this pipeline in a way so that I only get the items that are common in the two arrays. For the example above, only item1 would be returned.
Any ideas?
Related
I'm pretty new to MongoDb. I have a collection of some products, some of them contains an array of ids of other products in the same collection:
[
{
"id": 1,
"relatedProducts": [
"1", "2"
]
},
{
"id": 2,
"relatedProducts": [
"4", "5"
]
}
]
Problem is, not all of the products that are in that relatedProducts array are available in that collection.
I have to create an aggregation that will modify those arrays so only id of available products are present in it. So, for example, if product of id = 5 is not present in that collection, relatedProducts of object with id 2 above will have only one entry in the array ("4").
I recommend you first fetch all the product _id's, then update the documents accordingly.
Here's how I would do this:
const ids = await db.collection.distinct('id');
await db.collection.updateMany(
{
relatedProducts: {$nin: ids}
},
[
{
$set: {
relatedProducts: {
$setIntersection: [{$ifNull: ["$relatedProducts", []]}, ids]
}
}
}
]
)
I am working on a Node.js application that is using a MongoDB database with Mongoose. I've been stuck in this thing and didn't come up with the right query.
Problem:
There is a collection named chats which contain embedded documents (rooms) as an array of objects. I want to delete these embedded documents (rooms) through Ids which are in the array.
{
"_id": "ObjectId(6138e2b55c175846ec1e38c5)",
"type": "bot",
"rooms": [
{
"_id": "ObjectId(6138e2b55c145846ec1e38c5)",
"genre": "action"
},
{
"_id": "ObjectId(6138e2b545c145846ec1e38c5)",
"genre": "adventure"
}
]
},
{
"_id": "ObjectId(6138e2b55c1765846ec1e38c5)",
"type": "person",
"rooms": [
{
"_id": "ObjectId(6138e2565c145846ec1e38c5)",
"genre": "food"
},
{
"_id": "ObjectId(6138e2b5645c145846ec1e38c5)",
"genre": "sport"
}
]
},
{
"_id": "ObjectId(6138e2b55c1765846ec1e38c5)",
"type": "duo",
"rooms": [
{
"_id": "ObjectId(6138e21c145846ec1e38c5)",
"genre": "travel"
},
{
"_id": "ObjectId(6138e35645c145846ec1e38c5)",
"genre": "news"
}
]
}
I am converting my array of ids into MongoDB ObjectId so I can use these ids as match criteria.
const idsRoom = [
'6138e21c145846ec1e38c5',
'6138e2565c145846ec1e38c5',
'6138e2b545c145846ec1e38c5',
];
const objectIdArray = idsRoom.map((s) => mongoose.Types.ObjectId(s));
and using this query for the chat collection. But it is deleting the whole document and I only want to delete the rooms embedded document because the ids array is only for the embedded documents.
Chat.deleteMany({ 'rooms._id': objectIdArray }, function (err) {
console.log('Delete successfully')
})
I really appreciate your help on this issue.
You have to use $pull operator in a update query like this:
This query look for documents where exists the _id into rooms array and use $pull to remove the object from the array.
yourModel.updateMany({
"rooms._id": {
"$in": [
"6138e21c145846ec1e38c5",
"6138e2565c145846ec1e38c5",
"6138e2b545c145846ec1e38c5"
]
}
},
{
"$pull": {
"rooms": {
"_id": {
"$in": [
"6138e21c145846ec1e38c5",
"6138e2565c145846ec1e38c5",
"6138e2b545c145846ec1e38c5"
]
}
}
}
})
Example here.
Also you can run your query without the query parameter (in update queries the first object is the query) like this and result is the same. But is better to indicate mongo the documents using this first object.
Using MongoDB 4.2 and MongoDB Atlas to test aggregation pipelines.
I've got this products collection, containing documents with this schema:
{
"name": "TestProduct",
"relatedList": [
{id:ObjectId("someId")},
{id:ObjectId("anotherId")}
]
}
Then there's this cities collection, containing documents with this schema :
{
"name": "TestCity",
"instructionList": [
{ related_id: ObjectId("anotherId"), foo: bar},
{ related_id: ObjectId("someId"), foo: bar}
{ related_id: ObjectId("notUsefulId"), foo: bar}
...
]
}
My objective is to join both collections to output something like this (the operation is picking each related object from the instructionList in the city document to put it into the relatedList of the product document) :
{
"name": "TestProduct",
"relatedList": [
{ related_id: ObjectId("someId"), foo: bar},
{ related_id: ObjectId("anotherId"), foo: bar},
]
}
I tried using the $lookup operator for aggregation like this :
$lookup:{
from: 'cities',
let: {rId:'$relatedList._id'},
pipeline: [
{
$match: {
$expr: {
$eq: ["$instructionList.related_id", "$$rId"]
}
}
},
]
}
But it's not working, I'm a bit lost with this complex pipeline syntax.
Edit
By using unwind on both arrays :
{
{$unwind: "$relatedList"},
{$lookup:{
from: "cities",
let: { "rId": "$relatedList.id" },
pipeline: [
{$unwind:"$instructionList"},
{$match:{$expr:{$eq:["$instructionList.related_id","$$rId"]}}},
],
as:"instructionList",
}},
{$group: {
_id: "$_id",
instructionList: {$addToSet:"$instructionList"}
}}
}
I am able to achieve what I want, however,
I'm not getting a clean result at all :
{
"name": "TestProduct",
instructionList: [
[
{
"name": "TestCity",
"instructionList": {
"related_id":ObjectId("someId")
}
}
],
[
{
"name": "TestCity",
"instructionList": {
"related_id":ObjectId("anotherId")
}
}
]
]
}
How can I group everything to be as clean as stated for my original question ?
Again, I'm completely lost with the Aggregation framework.
the operation is picking each related object from the instructionList in the city document to put it into the relatedList of the product document)
Given an example document on cities collection:
{"_id": ObjectId("5e4a22a08c54c8e2380b853b"),
"name": "TestCity",
"instructionList": [
{"related_id": "a", "foo": "x"},
{"related_id": "b", "foo": "y"},
{"related_id": "c", "foo": "z"}
]}
and an example document on products collection:
{"_id": ObjectId("5e45cdd8e8d44a31a432a981"),
"name": "TestProduct",
"relatedList": [
{"id": "a"},
{"id": "b"}
]}
You can achieve try using the following aggregation pipeline:
db.products.aggregate([
{"$lookup":{
"from": "cities",
"let": { "rId": "$relatedList.id" },
"pipeline": [
{"$unwind":"$instructionList"},
{"$match":{
"$expr":{
"$in":["$instructionList.related_id", "$$rId"]
}
}
}],
"as":"relatedList",
}},
{"$project":{
"name":"$name",
"relatedList":{
"$map":{
"input":"$relatedList",
"as":"x",
"in":{
"related_id":"$$x.instructionList.related_id",
"foo":"$$x.instructionList.foo"
}
}
}
}}
]);
To get a result as the following:
{ "_id": ObjectId("5e45cdd8e8d44a31a432a981"),
"name": "TestProduct",
"relatedList": [
{"related_id": "a", "foo": "x"},
{"related_id": "b", "foo": "y"}
]}
The above is tested in MongoDB v4.2.x.
But it's not working, I'm a bit lost with this complex pipeline syntax.
The reason why it's slightly complex here is because you have an array relatedList and also an array of subdocuments instructionList. When you refer to instructionList.related_id (which could mean multiple values) with $eq operator, the pipeline doesn't know which one to match.
In the pipeline above, I've added $unwind stage to turn instructionList into multiple single documents. Afterward, using $in to express a match of single value of instructionList.related_id in array relatedList.
I believe you just need to $unwind the arrays in order to lookup the relation, then $group to recollect them. Perhaps something like:
.aggregeate([
{$unwind:"relatedList"},
{$lookup:{
from:"cities",
let:{rId:"$relatedList.id"}
pipeline:[
{$match:{$expr:{$eq:["$instructionList.related_id", "$$rId"]}}},
{$unwind:"$instructionList"},
{$match:{$expr:{$eq:["$instructionList.related_id", "$$rId"]}}},
{$project:{_id:0, instruction:"$instructionList"}}
],
as: "lookedup"
}},
{$addFields: {"relatedList.foo":"$lookedup.0.instruction.foo"}},
{$group: {
_id:"$_id",
root: {$first:"$$ROOT"},
relatedList:{$push:"$relatedList"}
}},
{$addFields:{"root.relatedList":"$relatedList"}},
{$replaceRoot:{newRoot:"$root"}}
])
A little about each stage:
$unwind duplicates the entire document for each element of the array,
replace the array with the single element
$lookup can then consider each element separately. The stages in $lookup.pipeline:
a. $match so we only unwind the document with matching ID
b. $unwind the array so we can consider individual elements
c. repeat the $match so we are only left with matching elements (hopefully just 1)
$addFields assigns the foo field retrieved from the lookup to the object from relatedList
$group collects together all of the documents with the same _id (i.e. that were unwound from a single original document), stores the first as 'root', and pushes all of the relatedList elements back into an array
$addFields moves the relatedList in to root
$replaceRoot returns the root, which should now be the original document with the matching foo added to each relatedList element
in MongoDB, I have many documents in 2-level array as below:
{
_id:1,
"toPerson": [
[
{
"userid": "test1"
},
{
"userid": "test2"
}
],
[
{
"userid": "test10"
},
{
"userid": "test11"
}
]
]
}
.....
{
_id:99,
"toPerson": [
[
{
"userid": "test2"
},
{
"userid": "test3"
}
],
[
{
"userid": "test100"
},
{
"userid": "test101"
}
]
]
}
Question is how to query all documents that have userid say test2 ?
Have tried:
col.find({'toPerson.userid':'test2'})
it's return nothing. also I have tried using aggregate but found maybe it's not the right direction.
Anyone can help with this?
UPDATE 1
Just read this post
Retrieve only the queried element in an object array in MongoDB collection
but it's different
Structure different: is {field:[ [{ }], [{ }], .... ]}, not { field:[ {}, {} ] }
I want to keep all returned documents structure untouched, $unwind(make toPerson to be 1-level array) or $$PRUNE(remove some fields) will change the structure returned.
UPDATE 2
What I want is to get following result in ONE statement:
col.find({ 'toPerson.0.userid':'test2' })
+ col.find({ 'toPerson.1.userid':'test2' })
+ ... ...
Is there any precise counterpart statement of above results combined together ?
You can query nested arrays like this using two levels of $elemMatch:
db.test.find({toPerson: {$elemMatch: {$elemMatch: {userid: 'test2'}}}})
The outer $elemMatch says match an array element of toPerson where the value passes the inner array $elemMatch test of an element matching {userid: 'test'}.
If I have the payload:
{
"objs": [
{ "_id": "1234566", "some":"data" },
{ "_id": "1234566", "some":"data" },
{ "_id": "2345666", "some":"otherdata" },
{ "_id": "4566666", "some":"yetotherdata" },
]
}
What would be the best filter to get all objects with id: "1234566"?
To find all the documents having the an obj with _id as 1234566:
db.collection.find({"objs._id":"1234566"});
To filter the obj items, having the specified _id, for the document. Assuming your document has the _id attribute.
db.collection.aggregate([
{$unwind:"$objs"},
{$match:{"objs._id":"1234566"}},
{$group:{"_id":"_id","objs":{$push:{"id":"$objs._id","some":"$objs.some"}}}},
{$project:{"_id":0,"objs":1}}
])
You can change the _id in the $group stage, if you want to group based on some different field.