MongoDB $lookup: Multiple Join Conditions on field in array - mongodb

I have an issue with joining documents in mongo based on conditions on a nested field in an array. Basically, I want to filter the documents of the foreign collection.
I was following the example on the official MongoDB documentation for Multiple Join Conditions with $lookup and extended it to fit my requirements.
Testdata (Note, the warehouse documents are extended by the field foo):
db.orders.insert([
{ "_id" : 1, "item" : "almonds", "price" : 12, "ordered" : 2 },
{ "_id" : 2, "item" : "pecans", "price" : 20, "ordered" : 1 },
{ "_id" : 3, "item" : "cookies", "price" : 10, "ordered" : 60 }
])
db.warehouses.insert([
{ "_id" : 1, "stock_item" : "almonds", warehouse: "A", "instock" : 120, 'foo': [{ 'bar': 1 }] },
{ "_id" : 2, "stock_item" : "pecans", warehouse: "A", "instock" : 80, 'foo': [{ 'bar': 1 }] },
{ "_id" : 3, "stock_item" : "almonds", warehouse: "B", "instock" : 60, 'foo': [{ 'bar': 1 }] },
{ "_id" : 6, "stock_item" : "almonds", warehouse: "A", "instock" : 61, 'foo': [{ 'bar': 2 }] },
{ "_id" : 4, "stock_item" : "cookies", warehouse: "B", "instock" : 40, 'foo': [{ 'bar': 1 }] },
{ "_id" : 5, "stock_item" : "cookies", warehouse: "A", "instock" : 80, 'foo': [{ 'bar': 1 }] }
])
On the lookup, I want to add the additional condition foo.bar: 1:
db.orders.aggregate([
{
$lookup: {
from: "warehouses",
let: { order_item: "$item", order_qty: "$ordered" },
pipeline: [{
$match: {
$expr: {
$and: [
{ $eq: [ "$stock_item", "$$order_item" ] },
{ $gte: [ "$instock", "$$order_qty" ] },
{ $eq: [ "$foo.bar", 1 ] }
]
}
}
}, {
$project: { stock_item: 0, _id: 0 }
}],
as: "stockdata"
}
}
])
Unfortunately this extra condition does not work. Furthermore not a sigle document is returned by the lookup.
Can someone point me in the right direction? I know the problem can also be done by a pipeline using unwind, filter and group.
Thanks!

Related

How to use lookup with custom condition inside subelement in MongoDB aggregation?

Helo everyone!
I have a products collection like this:
db.products.insertMany( [
{ "_id" : 1, "name" : "Apple", "variants" : [ { "_id" : 1, "name" : "Red Apple" }, { "_id" : 2, "name" : "Green Apple" }] },
{ "_id" : 2, "name" : "Banana", "variants" : [ { "_id" : 3, "name" : "Yellow Banana" }, { "_id" : 4, "name" : "Green Banana" }] },
] )
and a orders collection
db.orders.insertMany( [
{ "_id" : 1, "price" : 123, "itemId": 2},
] )
How to join products collection to orders collection by itemId (itemId == variants._id) with aggregate?
I try with this way but it's not working
db.orders.aggregate([
{
$lookup: {
from: 'products',
as: 'product',
let: { variantId: '$_id' },
pipeline: [
{
$match: {
$expr: { $eq: ['$$variantId', '$variants._id'] },
},
}
],
},
},
])
maybe issues from $expr { $eq: ['$$variantId', '$variants._id'] } but i cannot resolve it. anybody can help?
Thanks for help!

Mongo ranking results

I have a collection like
db.books.insertMany([
{"products" : [{"name": "name1", "ids": [4, 5, 6]}], "author" : "Dante", "shelf": "a" },
{ "products" : [{"name": "name1", "ids": [4, 5]}], "author" : "Homer", "shelf": "a" },
{ "products" : [{"name": "name1", "ids": [2]}], "author" : "Dante", "shelf": "b" },
])
and I want to retrieve all documents where "shelf" is 'a'
and sort by 2 conditions:
1 - by Author
2 - documents where products.ids not contains 6 should be the first.
Could anyone help?
You can try this query:
First $match the shelf value with "a".
Then create an auxiliar value where will be true if 6 not exists into products.ids, otherwise false.
Then $sort by values you want.
And use $project to remove the auxiliar value.
db.collection.aggregate([
{
"$match": {"shelf": "a"}
},
{
"$set": {
"rank": {
"$eq": [
{
"$filter": {
"input": "$products",
"cond": {"$in": [6,"$$this.ids"]}
}
},[]
]
}
}
},
{
"$sort": {
"rank": -1,
"author": 1
}
},
{
"$project": {"rank": 0}
}
])
Example here
Here is a variation that sorts more granularly on author+"not containing 6".
db.foo.aggregate([
{$match: {shelf:'a'}}
,{$unwind: '$products'}
,{$addFields: {sortMarker: {$cond: [
{$in: [6, '$products.ids']},
"Z", // THEN make sortMarker at the end
"A" // ELSE make sortMarker at the start
]}
}}
,{$sort: {'author':1, 'sortMarker':1}}
]);
which given this input set:
{"products" : [
{"name": "name3", "ids": [6, 7]},
{"name": "name2", "ids": [4, 5]}
],
"author" : "Homer",
"shelf": "a" },
{"products" : [
{"name": "name1", "ids": [4, 5, 6]},
{"name": "name4", "ids": [9]},
{"name": "name7", "ids": [9,6]},
{"name": "name7", "ids": [10]}
],
"author" : "Dante",
"shelf": "a"},
{ "products" : [
{"name": "name1", "ids": [2]}
], "author" : "Dante",
"shelf": "b"}
yields this result:
{
"_id" : 1,
"products" : {
"name" : "name4",
"ids" : [
9
]
},
"author" : "Dante",
"shelf" : "a",
"sortMarker" : "A"
}
{
"_id" : 1,
"products" : {
"name" : "name7",
"ids" : [
10
]
},
"author" : "Dante",
"shelf" : "a",
"sortMarker" : "A"
}
{
"_id" : 1,
"products" : {
"name" : "name1",
"ids" : [
4,
5,
6
]
},
"author" : "Dante",
"shelf" : "a",
"sortMarker" : "Z"
}
{
"_id" : 1,
"products" : {
"name" : "name7",
"ids" : [
9,
6
]
},
"author" : "Dante",
"shelf" : "a",
"sortMarker" : "Z"
}
{
"_id" : 0,
"products" : {
"name" : "name2",
"ids" : [
4,
5
]
},
"author" : "Homer",
"shelf" : "a",
"sortMarker" : "A"
}
{
"_id" : 0,
"products" : {
"name" : "name3",
"ids" : [
6,
7
]
},
"author" : "Homer",
"shelf" : "a",
"sortMarker" : "Z"
}
Optionally, this stage can be added after the $sort:
{$group: {_id: '$author', products: {$push: '$products'}}}
And this will bring the sorted "not containing 6 then containing 6" items together again as an array packaged by author; the $push retains the order. Note we need only need author in _id because the match was for one shelf. If more than one shelf is in the match, then we would need:
{$group: {_id: {author:'$author',shelf:'$shelf'}, products: {$push: '$products'}}}

Adding separate conditions for each item in Mongodb

I have 3 collections.
The user_movie collection keeps the relationship of the user and the movies he added to his list. The membership_date field is when the user is subscribed.
The user wants to see the reviews of the movies he added to his list.
But when showing these reviews, I want to show the comments after the subscription date.
With the query I tried, the user sees all the reviews.
Collection structure
db={
"user_movie": [
{
"_id" : 1,
"movie_id" : 1,
"user_id" : 1,
"status" : true,
"membership_date" : ISODate("2021-01-01")
},
{
"_id" : 2,
"movie_id" : 2,
"user_id" : 1,
"status" : true,
"membership_date" : ISODate("2021-01-01")
},
{
"_id" : 3,
"movie_id" : 3,
"user_id" : 1,
"status" : true,
"membership_date" : ISODate("2022-01-02")
}
],
"movie": [
{
"_id" : 1,
"movie_name" : "fugiat nulla",
},
{
"_id" : 2,
"movie_name" : "sint occaecat",
},
{
"_id" : 3,
"movie_name" : "cupidatat non",
}
],
"movie_reviews": [
{
"_id" : 1,
"movie_id" : 1,
"review": "Lorem ipsum dolor"
"review_date" : ISODate("2021-01-02"),
},
{
"_id" : 2,
"movie_id" : 2,
"review": "Consectetur adipiscing elit"
"review_date" : ISODate("2021-01-02"),
},
{
"_id" : 3,
"movie_id" : 3,
"review": "Do eiusmod tempor"
"review_date" : ISODate("2021-01-02"),
},
{
"_id" : 4,
"movie_id" : 3,
"review": "Abore et dolore magna"
"review_date" : ISODate("2022-01-01"),
}
]
}
The query I tried gives this output.
[
{
"_id" : 1,
"movie_id" : 1,
"date" : ISODate("2021-01-02"),
},
{
"_id" : 2,
"movie_id" : 2,
"date" : ISODate("2021-01-02"),
},
{
"_id" : 3,
"movie_id" : 3,
"date" : ISODate("2021-01-02"),
},
{
"_id" : 4,
"movie_id" : 3,
"date" : ISODate("2022-01-01"),
}
]
But this is not the output I was expecting. The movie reviews with id 3,4 should not be seen by the user. Because it was written before the membership_date date, so the user should not see these reviews.
How can I get all movie reviews of the user with id 1 as mentioned?
Stage 1: $lookup with pipeline
Join conditions:
By movie_id.
movie_review's review_date must not be earlier ($gte) than membership_date (from user_movie).
Stage 2: $unwind
Deconstruct movie_reviews array field to multiple document.
Stage 3: $replaceWith
Decorate the output document to show movie_review document.
db.user_movie.aggregate([
{
"$lookup": {
"from": "movie_reviews",
let: {
movie_id: "$movie_id",
membership_date: "$membership_date"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$movie_id",
"$$movie_id"
]
},
{
$gte: [
"$review_date",
"$$membership_date"
]
}
]
}
}
}
],
as: "movie_reviews"
}
},
{
$unwind: "$movie_reviews"
},
{
"$replaceWith": "$movie_reviews"
}
])
Sample Mongo Playground
Updated: Lookup join movie_reviews with user_movie
Since you are just to get movie_reviews, I think it is better to join from movie_reviews with user_movie instead of joining from user_movie with movie_reviews to get rid of the use of $unwind.
db.movie_reviews.aggregate([
{
"$lookup": {
"from": "user_movie",
let: {
movie_id: "$movie_id",
review_date: "$review_date"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$movie_id",
"$$movie_id"
]
},
{
$gte: [
"$$review_date",
"$membership_date"
]
}
]
}
}
}
],
as: "user_movie"
}
},
{
$match: {
"user_movie": {
$ne: []
}
}
},
{
$project: {
user_movie: 0
}
}
])
Sample Mongo Playground (Get movie_reviews)

MongoDB aggregation project the specific fields from lookup

This example is following https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/#use-lookup-with-mergeobjects
db.orders.insert([
{ "_id" : 1, "item" : "almonds", "price" : 12, "quantity" : 2 },
{ "_id" : 2, "item" : "pecans", "price" : 20, "quantity" : 1 }
])
db.items.insert([
{ "_id" : 1, "item" : "almonds", description: "almond clusters", "instock" : 120 },
{ "_id" : 2, "item" : "bread", description: "raisin and nut bread", "instock" : 80 },
{ "_id" : 3, "item" : "pecans", description: "candied pecans", "instock" : 60 }
])
Aggregation:
db.orders.aggregate([
{
$lookup: {
from: "items",
localField: "item", // field in the orders collection
foreignField: "item", // field in the items collection
as: "fromItems"
}
},
{
$replaceRoot: { newRoot: { $mergeObjects: [ { $arrayElemAt: [ "$fromItems", 0 ] }, "$$ROOT" ] } }
},
{ $project: { fromItems: 0 } }
])
Result:
{ "_id" : 1, "item" : "almonds", "description" : "almond clusters", "instock" : 120, "price" : 12, "quantity" : 2 }
{ "_id" : 2, "item" : "pecans", "description" : "candied pecans", "instock" : 60, "price" : 20, "quantity" : 1 }
Question: How to modify the aggregation to project the specific fields? e.g. project "_id", "item" and "description" only:
{ "_id" : 1, "item" : "almonds", "description" : "almond clusters" }
{ "_id" : 2, "item" : "pecans", "description" : "candied pecans" }
You're getting an empty array, because the $lookup catching anything.
match the types
$addFields to convert
PLAYGROUND
This should be the first stage:
{
$addFields: {
itemId: {
$convert: {
input: "$itemId",
to: "int"
}
}
}
},
If you prefer, there is no need to add a stage
You could also remove addFields and use $lookup+let.
Modify the lookup this way:
{
$lookup: {
from: "items",
let: {
itemId: {
$convert: {
input: "$itemId",
to: "int"
}
}
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$_id",
"$$itemId"
]
}
}
}
],
/** field in the items collection*/
as: "fromItems"
}
}
PLAYGROUND2

MongoDB aggregation - show values from different arrays in one array

I have this type of data
{
"_id" : 6444,
"name" : [
{
"name" : "John",
"sourcesID" : [
1,
2
]
},
{
"name" : "Jack",
"sourcesID" : [
3,
4
]
}
],
"address" : [
{
"city" : "Chicago",
"sourcesID" : [
3,
4
]
},
{
"city" : "Boston",
"sourcesID" : [
5,
6
]
}
]
}
I want to aggregate the data so that I will be able to match a certain sourcesID and find all the information types that came from this source.
This is what I am looking to achieve
{"type" : "name", "sourceID" : 1}
{"type" : "name", "sourceID" : 2}
{"type" : "name", "sourceID" : 3}
{"type" : "name", "sourceID" : 4}
{"type" : "address", "sourceID" : 3}
{"type" : "address", "sourceID" : 4}
{"type" : "address", "sourceID" : 5}
{"type" : "address", "sourceID" : 6}
Thanks for your help.
Assuming that there's always sourceID field you can run $objectToArray to transform be able to read object keys dynamically and then run $unwind three times to get single document per sourceID:
db.collection.aggregate([
{
$project: {
data: {
$filter: {
input: { $objectToArray: "$$ROOT" },
cond: {
$ne: [ "$$this.k", "_id" ]
}
}
}
}
},
{ $unwind: "$data" },
{ $unwind: "$data.v" },
{ $unwind: "$data.v.sourcesID" },
{
$project: {
_id: 0,
type: "$data.k",
sourceID: "$data.v.sourcesID"
}
}
])
Mongo Playground