I'm not sure this has been asked before, I tried looking for similar questions here on SO but unfortunately I don't find anything - perhaps for a lack of better googling skills or a lack of better wording on my part.
I have a multi-lookup query that looks like this:
const auction_id = "94a3cfb7b05c73fb5e746e21";
const record = await AuctionModel.aggregate([
{ $match: { _id: new Types.ObjectId(auction_id) } },
{
$lookup: {
from: "lots",
as: "lots",
let: { LOTID: "$_id" },
pipeline: [
{
$match: { auction_id: new Types.ObjectId(auction_id) },
},
{
$lookup: {
from: "bids",
as: "bids",
pipeline: [
{
$match: { lot_id: "$$LOTID" },
},
],
},
},
],
},
},
]);
I want to access that variable LOTID inside the inner pipeline.
Is this possible? I also tried this:
{
$lookup: {
from: "lots",
...
let: { LOTID1: "$_id" },
pipeline: [
...
{
$lookup: {
from: "bids",
...
let: { LOTID2: "$$LOTID1" },
pipeline: [
{
$match: { lot_id: "$$LOTID2" },
},
],
},
},
],
},
},
I also read here in the Mongo Docs:
https://www.mongodb.com/docs/manual/reference/operator/aggregation/lookup/#std-label-lookup-concise-correlated-subquery-let
A $match stage requires the use of an $expr operator to access the variables. The $expr operator allows the use of aggregation expressions inside of the $match syntax.
{
$lookup: {
from: "lots",
...
let: { LOTID1: "$_id" },
pipeline: [
...
{
$lookup: {
from: "bids",
...
let: { LOTID2: "$$LOTID1" },
pipeline: [
{
$match: { $expr: { $eq: ["$lot_id", "$$LOTID1"] } },
},
],
},
},
],
},
},
Also not working.
Is there any other way? Any assistance would be greatly appreciated!
Related
this.movie= {
name:"",
playlist:{
category:ObjectId
}
}
$lookup:{
from: 'categories',
let: { catId: '$movie.playlist.category' },
pipeline: [
{
$match: {
$expr: {
$and:[
{$eq:["$_id","$$catId"]}
]
},
},
},
],
as: 'category',
}
it's returning an empty array to the aggregated field category, could someone point out where I went wrong with this ?
I'm trying to do a conditional lookup with aggregration framework. In my local and foreign collections I have two fields: fieldA and fieldB. If fieldA != 0 my $lookup should be:
{ from: 'collectionA', localField: 'fieldA', foreignField: 'fieldA', as: 'agg' }
Otherwise, if fieldA = 0 my $lookup should be:
{ from: 'collectionA', localField: 'fieldB', foreignField: 'fieldB', as: 'agg' }
Is it possible to combine these conditions with a single $lookup?
It's not really possible OOB, but you can work around this in multiple ways.
for example add a new "temporary" field based on this condition:
db.colleciton.aggregate([
{
$lookup: {
from: 'collectionA',
let: {
fieldA: "$fieldA", fieldB: "$fieldB"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
{
$cond: [
{
$eq: [
'$$fieldA',
0,
],
},
'$$fieldB',
'$$fieldA',
],
},
{
$cond: [
{
$eq: [
'$$fieldA',
0,
],
},
'$fieldB',
'$fieldA',
],
}
],
},
},
},
],
as: 'agg',
},
}
])
The issue with this approach is that indexes won't be utilized for the lookup for older Mongo versions, which in some cases can be crucial.
You can work around for performance purposes like so:
db.collection.aggregate([
{
$facet: {
one: [
{
$match: {
fieldA: { $ne: 0 },
},
},
{
$lookup: { from: 'collectionA', localField: 'fieldA', foreignField: 'fieldA', as: 'agg' },
},
{
$match: {
'agg.0': { $exists: true },
},
},
],
two: [
{
$match: {
fieldA: { $eq: 0 },
},
},
{
$lookup: { from: 'collectionA', localField: 'fieldB', foreignField: 'fieldB', as: 'agg' },
},
{
$match: {
'agg.0': { $exists: true },
},
},
],
},
},
{
$addFieldS: {
combined: {
$concatArrays: [
'$one',
'$two',
],
},
},
},
{
$unwind: '$combined',
},
{
$replaceRoot: {
newRoot: "$combined"
},
},
]);
While there is some overhead here it will still work faster than an unindexed lookup.
Here is a link to understand the structure : https://mongoplayground.net/p/TUsBjiboKLk
I'm basically looking for the communities that the user is following which work just fine in mongoplayground.
But replacing the "userId1" with a custom javascript variable is not working.
pipeline: [
{
$match: {
userId: "userId1", // <================
$expr: {
$eq: [
"$userId",
"$$userId"
],
},
},
},
],
pipeline: [
{
$match: {
userId: myCustomVarible, // <================
$expr: {
$eq: [
"$userId",
"$$userId"
],
},
},
},
],
Here is the full code for further explanation:
app.get("/communities/following", (req, res) => {
const userId = req.query.userId;
Community.aggregate(
[
{
$lookup: {
from: "follows",
localField: "communityId",
foreignField: "communityId",
as: "follows",
},
},
{
$unwind: "$follows",
},
{
$lookup: {
from: "users",
let: {
userId: "$follows.userId",
},
pipeline: [
{
$match: {
userId: userId,
$expr: {
$eq: ["$userId", "$$userId"],
},
},
},
],
as: "users",
},
},
{
$unwind: "$users",
},
{
$project: {
name: 1,
avatar: 1,
},
},
],
(err, data) => {
if (err) res.status(500).send(err);
else res.status(201).send(data);
}
);
});
If i replace const userId = req.query.userId; width const userId = "myUserId"; it's working
Ok, i found the problem, i was testing this using :
http://localhost:9000/communities/following?userId="7arvEbQx3wVttqnbwmOionRsPBz1"
where it should be without the quotes
http://localhost:9000/communities/following?userId=7arvEbQx3wVttqnbwmOionRsPBz1
Like if i'm applying many lookup stages in aggregation pipeline and each lookup is followed by an unwind(just to covert into object) first question does it affect query performance? and if yes how to do that in optimised manner
Note: all lookup's will return only one object
For Ex:
xyz.aggregate([
{ $lookup:{ ----}} //first lookup
{$unwind :{----}} //first unwind
{ $lookup:{ ----}} //second lookup
{$unwind :{----}} //second unwind
{ $lookup:{ ----}} //third lookup
{$unwind :{----}} //third unwind
{ $lookup:{ ----}} //fourth lookup
{$unwind :{----}} //fourth unwind
])
In reference to comments, here is advanced $lookup:
$lookup: {
from: 'accounts',
let: { "localAccountField": "$account" },
pipeline: [
{
$match: {
$expr: {
$eq: ["$_id", "$$localAccountField"]
}
}
},
{
$project: {
_id: 1,
user: 1
}
},
{
$lookup: {
from: 'users',
let: { 'localUserField': "$user" },
pipeline: [
{
$match: {
$expr: {
$eq: ["$_id", "$$localUserField"]
}
}
},
{
$project: {
_id: 1,
username: "$uid",
phone:"$phoneNumber",
email: "$email.add",
name: {
$concat: [
"$profile.name.first",
' ',
"$profile.name.last"
]
},
}
}
],
as: "users"
}
},
{
$lookup: {
from: 'documents',
let: { 'localDocumentField': "$user" },
pipeline: [
{
$match: {
$expr: {
$eq: ["$user", "$$localDocumentField"]
},
status:"verified",
"properties.expirydate": { $exists: true, $ne: "" },
name: "idcard"
}
},
{
$project: {
_id: 0,
cnic: "$properties.number"
}
}
],
as: "documents"
}
}
],
as: 'account'
}
I have next mongo code:
db.users.aggregate([
{
$match: {
$and: [
{ UserName: { $eq: 'administrator' } },
{ 'Company.CompanyName': { $eq: 'test' } }
]
}
},
{
$lookup: {
from: "companies",
localField: "CompanyID",
foreignField: "CompanyID",
as: "Company"
}
},
])
The $lookup part of the code working great. I got next result:
But if I add $match to the code, it brings nothing.
I found that the problem is in the second match: { 'Company.CompanyName': { $eq: 'test' } }, but I can not realize what is wrong with it.
Any ideas?
UPDATE:
I had also tried $unwind on the $lookup result, but no luck:
db.users.aggregate([
{
$match: {
$and: [
{ UserName: { $eq: 'administrator' } },
{ 'Company.CompanyName': { $eq: 'edt5' } }
]
}
},
{ unwind: '$Company' },
{
$lookup: {
from: 'companies',
localField: 'CompanyID',
foreignField: 'CompanyID',
as: 'Company'
}
},
])
With MongoDB 3.4, you can run an aggregation pipeline that uses the $addFields pipeline and a $filter operator to only return the Company array with elements that match the given condition. You can then wrap the $filter expression with the $arrayElemAt operator to return a single document which in essence incorporates the $unwind functionality by flattening the array.
Follow this example to understand the above concept:
db.users.aggregate([
{ "$match": { "UserName": "administrator" } },
{
"$lookup": {
"from": 'companies',
"localField": 'CompanyID',
"foreignField": 'CompanyID',
"as": 'Company'
}
},
{
"$addFields": {
"Company": {
"$arrayElemAt": [
{
"$filter": {
"input": "$Company",
"as": "comp",
"cond": {
"$eq": [ "$$comp.CompanyName", "edt5" ]
}
}
}, 0
]
}
}
}
])
Below answer is for mongoDB 3.6 or later.
Given that:
You have a collection users with a field CompanyID and a collection of companies with a field CompanyID
you want to lookup Companies on Users by matching CompanyID, where additionally:
each User must match condition: User.UserName equals administrator
each Company on User must match condition: CompanyName equals edt5
The following query will work for you:
db.users.aggregate([
{ $match: { UserName: 'administrator' } },
{
$lookup: {
from: 'companies',
as: 'Company',
let: { CompanyID: '$CompanyID' },
pipeline: [
{
$match: {
$expr: {
$and: [
{ $eq: ['$CompanyID', '$$CompanyID'] },
{ $eq: ['$CompanyName', 'edt5'] },
]
}
}
}
]
}
},
])
Explanation:
This is the way to perform left join queries with conditions more complex than simple foreign / local field equality match.
Instead of using localField and foreignField, you use:
let option where you can map local fields to variables,
pipeline option where you can specify aggregation Array.
In pipeline you can use $match filter, with $expr, where you can reuse variables defined earlier in let.
More info on $lookup
Nice tutorial
here is code for fitering array inside lookup.
const userId = req.userData.userId;
const limit = parseInt(req.params.limit);
const page = parseInt(req.params.page);
Collection.aggregate([
{ $match: {} },
{ $sort: { count: -1 } },
{ $skip: limit * page },
{ $limit: limit },
{
$lookup: {
from: Preference.collection.name,
let: { keywordId: "$_id" },
pipeline: [
{
$match: {
$expr: {
$and: [
{ $eq: ["$keyword", "$$keywordId"] },
{
$eq: ["$user", mongoose.Types.ObjectId(userId)],
},
],
},
},
},
],
as: "keywordData",
},
},
{
$project: {
_id: 0,
id: "$_id",
count: 1,
for: 1,
against: 1,
created_at: 1,
updated_at: 1,
keyword: 1,
selected: {
$cond: {
if: {
$eq: [{ $size: "$keywordData" }, 0],
},
then: false,
else: true,
},
},
},
}])