I have an aggregate query where I join 3 collections. I'd like to filter the search based on fields from two of those collections. The problem is, I'm only able to use $match on the initial collection that mongoose initialized with.
Here's the query:
var pipeline = [
{
$lookup: {
from: 'blurts',
localField: 'followee',
foreignField: 'author.id',
as: 'followerBlurts'
}
},
{
$unwind: '$followerBlurts'
},
{
$lookup: {
from: 'users',
localField: 'followee',
foreignField: '_id',
as: 'usertbl'
}
},
{
$unwind: '$usertbl'
},
{
$match: {
'follower': { $eq: req.user._id },
//'blurtDate': { $gte: qryDateFrom, $lte: qryDateTo }
}
},
{
$sample: { 'size': 42 }
},
{
$project: {
_id: '$followerBlurts._id',
name: '$usertbl.name',
smImg: '$usertbl.smImg',
text: '$followerBlurts.text',
vote: '$followerBlurts.vote',
blurtDate: '$followerBlurts.blurtDate',
blurtImg: '$followerBlurts.blurtImg'
}
}
];
keystone.list('Follow').model.aggregate(pipeline)
.sort({blurtDate: -1})
.cursor().exec()
.toArray(function(err, data) {
if (!err) {
res.json(data);
} else {
console.log('Error getting following blurts --> ' + err);
}
});
Within the pipeline, I can only use $match on the 'Follow' model. When I use $match on the 'Blurt' model, it simply ignores the condition (you can see where I tried to include it in the commented line under $match).
What's perplexing is that I can utilize this field in the .sort method, but not in the $match conditions.
Any help much appreciated.
You can use the mongo dot notation to access elements of the collection that is being looked up via $lookup.
https://docs.mongodb.com/manual/core/document/#dot-notation
So, in this case followerBlurts.blurtDate should give you the value you are looking for.
Related
I have two collections that I want to join with $lookup based on two id fields. Both fields are from type guid and looke like this in mongodb compass: 'Binary('cavTZa/U2kqfHtf08sI+Fg==', 3)'
This syntax in the compass aggregation pipeline builder gives the expected result:
{
from: 'clients',
localField: 'ClientId',
foreignField: '_id',
as: 'ClientData'
}
But i want to add some projection and tried to change it like this:
{
from: 'clients',
'let': {
id: '$_id.clients'
},
pipeline: [
{
$match: {
$expr: {
$eq: [
'$ClientId',
'$$id'
]
}
}
},
{
$project: {
Name: 1,
_id: 0
}
}
],
as: 'ClientData'
}
But the result here is that every client from collection 'clients' is added to every document in the starting table. I have to use MongoDB 3.6 so the new lookup syntax from >=5.0 is not available.
Any ideas for me? Does $eq work for binary stored guid data?
In the first example, you say that the local field is ClientId and the foreign field is _id. But that's not what you used in your second example.
This should work better:
{
from: 'clients',
'let': {
ClientId: '$ClientId'
},
pipeline: [
{
$match: {
$expr: {
$eq: [
'$$ClientId',
'$_id'
]
}
}
},
{
$project: {
Name: 1,
_id: 0
}
}
],
as: 'ClientData'
}
I have two collections:
cats
balls
"cats" collection has documents with key "ballId" of type string
"balls" collection has documents with key "_id" of type ObjectId
An $lookup inside an aggregation is able to retrieve results if the join is done on keys with the same data type. However in my case, "ballId" and "_id" are of different types. This code retrieves the cats but doesn't retrieve the related balls:
collection('cats').aggregate([
{ $match:{} },
{
$lookup: {
from: "balls",
localField: "ballId",
foreignField: "_id",
as: "balls"
}
}
]);
How can I use $lookup with lossy data type?
Use $lookup with pipeline stage.
Join both collections by converting balls' _id to string ($toString) and next compare both values as string ($eq).
db.cats.aggregate([
{
$match: {}
},
{
$lookup: {
from: "balls",
let: {
ballId: "$ballId"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
{
"$toString": "$_id"
},
"$$ballId"
]
},
}
}
],
as: "balls"
}
}
])
Sample Mongo Playground
i have a collection named 'crew'
{
_id:
name:
description:
person_id:
}
and collection named 'person'
{
_id:
name:
description:
}
My goal is to make recommend system by searching crew.name, crew.description and person.name.
first, i create index for each collection ( collection > search indexes ), and i use aggregate as follows.
db.crew.aggregate([
{
$lookup: {
from: 'person',
localField: 'person_id',
foreignField: '_id',
as: 'person_info'
}
}
, {
$unwind: {
path: '$person_info'
}
}
, {
$project: {
'crewId': '$_id',
'crewName': '$name',
'crewDescription': '$description',
'personName': '$person_info.name'
}
}
, {
$search: {
"text": {
"query": "SEARCH_WORD",
"path": ["crewName","crewDescription","personName"]
}
}
}
])
as a resutl I got a error message as follows.
MongoError: $_internalSearchBetaMongotRemote is only valid as the first stage in a pipeline.
Is it impossible to use "$search in $lookup result" in mongodb atalas?
Yes it is impossible to use $search with $lookup. As per documentation,
$search must be the first stage of any pipeline it appears in. $search cannot be used in:
a view definition
a $lookup subpipeline
a $facet pipeline stage
I'm trying to get a department by it's _id from collection in Mongodb using $lookup and $match with the _id by receiving the _id from the request parameters.
However I get an empty array as my result, and when I change the request.params.id with static value like 2 or 1 it returns the data with no problem .. any solution ?
DepartmentRouter.get("/departments/:id", (request, response) => {
departmentSchema.aggregate([
{
$lookup: {
from: "students",
localField: "_id",
foreignField: "Department",
as: "studentsObject"
}
},
{
$match: {
_id: request.params.id
}
}
]).then(data => {
response.send(data);
}).catch(error => {
response.send(error);
});
});
request.params.id comes as string (assuming you don't use any specific middleware to re-cast it).
All you have to do is cast is to an ObjectId type. like so:
var ObjectID = require('mongodb').ObjectID
{
$match: {
_id: new ObjectId(request.params.id)
}
}
Also a performance tip, Move the $match stage to be before the $lookup stage so you don't perform a lookup (which is expensive) to all the redundant documents.
guys i found the solution , i had to make a new number object using Number
$match: {
_id: Number(request.params.id)
}
Aggregate1:
db.collection.aggregate([
{
$lookup: {
...
}
},
{
$limit: 1
}
])
Aggregate2:
db.collection.aggregate([
{
$limit: 1
},
{
$lookup: {
...
}
}
])
Aggregate1 and Aggregate2 are different?
In Aggregate1,is the whole collection scanned firstly, then do $lookup?
If it is different,how to lookup with some query?just like this:
db.collection.aggregate([
{
$lookup: {
from: 'collection2',
localField: 'field',
foreignField: 'field',
as: 'newField',
// do some query when lookup
query: {'newField.xxx': 1}
}
}
])
I know, i can do this:
db.collection.aggregate([
{
$lookup: {
from: 'collection2',
localField: 'field',
foreignField: 'field',
as: 'newField'
}
},
{$unwind: '$newField'},
{$match: {'newField.xxx': 1}}
])
But I'm afraid that, like the example above, the entire collection will be scanned。
Look forward to your reply!
Now,i find this api:$graphLookup.restrictSearchWithMatch,but:
NOTE
You cannot use any aggregation expression in this filter.
For example, a query document such as
{ lastName: { $ne: "$lastName" } }
will not work in this context to find documents in which the lastName
value is different from the lastName value of the input document,
because "$lastName" will act as a string literal, not a field path.