Related
I have documents like this in DB. And I need to grab the data of each item based on the itemType from their own collection.
{ listId: 2, itemType: 'book', itemId: 5364 },
{ listId: 2, itemType: 'car', itemId: 354 },
{ listId: 2, itemType: 'laptop', itemId: 228 }
Based on MongoDB docs and some search, I figured out that I need to use let and $expr in lookup, to make some condition.
ListItemsModel.aggregate([
{ $match: { listId: 2 } },
{ $lookup:
{
from: 'books',
localField: 'itemId',
foreignField: '_id',
let: { "itemType": "$itemType" },
pipeline: [
{ $project: { _id: 1, title: 1 }},
{ $match: { $expr: { $eq: ["$$itemType", "book"] } }}
],
as: 'data'
}
},
{ $lookup:
{
from: 'cars',
localField: 'itemId',
foreignField: '_id',
let: { "itemType": "$itemType" },
pipeline: [
{ $project: { _id: 1, title: 1 }},
{ $match: { $expr: { $eq: ["$$itemType", "car"] } }}
],
as: 'data'
}
},
{ $lookup:
{
from: 'laptops',
localField: 'itemId',
foreignField: '_id',
let: { "itemType": "$itemType" },
pipeline: [
{ $project: { _id: 1, title: 1 }},
{ $match: { $expr: { $eq: ["$$itemType", "laptop"] } }}
],
as: 'data'
}
}
]);
The problem is, in the result all data fields are empty as data: [].
The syntax seems correct to me. What's wrong?
Any subsequent reassignment of field values will eliminate any previous value.
So, for your aggregation pipeline, you need to assign different values to each "$lookup" "as" field.
For example:
// ...
{ $lookup:
{
from: 'books',
// ...
as: 'booksData'
}
},
{ $lookup:
{
from: 'cars',
// ...
as: 'carsData'
}
},
{ $lookup:
{
from: 'laptops',
// ...
as: 'laptopsData'
}
},
// ...
I have two collections. articles and bookmarks.
articles
{
_id: "5faa889ade5e0a6326a873d3",
name: "article 1"
},
{
_id: "5faa889ade5e0a6326a873d",
name: "article 2"
}
bookmarks
{
_id: "5faa889ade5e0a6326a873d1",
user_id: "5fc7b50da483a66a86aa7e9e",
model_id: "5faa889ade5e0a6326a873d3"
}
I want to join article with bookmark. if user bookmarked a article.
what i have tried
const aggregate = await Articles.aggregate([{
$lookup: {
from: "categories",
localField: "category_id",
foreignField: "_id",
as: "category_id"
}
},
{
$lookup: {
from: "bookmarks",
localField: "_id",
foreignField: "model_id",
as: "bookmarks"
}
}
]);
but it will gives all bookmark for the article not only logged in user bookmark. so how can I add a condition.
{ "user_id": objectId(req.user._id) } // logged user id
You can use $lookup with pipeline starting from MongoDB v3.6,
let to pass localField _id as model_id variable, you can use the field inside lookup pipeline using $$ reference,
pipeline to put $match stage and match your required conditions and user_id condition
{
$lookup: {
from: "bookmarks",
let: { model_id: "$_id" },
pipeline: [
{
$match: {
$expr: { $eq: ["$$model_id", "$model_id"] },
user_id: objectId(req.user._id)
}
}
],
as: "bookmarks"
}
}
Other option for MongoDB v3.4,
$filter to iterate loop of bookmarks and get filtered bookmarks on the base of condition
{
$lookup: {
from: "bookmarks",
localField: "_id",
foreignField: "model_id",
as: "bookmarks"
}
},
{
$addFields: {
bookmarks: {
$filter: {
input: "$bookmarks",
cond: { $eq: ["$$this.user_id", objectId(req.user._id)] }
}
}
}
}
You can have nested pipeline inside $lookup,
db.articles.aggregate([
{
$lookup: {
from: "bookmarks",
let: {
article_id: "$_id"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$model_id",
"$$article_id"
]
},
{
$eq: [
"$user_id",
"5fc7b50da483a66a86aa7e9a"
]
}
]
}
}
}
],
as: "bookmarks"
}
}
])
Here's a working playground
I'm trying to set up a query on the mongo that searches in 3 different collections.
Document of client:
{
"_id": ObjectId("1a")
"razaosocial:"32423424",
"prepository": [
{
"$ref": "prepository",
"$id": ObjectId("2a")
}
]
}
Document of prepository:
{
"_id": ObjectId("2a")
"name:"Jonh",
"prepository": {
"$ref": "representative",
"$id": ObjectId("3a")
}
}
Document of representative:
{
"_id": ObjectId("3a")
"name:"Josh"
}
I'm doing it this way, but it doesn't return anything:
db.clients.aggregate(
[
{
$lookup: {
from: 'prepository',
localField: 'prepository',
foreignField: 'id',
as: 'prepository'
}
},
{ $unwind: "$prepository" },
{ $lookup: {
from: 'representative',
localField: 'id',
foreignField: 'prepository._id',
as: 'prepository.repre'
}
},
{ $group: {
_id: "$_id",
client: { $first: "$razaosocial" },
users: {
$push: "$prepository"
}
} }
])
I'm trying to return
{
"_id": "1a"
"razaosocial:"32423424",
"prepository": [
{
"_id": "2a"
"name:"Jonh",
"representative": {
"_id": "3a"
"name:"Josh"
}
}
]
}
I am grateful for any help
You can use nested lookup,
$lookup with prepository collection
$match with prepository's id
$lookup with representative collection
$unwind deconstruct representative array
db.client.aggregate([
{
$lookup: {
from: "prepository",
as: "prepository",
let: { prepository: "$prepository" },
pipeline: [
{ $match: { $expr: { $in: ["$_id", "$$prepository"] } } },
{
$lookup: {
from: "representative",
localField: "representative",
foreignField: "_id",
as: "representative"
}
},
{ $unwind: "$representative" }
]
}
}
])
Playground
I have an aggregate call as listed below. I believe the massive delay is caused when the roles lookup happens because there are applicants, hired, and rejected people to lookup from the resources table. There are over 10,000 entries in the resource collection so the query is taking about 6 seconds. Is there something I am doing that is incredibly wrong here? I don't see how I can use indexes because all of the lookups are done by with the _id which is already indexed by default.
The applications, hired, and rejected fields are just arrays of object Ids e.g.
applicants: [
ObjectId('asldkajsdlkj'),
ObjectId('asldkjaoksdjak')
]
Any help would be greatly appreciated.It was taking 6 seconds on an M0 instance and is no faster on an M10 instance.
return db.collection('projects').aggregate([
{
$match: {
agents: ObjectId('SOMETHING')
}
},
{
$lookup: {
from: "agents",
localField: "agents",
foreignField: "_id",
as: "agents"
}
},
{
$lookup: {
from: "agencies",
localField: "agency",
foreignField: "_id",
as: "agency"
}
},
{
$lookup: {
from: "roles",
let: { "roles": "$roles" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$roles" ] } } },
{
$lookup: {
from:"resources",
let: { "applicants": "$applicants" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$applicants" ] } } }
],
as: "applicants"
}
},
{
$lookup: {
from: "resources",
let: { "hired": "$hired" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$hired" ] } } }
],
as: "hired"
}
},
{
$lookup: {
from: "resources",
let: { "rejected": "$rejected" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$rejected" ] } } }
],
as: "rejected"
}
},
{
$lookup: {
from: "agents",
localField: "hiring_agent",
foreignField: "_id",
as: "hiring_agent"
}
}
],
as: "roles"
}
}
], {
allowDiskUse: true
})
MongoDB is designed to store "boiled" data. It means, in project collection, you need to store redundant information to transform into desired result.
In your case, performance decreases with inner $lookup with let - pipeline for role collection due to this error.
Try to change:
db.collection('projects').aggregate([
{
$match: {
agents: ObjectId('SOMETHING')
}
},
{
$lookup: {
from: "agents",
localField: "agents",
foreignField: "_id",
as: "agents"
}
},
{
$lookup: {
from: "agencies",
localField: "agency",
foreignField: "_id",
as: "agency"
}
},
{
$lookup: {
from: "roles",
let: { "roles": "$roles" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$roles" ] } } },
{
$lookup: {
from:"resources",
localField: "applicants",
foreignField: "_id",
as: "applicants"
}
},
{
$lookup: {
from: "resources",
localField: "hired",
foreignField: "_id",
as: "hired"
}
},
{
$lookup: {
from: "resources",
localField: "rejected",
foreignField: "_id",
as: "rejected"
}
},
{
$lookup: {
from: "agents",
localField: "hiring_agent",
foreignField: "_id",
as: "hiring_agent"
}
}
],
as: "roles"
}
}
], {
allowDiskUse: true
})
I have the following mongo aggregate query:
return db.collection('projects').aggregate([
{
$match: {
agents: ObjectId(agent)
}
},
{
$lookup: {
from: "agents",
localField: "agents",
foreignField: "_id",
as: "agents"
}
},
{
$lookup: {
from: "roles",
localField: "roles",
foreignField: "_id",
as: "roles"
}
},
{
$lookup: {
from: "agencies",
localField: "agency",
foreignField: "_id",
as: "agency"
}
},
{
$lookup: {
from: "resources",
localField: "roles.applicants",
foreignField: "_id",
as: "roles.applicants"
}
}
])
It works as it should, embedding the proper documents. However, the "password_hash" field is showing for each applicant. I want to remove that field. If I try to project and set roles.applicants.password_hash: 0 I actually end up getting the entire applicant without the password hash, but the rest of the roles fields are no longer there. So I get something that looks like:
roles: {
applicants: {
name: "Josh"
}
}
That should be
roles: {
title: "Super Hero",
applicants: {
name: "Josh"
}
}
I figured it out. Here is how I did it. First, the projection wasn't the issue with why the roles document was missing fields. It was the lookup.
What I did was changed the roles lookup to use the pipeline method on the nested documents and then did another pipeline on applicants, which first matched the applicants from the resource collection and then handled the projection. It looks like this.
{
$lookup: {
from: "roles",
let: { "roles": "$roles" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$roles" ] } } },
{
$lookup: {
from: "resources",
let: { "applicants": "$applicants" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$applicants" ] } } },
{
$project: {
first_name: 1
}
}
],
as: "applicants"
}
}
],
as: "roles"
}
}
The entire aggregate looks like this
return db.collection('projects').aggregate([
{
$match: {
agents: ObjectId(agent)
}
},
{
$lookup: {
from: "agents",
localField: "agents",
foreignField: "_id",
as: "agents"
}
},
{
$lookup: {
from: "agencies",
localField: "agency",
foreignField: "_id",
as: "agency"
}
},
{
$lookup: {
from: "roles",
let: { "roles": "$roles" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$roles" ] } } },
{
$lookup: {
from: "resources",
let: { "applicants": "$applicants" },
pipeline: [
{ $match: { $expr: { $in: [ "$_id", "$$applicants" ] } } },
{
$project: {
first_name: 1
}
}
],
as: "applicants"
}
}
],
as: "roles"
}
}
])