I'm trying to do an aggregation on two collections that has a linkage between them, and I need to access information in an array of objects in one of those collections.
Here are the schemas:
User Schema:
{
_id: ObjectId,
username: String,
password: String,
associatedEvents: [
{
event_id: ObjectId,
isCreator: boolean,
access_level: String,
}
]
}
Event Schema:
{
_id: ObjectId,
title: String,
associated_users: [
{
user_id: ObjectId
}
]
}
I'm attempting to get the users associated to an event for a specific user, and then get their access level information. Here's the aggregation I have:
const eventsJoined = await Event.aggregate([
{
$match: {
$expr: { $in: [id, "$associatedUserIds"] },
},
},
{
$lookup: {
from: "users",
localField: "associatedUserIds",
foreignField: "_id",
as: "user_info",
},
},
{ $unwind: "$user_info" },
{
$unwind: {
path: "$user_info.associatedEvents",
preserveNullAndEmptyArrays: true,
},
},
{
$group: {
_id: "$_id",
title: { $first: "$title" },
description: { $first: "$description" },
startDate: { $first: "$startdate" },
userInfo: { $first: "$user_info" },
usersAssociatedEvents: { $push: "$user_info.associatedEvents" },
},
},
{
$project: {
title: 1,
description: 1,
startDate: 1,
userInfo: 1,
usersAssociatedEvents: "$usersAssociatedEvents",
},
},
]);
And this is the result I'm getting:
[
{
_id: 609d5ad1ef4cdbeb32987739,
title: 'hello',
description: 'desc',
startDate: null,
usersAssociatedEvents: [ [Object] ]
}
]
As you can see, the query is already aggregating the correct data. But the last thing that's tripping me up is the fact that the aggregation is [ [Object] ] for usersAssociatedEvents instead of the actual contents of the object. Any idea on why that would be?
Related
I have the following collections:
const movieSchema = new Schema({
title: String
...
})
const userSchema = new Schema({
firstName: String,
lastName: String,
movies: [
movie: {
type: Schema.Types.ObjectId,
ref: 'Movie'
},
status: String,
feeling: String
]
...
})
I am trying to match up the movie (with all its details) with the user status and feeling for that movie, with the aggregation:
Movie.aggregate([
{ $match: { _id: ObjectId(movieId) } },
{
$lookup: {
from: 'users',
as: 'user_status',
pipeline: [
{ $match: { _id: ObjectId(userId) } },
{
$project: {
_id: 0,
movies: 1
}
},
{ $unwind: '$movies' }
]
}
},
])
Which returns:
[
{
_id: 610b678702500b0646925542,
title: 'The Shawshank Redemption',
user_status: [
{
"movies": {
"_id": "610b678702500b0646925542",
"status": "watched",
"feeling": "love"
}
},
{
"movies": {
"_id": "610b678502500b0646923627",
"status": "watched",
"feeling": "like"
}
},
{
"movies": {
"_id": "610b678502500b0646923637",
"status": "watched",
"feeling": "like"
}
},
]
}
]
My desired result is to match the first movie in user_status to get the eventual final result:
[
{
_id: 610b678702500b0646925542,
title: 'The Shawshank Redemption',
status: "watched",
feeling: "love"
}
]
I thought the next step in my pipeline would be:
{
$addFields: {
user_status: {
$filter: {
input: '$user_status',
cond: {
$eq: ['$$this.movies._id', '$_id']
}
}
}
}
}
But it doesn't work - Not sure if this $addFields is correct, and one problem I know is that my first _id is an ObjectId and the second appears to be a string.
If I understand correctly, you can $filter the user in the already existing $lookup pipeline, which will make things more simple later:
db.movies.aggregate([
{$match: {_id: ObjectId(movieId)}},
{
$lookup: {
from: "users",
as: "user_status",
pipeline: [
{$match: {_id: ObjectId(userId)}},
{$project: {
movies: {
$first: {
$filter: {
input: "$movies",
cond: {$eq: ["$$this.movie", ObjectId(movieId)]}
}
}
}
}
}
]
}
},
{
$project: {
title: 1,
feeling: {$first: "$user_status.movies.feeling"},
status: {$first: "$user_status.movies.status"}
}
}
])
See how it works on the playground example
Hello I have the following collections
const TransactionSchema = mongoose.Schema({
schedule: {
type: mongoose.Schema.ObjectId,
required: true,
ref: "Schedule"
},
uniqueCode: {
type: String,
required: true
},
created: {
type: Date,
default: Date.now
},
status: {
type: String,
required: false
},
})
const ScheduleSchema = mongoose.Schema({
start: {
type: Date,
required: true,
},
end: {
type: Date,
required: false,
},
location: {
type: mongoose.Schema.ObjectId,
required: true,
ref: "Location"
},
})
and I want to return how many times the schedule appear in transaction ( where the status is equal to 'Active') and group it based on its location Id and then lookup the location collection to show the name.
For example I have the following data.
transaction
[
{
"_id":"identifier",
"schedule":identifier1,
"uniqueCode":"312312312312",
"created":"Date",
"status": 'Active'
},
{
"_id":"identifier",
"schedule":identifier1,
"uniqueCode":"1213123123",
"created":"Date",
"status": "Deleted"
}
]
schedule
[
{
"_id":identifier1,
"start":"date",
"end":"date",
"location": id1
},
{
"_id":identifier2,
"start":"date",
"end":"date",
"location": id2
}
]
and I want to get the following result and limit the result by 10 and sort it based on its total value:
[
{
"locationName":id1 name,
"total":1
},
{
"locationName":id2 name,
"total":0
}
]
thank you. Sorry for my bad english.
A bit complex and long query.
$lookup - schedule collection joins with transaction collection by matching:
_id (schedule) with schedule (transaction)
status is Active
and return a transactions array.
$lookup - schedule collection joins with location collection to return location array.
$set - Take the first document in location array so this field would be a document field instead of an array. [This is needed to help further stage]
$group - Group by location._id. And need the fields such as location and total.
$sort - Sort by total DESC.
$limit - Limit to 10 documents to be returned.
$project - Decorate the output documents.
db.schedule.aggregate([
{
$lookup: {
from: "transaction",
let: {
scheduleId: "$_id"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$schedule",
"$$scheduleId"
]
},
{
$eq: [
"$status",
"Active"
]
}
]
}
}
}
],
as: "transactions"
}
},
{
$lookup: {
from: "location",
localField: "location",
foreignField: "_id",
as: "location"
}
},
{
$set: {
location: {
$first: "$location"
}
}
},
{
$group: {
_id: "$location._id",
location: {
$first: "$location"
},
total: {
$sum: {
$size: "$transactions"
}
}
}
},
{
$sort: {
"total": -1
}
},
{
$limit: 10
},
{
$project: {
_id: 0,
locationName: "$location.name",
total: 1
}
}
])
Sample Mongo Playground
So I have 2 models user & form.
User Schema
firstName: {
type: String,
required: true,
},
lastName: {
type: String,
required: true,
},
email: {
type: String,
required: true,
}
Form Schema
approvalLog: [
{
attachments: {
type: [String],
},
by: {
type: ObjectId,
},
comment: {
type: String,
},
date: {
type: Date,
},
},
],
userId: {
type: ObjectId,
required: true,
},
... other form parameters
When returning a form, I'm trying to aggregate the user info of every user in the approvalLog into their respective objects as below.
{
...other form info
approvalLog: [
{
attachments: [],
_id: '619cc4953de8413b548f61a6',
by: '619cba9cd64af530448b6347',
comment: 'visit store for disburement',
date: '2021-11-23T10:38:13.565Z',
user: {
_id: '619cba9cd64af530448b6347',
firstName: 'admin',
lastName: 'user',
email: 'admin#mail.com',
},
},
{
attachments: [],
_id: '619cc4ec3ea3e940a42b2d01',
by: '619cbd7b3de8413b548f61a0',
comment: '',
date: '2021-11-23T10:39:40.168Z',
user: {
_id: '619cbd7b3de8413b548f61a0',
firstName: 'sam',
lastName: 'ben',
email: 'sb#mail.com',
},
},
{
attachments: [],
_id: '61a9deab8f472c52d8bac095',
by: '61a87fd93dac9b209096ed94',
comment: '',
date: '2021-12-03T09:08:59.479Z',
user: {
_id: '61a87fd93dac9b209096ed94',
firstName: 'john',
lastName: 'doe',
email: 'jd#mail.com',
},
},
],
}
My current code is
Form.aggregate([
{
$lookup: {
from: 'users',
localField: 'approvalLog.by',
foreignField: '_id',
as: 'approvedBy',
},
},
{ $addFields: { 'approvalLog.user': { $arrayElemAt: ['$approvedBy', 0] } } },
])
but it only returns the same user for all objects. How do I attach the matching user for each index?
I've also tried
Form.aggregate([
{
$lookup: {
from: 'users',
localField: 'approvalLog.by',
foreignField: '_id',
as: 'approvedBy',
},
},
{
$addFields: {
approvalLog: {
$map: {
input: { $zip: { inputs: ['$approvalLog', '$approvedBy'] } },
in: { $mergeObjects: '$$this' },
},
},
},
},
])
This adds the right user to their respective objects, but I can only add the to the root object and not a new one.
You can try the approach,
$map to iterate loop of approvalLog
$filter to iterate loop of approvedBy array and search for user id by
$arrayElemAt to get first element from above filtered result
$mergeObjects to merge current object properties of approvalLog and filtered user
$$REMOVE don't need approvedBy now
await Form.aggregate([
{
$lookup: {
from: "users",
localField: "approvalLog.by",
foreignField: "_id",
as: "approvedBy"
}
},
{
$addFields: {
approvalLog: {
$map: {
input: "$approvalLog",
as: "a",
in: {
$mergeObjects: [
"$$a",
{
user: {
$arrayElemAt: [
{
$filter: {
input: "$approvedBy",
cond: { $eq: ["$$a.by", "$$this._id"] }
}
},
0
]
}
}
]
}
}
},
approvedBy: "$$REMOVE"
}
}
])
Playground
The second approach using $unwind,
$unwind deconstruct the approvalLog array
$lookup with user collection
$addFields and $arrayElemAt to get first element from lookup result
$group by _id and reconstruct the approvalLog array and get first value of other required properties
await Form.aggregate([
{ $unwind: "$approvalLog" },
{
$lookup: {
from: "users",
localField: "approvalLog.by",
foreignField: "_id",
as: "approvalLog.user"
}
},
{
$addFields: {
"approvalLog.user": {
$arrayElemAt: ["$approvalLog.user", 0]
}
}
},
{
$group: {
_id: "$_id",
approvalLog: { $push: "$approvalLog" },
userId: { $first: "$userId" },
// add your other properties like userId
}
}
])
Playground
Hello i am trying to join two collections...
#COLLECTION 1
const valuesSchema= new Schema({
value: { type: String },
})
const categoriesSchema = new Schema({
name: { type: String },
values: [valuesSchema]
})
mongoose.model('categories', categoriesSchema )
#COLLECTION 2
const productsSchema = new Schema({
name: { type: String },
description: { type: String },
categories: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'categories',
}]
})
mongoose.model('productos', productsSchema )
Now, what i pretend to do is join these collections and have an output like this.
#Example Product Document
{
name: 'My laptop',
description: 'Very ugly laptop',
categories: ['5f55949054f3f31db0491b5c','5f55949054f3f31db0491b5b'] // these are _id of valuesSchema
}
#Expected Output
{
name: 'My laptop',
description: 'Very ugly laptop',
categories: [{value: 'Laptop'}, {value: 'PC'}]
}
This is what i tried.
{
$lookup: {
from: "categories",
let: { "categories": "$categories" },
as: "categories",
pipeline: [
{
$match: {
$expr: {
$in: [ '$values._id','$$categories']
},
}
},
]
}
}
but this query is not matching... Any help please?
You can try,
$lookup with categories
$unwind deconstruct values array
$match categories id with value id
$project to show required field
db.products.aggregate([
{
$lookup: {
from: "categories",
let: { cat: "$categories" },
as: "categories",
pipeline: [
{ $unwind: "$values" },
{ $match: { $expr: { $in: ["$values._id", "$$cat"] } } },
{
$project: {
_id: 0,
value: "$values.value"
}
}
]
}
}
])
Playground
Since you try to use the non-co-related queries, I appreciate it, you can easily achieve with $unwind to flat the array and then $match. To regroup the array we use $group. The $reduce helps to move on each arrays and store some particular values.
[
{
$lookup: {
from: "categories",
let: {
"categories": "$categories"
},
as: "categories",
pipeline: [
{
$unwind: "$values"
},
{
$match: {
$expr: {
$in: [
"$values._id",
"$$categories"
]
},
}
},
{
$group: {
_id: "$_id",
values: {
$addToSet: "$values"
}
}
}
]
}
},
{
$project: {
categories: {
$reduce: {
input: "$categories",
initialValue: [],
in: {
$concatArrays: [
"$$this.values",
"$$value"
]
}
}
}
}
}
]
Working Mongo template
The code example is in Mongo Playground
https://mongoplayground.net/p/W4Qt4oX0ZRP
Assume the following documents
[
{
_id: "5df1e6f75de2b22f8e6c30e8",
user: {
name: "Tom",
sex: 1,
age: 23
},
dream: [
{
label: "engineer",
industry: "5e06b16fb0670d7538222909",
type: "5e06b16fb0670d7538222951",
},
{
label: "programmer",
industry: "5e06b16fb0670d7538222909",
type: "5e06b16fb0670d7538222951",
}
],
works: [
{
name: "any engineer",
company: "5dd7fd51b0ae1837a08d00c8",
skill: [
"5dc3998e2cf66bad16efd61b",
"5dc3998e2cf66bad16efd61e"
],
},
{
name: "any programmer",
company: "5dd7fd9db0ae1837a08d00e2",
skill: [
"5dd509e05de2b22f8e67e1b7",
"5dd509e05de2b22f8e67e1bb"
],
}
]
}
]
I tried to use aggregate $lookup $unwind
db.coll.aggregate([
{
$unwind: {
path: "$dream",
}
},
{
$lookup: {
from: "industry",
localField: "dream.industry",
foreignField: "_id",
as: "dream.industry"
},
},
{
$unwind: {
path: "$dream.industry",
}
},
{
$lookup: {
from: "type",
localField: "dream.type",
foreignField: "_id",
as: "dream.type"
},
},
{
$unwind: {
path: "$dream.type",
}
},
{
$unwind: {
path: "$works",
}
},
{
$lookup: {
from: "company",
localField: "works.company",
foreignField: "_id",
as: "works.company"
},
},
{
$unwind: {
path: "$works.company",
}
},
{
$lookup: {
from: "skill",
localField: "works.skill",
foreignField: "_id",
as: "works.skill"
},
},
])
Executing the above code did not get the desired result!
This is what i expect
{
_id: "5df1e6f75de2b22f8e6c30e8",
user: {
name: 'Tom',
sex: 1,
age: 23
},
dream: [
{
label: 'engineer',
industry: {
_id: "5e06b16fb0670d7538222909", // Industry doc _id
name: 'IT',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
type: {
_id: "5e06b16fb0670d7538222951", // Type doc _id
name: 'job',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
},
{
label: 'programmer',
industry: {
_id: "5e06b16fb0670d7538222909", // Industry doc _id
name: 'IT',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
},
type: {
_id: "5e06b16fb0670d7538222951", // Type doc _id
name: 'job',
createdAt: "2019-12-28T01:35:44.070Z",
updatedAt: "2019-12-28T01:35:44.070Z"
}
}
],
works: [
{
name: 'any engineer',
company: {
_id: "5dd7fd51b0ae1837a08d00c8", // Company doc _id
name: 'alibaba',
area: 'CN',
},
skill: [
{
_id: "5dc3998e2cf66bad16efd61b", // Skill doc _id
name: 'Java'
},
{
_id: "5dc3998e2cf66bad16efd61e", // Skill doc _id
name: 'Php'
},
]
},
{
name: 'any programmer',
company: {
_id: "5dd7fd9db0ae1837a08d00e2", // Company doc _id
name: 'microsoft',
area: 'EN',
},
skill: [
{
_id: "5dd509e05de2b22f8e67e1b7", // Skill doc _id
name: 'Golang'
},
{
_id: "5dd509e05de2b22f8e67e1bb", // Skill doc _id
name: 'Node.js'
}
]
},
]
}
The expected result is dream is an array, works is an array, and dream.industry changed from ObjectId to document, dream.type changed from ObjectId to document, works.company changed from ObjectId to document
When I use populate, I can do it easily
Model.find()
.populate('dream.industry')
.populate('dream.type')
.populate('works.company')
.populate('works.skill')
.lean()
I refer to the following questions
mongoose aggregate lookup array (Almost the same as my question, But not resolved)
$lookup on ObjectId's in an array
hope to get everyone's help, thank you!
To make it easier i would not change the current pipeline but just add a $group stage to end of it in order to re-structure the data.
{
$group: {
_id: "$_id",
user: {$first: "$user"},
dream: {$addToSet: "$dream"},
works: {$addToSet: "$works"}
}
}
With that said if you are using Mongo version 3.6+ i do recommend you use the "newer" version of $lookup to re-write your pipeline to be a bit more efficient by avoiding all these $unwind's.