I have the following example JSON-object saved in my mongodb-collection named "Profile"
{
name: "Test",
relations: [
{
personid: <MongoDB-ID>,
type: "Friend",
duration: 5
},
{
personid: <MongoDB-ID>,
type: "Family",
duration: 9
},
]
}
I've used the mongoose-Aggregate function because i need to add artificial fields based on caluclation in the documents saved. At the end of my aggregation i use the $lookup-function to replace the property "personID" in the objects inside of the "relations"-array.
{
$lookup:
{
from: PersonModel.collection.name,
localField: 'relations.personid',
foreignField: '_id',
as: "relations.personid"
}
}
Because i want each "person" in the array of objects to replaced within the populated from the specific person-document.
This does not work as expected.
I also tried to call ".populate()" on the result returned by the aggregate function which also not worked.
Setting localField to relations.personid is not supported. localField either needs to point to an array where each member is used for the join or be a plain value. The usual way of getting around this is to $unwind first, perform the lookup, and then $group back if needed.
Related
I am having 2 collections -
Collection name is content
Collection name is surveys
I actually want to add the "type" information from the content collection to every content in every survey in the "surveys" collection.
I know in SQL we have to join these 2 tables first on _id and content_id commun columns and then add type column to suryeys table but not sure how should i do it here. I want the type value from content collection to add in every content field inside surveys collection. Please help me.
One option is:
Using $lookup to get the data from content collection
Merge the arrays using $mergeObjects and $indexOfArray. After the lookup, the content data is on someField array. This step finds the releant content from someField and put it inside the matching survey's content item, under the key type. This step also removes someField.
Format the result using $map. This step uses $map to iterate over the content array and format the data under type to contain only the wanted part.
Save it back using $merge. This step saves the result back into the collection.
db.surey.aggregate([
{$lookup: {
from: "content",
localField: "content.content_id",
foreignField: "_id",
as: "someField"
}},
{$set: {
someField: "$$REMOVE",
content: {$map: {
input: "$content",
in: {$mergeObjects: [
"$$this",
{type: {
$arrayElemAt: [
"$someField",
{$indexOfArray: ["$someField._id", "$$this.content_id"]}
]
}}
]}
}}
}},
{$set: {content: {$map: {
input: "$content",
in: {$mergeObjects: ["$$this", {type: "$$this.type.type"}]}
}}}},
{$merge: {into: "surey"}}
])
See how it works on the playground example
In mongoose (I assume you could be using mongoose, sind you added the tag) you can specify the relation in Schema definition and just just populate to add these fields to the output.
If you are not using mongoose you can use $lookup (https://www.mongodb.com/docs/manual/reference/operator/aggregation/lookup/) in your aggregation pipeline.
{
from: "content",
localField: "content._id",
foreignField: "_id",
as: "someField"
}
I'm performing a query on a large number of documents from a collection whose schema looks like this:
let mySchema = mongoose.Schema({
title: {
type: String
},
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
company: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Company'
},
....
});
Many of the documents in the above schema will have similar values for user and company and I'm trying to find the most efficient way to perform this query. Here is what I have, but I feel like it isn't as performant as it could be:
this.model('MyModel')
.aggregate([
{$match: match_stuff_here},
$lookup: {
from: "users",
localField: "user",
foreignField: "_id",
as: "user"
}
$lookup: {
from: "companies",
localField: "company",
foreignField: "_id",
as: "company"
}
{$project: {
_id: 1,
title: 1,
"user._id": 1,
"user.name": 1,
"company._id": 1,
"company.name": 1
}}
])
.exec(function (err, data) {
...
});
It seems like maybe I should somehow group all of the common user ids and company ids together, do a lookup on them as a group, then add the results back in to the original match results. I'm not sure how to do this, or even if that's the right approach. Any ideas on how to optimize this query? Thanks ahead of time!
Since you already have it set up with ObjectId and ref, you can use .populate to make life easier https://mongoosejs.com/docs/populate.html
In your case, this should work for you:
var results = await Model.find(matchQuery).populate('user').populate('company');
console.log(results);
UPDATE
Assuming that you want to pull data from two collections - 'user' and 'company' there is no single query that will retrieve all data needed. You will need atleast 3 calls - match query, get user data, get company data. To my understand, it will map what it needs to populate, then query for the data. You can view the function in source https://github.com/Automattic/mongoose/blob/4.3.7/lib/model.js#L2456
You can also see the queries mongoose uses during the population by enabling debug
mongoose.set('debug', true);
This guy explains it pretty well http://frontendcollisionblog.com/mongodb/2016/01/24/mongoose-populate.html
For an existing mongo database, the link between 2 collections is done by :
collA : field collB_id
collB : field _id = ObjectId("a string value")
where collB_id is _id.valueOf()
i.e. : the value of collB_id of collA is "a string value"
but in a $lookup :
localField: "collB_id",
foreignField: _id.valueOf(),
don't work, so what can I do ?
Mongodb v3.6
If i understood you correctly, you have two collections where documents from first collection (collA) reference documents from second collection (collB). And the problem is that you store reference as a string value of that objectId, so you cant use $lookup to join those docs.
collA:
{
"_id" : ObjectId(...),
"collB_id" : "123456...",
...
}
collB:
{
"_id" : ObjectId("123456..."),
...
}
If you are using mongo 4.0+ you can do it with following aggregation:
db.getCollection('collA').aggregate([
{
$addFields: {
converted_collB_id: { $toObjectId: "$collB_id" }
}
},
{
$lookup: {
from: 'collB',
localField: 'converted_collB_id',
foreignField: '_id',
as: 'joined'
}
}
]);
Mongo 4.0 introduces new aggregation pipeline operators $toObjectId and $toString.
That allows you to add new field which will be an ObjectId created from the string value stored in collB_id, and use that new field as localField in lookup.
I would strongly advise you not to store ObjectIds as strings.
You already experienced the problem with $lookup.
Also, size of ObjectId is 12 bytes while its hex representation is 24 bytes (that is twice the size). You will probably want to index that field as well, so you want it to be as small as possible.
ObjectId also contains timestamp which you can get by calling getTimestamp()
Make your life easier by using native types when possible!
Hope this helps!
I have two collections, products and properties.
I'm doing a lookup such as:
[{
$match: {
category_id: mongoose.Types.ObjectId(category_id)
}
},
{
$lookup: {
from: "properties",
localField: "category_id",
foreignField: "category_id",
as: "properties"
}
}
]
This is basically getting me all of the products that match the category_id and including the properties that match the same category_id.
I need to add an additional check, for some_id on the properties result. In otherwords, the properties should be grouped by the some_id that is returned from the products collection and matches the same key in properties. Does that make sense? Basically having the ability to have multiple local/foreign field definitions.
any idea how I could this?
Since version 3.6 we can use uncorrelated sub-queries
{
$lookup:
{
from: '<collection to join>',
let: { <var_1>: '<expression>', …, <var_n>: '<expression>' },
pipeline: [ '<pipeline to execute on the collection to join>' ],
as: <output array field>
}
}
This allows us to to have more than a single equality match on the lookup.
See also: https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/
I am beginner in the MongoDB.
Right now, I am making one query by using mongo. Please look this and let me know is it possible? If it is possible, how can I do?
collection:students
[{id:a, name:a-name}, {id:b, name:b-name}, {id:c, name:c-name}]
collection:school
[{
name:schoolA,
students:[a,b,c]
}]
collection:room
[{
name:roomA,
students:[c,a]
}]
Expected result for roomA
{
name:roomA,
students:[
{id:a name:a-name isRoom:YES},
{id:b name:b-name isRoom:NO},
{id:c name:c-name isRoom:YES}
]
}
Not sure about the isRoom property, but to perform a join across collections, you'd have two basic options:
code it yourself, with multiple queries
use the aggregation pipeline with $lookup operator
As a quick example of $lookup, you can take a given room, unwind its students array (meaning separate out each student element into its own entity), and then look up the corresponding student id in the student collection.
Assuming a slight tweak to your room collection document:
[{
name:"roomA",
students:[ {studentId: "c"}, {studentId: "a"}]
}]
Something like:
db.room.aggregate([
{
$unwind: "$students"
},
{
$lookup:
{
from: "students",
localField: "studentid",
foreignField: "id",
as: "classroomStudents"
}
},
{
$project:
{ _id: 0, name : 1 , classroomStudents : 1 }
}
])
That would yield something like:
{
name:"roomA",
classroomStudents: [
{id:"a", name:"a-name"},
{id:"c", name:"c-name"}
]
}
Disclaimer: I haven't actually run this aggregation, so there may be a few slight issues. Just trying to give you an idea of how you'd go about solving this with $lookup.
More info on $lookup is here.