I have two collections, one named Exports and one named Service. Inside the Exports collection there is an object that holds inside of it an array of servicesIds.
I want to aggregate and lookup for the corressponding matching _ids from the Exports collection with the Service collection to find the name of the services.
The structure of the each document for the two collection is as follows:
Exports:
{
"_id" : "818a2c4fc4",
"companyId" : "7feb1812d8",
"filter" : {
"servicesIds" : [
"0111138dc679d",
"0c18c499435e9",
],
},
"_created_at" : ISODate("2019-10-27T09:06:03.102+0000"),
"_updated_at" : ISODate("2019-10-27T09:06:05.099+0000"),
}
Service:
An example of one document with its _id is a foreign key inside the filters object then inside the servicesIds array
{
"_id" : "0111138dc679d",
"name" : "Bay Services",
"character" : "B",
"company" : {
"id" : "f718a1c385",
"name" : "xxx"
},
"active" : true,
"tags" : [
],
"_created_at" : ISODate("2020-04-09T06:36:14.442+0000"),
"_updated_at" : ISODate("2020-06-06T03:52:16.770+0000"),
}
How can i do that?
Here is what i tried, but it keeps giving me and error reading
Mongo Server error '$in requires an array as a second argument, found: missing' on server
Here is my code:
db.getCollection("Exports").aggregate([
{
"$match": { "companyId":"818a2c4fc4" },
},
{
"$lookup": {
"from": "Service",
"let":{ id : "$_id" },
"pipeline": [
{
"$match":
{
"$expr":
{
"$in": ["$$id","$filter.servicesIds"]
}
}
}
],
"as":"services"
}
},
])
$unwind the array first, or you can edit your answer with an expected result you want, then I will correct my answer.
db.Exports.aggregate([
{
"$match": {
"companyId": "7feb1812d8"
}
},
{
"$unwind": "$filter.servicesIds"
},
{
"$lookup": {
"from": "Service",
"localField": "filter.servicesIds",
"foreignField": "_id",
"as": "docs"
}
}
])
https://mongoplayground.net/p/l2VweVYz1Fy
Related
how to correct lookup collection product which has array of ids of prices and need execute query exactly from prices collection query. Need to show price record with lookup product record
so, I have prices record
{
"_id" : "813f02ff-882e-44f7-b2bc-2f067427daf6",
"unit_amount" : 333,
"currency" : "USD",
"interval" : "year",
"active" : true
}
and product
"_id" : "3c46f277-8953-4f96-baf1-bd871ee3301f",
"name" : "test",
"prices" : [
"813f02ff-882e-44f7-b2bc-2f067427daf6",
"f5c76122-6132-4e4b-a26b-41bbd6325acc",
"3e4be68e-fbed-47f7-b871-92de72cb00df"
]
and my query, I thought it should be like that
db.getCollection('price').aggregate([
{
"$lookup": {
"from": "product",
"let": { "prid": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$in": ["$$prid", '$prices'] } } }
],
"as": "product_tbl"
}
}
])
faced with
{
"ok" : 0,
"errmsg" : "PlanExecutor error during aggregation :: caused by :: $in requires an array as a second argument, found: missing",
"code" : 40081,
"codeName" : "Location40081"
}
but it's not works. How it shoul be look ?
Seems like some of the documents in your product collection are missing prices
key. You can try this:
db.prices.aggregate([
{
"$lookup": {
"from": "product",
"let": {
"prid": "$_id"
},
"pipeline": [
{
"$addFields": {
"prices": {
"$ifNull": [
"$prices",
[]
]
}
}
},
{
"$match": {
"$expr": {
"$in": [
"$$prid",
"$prices"
]
}
}
}
],
"as": "product_tbl"
}
}
])
Here, we recompute the prices and set it to empty array, if it's missing, before the $match. Playground link.
I have a single mongo collection called "CourseCollection" and it contains both parent and child doc. Any document with the key "Parent" is a child doc and a parent can have multiple child doc.
{
"_id" : "abracadavra",
"Name" : "abracadavra",
"Description" : "",
"Type" : "Spell",
"Parent" : {
"_id" : "Magic",
"Type" : "Course",
"Name" : "Magic"
}
},
{
"_id" : "Magic",
"Name" : "Magic",
"Type" : "Course",
"Access" : [
{
"_id" : "2sssdw5oe",
"Name" : "Abc"
},
{
"_id" : "4fddfye42",
"Name" : "Xyz"
}
]
}
What I'm trying to do is, based on the Access of Parent doc, I'm trying to get all the child doc.
Existing and working solution:
The solution that I have currently is to perform 2 queries.
Query 1. Get all the courses that the user has access to.
db.getCollection("CourseCollection").find({"Type": "Course", "Access._id": {"$in": ["2sssdw5oe"]}})
Query 2. Since I'm using Python, I do a list comprehension to get only the IDs of the course and then perform another query with this list
db.getCollection("CourseCollection").find({"Type": "Spell", "Parent._id": {"$in": course_list_id}})
Is there a way to get the child data after filtering out the parent in a single query. I also tried aggregation but only the results of the previous stage are passed to the next stage.
I guess you're trying to do something like this:
db.CourseCollection.aggregate([
{
"$match": {
"Type": "Spell"
}
},
{
"$lookup": {
"from": "CourseCollection",
"localField": "Parent._id",
"foreignField": "_id",
"as": "Parents"
}
},
{
"$match": {
"Parents": {
"$elemMatch": {
"Type": "Course",
"Access._id": {
"$in": [
"2sssdw5oe"
]
}
}
}
}
}
])
You can achieve the same result doing this too:
db.CourseCollection.aggregate([
{
"$match": {
"Type": "Spell"
}
},
{
"$lookup": {
"from": "CourseCollection",
"localField": "Parent._id",
"foreignField": "_id",
"as": "Parents",
"pipeline": [
{
"$match": {
"Type": "Course",
"Access._id": {
"$in": [
"2sssdw5oe"
]
}
}
}
]
}
},
{
"$match": {
"Parents.0": {
"$exists": true
}
}
}
])
I'm a newbie to MongoDB, I'm trying to aggregate complete details of students in referencing with other collections.
students collection structure:
{
"_id" : ObjectId("5cc973dd008221192148177a"),
"name" : "James Paulson",
"teachers" : [
ObjectId("5cc973dd008221192148176f"),
ObjectId("5cc973dd0082211921481770")
],
"attenders": [
ObjectId("5cc973dd0082211921481732"),
ObjectId("5cc973dd008221192148173f")
]
}
staff collection structure:
{
"_id" : ObjectId("5cc973dd008221192148176f"),
"name" : "John Paul",
"subject" : [
"english",
"maths"
]
}
{
"_id" : ObjectId("5cc973dd0082211921481771"),
"name" : "Pattrick",
"subject" : [
"physics",
"history"
]
}
{
"_id" : ObjectId("5cc973dd0082211921481732"),
"name" : "Roger",
"subject" : [
"sweeper"
]
}
{
"_id" : ObjectId("5cc973dd008221192148173f"),
"name" : "Ken",
"subject" : [
"dentist"
]
}
This is the query I used for the retrieval of all teacher details of a particular student.
Query:
db.getCollection('students').aggregate([
{
$unwind: "$teachers"
},
{
$lookup:
{
from: 'staff',
localField: 'teachers',
foreignField: '_id',
as: 'teachers'
}
}
]);
Result:
{
"_id" : ObjectId("5cc973dd008221192148177a"),
"name" : "James Paulson",
"teachers" : [
{
"_id" : ObjectId("5cc973dd008221192148176f"),
"name" : "John Paul",
"subject" : [
"english",
"maths"
]
},
{
"_id" : ObjectId("5cc973dd008221192148176f"),
"name" : "Pattrick",
"subject" : [
"physics",
"history"
]
}
],
"attenders": [
ObjectId("5cc973dd0082211921481732"),
ObjectId("5cc973dd008221192148173f")
]
}
As you can see, the attenders array is also similar to teachers except the difference in column name in students table. So how to applying a similar query to the second column (attenders)? Also is there any way to select specific columns from the second table (like only _id and name from staff collection)?
Any help on this would be greatly appreciated.
You can use below aggregation with mongodb 3.6 and above
Firstly you don't need to use $unwind here as your field already contains array of ObjectIds. And to select the specific field from the referenced collection you can use the custom $lookup with pipeline and $project the fields inside it.
db.getCollection('students').aggregate([
{ "$lookup": {
"from": "staff",
"let": { "teachers": "$teachers" },
"pipeline": [
{ "$match": { "$expr": { "$in": [ "$_id", "$$teachers" ] } } }
{ "$project": { "name": 1 }}
],
"as": "teachers"
}},
{ "$lookup": {
"from": "attenders",
"let": { "attenders": "$attenders" },
"pipeline": [
{ "$match": { "$expr": { "$in": [ "$_id", "$$attenders" ] } } }
],
"as": "attenders"
}}
])
I am trying to understand why a $lookup I'm using in my MongoDB aggregation is producing the result it is.
First off, my initial data looks like this:
"subscriptions": [
{
"agency": "3dg2672f145d0598be095634", // This is an ObjectId
"memberType": "primary"
}
]
Now, what I want to do is a simple $lookup, pulling in the related data for the ObjectId that's currently being populated as the value to the "agency" field.
What I tried doing was a $lookup like this:
{
"from" : "agencies",
"localField" : "subscriptions.0.agency",
"foreignField" : "_id",
"as" : "subscriptions.0.agency"
}
So, basically what I want to do is go get that info related to that ObjectId ref, and populate it right here, in place of where the ObjectId currently resides.
What I'd expect as a result is something like this:
"subscriptions": [
{
"agency": [
{
_id: <id-value>,
name: <name-value>,
address: <address-value>
}
],
"memberType": "primary"
}
]
Instead, I end up with this (with my "memberType" prop now nowhere to be found):
"subscriptions" : {
"0" : {
"agency" : [ <agency-data> ]
}
}
Why is this the result of the $lookup, and how can I get the data structure I'm looking for here?
To clarify further, in the docs they mention using an $unwind BEFORE the $lookup when it's an array field. But in this case, the actual local field being targeted and replaced by the $lookup is NOT an array, but it is within an array. So I'm not clear on what the problem is.
You need to use $unwind to match your "localField" with to the "foreignField" and then $group to rollback again to the array
db.collection.aggregate([
{ "$unwind": "$subsciption" },
{ "$lookup": {
"from": Agency.collection.name,
"localField": "subsciption.agency",
"foreignField": "_id",
"as": "subsciption.agency"
}},
{ "$group": {
"_id": "$_id",
"memberType": { "$first": "$memberType" },
"subsciption": { "$push": "$subsciption" },
}}
])
Basically, what OP is looking for is to transform data in desired format after looking up into another collection. Assuming there are two collections C1 and C2 where C1 contains document
{ "_id" : ObjectId("5b50b8ebfd2b5637081105c6"), "subscriptions" : [ { "agency" : "3dg", "memberyType" : "primary" } ] }
and C2 contains
{ "_id" : ObjectId("5b50b984fd2b5637081105c8"), "agency" : "3dg", "name" : "ABC", "address" : "1 some street" }
if following query is executed against database
db.C1.aggregate([
{$unwind: "$subscriptions"},
{
$lookup: {
from: "C2",
localField: "subscriptions.agency",
foreignField: "agency",
as: "subscriptions.agency"
}
}
])
We get result
{
"_id": ObjectId("5b50b8ebfd2b5637081105c6"),
"subscriptions": {
"agency": [{
"_id": ObjectId("5b50b984fd2b5637081105c8"),
"agency": "3dg",
"name": "ABC",
"address": "1 some street"
}],
"memberyType": "primary"
}
}
which is pretty close to what OP is looking forward.
Note: there may be some edge cases but with minor tweaks, this solution should work
I'm Strugling with some aggregation functions in mongodb.
I want to get books Documents in author's document that has just books ids as array of strings ids like this :
Author Document
{
"_id" : "10",
"full_name" : "Joi Dark",
"books" : ["100", "200", "351"],
}
And other documents (books) :
{
"_id" : "100",
"title" : "Node.js In Action",
"ISBN" : "121215151515154",
"date" : "2015-10-10"
}
So in result i want this :
{
"_id" : "10",
"full_name" : "Joi Dark",
"books" : [
{
"_id" : "100",
"title" : "Node.js In Action",
"ISBN" : "121215151515154",
"date" : "2015-10-10"
},
{
"_id" : "200",
"title" : "Book 2",
"ISBN" : "1212151454515154",
"date" : "2015-10-20"
},
{
"_id" : "351",
"title" : "Book 3",
"ISBN" : "1212151454515154",
"date" : "2015-11-20"
}
],
}
Use $lookup which retrieves data from the nominated collection in from based on the matching of the localField to the foreignField:
db.authors.aggregate([
{ "$lookup": {
"from": "$books",
"foreignField": "_id",
"localField": "books",
"as": "books"
}}
])
The as is where in the document to write an "array" containing the related documents. If you specify an existing property ( such as is done here ) then that property is overwritten with the new array content in output.
If you have a MongoDB before MongoDB 3.4 then you may need to $unwind the array of "books" as the localField first:
db.authors.aggregate([
{ "$unwind": "$books" },
{ "$lookup": {
"from": "$books",
"foreignField": "_id",
"localField": "books",
"as": "books"
}}
])
Which creates a new document for each array member in the original document, therefore use $unwind again and $group to create the original form:
db.authors.aggregate([
{ "$unwind": "$books" },
{ "$lookup": {
"from": "$books",
"foreignField": "_id",
"localField": "books",
"as": "books"
}},
{ "$unwind": "$books" },
{ "$group": {
"_id": "$_id",
"full_name": { "$first" "$full_name" },
"books": { "$push": "$books" }
}}
])
If in fact your _id values in the foreign collection of of ObjectId type, but you have values in the localField which are "string" versions of that, then you need to convert the data so the types match. There is no other way.
Run something like this through the shell to convert:
var ops = [];
db.authors.find().forEach(doc => {
doc.books = doc.books.map( book => new ObjectId(book.valueOf()) );
ops.push({
"updateOne": {
"filter": { "_id": doc._id },
"update": {
"$set": { "books": doc.books }
}
}
});
if ( ops.length >= 500 ) {
db.authors.bulkWrite(ops);
ops = [];
}
});
if ( ops.length > 0 ) {
db.authors.bulkWrite(ops);
ops = [];
}
That will convert all the values in the "books" array into real ObjectId values that can actually match in a $lookup operation.
Just adding on top of the previous answer. If your input consists of an array of strings and you want to convert them to ObjectIds, you can achieve this by using a projection, followed by a map and the $toObjectId method.
db.authors.aggregate([
{ $project: {
books: {
$map: {
input: '$books',
as: 'book',
in: { $toObjectId: '$$book' },
},
},
},},
{ $lookup: {
from: "$books",
foreignField: "_id",
localField: "books",
as: "books"
}
},
])
Ideally, your database would be formatted in such a manner that your aggregates are stored as ObjectIds, but in the case where that is not an option, this poses as a viable solution.