I'm fairly new to MongoDB and need help doing a select, or perhaps some sort of left join, on one collection based on another collection's data.
I have two collections, animals and meals, and I want to get the animal(s) that has had it's last registered meal after a certain date (let's say 20171001) to determine if the animal is still active.
collection animals:
{
name: mr floof,
id: 12345,
lastMeal: abcdef
},
{
name: pluto,
id: 6789,
lastMeal: ghijkl
}
collection meals:
{
id: abcdef,
created: 20171008,
name: carrots
},
{
id: ghijkl,
created: 20170918,
name: lettuce
}
So the expected output of the query in this case would be:
{
name: mr floof,
id: 12345,
lastMeal: abcdef
}
As Mr Floof has had his last meal 20171008, i.e. after 20171001.
Hope I was clear enough, but if not, don't hesitate to ask.
You can try below aggregation query.
db.animals.aggregate([ [
{
"$lookup": {
"from": "meals",
"localField": "lastMeal",
"foreignField": "id",
"as": "last_meal"
}
},
{
"$unwind": "$last_meal"
},
{
"$match": {
"last_meal.created": {
"$gt": 20171001
}
}
}
])
More info here.
You can use $project with exclusion after $match stage to format the response to exclude joined fields. Something like { $project: {"last_meal":0} }
MongoDB supports joins with $lookup , In your case you can use query like:-
db.animals.aggregate([
{
$lookup:
{
from: "meals",
localField: "lastMeal",
foreignField: "id",
as: "last_meal"
}
},
{
$match: {
"created" : {
$gt: "date" //your date format
}
}
}
])
thanks !
Related
Assuming a setup like this:
stores
{
name: "store1",
category: "category1"
},
{
name: "store2",
category: "category2"
}
products
{
store_name: "store1",
name: "product1"
},
{
store_name: "store2",
name: "product2"
}
with hundreds of millions of records in each collection.
I need to query all product names for stores with category != some_parameter. The example above for category category1 should return product2 because store2 has category2 (not category1).
I can't change the DB collections. Is it possible to use $lookup stage within an aggregation pipeline to perform "NOT IN" against another collection? something like: find all products with store not in (find all stores with category = category_param)
You can do it like this:
$lookup - to fetch store information for each product.
$set with $first - Since first step will return an array that will always have one item, we will take that item.
$match with $ne - to filter for all the documents where category is not equal to the requested category.
db.products.aggregate([
{
"$lookup": {
"from": "stores",
"localField": "store_name",
"foreignField": "name",
"as": "store"
}
},
{
"$set": {
"store": {
"$first": "$store"
}
}
},
{
"$match": {
"store.category": {
"$ne": "category1"
}
}
}
])
Working example
I have two table
master_document_table document_table
id | title id | master_document_id | userId
1 | Profile
select * from master_document_table as md
left join document_table as d on md.id = d.master_document_id and d.userId = 2
Result:
id | title | master_document_id | userId
1 | Profile null null
how can this be achieved using mongodb i have tried & also did some research from stack overflow and did not got the expected result.
From mongo v3.2, it is possible to aggregate two or more than two collections.
But here one thing, you need to know is, you won't get the same structure as sql. You will get your data on nested document.
Your tables will look something like this in mongodb:
master_document_table (collection / table)
{
_id: ObjectId("5f28d53c00613e6ada45f702"),
title: "Profile",
}
document_table (collection / table)
{
_id: ObjectId("5f28d53c00613e6ada45e790"),
master_document_id: ObjectId("5f28d53c00613e6ada45f702"), // master_doc_table's ref id.
userId: ObjectId("5f28d53c00613e6ada327f231"), // user's ref id.
}
Your query will be:
db.document_table.aggregate([
{$match: {title: "Profile"}}, // you can give any condition here.
{
$lookup:
{
from: "master_document_table",
localField: "master_document_id",
foreignField: "_id",
as: "master_document_id"
}
}
])
Your query result will look something like:
[{
_id: ObjectId("5f28d53c00613e6ada45e790"),
master_document_id: {
_id: ObjectId("5f28d53c00613e6ada45f702"),
title: "Profile"
},
userId: ObjectId("5f28d53c00613e6ada327f231"),
}]
Now you can restructure it based on your requirement.
I would like to recommend you
https://docs.mongodb.com/manual/reference/operator/aggregation/match/
&
https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/
article from mongodb docs.
Hope this will help to resolve your problem.
Inorder to join two tables you can use the $lookup aggregation pipeline operator in version 3.6 and newer. Since you need the results like LEFT OUTER JOIN, you have to $unwind the array to objects and filter out the desired results using $match conditions (ie document_table.userId = 2). Dont forget to use preserveNullAndEmptyArrays to true when unwinding.
Sample code for reference:
db.master_document_table.aggregate([
{ "$lookup": {
"from": "document_table",
"localField": "id",
"foreignField": "master_document_id",
"as": "document_table"
}},
{ $unwind: { path: "$document_table", preserveNullAndEmptyArrays: true } },
{
$match:{
$or:[
{ "document_table": { $exists: false } },
{"document_table.userId" : 2 }
]
}
}
])
I some how managed to get desire result thanks for the help
db.master_document_table.aggregate([
{
$lookup: {
from: 'document_table',
localField: 'id',
foreignField: 'master_document_id',
as: 'users'
}
},
{
$project: {
title: 1,
users: {
$filter: {
input: '$users',
as: 'user',
cond: {
$eq: ['$$user.userId', 2]
}
}
}
}
}
]);
Suppose, In MongoDB i have two collections. one is "Students" and the another is "Course".
Student have the document such as
{"id":"1","name":"Alex"},..
and Course has the document such as
{"course_id":"111","course_name":"React"},..
and there is a third collection named "students-courses" where i have kept student's id with their corresponding course id. Like this
{"student_id":"1","course_id":"111"}
i want to make a query with student's id so that it gives the output with his/her enrolled course. like this
{
"id": "1",
"name":"Alex",
"taken_courses": [
{"course_id":"111","course_name":"React"},
{"course_id":"112","course_name":"Vue"}
]
}
it will be many to many relationship in MongoDB without using ORM. How can i make this query?
Need to use $loopup with pipeline,
First $group by student_id because we are going to get courses of students, $push all course_id in course_ids for next step - lookup purpose
db.StudentCourses.aggregate([
{
$group: {
_id: "$student_id",
course_ids: {
$push: "$course_id"
}
}
},
$lookup with Student Collection and get the student details in student
$unwind student because its an array and we need only one from group of same student record
$project required fields
{
$lookup: {
from: "Student",
localField: "_id",
foreignField: "id",
as: "student"
}
},
{
$unwind: "$student"
},
{
$project: {
id: "$_id",
name: "$student.name",
course_ids: 1
}
},
$lookup Course Collection and get all courses that contains course_ids, that we have prepared in above $group
$project the required fields
course details will store in taken_courses
{
$lookup: {
from: "Course",
let: {
cId: "$course_ids"
},
pipeline: [
{
$match: {
$expr: {
$in: [
"$course_id",
"$$cId"
]
}
}
},
{
$project: {
_id: 0
}
}
],
as: "taken_courses"
}
},
$project details, removed not required fields
{
$project: {
_id: 0,
course_ids: 0
}
}
])
Working Playground: https://mongoplayground.net/p/FMZgkyKHPEe
For more details related syntax and usage, check aggregation
I am doing a lookup where the local field is an ObjectId and the foreign field is an array of ObjectId's. Performing the lookup gives me the error:
arguments to $lookup must be strings
I have done similar lookups where the foreign field is not an array (but is an ObjectId) so the error seems to be ambiguous. my database consists two collections: Song and Playlist. A Song can belong to many playlists. I am trying to write an aggregation that returns a matching song that contains an array of playlists the song belongs to:
Songs:
[
{
songName: "In Da Club",
_id: ObjectId(1)
},
{
songName: "Happy Birthday",
_id: ObjectId(2)
},
{
songName: "Ode to Joy",
_id: ObjectId(3)
}
]
Playlists:
[
{
_id: ObjectId(4)
playlistName: "PlaylistOne,
songs: [ObjectId(1), ObjectId(3)]
},
{
_id: ObjectId(5)
playlistName: "PlaylistTwo,
songs: [ObjectId(1)]
}
]
Desired outcome:
{
songName: "In Da Club",
_id: ObjectId(1),
playlists: [
{
_id: ObjectId(4),
playlistName: "PlaylistOne,
},
{
_id: ObjectId(5),
playlistName: "PlaylistTwo"
}
]
}
The query I tried:
db.songs.aggregate([
{
$match: {
songName: "In Da Club"
}
},
{
$lookup: {
from: 'playlists',
let: { songId: '$_id'},
pipeline: [
{
$match: {
$expr: {
{
$in: ["$$songId", "$songs"]
}
}
}
}
],
as: 'playlists'
}
}
])
It seems to be a relatively simple query and I'm not sure how I can get around the "arguments passed into lookup must be strings" error since my lookup is based on ObjectId's. Any help would be greatly appreciated! TIA!
Though $lookup is introduced in version 3.2 but couple of enhancements are done on it in sub-sequent updates :
As per docs :
Starting MongoDB 3.4, if the localField is an array, you can match the
array elements against a scalar foreignField without needing an
$unwind stage.
MongoDB 3.6, adds support for executing a pipeline on the joined
collection, which allows for specifying multiple join conditions as
well as uncorrelated sub-queries.
So the issue might be from mongodb version being low. Anyway for your requirement you can use $lookup which is used for single equality join as array is on foreignField.
{
$lookup: {
from: "playlists",
localField: "_id", // Scalar value
foreignField: "songs", // Against an array
as: "playlists"
}
}
Test : mongoplayground
I would like unmatched data from the USERS collection. Here I have two collections 1) USERS, 2) COMPANY.I am able to get the matched data from both USERS using aggregate function. but in this case I want data from USERS table which are not assigned to a company.
USERS table
{
_id: "AAA",
fullName:"John Papa"
},
{
_id: "BBB",
fullName:"Robin Son"
}
COMPANY table
{
_id: "1sd1s",
Name:"Lumbar Company"
User:"AAA"
},
{
_id: "23s1dfs3",
Name:"Patricia"
User:"AAA"
}
$lookup works like LEFT OUTER JOIN so it will remove empty array when there's no match. Then you can use $size to get only empty arrays:
db.users.aggregate([
{
$lookup: {
from: "company",
localField: "_id",
foreignField: "User",
as: "companies"
}
},
{
$match: {
$expr: {
$eq: [ { "$size": "$companies" }, 0 ]
}
}
}
])
Mongo Playground