Assuming a setup like this:
stores
{
name: "store1",
category: "category1"
},
{
name: "store2",
category: "category2"
}
products
{
store_name: "store1",
name: "product1"
},
{
store_name: "store2",
name: "product2"
}
with hundreds of millions of records in each collection.
I need to query all product names for stores with category != some_parameter. The example above for category category1 should return product2 because store2 has category2 (not category1).
I can't change the DB collections. Is it possible to use $lookup stage within an aggregation pipeline to perform "NOT IN" against another collection? something like: find all products with store not in (find all stores with category = category_param)
You can do it like this:
$lookup - to fetch store information for each product.
$set with $first - Since first step will return an array that will always have one item, we will take that item.
$match with $ne - to filter for all the documents where category is not equal to the requested category.
db.products.aggregate([
{
"$lookup": {
"from": "stores",
"localField": "store_name",
"foreignField": "name",
"as": "store"
}
},
{
"$set": {
"store": {
"$first": "$store"
}
}
},
{
"$match": {
"store.category": {
"$ne": "category1"
}
}
}
])
Working example
Related
I have two mongoDb collections, one contains data about cards and the other contains data about a field of cards and called list.
structure of firstCollection :
{
"cardType":"card",
"xyz":"XYZ",
"fields":[
{"abc":"abc", "xyz":"XYZ", "inputMethod" : "Entry", "xyz":"xyz"},
{"abc":"abc", "xyz":"XYZ", "inputMethod" : "List", "xyz":"xyz", "ListId":"1234"}
// ListId will only be present incase of inputMethod=List
]
}
Structure of secondCollection:
{ "abc":"abc", "xyz":"xyz, "itemId": "1234" }
Now what I want is
all the firstCollection where cardType = "card", complete card object
and
all the items from secondCollection where itemId in (select ListId from firstCollection where fields.inputmethod = "List").
Need to write MongoDB pipeline for this situation. I am quite new to mongo, it can be done using an aggregation pipeline with $loopup but I can write the pipeline.
the result I want :
{
firstCollection:{complete collection },
secondCollection:[
array of matching records from second collection where
secondelement.itemId in(records from array of firstcollection
where fields.inputmethod = "List" )
]
}
db.first.aggregate([
{
$match: {}
},
{
$project: {
firstCollection: "$$ROOT"
}
},
{
$lookup: {
"from": "second",
"localField": "firstCollection.fields.ListId",
"foreignField": "itemId",
"as": "secondCollection"
}
}
])
mongoplayground
I'm trying to figure out how to essentially do a join in MongoDB. I've read about doing aggregates: https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/, but that doesn't seem to be what I'm looking for.
I'm also very new to NoSQL, so I'm not exactly sure what I should be using here.
I have two collections in MongoDB, structured as follows:
db collection - employees:
{
_id: 1,
name: 'John Doe',
filesAccess: {
web: true
},
fileIds: [
'fileId1',
'fileId2'
]
},
{
_id: 2,
name: 'Bob Jones',
filesAccess: {
web: false
},
fileIds: [
'fileId3',
'fileId4'
]
}
db collection - files:
{
_id: fileId1,
fileMetaData: {
location: 'NE'
}
},
{
_id: fileId2,
fileMetaData: {
location: 'NE'
}
},
{
_id: fileId3,
fileMetaData: {
location: 'SW'
}
},
{
_id: fileId4,
fileMetaData: {
location: 'SW'
}
}
I want to be able to query for all employees who have fileAccess.web = true and get their employee ID, name, and fileMetaData.location. The location for all of an employee's files will be the same. So the query only needs to use the first fileId from an employee to get the location from the files collection
So I'd like my result should look like:
{
_id: 1,
name: 'John Doe',
location: 'NE'
}
How would you structure a query to accomplish this in MongoDB? I'm using Studio3T as my interface to the db. Any help is greatly appreciated
You can use this aggregation query:
First $match to get only documents where filesAccess.web is true.
The join based on values on fileIds and _id. This give an array calling result.
Then get the first position
And $project to output the fields you want.
db.employess.aggregate([
{
"$match": {
"filesAccess.web": true
}
},
{
"$lookup": {
"from": "files",
"localField": "fileIds",
"foreignField": "_id",
"as": "result"
}
},
{
"$set": {
"result": {
"$arrayElemAt": [
"$result",
0
]
}
}
},
{
"$project": {
"_id": 1,
"name": 1,
"location": "$result.fileMetaData.location"
}
}
])
Example here
Suppose, In MongoDB i have two collections. one is "Students" and the another is "Course".
Student have the document such as
{"id":"1","name":"Alex"},..
and Course has the document such as
{"course_id":"111","course_name":"React"},..
and there is a third collection named "students-courses" where i have kept student's id with their corresponding course id. Like this
{"student_id":"1","course_id":"111"}
i want to make a query with student's id so that it gives the output with his/her enrolled course. like this
{
"id": "1",
"name":"Alex",
"taken_courses": [
{"course_id":"111","course_name":"React"},
{"course_id":"112","course_name":"Vue"}
]
}
it will be many to many relationship in MongoDB without using ORM. How can i make this query?
Need to use $loopup with pipeline,
First $group by student_id because we are going to get courses of students, $push all course_id in course_ids for next step - lookup purpose
db.StudentCourses.aggregate([
{
$group: {
_id: "$student_id",
course_ids: {
$push: "$course_id"
}
}
},
$lookup with Student Collection and get the student details in student
$unwind student because its an array and we need only one from group of same student record
$project required fields
{
$lookup: {
from: "Student",
localField: "_id",
foreignField: "id",
as: "student"
}
},
{
$unwind: "$student"
},
{
$project: {
id: "$_id",
name: "$student.name",
course_ids: 1
}
},
$lookup Course Collection and get all courses that contains course_ids, that we have prepared in above $group
$project the required fields
course details will store in taken_courses
{
$lookup: {
from: "Course",
let: {
cId: "$course_ids"
},
pipeline: [
{
$match: {
$expr: {
$in: [
"$course_id",
"$$cId"
]
}
}
},
{
$project: {
_id: 0
}
}
],
as: "taken_courses"
}
},
$project details, removed not required fields
{
$project: {
_id: 0,
course_ids: 0
}
}
])
Working Playground: https://mongoplayground.net/p/FMZgkyKHPEe
For more details related syntax and usage, check aggregation
I'm fairly new to MongoDB and need help doing a select, or perhaps some sort of left join, on one collection based on another collection's data.
I have two collections, animals and meals, and I want to get the animal(s) that has had it's last registered meal after a certain date (let's say 20171001) to determine if the animal is still active.
collection animals:
{
name: mr floof,
id: 12345,
lastMeal: abcdef
},
{
name: pluto,
id: 6789,
lastMeal: ghijkl
}
collection meals:
{
id: abcdef,
created: 20171008,
name: carrots
},
{
id: ghijkl,
created: 20170918,
name: lettuce
}
So the expected output of the query in this case would be:
{
name: mr floof,
id: 12345,
lastMeal: abcdef
}
As Mr Floof has had his last meal 20171008, i.e. after 20171001.
Hope I was clear enough, but if not, don't hesitate to ask.
You can try below aggregation query.
db.animals.aggregate([ [
{
"$lookup": {
"from": "meals",
"localField": "lastMeal",
"foreignField": "id",
"as": "last_meal"
}
},
{
"$unwind": "$last_meal"
},
{
"$match": {
"last_meal.created": {
"$gt": 20171001
}
}
}
])
More info here.
You can use $project with exclusion after $match stage to format the response to exclude joined fields. Something like { $project: {"last_meal":0} }
MongoDB supports joins with $lookup , In your case you can use query like:-
db.animals.aggregate([
{
$lookup:
{
from: "meals",
localField: "lastMeal",
foreignField: "id",
as: "last_meal"
}
},
{
$match: {
"created" : {
$gt: "date" //your date format
}
}
}
])
thanks !
My goal is to get all 'student' documents that belong to 'classes'
that have at least one student of grade "blue" and at least one
of grade "red".
I am inclined to simply do a sequence of queries in Python (pymongo), tackling the task directly.
I wonder if there is some clever aggregation pipeline that I could use!
Given:
Classes collection:
{ class_id: 'a' }
{ class_id: 'b' }
Students collection:
{ class_id: 'a',
grade: 'blue' }
{class_id: 'a',
grade: 'red' }
You could use :
a $group to group by class_id and $push all grade in an array so we can deduce easily in the next step which class "contains" blue. Preserve the current document with $$ROOT because we'll need the students that match the class_id
a $match to match only classes that have grade blue in it
an $unwind to remove the array of students generated by previous $$ROOT
a $project to reorganize your document nicely
Query would be :
db.students.aggregate([{
"$group": {
"_id": "$class_id",
"grades": { "$push": "$grade" },
"students": { "$push": "$$ROOT" }
}
}, {
"$match": {
"grades": { "$all": ["blue","red"] }
}
}, {
"$unwind": "$students"
}, {
"$project": {
"_id": "$students._id",
"class_id": "$students.class_id",
"grade": "$students.grade",
}
}])
If you need to match other color than ["blue","red"] you can add more in the $match aggregation ($in: ["blue","red","yellow"])
For implementing it in PyMongo, it is very straightforward :
from pymongo import MongoClient
import pprint
db = MongoClient().testDB
pipeline = [ <the_aggregation_query_here> ]
pprint.pprint(list(db.students.aggregate(pipeline)))
Additionnaly, to match only students that belong to classes collection, perform a $lookup and match those that are not empty. Add the following at the aggregation query :
{
$lookup: {
from: "classes",
localField: "class_id",
foreignField: "class_id",
as: "class"
}
}, {
$match: {
"class": { $not: { $size: 0 } }
}
}