I have few referenced collections: Employees, Roles, Schools...
I need to query the database, to list all employees that are employed by a certain school. That means all Employees, that has certain School ID in their corresponding role document must be returned in the list or array of results.
So far I have tried to find it like this:
const employees = mongoose.schema("Employees");
employees.find({})
.populate({
"path": "roles"
})
.then(function(docs){
docs.filter(function(){
// write my filters here....
})
});
but this still is inefficient and I can't make it work.
It has to be a smarter way...
This is my Employee document, which references a document in Roles Collection:
{
"_id" : { "$oid" : "57027f1b9522d363243abr42" },
"assignedRoles": [
{
"type": "teacher",
"schools": [
{
"schoolId": "57027f1b9522d3632423rf32",
"classes" : ["math", "science"],
"active": true
},
{
"schoolId": "57027f1b9522d36324252fs2",
"classes" : ["science"],
"active": true
},
{
"schoolId": "57027f1b9522d36324242f35",
"classes" : ["math"],
"active": true
},
]
},
{
"type": "manager",
"schools": [
{
"schoolId": "57027f1b9522d3632423rf32",
"active": true
},
{
"schoolId": "57027f1b9522d36324252fs2",
"active": true
}
]
}
{
"type": "principal",
"schools": [
{
"schoolId": "57027f1b9522d3632423rf32",
"active": true
}
]
}
],
"rolesMeta": "Meta Info for the Roles"
}
The following is the list of Schools - irrelevant for the task, I am just adding that for the completion:
{
"_id": { "$oid" : "57027f1b9522d3632423rf32" },
"name": "G.Washington HS",
"district": "six",
"state": "New Mexico"
},
{
"_id": { "$oid" : "57027f1b9522d36324252fs2" },
"name": "A. Lincoln HS",
"district": "six",
"state": "New Mexico"
},
{
"_id": { "$oid" : "57027f1b9522d36324242f35" },
"name": "T. Roosvelt HS",
"district": "four",
"state": "New Mexico"
}
The Resolution to the problem is to use MongoDB Aggregate framework to be able to drill down and "flatten" the array structure.
Here is how I have resolved it:
db.employees.aggregate([
// I use $project to specify which field I want selected from the collection
{
$project: {_id: 1, "roles": 1}
},
// $lookup helps me get results and populate the 'role' field from the Roles collection. This returns an array
{
$lookup: {
"from": "roles",
"localField": "roles",
"foreignField": "_id",
"as": "roles"
}
},
// $unwind is self explanatory - it flattens the array and creates multiple objects, instances of the parent object for each array element.
{
$unwind: "$roles"
},
// now repeat for each array in this document
{
$unwind: "$roles.assignedRoles"
},
// repeat again, since I need to look in the schools array.
{
$unwind: "$roles.assignedRoles.schools"
},
// The following line will return all records that the school ID checks out.
{
$match: {"roles.assignedRoles.schools.schoolId": school}
},
// $group is a necessary cleanup task so we avoid repeating employee records... Nicely Done! :)
{
$group: {
"_id": "$_id",
"roles": {"$push": "$roles"},
}
}
])
.exec()
.then(function (docs) {
// here you process the resulted docs.
});
Related
I have 2 collection in mongodb: Account, Information.
Account:
{
"_id": {
"$oid": "6348dc197a7b552660170d8b"
},
"username": "12345",
"password": "123dsgfdsgdfsg",
"email": "1243",
"role": "123",
"_infoid": {
"$oid": "6348dc197a7b552660170d8a"
}
}
Information:
{
"_id": {
"$oid": "6348dc197a7b552660170d8a"
},
"avatar": "hello",
"name": "Abcd",
"phonenumber": "012345678",
"address": "abcd"
}
I wanna change "phonenumber" to "123" but I just have "_id" of Account. Can I change it with aggregation pipeline?
Does this seem what you try to achieve?
// populate database with test data
use("test_db")
db.account.insertOne({
"_id": "6348dc197a7b552660170d8b",
"username": "12345",
"password": "123dsgfdsgdfsg",
"email": "1243",
"role": "123",
"_infoid": "6348dc197a7b552660170d8a"
})
db.information.insertOne({
"_id": "6348dc197a7b552660170d8a",
"avatar": "hello",
"name": "Abcd",
"phonenumber": "012345678",
"address": "abcd"
})
// define some test variables to use
let some_account_id = "6348dc197a7b552660170d8b"
let new_phone_number = "+9876543210"
// change data
db.account.aggregate([
{
$match: {_id: some_account_id}
},
{
$addFields: {phonenumber: new_phone_number}
},
{
$project: {phonenumber: 1, _id: "$_infoid"}
},
{
$merge:{
into: "information",
whenNotMatched: "fail",
}
}
])
// show final results
db.information.find()
Result:
[
{
"_id": "6348dc197a7b552660170d8a",
"avatar": "hello",
"name": "Abcd",
"phonenumber": "+9876543210",
"address": "abcd"
}
]
You don't need to duplicate _id, findOneAndUpdate() can be executed for this use case.
The definition of it is:
db.collection.findOneAndUpdate( filter, update, options )
which updates a single document based on the filter and sort criteria.
Below you can refer to the link for more details:
https://www.mongodb.com/docs/manual/reference/method/db.collection.findOneAndUpdate/
.
Performa a $lookup and perform some wrangling to your desired format. Finally do a $merge to update to the collection Information
db.Information.aggregate([
{
"$lookup": {
"from": "Account",
"localField": "_id",
"foreignField": "_infoid",
"pipeline": [
{
"$project": {
role: 1
}
}
],
"as": "AccountLookup"
}
},
{
"$unwind": "$AccountLookup"
},
{
$set: {
phonenumber: "$AccountLookup.role"
}
},
{
$unset: "AccountLookup"
},
{
"$merge": {
"into": "Information",
"on": "_id",
"whenMatched": "merge",
"whenNotMatched": "discard"
}
}
])
Here is the Mongo Playground for your reference.
I have 3 collections.
User:
{
"_id":ObjectId("60a495cdd4ba8b122899d415"),
"email":"br9#gmail.com",
"username":"borhan"
}
Panel:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415")
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
Team:
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
I want to receive a output from querying Panel colllection just like this:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415"),
"teams":[
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
]
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
I mean i want to add teams field (which is array of teams that user is existed on it) to each user in Panel collection
Here is my match query in mongoose to select specific panel:
panel_model.aggregate([
{
$match: {
users: {
$elemMatch: {user: ObjectId("60a495cdd4ba8b122899d415"), role:"admin"}
}
}
},
])
Is it possible to get my output with $lookup or $addFields aggregations?
You need to join all three collections,
$unwind to deconstruct the array
$lookup there are two kind of lookups which help to join collections. First I used Multiple-join-conditions-with--lookup, and I used standrad lookup to join Users and Teams collections.
$match to match the user's id
$expr - when you use $match inside lookup, u must use it.
$set to add new fields
$group to we already destructed using $unwind. No we need to restructure it
here is the code
db.Panel.aggregate([
{ $unwind: "$users" },
{
"$lookup": {
"from": "User",
"let": { uId: "$users.user" },
"pipeline": [
{
$match: {
$expr: {
$eq: [ "$_id", "$$uId" ]
}
}
},
{
"$lookup": {
"from": "Team",
"localField": "_id",
"foreignField": "users",
"as": "teams"
}
}
],
"as": "users.join"
}
},
{
"$set": {
"users.getFirstElem": {
"$arrayElemAt": [ "$users.join", 0 ]
}
}
},
{
$set: {
"users.teams": "$users.getFirstElem.teams",
"users.join": "$$REMOVE",
"users.getFirstElem": "$$REMOVE"
}
},
{
"$group": {
"_id": "$_id",
"name": { "$first": "name" },
"users": { $push: "$users" }
}
}
])
Working Mongo playground
Note : Hope the panel and user collections are in 1-1 relationship. Otherwise let me know
I have the following two collections:
{
"organizations": [
{
"_id": "1",
"name": "foo",
"users": { "1": "admin", "2": "member" }
},
{
"_id": "2",
"name": "bar",
"users": { "1": "admin" }
}
],
"users": [
{
"_id": "1",
"name": "john smith"
},
{
"_id": "2",
"name": "bob johnson"
}
]
}
The following query works to merge the users into members when I just use an array of the user ids to match, however, the users prop is an object.
{
"collection": "organizations",
"command": "aggregate",
"query": [
{
"$lookup": {
"from": "users",
"localField": "users",
"foreignField": "_id",
"as": "members"
}
}
]
}
What I'm hoping to do is lookup by id then create a members array from the results with the user object including the role (value of the users objects:
{
"_id": "1",
"name": "foo",
"users": {
"1": "admin",
"2": "member"
},
"members": [
{
"_id": "1",
"name": "john smith",
"role": "admin"
},
{
"_id": "2",
"name": "bob johnson",
"role": "user"
}
]
}
Here's the sandbox I have setup: https://mongoplayground.net/p/yhRpeRvJf3u
You really need to change your schema design, this will cause the performance on retrieving data,
$addFields to add new field usersArray convert users object to array using $objectToArray, the format will be k(key) and v(value),
$lookup to join users collection, set localField name to usersArray.k
$addFields, remove usersArray field using $$REMOVE,
$map iterate loop of members array and $reduce to iterate loop of usersArray and get matching role as per _id and merge current fields and role field using $mergeObjects
db.organizations.aggregate([
{
$addFields: {
usersArray: {
$objectToArray: "$users"
}
}
},
{
"$lookup": {
"from": "users",
"localField": "usersArray.k",
"foreignField": "_id",
"as": "members"
}
},
{
$addFields: {
usersArray: "$$REMOVE",
members: {
$map: {
input: "$members",
as: "m",
in: {
$mergeObjects: [
"$$m",
{
role: {
$reduce: {
input: "$usersArray",
initialValue: "",
in: { $cond: [{ $eq: ["$$this.k", "$$m._id"] }, "$$this.v", "$$value"] }
}
}
}
]
}
}
}
}
}
])
Playground
First of all, the problem with your query is you want to use a KEY to do the $lookup, then the members field always gonna be empty.
You are trying to use users as local field, but users is an object, so you need the key (users.1, users.2, ... )
To do this you need to use $objectToArray, which create an object array with two fields: k and v for key and value. So now, you can $lookup with the field users.k.
To get the query you need $unwind before $lookup because you also want the users filed into the new document.
With the new object created using $objectToArray, you can do $unwind to get the values in differents documents. And then $lookup to get the "join".
Here, localField uses the value k created by $objectToArray (the object key).
After that, $set to add the field with the role and $group again into one document.
Ive used _id to get the values without changes between stages, and into members push the members in each collection.
And then, $project to output the values you want. In this case, tha calues "stored" into _id and the array members in "one level" using $reduce.
So, the query you need I think is this:
db.organizations.aggregate([
{
"$match": {
"_id": "1"
}
},
{
"$set": {
"usersArray": {
"$objectToArray": "$users"
}
}
},
{
"$unwind": "$usersArray"
},
{
"$lookup": {
"from": "users",
"localField": "usersArray.k",
"foreignField": "_id",
"as": "members"
}
},
{
"$set": {
"members.role": "$usersArray.v"
}
},
{
"$group": {
"_id": {
"_id": "$_id",
"users": "$users",
"name": "$name"
},
"members": {
"$push": "$members"
}
}
},
{
"$project": {
"members": {
"$reduce": {
"input": "$members",
"initialValue": [],
"in": {
"$concatArrays": [
"$$value",
"$$this"
]
}
}
},
"users": "$_id.users",
"name": "$_id.name",
"_id": "$_id._id"
}
}
])
Example here
I have a document as below in Mongodb.
{
"vatInfo":{
"company": "apple"
},
"type": "manager",
"parent": "123"
}
And I have another document in same collection as below.
{
"type": "member",
"parentId": "123",
"id": "3"
}
And when I make client.find({id: 3, type: 'member'}), I want to get this that finds vatInfo automatically inside of find.
{
"type": "member",
"parentId": "123",
"id": "3",
"vatInfo":{
"company": "apple"
},
}
How should I make aggregate for this find? I dont want to make double find.
Thank you so much for reading it.
You can use the aggregation pipelines for that.
$match - find the parent documen.
$lookup - join the document from the other collection.
$project - modify the structure of the result.
// collection `client`
{
"vatInfo":{ "company": "apple" },
"type": "manager",
"parent": "123"
},
{
"type": "member",
"parentId": "123",
"id": "3"
}
// query example
db.getCollection('client').aggregate([
{ $match: { 'id': '3', 'type': 'member' } },
{
$lookup: {
from: 'client',
localField: 'parentId',
foreignField: 'parent',
as: 't01'
}
},
{
$project: {
'_id': 0,
'type': 1,
'parentId': 1,
'id': 1,
'vatInfo': { $arrayElemAt: [ "$t01.vatInfo", 0 ] }
}
}
])
// result
{
"type" : "member",
"parentId" : "123",
"id" : "3",
"vatInfo" : {
"company" : "apple"
}
}
To add the field of another document, you can use populate in Mongoose :
For this to work you need to have a reference in your schema between the client document and the parent document :
const clientSchema = Schema({
[...]
parentId: { type: Schema.Types.ObjectId, ref: 'ParentCollection' }
});
Then, you can use populate anywhere in your code :
client.findOne({id: 3, type: 'member'}).populate('parentId');
I have a database with some users, who belong to teams. Each team has a leader. Each user has a subject.
I want to collate teams by the leader's subject.
My data looks like this:
db={
"teams": [
{
_id: "t1",
members: [
{
"_id": "u1",
"leader": true
},
{
"_id": "u2"
},
{
"_id": "u3"
}
],
},
{
_id: "t2",
members: [
{
"_id": "u2",
"leader": true
},
{
"_id": "u4"
}
],
},
{
_id: "t3",
members: [
{
"_id": "u1",
"leader": true
},
{
"_id": "u4"
}
],
},
{
_id: "t4",
members: [
{
"_id": "u2",
"leader": true
}
],
},
],
"users": [
{
"_id": "u1",
"subject": "history"
},
{
"_id": "u2",
"subject": "maths"
},
{
"_id": "u3",
"subject": "geography"
},
{
"_id": "u4",
"subject": "french"
}
]
}
The result I want is:
{
"history": ["t1", "t3"],
"maths": ["t2", "t4"]
}
I have an aggregation that gets me the _id of every leader, and from there I can get the result I want in stages, by first finding the subject of every leader, then going back through the projects and assigning a subject to each project based on the identify of the leader. It works but it is inelegant and I think it will be slow. It seems to me there should be some better way to do this, maybe something like a join?
Is there a nifty way to get the result I want from a single MongoDB operation?
Here is a Mongo Playground with my data:
https://mongoplayground.net/p/SIJv9-hVNzJ
Many thanks for any help.
Edit: my test data are confusing because '_id' is used in both collections, making it hard to unpack the answer. Here is an updated Mongo Playground that uses different key names for each collection and helped me to understand the perfect answer.
Yes, you should join your collections on users._id with a $lookup, and then transform value to key with $arrayToObject (introduced in Mongodb 3.4.4)
Here is a possible way to do this :
db.teams.aggregate([
{
"$unwind": "$members"
},
{
"$match": {
"members.leader": true
}
},
{
"$lookup": {
"from": "users",
"localField": "members._id",
"foreignField": "_id",
"as": "users"
}
},
{
"$unwind": "$users"
},
{
"$group": {
"_id": "$users.subject",
"team": {
"$push": "$_id"
}
}
},
{
"$replaceRoot": {
"newRoot": {
"$arrayToObject": [
[
{
k: "$_id",
v: "$team"
}
]
]
}
}
}
])
try it online: mongoplayground.net/p/TuEpMzHkI-0