Here is the problem, I am unable to get the following result. Please look into the piece of json and help me out.
This is my data:
[
{
"user_id": "65asdfksadjfk3u4",
"lat": 23.4343,
"long": 15.2382
}
]
Currently my result is:
[
{
"_id": "65asdfksadjfk3u4",
"name": "Srini",
"age": 26,
"some other key": "some other values"
}
]
I need to get the collection from the user_id and add it to the same array object. As you can notice both lat and long are being removed in my current result.
[
{
"_id": "65asdfksadjfk3u4",
"name": "Srini",
"age": 26,
"some other keys": "some other values",
"lat": 23.4343,
"long": 15.2382
}
]
You can append the $lookup stage to join the current pipeline results with the users collections by the user_id fields and then use $mergeObjects in the $replaceRoot to merge the joined documents from users and the current results:
db.collection.aggregate([
/* current pipeline here */
{ "$lookup": {
"from": "users",
"localField": "_id",
"foreignField": "user_id",
"as": "user"
} },
{ "$replaceRoot": {
"newRoot": {
"$mergeObjects": [
{ "$arrayElemAt": [ "$user", 0 ] },
"$$ROOT"
]
}
} },
{ "$project": { "user": 0, "user_id": 0 } }
]);
Related
Collection A:
[{
"_id": 1,
"operation":"SEC",
"name":"x"
},{
"_id": 2,
"operation": "SEC",
"name": "y"
},
{
"_id": 3,
"operation": "SEC",
"name": "z"
}]
Collection B:
[{
"user": 1,
"operation":"SEC",
"name":"x",
"date": "2022-10-25"
},{
"user": 2,
"operation":"SEC",
"name":"y",
"date": "2022-10-25"
}
]
Expected output:
[
{
"_id": 3,
"operation": "SEC",
"name": "z"
}
]
I have two collections and I want to match from the first collection to the second collection by date and want to get only those that are not in the second collection.
You can use the following aggregation pipeline in order to achieve your desired outpu:
[
{
"$lookup": {
"from": "collectionB",
"localField": "_id",
"foreignField": "user",
"as": "collectionB"
}
},
{
$match: {
collectionB: {
$size: 0
}
}
},
{
$project: {
collectionB: 0
}
}
]
Please note that this is an efficient solution. You probably should add a $match step at the beginning in order to limit your results.
Hi I am looking to join data from two tables
What I am looking to do is create a "friend" relationship between two users using a lookup and find out how many emails each friend (if any sent)
here is a mongo DB playground... I am close, I just can't figure out how to get the 'other users' email information
I should only see emails from a's friends (b and c) and NOT any emails sent from a
This playground almost does what I want...
https://mongoplayground.net/p/KKocPm3fzEv
Here is the input for the above playground
db={
"users": [
{
"_id": "a",
"email": "a#test.com",
},
{
"_id": "b",
"email": "b#test.com",
},
{
"_id": "c",
"email": "c#test.com",
}
],
"friends": [
{
"userId": "a",
"otherUserId": "b"
},
{
"userId": "a",
"otherUserId": "c"
},
],
"emailsSent": [
{
"userId": "a",
"number": "25"
},
{
"userId": "b",
"number": "3"
},
]
}
Here is the output from the above playground
[
{
"_id": "a",
"a_myfriends": [
{
"_id": ObjectId("5a934e000102030405000002"),
"otherUserId": "b",
"userId": "a"
},
{
"_id": ObjectId("5a934e000102030405000003"),
"otherUserId": "c",
"userId": "a"
}
],
"email": "a#test.com",
"emailaddr": [
{
"_id": "b",
"email": "b#test.com"
},
{
"_id": "c",
"email": "c#test.com"
}
],
"emailsent": [
{
"_id": ObjectId("5a934e000102030405000001"),
"number": "3",
"userId": "b"
}
]
}
]
There are three arrays of information now... how do I join them all together so each entry in the array is only for 'that' friend?
this is what I'd like to end up with
{
"_id": "a",
"a_myfriends": [
{
"otherUserId": "b",
"email": "b#test.com",
"number": "3"
},
{
"otherUserId": "c",
"email": "c#test.com"
}
]
}
NOTE: I tried concatenating unions from this article, but I think it's not working due to the disparity of IDs for the user (eg _id and userId)
MongoDB: Combine data from multiple collections into one..how?
I think you can reduce the number of collections => reduce the $lookups and data will look more simple
How about this schema of 1 collection?
Even if someone has too many friends like 200.000, you could have an extra field {"extra_friends" "_id"}, and rarely use the a secondary friends collections going rarely max to 2 collections.
users=
[
{
"_id": "a",
"email": "a#test.com",
"emails-send" 25,
"friends" ["b" "c"]
},
{
"_id": "b",
"email": "b#test.com",
"emails-send" 3,
"friends" [....]
},
{
"_id": "c",
"email": "c#test.com",
"emails-send" 0
"friends" [....]
}
]
Query
(for your schema, produces the expected data)
a series of $lookup and $unwind
from users
-to friends (get friends)
-to users (get friends email)
-to emailsSent (get number of emails for those that sended)
some $set in the middle to keep the schema simple
group by _id back and combine those friends
*query could become smaller a bit, but this way its easy to follow, stage by stage and see what is happening
Test code here
db.users.aggregate([
{
"$match": {
"$expr": {
"$eq": [
"$_id",
"a"
]
}
}
},
{
"$lookup": {
"from": "friends",
"localField": "_id",
"foreignField": "userId",
"as": "myfriends"
}
},
{
"$unwind": {
"path": "$myfriends"
}
},
{
"$lookup": {
"from": "users",
"localField": "myfriends.otherUserId",
"foreignField": "_id",
"as": "myfriendsEmails"
}
},
{
"$unwind": {
"path": "$myfriendsEmails"
}
},
{
"$set": {
"myfriends.email": "$myfriendsEmails.email"
}
},
{
"$unset": [
"myfriendsEmails",
"myfriends._id",
"myfriends.userId"
]
},
{
"$lookup": {
"from": "emailsSent",
"localField": "myfriends.otherUserId",
"foreignField": "userId",
"as": "friendsEmails"
}
},
{
"$unwind": {
"path": "$friendsEmails",
"preserveNullAndEmptyArrays": true
}
},
{
"$set": {
"myfriends.number": "$friendsEmails.number"
}
},
{
"$unset": [
"friendsEmails"
]
},
{
"$group": {
"_id": "$_id",
"email": {
"$first": "$email"
},
"friends": {
"$push": "$myfriends"
}
}
}
])
I have 3 collections.
User:
{
"_id":ObjectId("60a495cdd4ba8b122899d415"),
"email":"br9#gmail.com",
"username":"borhan"
}
Panel:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415")
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
Team:
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
I want to receive a output from querying Panel colllection just like this:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415"),
"teams":[
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
]
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
I mean i want to add teams field (which is array of teams that user is existed on it) to each user in Panel collection
Here is my match query in mongoose to select specific panel:
panel_model.aggregate([
{
$match: {
users: {
$elemMatch: {user: ObjectId("60a495cdd4ba8b122899d415"), role:"admin"}
}
}
},
])
Is it possible to get my output with $lookup or $addFields aggregations?
You need to join all three collections,
$unwind to deconstruct the array
$lookup there are two kind of lookups which help to join collections. First I used Multiple-join-conditions-with--lookup, and I used standrad lookup to join Users and Teams collections.
$match to match the user's id
$expr - when you use $match inside lookup, u must use it.
$set to add new fields
$group to we already destructed using $unwind. No we need to restructure it
here is the code
db.Panel.aggregate([
{ $unwind: "$users" },
{
"$lookup": {
"from": "User",
"let": { uId: "$users.user" },
"pipeline": [
{
$match: {
$expr: {
$eq: [ "$_id", "$$uId" ]
}
}
},
{
"$lookup": {
"from": "Team",
"localField": "_id",
"foreignField": "users",
"as": "teams"
}
}
],
"as": "users.join"
}
},
{
"$set": {
"users.getFirstElem": {
"$arrayElemAt": [ "$users.join", 0 ]
}
}
},
{
$set: {
"users.teams": "$users.getFirstElem.teams",
"users.join": "$$REMOVE",
"users.getFirstElem": "$$REMOVE"
}
},
{
"$group": {
"_id": "$_id",
"name": { "$first": "name" },
"users": { $push: "$users" }
}
}
])
Working Mongo playground
Note : Hope the panel and user collections are in 1-1 relationship. Otherwise let me know
I have a question about how to aggregate mongoose deep collection, for example i have 3 collections:
Specialization job
[
{
"id": 122,
"name": "Administration",
},
{
"id": 133,
"name": "IT/Computer"
}
]
Job Position (relation with specialization collection, one specialization job have many job position)
[
{
"id": 1,
"name": "Manager",
"id_specialization": 122
},
{
"id": 2,
"name": "Front End Developer",
"id_specialization": 133
}
]
Job (relation with job position)
[
{
"id": 1,
"id_job_position": "1",
"location": "New York"
},
{
"id": 2,
"id_job_position": "2",
"location": "Dallas"
}
]
I want to make a filter by "specialization", if i choose "122" id of specialization, then i want to show job data which is job position is in that specialization.
[
{
"id": 1,
"id_job_position": "1",
"location": "New York"
}
]
Thanks before.
Demo - https://mongoplayground.net/p/1Bn1OUOODrT
Use $lookup
Performs a left outer join to an unsharded collection in the same database to filter in documents from the "joined" collection for processing. To each input document, the $lookup stage adds a new array field whose elements are the matching documents from the "joined" collection. The $lookup stage passes these reshaped documents to the next stage.
db.jobPosition.aggregate([
{
$match: {
id_specialization: 122
}
},
{
"$lookup": {
"from": "job",
"localField": "id",
"foreignField": "id_job_position",
"as": "jobs"
}
},
{
$project: {
jobs: 1
}
},
{
$unwind: "$jobs" // break into individual documents can skip if only 1 document will come from lookup
},
{
"$replaceRoot": {
"newRoot": "$jobs"
}
},
{
$project: { _id: 0 }
}
])
if only 1 document will come from the lookup
Demo - https://mongoplayground.net/p/CNevmFlEWWZ
db.jobPosition.aggregate([
{
$match: {
id_specialization: 122
}
},
{
"$lookup": {
"from": "job",
"localField": "id",
"foreignField": "id_job_position",
"as": "jobs"
}
},
{
"$replaceRoot": {
"newRoot": {
"$first": "$jobs"
}
}
},
{
"$project": {
_id: 0
}
}
])
I have the following two collections:
{
"organizations": [
{
"_id": "1",
"name": "foo",
"users": { "1": "admin", "2": "member" }
},
{
"_id": "2",
"name": "bar",
"users": { "1": "admin" }
}
],
"users": [
{
"_id": "1",
"name": "john smith"
},
{
"_id": "2",
"name": "bob johnson"
}
]
}
The following query works to merge the users into members when I just use an array of the user ids to match, however, the users prop is an object.
{
"collection": "organizations",
"command": "aggregate",
"query": [
{
"$lookup": {
"from": "users",
"localField": "users",
"foreignField": "_id",
"as": "members"
}
}
]
}
What I'm hoping to do is lookup by id then create a members array from the results with the user object including the role (value of the users objects:
{
"_id": "1",
"name": "foo",
"users": {
"1": "admin",
"2": "member"
},
"members": [
{
"_id": "1",
"name": "john smith",
"role": "admin"
},
{
"_id": "2",
"name": "bob johnson",
"role": "user"
}
]
}
Here's the sandbox I have setup: https://mongoplayground.net/p/yhRpeRvJf3u
You really need to change your schema design, this will cause the performance on retrieving data,
$addFields to add new field usersArray convert users object to array using $objectToArray, the format will be k(key) and v(value),
$lookup to join users collection, set localField name to usersArray.k
$addFields, remove usersArray field using $$REMOVE,
$map iterate loop of members array and $reduce to iterate loop of usersArray and get matching role as per _id and merge current fields and role field using $mergeObjects
db.organizations.aggregate([
{
$addFields: {
usersArray: {
$objectToArray: "$users"
}
}
},
{
"$lookup": {
"from": "users",
"localField": "usersArray.k",
"foreignField": "_id",
"as": "members"
}
},
{
$addFields: {
usersArray: "$$REMOVE",
members: {
$map: {
input: "$members",
as: "m",
in: {
$mergeObjects: [
"$$m",
{
role: {
$reduce: {
input: "$usersArray",
initialValue: "",
in: { $cond: [{ $eq: ["$$this.k", "$$m._id"] }, "$$this.v", "$$value"] }
}
}
}
]
}
}
}
}
}
])
Playground
First of all, the problem with your query is you want to use a KEY to do the $lookup, then the members field always gonna be empty.
You are trying to use users as local field, but users is an object, so you need the key (users.1, users.2, ... )
To do this you need to use $objectToArray, which create an object array with two fields: k and v for key and value. So now, you can $lookup with the field users.k.
To get the query you need $unwind before $lookup because you also want the users filed into the new document.
With the new object created using $objectToArray, you can do $unwind to get the values in differents documents. And then $lookup to get the "join".
Here, localField uses the value k created by $objectToArray (the object key).
After that, $set to add the field with the role and $group again into one document.
Ive used _id to get the values without changes between stages, and into members push the members in each collection.
And then, $project to output the values you want. In this case, tha calues "stored" into _id and the array members in "one level" using $reduce.
So, the query you need I think is this:
db.organizations.aggregate([
{
"$match": {
"_id": "1"
}
},
{
"$set": {
"usersArray": {
"$objectToArray": "$users"
}
}
},
{
"$unwind": "$usersArray"
},
{
"$lookup": {
"from": "users",
"localField": "usersArray.k",
"foreignField": "_id",
"as": "members"
}
},
{
"$set": {
"members.role": "$usersArray.v"
}
},
{
"$group": {
"_id": {
"_id": "$_id",
"users": "$users",
"name": "$name"
},
"members": {
"$push": "$members"
}
}
},
{
"$project": {
"members": {
"$reduce": {
"input": "$members",
"initialValue": [],
"in": {
"$concatArrays": [
"$$value",
"$$this"
]
}
}
},
"users": "$_id.users",
"name": "$_id.name",
"_id": "$_id._id"
}
}
])
Example here