I need to join the faults array
{
"_id": "99812930-37CE-456F-A9D9-837E9E3F712A",
"faultsChanged": [
{
"_id": "7C628A46-7E80-4615-8B08-10C5E9A6B1D7",
"faults": [
"BF221A71-0217-42E7-B853-53112EDA9694",
"E4A54172-7E93-49C4-840B-8E6116116979"
],
"isDeleted": false,
"partition": "indego",
"sessionUuid": "A83CE9A1-7539-493F-8BA4-6FBE25B18B57",
"source": "1",
"timestamp": {
"$date": {
"$numberLong": "1630603342700"
}
},
"unmigratedNote": null,
"uuid": "7C628A46-7E80-4615-8B08-10C5E9A6B1D7"
}
]
}
So that it gets replaced by a document from another collection OOFaultEntry.
New to the aggregation pipeline, I have tried
{$lookup:{
from: "OOFaultEntry",
localField: 'faultsChanged.faults',
foreignField: "_id",
let:{faults:"$faultsChanged.faults"},
pipeline:[],
as:"faults"
}}
but this just created a key on the result and was not embedded inside of each respective faultChanged object
expected result
OOFaultEntry would be JOINED on _id . So ,it should look something like
{
"_id": "99812930-37CE-456F-A9D9-837E9E3F712A",
"faultsChanged": [
{
"_id": "7C628A46-7E80-4615-8B08-10C5E9A6B1D7",
"faults": [
{
"_id": "BF221A71-0217-42E7-B853-53112EDA9694",
"some_key": "value"
},
{
"_id": "E4A54172-7E93-49C4-840B-8E6116116979",
"some_key": "value"
}
],
"isDeleted": false,
"partition": "indego",
"sessionUuid": "A83CE9A1-7539-493F-8BA4-6FBE25B18B57",
"source": "1",
"timestamp": {
"$date": {
"$numberLong": "1630603342700"
}
},
"unmigratedNote": null,
"uuid": "7C628A46-7E80-4615-8B08-10C5E9A6B1D7"
}
]
}
One way would be $unwind the faultsChanged array first. Perform the $lookup and regroup the results.
db.Faults.aggregate([
{
"$unwind": "$faultsChanged"
},
{
"$lookup": {
"from": "00FaultEntry",
"localField": "faultsChanged.faults",
"foreignField": "_id",
"as": "faultsChanged.faults"
}
},
{
"$group": {
"_id": "$_id",
"faultsChanged": {
$push: "$faultsChanged"
}
}
}
])
Mongo Playground
Related
I have 3 collections.
User:
{
"_id":ObjectId("60a495cdd4ba8b122899d415"),
"email":"br9#gmail.com",
"username":"borhan"
}
Panel:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415")
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
Team:
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
I want to receive a output from querying Panel colllection just like this:
{
"_id": ObjectId("60a495cdd4ba8b122899d417"),
"name": "borhan",
"users": [
{
"role": "admin",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a495cdd4ba8b122899d418"),
"user": ObjectId("60a495cdd4ba8b122899d415"),
"teams":[
{
"_id":ObjectId("60a495e0d4ba8b122899d419"),
"title":"New Teams",
"users":[
ObjectId("60a495cdd4ba8b122899d415")
],
"panel":ObjectId("60a495cdd4ba8b122899d417")
}
]
},
{
"role": "member",
"joined": "2021-05-19T04:35:47.474Z",
"status": "active",
"_id": ObjectId("60a49600d4ba8b122899d41a"),
"user": ObjectId("60a34e167958972d7ce6f966")
}
],
}
I mean i want to add teams field (which is array of teams that user is existed on it) to each user in Panel collection
Here is my match query in mongoose to select specific panel:
panel_model.aggregate([
{
$match: {
users: {
$elemMatch: {user: ObjectId("60a495cdd4ba8b122899d415"), role:"admin"}
}
}
},
])
Is it possible to get my output with $lookup or $addFields aggregations?
You need to join all three collections,
$unwind to deconstruct the array
$lookup there are two kind of lookups which help to join collections. First I used Multiple-join-conditions-with--lookup, and I used standrad lookup to join Users and Teams collections.
$match to match the user's id
$expr - when you use $match inside lookup, u must use it.
$set to add new fields
$group to we already destructed using $unwind. No we need to restructure it
here is the code
db.Panel.aggregate([
{ $unwind: "$users" },
{
"$lookup": {
"from": "User",
"let": { uId: "$users.user" },
"pipeline": [
{
$match: {
$expr: {
$eq: [ "$_id", "$$uId" ]
}
}
},
{
"$lookup": {
"from": "Team",
"localField": "_id",
"foreignField": "users",
"as": "teams"
}
}
],
"as": "users.join"
}
},
{
"$set": {
"users.getFirstElem": {
"$arrayElemAt": [ "$users.join", 0 ]
}
}
},
{
$set: {
"users.teams": "$users.getFirstElem.teams",
"users.join": "$$REMOVE",
"users.getFirstElem": "$$REMOVE"
}
},
{
"$group": {
"_id": "$_id",
"name": { "$first": "name" },
"users": { $push: "$users" }
}
}
])
Working Mongo playground
Note : Hope the panel and user collections are in 1-1 relationship. Otherwise let me know
I'm using Python with pymongo to query from the database.
I have 3 different collections:
1st:
# Projects collection
{
"_id": "A",
},
{
"_id": "B",
},
{
"_id": "C"
},
..
2nd:
# Episodes collection
{
"_id": "A/Episode01",
"project": "A",
"name": "Episode01"
},
{
"_id": "A/Episode02",
"project": "A",
"name": "Episode02"
},
{
"_id": "B/Episode01",
"project": "B",
"name": "Episode01"
},
..
3rd:
# Sequences collection
{
"_id": "A/Episode01/Sequence01",
"project": "A",
"episode": "Episode01",
"name": "Sequence01"
},
{
"_id": "A/Episode02/Sequence02",
"project": "A",
"episode": "Episode02",
"name": "Sequence02"
},
{
"_id": "B/Episode01/Sequence01",
"project": "B",
"episode": "Episode01",
"name": "Sequence01"
},
..
I want to use aggregate to query project A and get all of its corresponding episodes and sequences like this:
{
"_id": "A",
"episodes":
[
{
"_id": "A/Episode01",
"project": "A",
"name": "Episode01",
"sequences":
[
{
"_id": "A/Episode01/Sequence01",
"project": "A",
"episode": "Episode01",
"name": "Sequence01"
},
]
},
{
"_id": "A/Episode02",
"project": "A",
"name": "Episode02",
"sequences":
[
{
"_id": "A/Episode02/Sequence02",
"project": "A",
"episode": "Episode02",
"name": "Sequence02"
},
]
},
]
}
I can get as far as getting the proper episodes, but I'm not sure how to add an embed field for any matching sequences. Is it possible to do this all in a single pipeline query?
Right now my query is looking like this:
[
{"$match": {
"_id": "A"}
},
{"$lookup": {
"from": "episodes",
"localField": "_id",
"foreignField": "project",
"as": "episodes"}
},
{"$group": {
"_id": {
"_id": "$_id",
"episodes": "$episodes"}
}}
]
You can do like following
use $match to match the document
use uncorrelated queries to join two collection. But normal joining also possible as you have written. This is easier when we get some complex situations.
Mongo script is given below
[
{
"$match": {
"_id": "A"
}
},
{
$lookup: {
from: "Episodes",
let: {
id: "$_id"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$project",
"$$id"
]
}
}
},
{
$lookup: {
from: "Sequences",
let: {
epi: "$name"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$episode",
"$$epi"
]
}
}
}
],
as: "sequences"
}
}
],
as: "episodes"
}
}
]
Working Mongo playground
Update 01
Using standard lookup
[
{
"$match": {
"_id": "A"
}
},
{
"$lookup": {
"from": "Episodes",
"localField": "_id",
"foreignField": "project",
"as": "episodes"
}
},
{
$unwind: "$episodes"
},
{
"$lookup": {
"from": "Sequences",
"localField": "episodes.name",
"foreignField": "episode",
"as": "episodes.sequences"
}
},
{
$group: {
_id: "$episodes._id",
episodes: {
$addToSet: "$episodes"
}
}
}
]
Working Mongo playground
I am using aggregate to group and populate the result like below:
{
"$group": {
"_id": "$userId",
"projectId": { "$push": "$projectId" }
}
},
{
"$lookup": {
"from": "users",
"localField": "_id",
"foreignField": "_id",
"as": "user"
}
},
{ $unwind:"$user" },
{
"$lookup": {
"from": "projects",
"localField": "projectId",
"foreignField": "_id",
"as": "projects"
}
}
But i want to populate specific fields from that result For this I tried
$project, But it combining projectId into one array and projectName into another array.Below is my result json:
[
{
"_id": "5c0a29e597e71a0d28b910aa",
"projectId": [
"5c0a2a8897e71a0d28b910ac",
"5c0a4083753a321c6c4ee024"
],
"user": {
"_id": "5c0a29e597e71a0d28b910aa",
"firstName": "Amit"
"lastName": "kumar",
"type": "developer",
"status": "active"
},
"projects": [
{
"_id": "5c0a2a8897e71a0d28b910ac",
"skypeId": "",
"projectName": "LN-PM",
"status": "ongoing",
"assignId": "5c0a2a0a97e71a0d28b910ab"
},
{
"_id": "5c0a4083753a321c6c4ee024",
"skypeId": "",
"status": "pending",
"assignId": "5c0a2a0a97e71a0d28b910ab"
}
]
}
]
Now i want to get the only "firstName and _id" field from user field and "projectName and _id" field from the projects field
You can use below aggregation with mongodb 3.6 and above
With the newer $lookup syntax you can use $projection inside the $lookup pipeline
db.collection.aggregate([
{ "$group": {
"_id": "$userId",
"projectId": { "$push": "$projectId" }
}},
{ "$lookup": {
"from": "users",
"let": { "userId": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$eq": [ "$_id", "$$userId" ] }}},
{ "$project": { "firstName": 1 }}
],
"as": "user"
}},
{ "$unwind": "$user" },
{ "$lookup": {
"from": "projects",
"let": { "projectId": "$projectId" },
"pipeline": [
{ "$match": { "$expr": { "$in": [ "$_id", "$$projectId" ] }}},
{ "$project": { "projectName": 1 }}
],
"as": "projects"
}}
])
Here i am try to get aggregated result from my challenge collection with challengeusers and challengeusers has the user_id and i used $lookup to join the users too.
When i use this query, i am getting following output.
"challenges": [
{
"_id": "5b7bf6fd87ec106308d7e3c1",
"start_date": "2018-08-09T12:40:21.470Z",
"end_date": "2018-08-05T12:40:21.470Z",
"challnegedusers": [
{
"chalenge_id": "5b7bf6fd87ec106308d7e3c1",
"user_id": "5b75623db457045e3bb12e0a",
"status": 1
},
{
"user_id": "5b75643c0a97791bcc9ed64c",
"status": 1
},
{
"user_id": "5b756144b457045e3bb12e08",
"status": 1
}
],
"users": [
{
"_id": "5b756144b457045e3bb12e08",
"first_name": "XYZ"
},
{
"_id": "5b75623db457045e3bb12e0a",
"first_name": "BAC"
},
{
"_id": "5b75643c0a97791bcc9ed64c",
"first_name": "YTA"
}
]
}
]
But i want the challengeusers and users to merge in a single object.
Most of all i want the status of challengeusers with user's info.
expected output:
"challenges": [
{
"_id": "5b7bf6fd87ec106308d7e3c1",
"start_date": "2018-08-09T12:40:21.470Z",
"end_date": "2018-08-05T12:40:21.470Z",
"challnegedusers": [
{
"user_id": "5b75623db457045e3bb12e0a",
"status": 1,
"first_name": "BAC"
},
{
"user_id": "5b75643c0a97791bcc9ed64c",
"status": 1,
"first_name": "YTA"
},
{
"user_id": "5b756144b457045e3bb12e08",
"status": 1,
"first_name": "XYZ"
}
]
}
]
MongoDB Aggregate Query that i am using.
let challenges = await ChallengeModel.aggregate([
{ $match: criteria },
{ $lookup: {
from: 'challengeusers',
localField: '_id',
foreignField: 'challenge_id',
as: 'challnegedusers'
} },
{ $lookup: {
from: 'appusers',
localField: 'challnegedusers.user_id',
foreignField: '_id',
as: 'users'
} },
{ $sort: {created_at: -1}}
]);
You can try below aggregation in mongodb 3.6
ChallengeModel.aggregate([
{ "$match": criteria },
{ "$lookup": {
"from": "challengeusers",
"let": { "challengeusersId": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$eq": [ "$challenge_id", "$$challengeusersId" ] } } },
{ "$lookup": {
"from": "appusers",
"let": { "user_id": "$user_id" },
"pipeline": [
{ "$match": { "$expr": { "$eq": [ "$_id", "$$user_id" ] } } },
],
"as": "user"
}},
{ "$unwind": "$user" },
{ "$addFields": { "first_name": "$user.first_name" }},
{ "$project": { "user": 0 }}
],
"as": "challnegedusers"
}}
])
And with your approach you can try this
ChallengeModel.aggregate([
{ "$match": criteria },
{ "$lookup": {
"from": "challengeusers",
"localField": "_id",
"foreignField": "challenge_id",
"as": "challnegedusers"
}},
{ "$unwind": "challnegedusers" },
{ "$lookup": {
"from": "appusers",
"localField": "challnegedusers.user_id",
"foreignField": "_id",
"as": "challnegedusers.user"
}},
{ "$unwind": "challnegedusers.user" },
{ "$addFields": { "challnegedusers.first_name": "$challnegedusers.user.first_name" }},
{ "$sort": { "created_at": -1 }},
{ "$group": {
"_id": "$_id",
"start_date": { "$first": "$start_date" }
"end_date": { "$first": "$end_date" },
"challnegedusers": { "$push": "$challnegedusers" }
}}
])
I'm a MongoDb novice. I'm getting pretty good, but no expert yet. I'm trying setup my collections in a way that makes sense. I'd like to keep some links to foreign docs inside arrays of just _ids and also arrays of objects that have _ids.
I created a JSON doc with notes that I think fully shows what I'm trying to do...
// ( item ) Character Inventory/Items collection
[
{
"_id": "1234",
"name": "Sword",
"descr": "Long sword, well worn, light rust",
"encumber": 2,
"del": false
},
{
"_id": "1271",
"name": "Pouch",
"descr": "Small leather waist pouch, suitable for coins",
"encumber": 0,
"del": false
}
]
// ( charnpcclass ) Character Classes collection
[
{ "_id": "2", "name": "Thief", "del": false },
{ "_id": "3", "name": "Cleric", "del": false }
]
// ( charnpcalign ) Character Alignments collection
[
{ "_id": "3", "name": "Lawful Good", "del": false },
{ "_id": "4", "name": "Neutral", "del": false }
]
// ( character ) Characters collection
[
{
"_id": "3345",
"name": "Offut 'Dead Dog' Dubro",
"description": "Halfling, scruffy, looks homeless",
"align": ObjectId("4"),
"classes": [
ObjectId("2"),
ObjectId("3")
],
"carrying": [
{ "itemId": ObjectId("1271"), "qty":1, "where": "Sheath inside vest", "visible": false }
{ "itemId": ObjectId("1234"), "qty":1, "where": "Sword scabbard at waist", "visible": true }
],
"del": false
}
]
// ------------------------------------------------------------
// This is my MongoDb aggregation in the REST api routes
var linkedModels = [
{
"$match": { "del": false }
}, {
"$lookup": {
from: "charnpcclass",
localField: "classes",
foreignField: "_id",
as: "linked_classes"
}
}, {
"$lookup": {
from: "charnpcalign",
localField: "alignId",
foreignField: "_id",
as: "linked_align"
}
}, {
"$lookup": {
from: "item",
localField: "carrying.itemId",
foreignField: "_id",
as: "linked_carrying"
}
}
];
db.collection('character').aggregate(linkedModels).toArray(function (err, docs) {
res.json(201, docs);
next();
});
// Query for Character, return items carrying with data from items collection
// ------------------------------------------------------------
// WHAT I *WANT* IN RESPONSE...
{
"id": "3345",
"name": "Offut 'Dead Dog' Dubro",
"description": "Halfling, scruffy, looks homeless",
"align": "4",
"classes": [
"2",
"3"
],
"carrying": [
{ "itemId": "1271", "qty":1, "where": "Sheath inside vest", "visible": false }
{ "itemId": "1234", "qty":1, "where": "Sword scabbard at waist", "visible": true }
],
"linked_align": [
{ "_id": "4", "name": "Neutral" },
],
"linked_classes": [
{ "_id": "2", "name": "Thief" },
{ "_id": "3", "name": "Cleric" }
],
"linked_carrying": [
{ "_id": "1271", "name": "Dagger", "encumber": 0 },
{ "_id": "1234", "name": "Sword", "encumber": 2 }
]
}
// ------------------------------------------------------------
// WHAT I ACTUALLY GET IN RESPONSE
{
"id": "3345",
"name": "Offut 'Dead Dog' Dubro",
"description": "Halfling, scruffy, looks homeless",
"align": "4",
"classes": [
"2",
"3"
],
"carrying": [
{ "itemId": "1271", "qty":1, "where": "Sheath inside vest", "visible": false }
{ "itemId": "1234", "qty":1, "where": "Sword scabbard at waist", "visible": true }
],
"linked_align": [
{ "_id": "4", "name": "Neutral" },
],
"linked_classes": [],
"linked_carrying": []
}
The problem that I hope you noticed is just above, at bottom of JSON response example. My linked arrays are empty and I'm not sure how to solve this.
I would greatly appreciate your expert MongoDb querying advice :-)
You have to $unwind to flatten the both scalar and sub document foreign _ids and add $group stage at the end of the pipeline to get back the original structure.
$first accumulator to keep the fields and $push with $arrayElemAt to accumulate the array values to adjust for $unwind
var linkedModels = [
{
"$match": { "del": false }
},
{
"$lookup": {
from: "charnpcalign",
localField: "align",
foreignField: "_id",
as: "linked_align"
}
},
{
"$unwind":"$classes"
},
{
"$lookup": {
from: "charnpcclass",
localField: "classes",
foreignField: "_id",
as: "linked_classes"
}
},
{
"$group": {
"_id": "$_id",
"name": {"$first":"$name"},
"align": {"$first":"$align"},
"classes":{"$push":"$classes"},
"carrying":{"$first":"$carrying"},
"linked_align":{"$first":"$linked_align"},
"linked_classes":{"$push":{"$arrayElemAt":["$linked_classes",0]}}
}
},
{
"$unwind":"$carrying"
},
{
"$lookup": {
from: "item",
localField: "carrying.itemId",
foreignField: "_id",
as: "linked_carrying"
}
},
{
"$group": {
"_id": "$_id",
"name": {"$first":"$name"},
"align": {"$first":"$align"},
"classes":{"$first":"$classes"},
"linked_align":{"$first":"$linked_align"},
"carrying":{"$push":"$carrying"},
"linked_carrying":{"$push":{"$arrayElemAt":["$linked_carrying",0]}}
}
}
]
You don't need the $unwind on the scalar array (classes) in 3.4 version and you can replace the {"classes":{"$push":"$classes"}} & {"linked_classes":{"$push":{$arrayElemAt:["$linked_classes",0]}}} with {"classes":{"$first":"$classes"}} & {"linked_classes":{"$first":"$linked_classes"}} respectively.