I am having a collection which contains the data like the following and want to have the desirable output which I have mentioned below.
db={
collectionA: [
{
"id": ObjectId("63b7c24c06ebe7a8fd11777b"),
"uniqueRefId": "UUID-2023-0001",
"products": [
{
"productIndex": 1,
"productCategory": ObjectId("63b7c24c06ebe7a8fd11777b"),
"productOwners": [
ObjectId("63b7c2fd06ebe7a8fd117781")
]
},
{
"productIndex": 2,
"productCategory": ObjectId("63b7c24c06ebe7a8fd11777b"),
"productOwners": [
ObjectId("63b7c2fd06ebe7a8fd117781"),
ObjectId("63b7c12706ebe7a8fd117778")
]
},
{
"productIndex": 3,
"productCategory": "",
"productOwners": ""
}
]
}
],
collectionB: [
{
"_id": ObjectId("63b7c2fd06ebe7a8fd117781"),
"fullname": "Jim Corbett",
"email": "jim.corbett#pp.com"
},
{
"_id": ObjectId("63b7c12706ebe7a8fd117778"),
"fullname": "Carry Minatti",
"email": "carry.minatty#pp.com"
},
]
}
Desirable Output = [
{
"id": ObjectId("507f1f77bcf86cd799439011"),
"uniqueRefId": "UUID-2023-0001",
"products": [
{
"productIndex": 1,
"productCategory": ObjectId('614g2f77bff86cd755439021'),
"productOwners": [
{
"_id": ObjectId("63ac1e59c0afb8b6f2d41acd"),
"fullname": "Jim Corbett",
"email": "jim.corbett#pp.com"
}
]
},
{
"productIndex": 2,
"productCategory": ObjectId('614g2f77bff86cd755439021'),
"productOwners": [
{
"_id": ObjectId("63ac1e59c0afb8b6f2d41acd"),
"fullname": "Jim Corbett",
"email": "jim.corbett#pp.com"
},
{
"_id": ObjectId("63ac1e59c0afb8b6f2d41ace"),
"fullname": "Carry Minatti",
"email": "carry.minatty#pp.com"
}
]
},
{
"productIndex": 3,
"productCategory": "",
"productOwners": ""
}
]
}
]
In the collectionA we are having other documents as well, its not just one document.
Similarly for collectionB we are having other documents too.
How we can get this desirable output?
I am expecting the mongodb query for getting this solution.
I have implemented the lookup like the following
db.collectionA.aggregate([
{
"$lookup": {
"from": "collectionB",
"localField": "products.productOwners",
"foreignField": "_id",
"as": "inventory_docs"
}
}
])
You can try this:
db.collectionA.aggregate([
{
"$unwind": "$products"
},
{
"$lookup": {
"from": "collectionB",
"localField": "products.productOwners",
"foreignField": "_id",
"as": "products.productOwners"
}
},
{
"$group": {
"_id": {
id: "$id",
uniqueRefId: "$uniqueRefId"
},
"products": {
"$push": "$products"
}
}
},
{
"$project": {
id: "$_id.id",
uniqueRefId: "$_id.uniqueRefId",
products: 1,
_id: 0
}
}
])
Playground link.
In this query, we do the following:
First we unwind the products array, using $unwind.
Then we calculate productOwners, using $lookup.
Then we group the unwinded elements, using $group.
Finally we, project the desired output using $project.
Related
I have a two collections "datasets" and "users". I tried to lookup for array of object both collections.
I want to join the "datasets.stateHistory.date" field and "users.prices.date" field. get the result of the datasets collection i want sum of "users.prices.price" sum values
Datasets json Data:
"datasets": [
{
"colorDescription": "braun, rose gold",
"stateHistory": [
{
"state": "scanning",
"date": "2022-02-22T13:06:13.493+00:00"
},
{
"state": "scanned",
"date": "2022-02-18T13:06:13.493+00:00"
},
{
"state": "reconstructing",
"date": "2022-02-16T13:06:13.493+00:00"
}
]
},
{
"colorDescription": "beige, silber",
"stateHistory": [
{
"state": "scanning",
"date": "2022-03-22T13:06:13.493+00:00"
},
{
"state": "scanned",
"date": "2022-03-18T13:06:13.493+00:00"
},
{
"state": "reconstructing",
"date": "2022-03-16T13:06:13.493+00:00"
}
]
}
]
Users json Data:
"users": [
{
"name": "Aravinth",
"prices": [
{
"date": "2022-02-16T13:06:13.493+00:00",
"price": 45
},
{
"date": "2022-03-22T13:06:13.493+00:00",
"price": 55
}
]
},
{
"name": "Raja",
"prices": [
{
"date": "2022-02-24T13:06:13.493+00:00",
"price": 75
},
{
"date": "2022-03-23T13:06:13.493+00:00",
"price": 85
}
]
}
]
Expected result json Data:
[
{
"colorDescription": "braun, rose gold",
"cgPrices: 45,
"stateHistory": [
{
"state": "scanning",
"date": "2022-02-22T13:06:13.493+00:00"
},
{
"state": "scanned",
"date": "2022-02-18T13:06:13.493+00:00"
},
{
"state": "reconstructing",
"date": "2022-02-16T13:06:13.493+00:00"
}
]
},
{
"colorDescription": "beige, silber",
"cgPrices: 0,
"stateHistory": [
{
"state": "scanning",
"date": "2022-03-22T13:06:13.493+00:00"
},
{
"state": "scanned",
"date": "2022-03-18T13:06:13.493+00:00"
},
{
"state": "reconstructing",
"date": "2022-03-16T13:06:13.493+00:00"
}
]
}
]
"cgPrice" field i need to sum of matched prices with date of two collection added.
my code:
db.datasets.aggregate([
{
"$lookup": {
"from": "users",
"as": "details",
"localField": "stateHistory.date",
"foreignField": "prices.date"
}
},
{
"$project": {
color: "$details.colorDescription",
prices: "$details"
}
}
])
How to join the lookup and get prices for matched field add the additional field "cgPrice" count sum.
mongo playground link: https://mongoplayground.net/p/vv8R3DlEDYo
You just need to do quite a lot of restructure, here is an example using the $map, $filter and $reduce operators:
db.datasets.aggregate([
{
"$lookup": {
"from": "users",
"as": "details",
"localField": "stateHistory.date",
"foreignField": "prices.date"
}
},
{
"$project": {
colorDescription: 1,
stateHistory: 1,
prices: {
$sum: {
$map: {
input: {
$filter: {
input: {
$reduce: {
input: {
$map: {
input: "$details",
in: "$$this.prices"
}
},
initialValue: [],
in: {
"$concatArrays": [
"$$this",
"$$value"
]
}
}
},
cond: {
$in: [
"$$this.date",
"$stateHistory.date"
]
}
}
},
in: "$$this.price"
}
}
}
}
}
])
Mongo Playground
I want to aggregate MongoDB documents which is having arrays inside of an array. my document was like the below.
{
"_id": "6257e31d11a9d5231c05c084",
"name": "Test Name 1",
"phone": "1234567891",
"visits": [
{
"_id": "6257e31d11a9d5231c05c069",
"date": "2-7-2021",
"samples": [
"6257f8855197613b641d494e",
....
],
"products_detailed": [
"5d725cd2c4ded7bcb480eab2",
.....
]
},
...........
]
}
and I want to get the output line below
{
"_id": "6257e31d11a9d5231c05c084",
"name": "Test Name 1",
"phone": "1234567891",
"visits": [
{
"_id": "6257e31d11a9d5231c05c069",
"date": "2-7-2021",
"samples": [
{
"_id": "6257f8855197613b641d494e",
"product_name": "Samor",
"price": 250
},
........
],
"products_detailed": [
{
"_id": "5d725cd2c4ded7bcb480eab2",
"product_name": "Pahad",
"price": 100
},
............
]
},
.........................
]
}
how can I get like this? I tried to use $lookup & group to get the output, but I am not getting the output as required me.
Since you have a list of visits on each document, one way to go is to $unwind and then $group at the end, like this:
db.Main.aggregate([
{
$unwind: "$visits"
},
{
"$lookup": {
"from": "Samples",
"localField": "visits.samples",
"foreignField": "_id",
"as": "samples"
}
},
{
"$lookup": {
"from": "Product Detailed",
"localField": "visits.products_detailed",
"foreignField": "_id",
"as": "products_detailed"
}
},
{
$project: {
name: 1,
phone: 1,
"visits._id": 1,
"visits.date": 1,
"visits.products_detailed": "$products_detailed",
"visits.samples": "$samples"
}
},
{
$group: {
_id: 0,
name: {$first: "$name"},
visits: {$push: "$visits"}
}
}
])
As you can see on the playground, on your data sample it will return:
[
{
"_id": 0,
"name": "Test Name 1",
"visits": [
{
"_id": "6257e31d11a9d5231c05c069",
"date": "2-7-2021",
"products_detailed": [
{
"_id": "5d725cd2c4ded7bcb480eab2",
"price": 100,
"product_name": "Pahad"
}
],
"samples": [
{
"_id": "6257f8855197613b641d494e",
"price": 250,
"product_name": "Samor"
}
]
}
]
}
]
I want to join two mongodb collections, collectionA and collectionB.
For each document in collectionA I want to check if exists a coincidence in collectionB.
If I do it in a $lookup, it returns all the documents joined, but I would like the search in collectionB stops as soon as one coincidence is found (kind of a mongodb findOne). My concern is the performance, I know I could get just the element 0 from the array.
Is there a way to do it using the mongodB aggregation framework?
Example:
collectionA:
[
{
"_id": 1,
"item": "almonds"
},
{
"_id": 2,
"item": "pecans"
}
]
colectionB:
[
{
"_fid": 1,
"date": "2021-01-10"
},
{
"_fid": 1,
"date": "2021-01-11"
},
{
"_fid": 1,
"date": "2021-01-12"
},
{
"_fid": 2,
"date": "2021-01-03"
}
]
$lookup mongoDb
db.colectionA.aggregate([
{
"$lookup": {
"from": "colectionB",
"localField": "_id",
"foreignField": "_fid",
"as": "matches"
}
}
])
Result
[
{
"_id": 1,
"item": "almonds",
"matches": [
/* I don't want this array, with 1 element would be enough */
{
"_fid": 1,
"_id": ObjectId("5a934e000102030405000002"),
"date": "2021-01-10"
},
{
"_fid": 1,
"_id": ObjectId("5a934e000102030405000003"),
"date": "2021-01-11"
},
{
"_fid": 1,
"_id": ObjectId("5a934e000102030405000004"),
"date": "2021-01-12"
}
]
},
{
"_id": 2,
"item": "pecans",
"matches": [
{
"_fid": 2,
"_id": ObjectId("5a934e000102030405000005"),
"date": "2021-01-03"
}
]
}
]
You can test on this mongo playground.
Thanks in advance
If you're using at least MongoDB 3.6, you can execute an aggregation pipeline on a joined collection. It might look like this:
db.colectionA.aggregate([
{
"$lookup": {
"from": "colectionB",
"as": "matches",
"let": {
"fid": "$_id"
},
"pipeline": [
{
"$match": {
"$expr": {
"$eq": [
"$_fid",
"$$fid"
]
}
}
},
{
"$limit": 1
}
]
}
}
])
Working Mongo playground
I'm using Python with pymongo to query from the database.
I have 3 different collections:
1st:
# Projects collection
{
"_id": "A",
},
{
"_id": "B",
},
{
"_id": "C"
},
..
2nd:
# Episodes collection
{
"_id": "A/Episode01",
"project": "A",
"name": "Episode01"
},
{
"_id": "A/Episode02",
"project": "A",
"name": "Episode02"
},
{
"_id": "B/Episode01",
"project": "B",
"name": "Episode01"
},
..
3rd:
# Sequences collection
{
"_id": "A/Episode01/Sequence01",
"project": "A",
"episode": "Episode01",
"name": "Sequence01"
},
{
"_id": "A/Episode02/Sequence02",
"project": "A",
"episode": "Episode02",
"name": "Sequence02"
},
{
"_id": "B/Episode01/Sequence01",
"project": "B",
"episode": "Episode01",
"name": "Sequence01"
},
..
I want to use aggregate to query project A and get all of its corresponding episodes and sequences like this:
{
"_id": "A",
"episodes":
[
{
"_id": "A/Episode01",
"project": "A",
"name": "Episode01",
"sequences":
[
{
"_id": "A/Episode01/Sequence01",
"project": "A",
"episode": "Episode01",
"name": "Sequence01"
},
]
},
{
"_id": "A/Episode02",
"project": "A",
"name": "Episode02",
"sequences":
[
{
"_id": "A/Episode02/Sequence02",
"project": "A",
"episode": "Episode02",
"name": "Sequence02"
},
]
},
]
}
I can get as far as getting the proper episodes, but I'm not sure how to add an embed field for any matching sequences. Is it possible to do this all in a single pipeline query?
Right now my query is looking like this:
[
{"$match": {
"_id": "A"}
},
{"$lookup": {
"from": "episodes",
"localField": "_id",
"foreignField": "project",
"as": "episodes"}
},
{"$group": {
"_id": {
"_id": "$_id",
"episodes": "$episodes"}
}}
]
You can do like following
use $match to match the document
use uncorrelated queries to join two collection. But normal joining also possible as you have written. This is easier when we get some complex situations.
Mongo script is given below
[
{
"$match": {
"_id": "A"
}
},
{
$lookup: {
from: "Episodes",
let: {
id: "$_id"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$project",
"$$id"
]
}
}
},
{
$lookup: {
from: "Sequences",
let: {
epi: "$name"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$episode",
"$$epi"
]
}
}
}
],
as: "sequences"
}
}
],
as: "episodes"
}
}
]
Working Mongo playground
Update 01
Using standard lookup
[
{
"$match": {
"_id": "A"
}
},
{
"$lookup": {
"from": "Episodes",
"localField": "_id",
"foreignField": "project",
"as": "episodes"
}
},
{
$unwind: "$episodes"
},
{
"$lookup": {
"from": "Sequences",
"localField": "episodes.name",
"foreignField": "episode",
"as": "episodes.sequences"
}
},
{
$group: {
_id: "$episodes._id",
episodes: {
$addToSet: "$episodes"
}
}
}
]
Working Mongo playground
I am using below query to get combined data from users and project collections:
db.collection.aggregate([
{
"$group": {
"_id": "$userId",
"projectId": { "$push": "$projectId" }
}
},
{
"$lookup": {
"from": "users",
"let": { "userId": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$eq": [ "$_id", "$$userId" ] }}},
{ "$project": { "firstName": 1 }}
],
"as": "user"
}
},
{ "$unwind": "$user" },
{
"$lookup": {
"from": "projects",
"let": { "projectId": "$projectId" },
"pipeline": [
{ "$match": { "$expr": { "$in": [ "$_id", "$$projectId" ] }}},
{ "$project": { "projectName": 1 }}
],
"as": "projects"
}
}
])
and it results like below:
[
{
"_id": "5c0a29e597e71a0d28b910aa",
"projectId": [
"5c0a2a8897e71a0d28b910ac",
"5c0a4083753a321c6c4ee024"
],
"user": {
"_id": "5c0a29e597e71a0d28b910aa",
"firstName": "Amit"
},
"projects": [
{
"_id": "5c0a2a8897e71a0d28b910ac",
"projectName": "LN-PM"
},
{
"_id": "5c0a4083753a321c6c4ee024",
"projectName": "fallbrook winery"
}
]
},
{
"_id": "5c0a29c697e71a0d28b910a9",
"projectId": [
"5c0a4083753a321c6c4ee024"
],
"user": {
"_id": "5c0a29c697e71a0d28b910a9",
"firstName": "Rajat"
},
"projects": [
{
"_id": "5c0a4083753a321c6c4ee024",
"projectName": "fallbrook winery"
}
]
}
]
Now i have another table "Worksheets" and want to include hours field in projects Array, which will be calculated from the worksheets table by specifying the projectId which is _id in the projects array. It will be find in worksheet table and hours will be incremented how many times this _id has in worksheets table. Below is my worksheet collection:
{
"_id" : ObjectId("5c0a4efa91b5021228681f7a"),
"projectId" : ObjectId("5c0a4083753a321c6c4ee024"),
"hours" : 8,
"userId" : ObjectId("5c0a29c697e71a0d28b910a9"),
"__v" : 0
}
{
"_id" : ObjectId("5c0a4f4191b5021228681f7c"),
"projectId" : ObjectId("5c0a2a8897e71a0d28b910ac"),
"hours" : 6,
"userId" : ObjectId("5c0a29e597e71a0d28b910aa"),
"__v" : 0
}
The result will look like below:
{
"_id": "5c0a29c697e71a0d28b910a9",
"projectId": [
"5c0a4083753a321c6c4ee024"
],
"user": {
"_id": "5c0a29c697e71a0d28b910a9",
"firstName": "Rajat"
},
"projects": [
{
"_id": "5c0a4083753a321c6c4ee024",
"projectName": "fallbrook winery",
"hours":8
}
]
}
You can use below aggregation
$lookup 3.6 nested syntax allows you to join nested collection inside the $lookup pipeline. You can perform all the aggregation inside the nested $lookup pipline
db.collection.aggregate([
{ "$group": {
"_id": "$userId",
"projectId": { "$push": "$projectId" }
}},
{ "$lookup": {
"from": "users",
"let": { "userId": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$eq": [ "$_id", "$$userId" ] }}},
{ "$project": { "firstName": 1 }}
],
"as": "user"
}},
{ "$unwind": "$user" },
{ "$lookup": {
"from": "projects",
"let": { "projectId": "$projectId" },
"pipeline": [
{ "$match": { "$expr": { "$in": [ "$_id", "$$projectId" ] }}},
{ "$lookup": {
"from": "worksheets",
"let": { "projectId": "$_id" },
"pipeline": [
{ "$match": { "$expr": { "$eq": [ "$projectId", "$$projectId" ] }}},
{ "$group": {
"_id": "$projectId",
"totalHours": { "$sum": "$hours" }
}}
],
"as": "workHours"
}}
{ "$project": {
"projectName": 1,
"hours": { "$arrayElemAt": ["$workHours.totalHours", 0] }
}}
],
"as": "projects"
}}
])