I have this sample data:
[
{
"customers": [
{"id": 100, "name": "a"},
{"id": 200, "name": "b"},
{"id": 300, "name": "c"},
{"id": 400, "name": "d"}
],
"sales": [
{
"sale_id": 9999,
"persons_related": [
{"id": 100},
{"id": 900},
{"id": 800}
]
},
{
"sale_id": 9998,
"persons_related": [
{"id": 500},
{"id": 550},
{"id": 560}
]
},
]
}
]
It represents two collections, customers and sales.
Imagine that I am working with the customers collection, I have selected just the customer 100
db.collection.aggregate([
{ $project: {
"customer": { "$arrayElemAt": [ "$customers", 0 ]
}}
}
Which returns:
[
{
"_id": ObjectId("5a934e000102030405000000"),
"customer": {
"id": 100,
"name": "a"
}
}
]
And I want to find up the sales where this id appears, so I lookup against the same collection, adding this stage to the pipeline:
{ $lookup: {
from: "collection",
let: { id: "$customer.id" },
pipeline: [
{ $match: {
$and: [
{ $expr: { $in: [ "$$id", "$sales.persons_related.id" ] } }
]
}}
],
as: "sales"
}
}
I need to use this lookup version (the one with let and pipeline, and not the other one with
localField/foreignField) because I need to add additional filters in the match stage of the pipeline. However, this part:
{ $expr: { $in: [ "$$id", "$sales.persons_related.id" ] } }
Doesn't work as expected, I have tried with other operators ($eq) with same result.
The expected output (using a pipeline in the lookup) should be:
[
{
"_id": ObjectId("5a934e000102030405000000"),
"customer": {
"id": 100,
"name": "a"
},
"sales": [
{
"sale_id": 9999,
"persons_related": [
{"id": 100},
{"id": 900},
{"id": 800}
]
}
]
}
]
Can you please lend me a hand? You can test on this mongo playground
Just FYI mongoplayground provides a multiple collection option, so instead of trying to hack your syntax to simulate it you can just us the dropdown at the top right side to change it.
Your syntax is fine, here is a working playground example:
db.customers.aggregate([
{
$match: {
id: 100
}
},
{
$lookup: {
from: "sales",
let: {
id: "$id"
},
pipeline: [
{
$match: {
$and: [
{
$expr: {
$in: [
"$$id",
"$persons_related.id"
]
}
}
]
}
}
],
as: "sales"
}
}
])
$lookup also flattens arrays so you can just use the simpler syntax for it and it simplifies the code:
db.customers.aggregate([
{
$match: {
id: 100
}
},
{
$lookup: {
from: "sales",
localField: "id",
foreignField: "persons_related.id",
as: "sales"
}
}
])
Mongo Playground
Related
I am having a collection which contains the data like the following and want to have the desirable output which I have mentioned below.
db={
collectionA: [
{
"id": ObjectId("63b7c24c06ebe7a8fd11777b"),
"uniqueRefId": "UUID-2023-0001",
"products": [
{
"productIndex": 1,
"productCategory": ObjectId("63b7c24c06ebe7a8fd11777b"),
"productOwners": [
ObjectId("63b7c2fd06ebe7a8fd117781")
]
},
{
"productIndex": 2,
"productCategory": ObjectId("63b7c24c06ebe7a8fd11777b"),
"productOwners": [
ObjectId("63b7c2fd06ebe7a8fd117781"),
ObjectId("63b7c12706ebe7a8fd117778")
]
},
{
"productIndex": 3,
"productCategory": "",
"productOwners": ""
}
]
}
],
collectionB: [
{
"_id": ObjectId("63b7c2fd06ebe7a8fd117781"),
"fullname": "Jim Corbett",
"email": "jim.corbett#pp.com"
},
{
"_id": ObjectId("63b7c12706ebe7a8fd117778"),
"fullname": "Carry Minatti",
"email": "carry.minatty#pp.com"
},
]
}
Desirable Output = [
{
"id": ObjectId("507f1f77bcf86cd799439011"),
"uniqueRefId": "UUID-2023-0001",
"products": [
{
"productIndex": 1,
"productCategory": ObjectId('614g2f77bff86cd755439021'),
"productOwners": [
{
"_id": ObjectId("63ac1e59c0afb8b6f2d41acd"),
"fullname": "Jim Corbett",
"email": "jim.corbett#pp.com"
}
]
},
{
"productIndex": 2,
"productCategory": ObjectId('614g2f77bff86cd755439021'),
"productOwners": [
{
"_id": ObjectId("63ac1e59c0afb8b6f2d41acd"),
"fullname": "Jim Corbett",
"email": "jim.corbett#pp.com"
},
{
"_id": ObjectId("63ac1e59c0afb8b6f2d41ace"),
"fullname": "Carry Minatti",
"email": "carry.minatty#pp.com"
}
]
},
{
"productIndex": 3,
"productCategory": "",
"productOwners": ""
}
]
}
]
In the collectionA we are having other documents as well, its not just one document.
Similarly for collectionB we are having other documents too.
How we can get this desirable output?
I am expecting the mongodb query for getting this solution.
I have implemented the lookup like the following
db.collectionA.aggregate([
{
"$lookup": {
"from": "collectionB",
"localField": "products.productOwners",
"foreignField": "_id",
"as": "inventory_docs"
}
}
])
You can try this:
db.collectionA.aggregate([
{
"$unwind": "$products"
},
{
"$lookup": {
"from": "collectionB",
"localField": "products.productOwners",
"foreignField": "_id",
"as": "products.productOwners"
}
},
{
"$group": {
"_id": {
id: "$id",
uniqueRefId: "$uniqueRefId"
},
"products": {
"$push": "$products"
}
}
},
{
"$project": {
id: "$_id.id",
uniqueRefId: "$_id.uniqueRefId",
products: 1,
_id: 0
}
}
])
Playground link.
In this query, we do the following:
First we unwind the products array, using $unwind.
Then we calculate productOwners, using $lookup.
Then we group the unwinded elements, using $group.
Finally we, project the desired output using $project.
I'm migrating from MySQL to MongoDB. In this process I want to rewrite the following MySQL query:
This query is used to add additional information to the minutes table (adding the program, title, broadcaster and program_id (prid) to the other table)
UPDATE etl_staging.sko_minutes minut
JOIN etl_staging.sko_daily AS daily ON
minut.channel = daily.channel AND
(minut.C BETWEEN daily.start_datetime and daily.end_datetime)
set minut.sko_frequency = daily.sko_frequency,
minut.programma = daily.programma,
minut.titel = daily.titel,
minut.omroep = daily.omroep,
minut.prid = daily.prid
WHERE daily.doelgroep = '6+'
AND daily.universe = 'Currency'
Now i'm trying to rewrite it to the following... Only i found out that i could not make a join in MongoDB. Was looking at something like this. But that returns an empty minutes array as field to the minutes document.
db.sko_minutes.aggregate(
[
{ $lookup: {
from: "sko_daily",
let: { channel: "$channel", start_datetime: "$start_datetime", end_datetime: "$end_datetime",
doelgroep: "$doelgroep", universe: "$universe"},
pipeline: [
{ $match: {
$expr: {
$and: [
{ $eq: ["$$universe", 'Currency']},
{ $eq: ["$$doelgroep", '6+']},
{ $eq: ["$channel", "$$channel"]},
{ $gte: ["$datetime", "$$start_datetime"]},
{ $lte: ["$datetime", "$$end_datetime"]}
]
}
}}
],
as: 'minutes'
}}
]
)
Does anybody have an idea what would be the best approach for this problem?
The documents of the sko_minutes looks like this:
{
"_id": {"$oid": "5fbb8b85336e42949248fb1b"},
"abs": 0,
"channel": "vicetv",
"date": {"$date": "2020-11-22T00:00:00.000Z"},
"datetime": {"$date": "2020-11-22T09:20:00.000Z"},
"hour": 9,
"kta": 0,
"minutes": "20",
"start_time": "09:20:00"
},
And the daily document looks like this:
{
"_id": {"$oid": "5fbb8afb4cab8a4ce5acac5f"},
"abs": 0,
"brk": 0,
"channel": "net5",
"date": "2020-11-18",
"doelgroep": "20-34",
"duration": "47",
"end_date": "2020-11-18",
"end_datetime": {"$date": "2020-11-18T01:12:00.000Z"},
"end_hour": "1",
"end_minutes": "12",
"end_time": "25:12:59",
"hour": "0",
"kdh": 0,
"kta": 0,
"minutes": "12",
"omroep": "net5",
"prid": "694949124",
"programma": "vtwonenverbouwenofverhuizen",
"sko_frequency": "overige herhaling",
"start_datetime": {"$date": "2020-11-18T00:12:00.000Z"},
"start_time": "24:12:00",
"titel": "Vtwonen verbouwen of verhuizen",
"universe": "UGK1-6",
"waardering": 0,
"webtv_gaas": 0,
"webtv_nstreams": 0
},
Brings part of sko_daily to sko_minutes
let fields are from sko_minutes: date, datetime, channel
this fields are matched to daily
You also require a subset of sko_daily, using filters.
Mind you that I tweaked the date fields, and some data, just to make mongoplayground work.
Playground
db.sko_minutes.aggregate([
{
$lookup: {
/**$$ are from sko_minutes*/
from: "sko_daily",
let: {
"start_datetime": "$date",
"end_datetime": "$datetime",
channel: "$channel"
},
pipeline: [
{
$match: {
$expr: {
$and: [
{
$eq: [
"$universe",
"Currency"
]
},
{
$eq: [
"$doelgroep",
"6+"
]
},
{
$eq: [
"$$channel",
"$channel"
]
},
{
$gte: [
"$$start_datetime",
"$start_datetime"
]
},
{
$lte: [
"$$end_datetime",
"$end_datetime"
]
}
]
}
}
}
],
as: "minutes"
}
}
])
i have collections which represent a tree hiearchy meta, site and geolocation collections, one meta can have multiple sites and one site can have multiple geolocation
meta collection
{
"_id": "1",
"meta_id": 1,
"meta_name": "yankung"
}
site collection
{
"_id": "1",
"meta_id": 1,
"site_id" :2,
"site_name": "uoop"
}
geo collection
{
"_id": "1",
"site_id": 2,
"geo_id" :3,
"geo_name": "toop"
}
i have to get the final result like this
{
"_id": "1",
"meta_id": 1,
"meta_name": "yankung",
"sites": [
{
"site_id": 2,
"site_name": "uoop",
"geos:": [
{
"geo_id": 3,
"geo_name": "toop"
},
{
"geo_id": 4,
"geo_name": "toop"
}
]
},
{
"site_id": 1000,
"site_name": "uoop",
"geos:": [
{
"geo_id": 5,
"geo_name": "toop"
},
{
"geo_id": 6,
"geo_name": "toop"
}
]
}
]
}
i tried using aggregation query with lookup and unwind was able to segregate sites and geos as list , thought of getting the required result from application level, but would have to iterate through each document and add which will increase the time complexity, any help on how should i proceed?
this is what i was able to achieve
{
"_id": "1",
"meta_id": 1,
"meta_name": "yankung",
"sites": [
{ "site_id": 2, "site_name": "uoop"
},
{"site_id": 1000,"site_name": "uoop"
}
],
"geos:": [
{ "geo_id": 5,"geo_name": "toop"
},
{"geo_id": 6,"geo_name": "toop"
}
]
}
The trick is to use $lookup with join conditions and uncorrelated subqueries. By this way you can define $lookup inside $lookup.
Here's the query :
db.meta_collection.aggregate([
{
$lookup: {
from: "site_collection",
let: {
meta: "$meta_id"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$meta_id",
"$$meta"
]
}
}
},
{
$lookup: {
from: "geo_collection",
let: {
site: "$site_id"
},
pipeline: [
{
$match: {
$expr: {
$eq: [
"$site_id",
"$$site"
]
}
}
},
],
as: "geos"
}
}
],
as: "sites"
}
}
])
You can test it here
I got two collections.
One contains an array of objects. These objects own a field with an id to a document in another collection.
The goal is to "replace" the ref by the document. Sounds simple but I have no clue how to archive this.
E.G.
Collection "Product"
{ "_id": 1,
"alias": "ProductA"
},
{ "_id": 2,
"alias": "ProductC"
}
Collection "Order"
{ "_id": 5765,
"cart": [
{
"product": 1,
"qty": 7
}, {
"product": 2,
"qty": 6
}
]
}
What I need by a query is this:
{ "_id": 5765,
"cart": [
{
"product": {
"_id": 1,
"alias": "ProductA"
},
"qty": 7
}, {
"product": {
"_id": 2,
"alias": "ProductC"
},
"qty": 6
}
]
}
I tried a simple lookup, but the array will only contains the products. What do I need to change?
{
$lookup: {
from: "products",
let: {
tmp: "$cart.product"
},
pipeline: [
{
$match: {
$expr: {
$in: ["$_id", "$$tmp"]
}
}
}
],
as: "cart.product"
}
}
Thanks for your help.
I added a new $addFields stage to transform the output from the $lookup stage - it gets the desired output:
db.order.aggregate([
{
$lookup: {
from: "product",
let: {
tmp: "$cart.product"
},
pipeline: [
{
$match: {
$expr: {
$in: ["$_id", "$$tmp"]
}
}
}
],
as: "products"
}
},
{
$addFields: {
cart: {
$map: {
input: "$cart", as: "ct",
in: {
product: {
$arrayElemAt: [
{ $filter: {
input: "$products", as: "pr",
cond: {
$eq: [ "$$ct.product", "$$pr._id" ]
}
}
}, 0 ]
},
qty: "$$ct.qty"
}
}
}
}
}
] ).pretty()
I have been searching on stackoverflow and cannot find exactly what I am looking for and hope someone can help. I want to submit a single query, get multiple counts back, for a single document, based on array of that document.
My data:
db.myCollection.InsertOne({
"_id": "1",
"age": 30,
"items": [
{
"id": "1",
"isSuccessful": true,
"name": null
},{
"id": "2",
"isSuccessful": true,
"name": null
},{
"id": "3",
"isSuccessful": true,
"name": "Bob"
},{
"id": "4",
"isSuccessful": null,
"name": "Todd"
}
]
});
db.myCollection.InsertOne({
"_id": "2",
"age": 22,
"items": [
{
"id": "6",
"isSuccessful": true,
"name": "Jeff"
}
]
});
What I need back is the document and the counts associated to the items array for said document. In this example where the document _id = "1":
{
"_id": "1",
"age": 30,
{
"totalIsSuccessful" : 2,
"totalNotIsSuccessful": 1,
"totalSuccessfulNull": 1,
"totalNameNull": 2
}
}
I have found that I can get this in 4 queries using something like this below, but I would really like it to be one query.
db.test1.aggregate([
{ $match : { _id : "1" } },
{ "$project": {
"total": {
"$size": {
"$filter": {
"input": "$items",
"cond": { "$eq": [ "$$this.isSuccessful", true ] }
}
}
}
}}
])
Thanks in advance.
I am assuming your expected result is invalid since you have an object literal in the middle of another object and also you have totalIsSuccessful for id:1 as 2 where it seems they should be 3. With that said ...
you can get similar output via $unwind and then grouping with $sum and $cond:
db.collection.aggregate([
{ $match: { _id: "1" } },
{ $unwind: "$items" },
{ $group: {
_id: "_id",
age: { $first: "$age" },
totalIsSuccessful: { $sum: { $cond: [{ "$eq": [ "$items.isSuccessful", true ] }, 1, 0 ] } },
totalNotIsSuccessful: { $sum: { $cond: [{ "$ne": [ "$items.isSuccessful", true ] }, 1, 0 ] } },
totalSuccessfulNull: { $sum: { $cond: [{ "$eq": [ "$items.isSuccessful", null ] }, 1, 0 ] } },
totalNameNull: { $sum: { $cond: [ { "$eq": [ "$items.name", null ]}, 1, 0] } } }
}
])
The output would be this:
[
{
"_id": "_id",
"age": 30,
"totalIsSuccessful": 3,
"totalNameNull": 2,
"totalNotIsSuccessful": 1,
"totalSuccessfulNull": 1
}
]
You can see it working here