Aggregation unwind multiple arrays $lookup as multiple fields for projection - mongodb

This is my current query:
Follow.aggregate([
{
$match: {
"user": Types.ObjectId(user_id)
}
},
{
$unwind: "$following"
},
{
$lookup: {
"from": "users",
"localField": "following",
"foreignField": "createdById",
"as": "followingUsers"
}
}, {
$project: {
"user": 1,
"followingUsers": 1
}
}
])
This query is run on a document like this:
{
"_id" : ObjectId("5a271a93a19d690b25a3e181"),
"user" : ObjectId("5a271a4f50261a3c1695a391"),
"following" : [
ObjectId("5a257c87086eb00712fd02ec"),
ObjectId("5a257a79086eb00712fd02eb")
],
"followers" : [
ObjectId("5a257a79086eb00712fd02eb")
]
}
And gives a result like this:
{
"_id" : ObjectId("5a271a93a19d690b25a3e181"),
"user" : ObjectId("5a271a4f50261a3c1695a391"),
"followingUsers" : [
{
"_id" : ObjectId("5a257a79086eb00712fd02eb"),
"email" : "email#gmail.com",
"username" : "username",
"email_verified" : true,
"created" : ISODate("2010-12-04T16:40:25.670Z"),
"__v" : 0,
"last_login" : ISODate("2010-12-06T21:14:25.538Z"),
"is_active" : false
}
]
}
The above unwinds only one of the arrays and follows up the lookup.
I want to, in one scoop (assuming that is possible)
unwind both the $following and $followers array fields
$lookup both of the array fields independently
then have both come out as, say, followingUsers and followersUsers
So end up with something like this:
{
"_id" : ObjectId("5a271a93a19d690b25a3e181"),
"user" : ObjectId("5a271a4f50261a3c1695a391"),
"followingUsers" : [
{
"_id" : ObjectId("5a257a79086eb00712fd02eb"),
....
}
],
"followersUsers": [
{
"_id" : ObjectId("5a257a79086eb00712fd02eb"),
....
}
]
}
Am I even sane to imagine something like this in MongoDB?

Related

Unable to aggregate two collections using lookup in MongoDB Atlas

I have an orders collection that looks like this:
{
"_id" : "wJNEiSYwBd5ozGtLX",
"orderId" : 52713,
"createdAt" : ISODate("2020-01-31T04:34:13.790Z"),
"status" : "closed",
"orders" : [
{
"_id" : "ziPzwLuZrz9MNkaRT",
"productId" : 10290,
"quantity" : 2
}
]
}
I have an products collection that looks like this
{
"_id" : "238cwwLkZa6gKNN86",
"productId" : 10290,
"title" : "Product Title",
"price" : 9.9
}
I am trying to merge the price information into the orders information.
Something like:
{
"_id" : "wJNEiSYwBd5ozGtLX",
"orderId" : 52713,
"createdAt" : ISODate("2020-01-31T04:34:13.790Z"),
"status" : "closed",
"orders" : [
{
"_id" : "ziPzwLuZrz9MNkaRT",
"productId" : 10290,
"quantity" : 2,
"price": 9.9
}
]
}
If I try a $lookup command on MongoDB Atlas Dashboard like this:
{
from: 'products',
localField: 'orders.productId',
foreignField: 'productId',
as: 'priceInfo'
}
The aggregated output is (not what I wanted):
{
"_id" : "wJNEiSYwBd5ozGtLX",
"orderId" : 52713,
"createdAt" : ISODate("2020-01-31T04:34:13.790Z"),
"status" : "closed",
"orders" : [
{
"_id" : "ziPzwLuZrz9MNkaRT",
"productId" : 10290,
}
],
"priceInfo": [
{
"_id" : "238cwwLkZa6gKNN86",
"productId" : 10290,
"title" : "Product Title",
"price" : 9.9
}
]
}
I do not need a separate priceInfo array. It will be best if I have the product details information merged into the "orders" array. What should be the aggregation lookup syntax to achieve the desired output?
Demo - https://mongoplayground.net/p/bLqcN7tauWU
Read - $lookup $unwind $first $set $push $group
db.orders.aggregate([
{ $unwind: "$orders" }, // break array of orders into individual documents
{
$lookup: { // join
"from": "products",
"localField": "orders.productId",
"foreignField": "productId",
"as": "products"
}
},
{
$set: {
"orders.price": { "$arrayElemAt": [ "$products.price", 0 ] } // set the price
}
},
{
$group: { // group records back
_id: "$_id",
createdAt: { $first: "$createdAt" },
status: { $first: "$status" },
orderId: { $first: "$orderId" },
orders: { $push: "$orders" }
}
}
])

How can I use MongoDB's aggregate with $lookup to replace an attribute that holds an id with the whole document?

I have two collections:
user:
{
"_id" : "9efb42e5-514d-44bd-a4b8-6f74e6313ec2",
"name" : "Haralt",
"age" : 21,
"bloodlineId" : "c59a2d02-f304-49a8-a52a-44018fc15fe6",
"villageId" : "foovillage"
}
bloodlines:
{
"_id" : "c59a2d02-f304-49a8-a52a-44018fc15fe6",
"name" : "Tevla",
"legacy" : 0
}
Now I'd like to do an aggregate to replace user.bloodlineId with the whole bloodline document.
This is what I tried to far:
db.getCollection('character').aggregate([
{
"$match": { _id: "9efb42e5-514d-44bd-a4b8-6f74e6313ec2" }
},
{
"$lookup": {
from: "bloodline",
localField: "bloodlineId",
foreignField: "_id",
as: "bloodline"
}
}])
The result is almost where I want it:
{
"_id" : "9efb42e5-514d-44bd-a4b8-6f74e6313ec2",
"name" : "Haralt",
"age" : 21,
"bloodlineId" : "c59a2d02-f304-49a8-a52a-44018fc15fe6",
"villageId" : "foovillage",
"bloodline" : [
{
"_id" : "c59a2d02-f304-49a8-a52a-44018fc15fe6",
"name" : "Tevla",
"legacy" : 0
}
]
}
Only two issues here. The first is that bloodlineId is still there and bloodline was just added to the result. I'd like to have bloodline replace the bloodlineId attribute.
The second problem is that bloodline is an array. I'd love to have it a single object.
I think this pipeline might do the trick:
[
{
"$match": {
_id: "9efb42e5-514d-44bd-a4b8-6f74e6313ec2"
}
},
{
"$lookup": {
from: "bloodlines",
localField: "bloodlineId",
foreignField: "_id",
as: "bloodline"
}
},
{
$project: {
"age": 1,
"bloodlineId": {
$arrayElemAt: [
"$bloodline",
0
]
},
"name": 1,
"villageId": 1
}
}
]
Mongo Playground
If there's anything I'm missing, please let me know!

MongoDB aggregate two collections, return additional field as count

(See edit below)
I am trying to aggregate data from two separate collections within the same MongoDB database.
The "accounts" collection contains user information (cleansed):
{
_id: ObjectId("5c0d64a4224a2900108c005f"),
"username" : "mike22",
"email" : "mike22#<domain>.com",
"country" : GB,
"created" : ISODate("2018-11-26T23:37:49.051Z")
},
{
_id: ObjectId("5a0d64a4527h2880108c0445"),
"username" : "mike23",
"email" : "mike23#<domain>.com",
"country" : DE,
"created" : ISODate("2018-11-26T23:37:49.051Z")
},
{
_id: ObjectId("5a3334a45zzz2884448c0445"),
"username" : "mike24",
"email" : "mike24#<domain>.com",
"country" : DE,
"created" : ISODate("2018-11-26T23:37:49.051Z")
}
The "devices" collection contains device definitions for all users. A user is likely to have many devices defined in this collection and many users devices are in this collection.
A single device within this collection is defined as follows:
{
"_id" : ObjectId("5c10138c73bbe0001018e415"),
"capabilities" : [
"BrightnessController",
"PowerController"
],
"displayCategories" : [
"LIGHT"
],
"friendlyName" : "Test1",
"description" : "Test device 1",
"reportState" : true,
"username" : "mike22",
"endpointId" : 11,
"__v" : 0
},
{
"_id" : ObjectId("5c10138c73bbe0001018e415"),
"capabilities" : [
"PowerController"
],
"displayCategories" : [
"SWITCH"
],
"friendlyName" : "Test2",
"description" : "Test device 2",
"reportState" : true,
"username" : "mike23",
"endpointId" : 12,
"__v" : 0
},
{
"_id" : ObjectId("5c10138c73bbe0001018e415"),
"capabilities" : [
"PowerController"
],
"displayCategories" : [
"SMARTPLUG"
],
"friendlyName" : "Test3",
"description" : "Test device 3",
"reportState" : true,
"username" : "mike22",
"endpointId" : 13,
"__v" : 0
}
I'm able to use the aggregate below to show me a count of device per-user:
db.accounts.aggregate([
{
$lookup: {
from : "devices",
localField : "username",
foreignField : "username",
as : "userdevs"
},
},
{ $unwind:"$userdevs" },
{ $group : { _id : "$username", count : { $sum : 1 } } }
])
Example output from the data/ aggregate above:
{ "_id" : "mike22", "count" : 2 },
{ "_id" : "mike23", "count" : 1 }
(Note user with no devices is now missing/ should be there with a zero count?!)
However, I want to return all fields for each user plus a new field which shows me the count of devices they have in the "devices" collection. The output I am looking for is as below:
{
"_id" : ObjectId("5c0d64a4224a2900108c005f"),
"username" : "mike22",
"email" : "mike22#<domain>.com",
"country" : GB,
"created" : ISODate("2018-11-26T23:37:49.051Z"),
"countDevices": 2
},
{
"_id" : ObjectId("5a0d64a4527h2880108c0445"),
"username" : "mike23",
"email" : "mike23#<domain>.com",
"country" : DE,
"created" : ISODate("2018-11-26T23:37:49.051Z"),
"countDevices": 1
},
{
"_id" : ObjectId("5a0d64a4527h2880108c0445"),
"username" : "mike24",
"email" : "mike24#<domain>.com",
"country" : DE,
"created" : ISODate("2018-11-26T23:37:49.051Z"),
"countDevices": 0
}
Edit 16/12: So I am nearly there with the aggregate below. Zero-count users are missing though.
use users
db.accounts.aggregate([
{
$lookup: {
from : "devices",
localField : "username",
foreignField : "username",
as : "userdevs"
},
},
{ $unwind: "$userdevs"},
{ $group : { _id : {
_id: "$_id",
username: "$username",
email: "$email",
country: "$country",
region: "$region",
},
countDevices : { $sum : 1 } } }
])
2nd Edit 16/12:
I have found the aggregate needed below:
db.accounts.aggregate([
{ "$lookup": {
"from": "devices",
"let": { "username": "$username" },
"pipeline": [
{ "$match": {
"$expr": { "$eq": [ "$$username", "$username" ] }
}},
{ "$count": "count" }
],
"as": "deviceCount"
}},
{ "$addFields": {
"countDevices": { "$sum": "$deviceCount.count" }
}}
])
First of All, you can flatten the answer you have got with a projection like below:
{ $project : {
_id : '$_id._id',
username : '$_id.username',
email : '$_id.email',
country : '$_id.country',
region : '$_id.region',
countDevices: 1
}
}
add this after the $group in your pipeline, you will get your result as you wanted in the question.
About zero-count users, there is a way to handle this in database using mongoDB, as explained in detail here but I do not recommend it, its better that you handle this kind of problem client side.
As-per second edit, the aggregate I used is as below:
db.accounts.aggregate([
{ "$lookup": {
"from": "devices",
"let": { "username": "$username" },
"pipeline": [
{ "$match": {
"$expr": { "$eq": [ "$$username", "$username" ] }
}},
{ "$count": "count" }
],
"as": "deviceCount"
}},
{ "$addFields": {
"countDevices": { "$sum": "$deviceCount.count" }
}}
])

Combining array of objects according to field in MongoDb

I am trying to build an aggregation that will return the documents with the biggest number of objects in one of the fields, after combining the field.
Let's say that have this following documents that two of them contains the same id inside the movie field
{
"_id" : ObjectId("5b79bb0c15d2d0697885467c"),
"movie" : [
ObjectId("5b79b8387d467d5ab860544f")
],
"takenSeats" : [{"id" : 1},{"id" : 2},{"id" : 4},
],
"creteDate" : ISODate("2018-08-14T18:46:36.090Z"),
}
{
"_id" : ObjectId("5b79bb0c15d2d069788ef48d"),
"movie" : [
ObjectId("5b79b8387d467d5ab860544f")
],
"takenSeats" : [{"id" : 2},{"id" : 7},{"id" : 4},
],
"creteDate" : ISODate("2018-08-14T18:46:36.090Z"),
}
{
"_id" : ObjectId("5b79bb0c15d2d069788fg54hq"),
"movie" : [
ObjectId("5b79b8387d467d5ab8df54h43")
],
"takenSeats" : [{"id" : 6},{"id" : 2},{"id" : 5},
],
"creteDate" : ISODate("2018-08-14T18:46:36.090Z"),
}
As you can see, two of the documents contains the same id in the field movie.
What I trying to do is: to take those document that contains the same id in movie field and combine the takenSeats field
The wanted result should looks like
{
"_id" : ObjectId("5b79b8387d467d5ab860544f"),
"takenSeats" : [{"id" : 2},{"id" : 7},{"id" : 4},{"id" : 1},{"id" : 2},
{"id" : 4}
],
"creteDate" : ISODate("2018-08-14T18:46:36.090Z"),
}
{
"_id" : ObjectId("5b79b8387d467d5ab8df54h43"),
"takenSeats" : [{"id" : 6},{"id" : 2},{"id" : 5},
],
"creteDate" : ISODate("2018-08-14T18:46:36.090Z"),
}
In the last hours I tried to achieve it with different operators like $push and $addToSet. This is the query that I did that was the closest to the result that I want, but the issue that the documents that I was receiving in the result are with duplicated ids
db.orders.aggregate([
{$match:{ "created":{$gt: new Date(ISODate().getTime() - 1000*60*60*24*15)}}},
{ $lookup: { from: "shows", localField: "showId", foreignField: "_id", as: "acociatedShow" } },
{ "$project": { "acociatedShow": 1 } },
{ $unwind : "$acociatedShow" },
{ "$group": {"_id": { "movie": "$acociatedShow.movie"},
"takenSeats": { "$addToSet": "$acociatedShow.takenSeats"}}},
{ $unwind : "$takenSeats" },
{ $group : { _id : "$takenSeats", movieId : { $first: '$_id.movie' },len : { $sum : 1 } } },
{ $limit : 3 },
{ $lookup: { from: "movies", localField: "movieId", foreignField: "_id", as: "topMovie" } },
{ $unwind: "$topMovie" }, { $replaceRoot: { newRoot: "$topMovie" } }
])

MongoDB projection into array

The below document has the dob of student and its parent's dob.
{
"_id" : ObjectId("56a31573a3b1f89cb895abd3"),
"dob" : {
"isodate" : ISODate("1996-01-21T18:30:00.000+0000")
},
"parent" : [
{
"dob" : {
"isodate" : ISODate("1956-07-21T18:30:00.000+0000")
},
"type" : "father"
},
{
"dob" : {
"isodate" : ISODate("1958-11-01T18:30:00.000+0000")
},
"type" : "mother"
}
]
}
In one of the application use case, it is better to receive output in the below format
{
"_id" : ObjectId("56a31573a3b1f89cb895abd3"),
"dob" : {
"isodate" : ISODate("1996-01-21T18:30:00.000+0000")
},
"type" : "student"
},
{
"_id" : ObjectId("56a31573a3b1f89cb895abd3"),
"dob" : {
"isodate" : ISODate("1956-07-21T18:30:00.000+0000")
},
"type" : "father"
},
{
"_id" : ObjectId("56a31573a3b1f89cb895abd3"),
"dob" : {
"isodate" : ISODate("1958-11-01T18:30:00.000+0000")
},
"type" : "mother"
}
The approach is to $project the fields into array and then $unwind that array. However, projection doesn't allow me to create array.
I believe $group and its associated aggregation cannot be used as my operations are on the same document in the pipeline.
Is this possible?
Note - i have the flexibility to change the document design as well.
For Mongo 3.0
Here I have included a [null] array which gives me the option to insert array in projection using a combination of $setDiffernce and $cond. The output of this is given to $setUnion with $parent array.
db.p1.aggregate(
{ "$project": {
"allVal": {
'$setUnion': [
{"$setDifference": [
{ "$map": {
"input": [null],
"as": "type",
"in": { "$cond": [
{"$eq": ["$$type", null]},
{dob:"$dob", type:{$literal:'student'}},
null
]}
}},
[null]
]}
,
'$parent'
]
}
}},
{$unwind : '$allVal'}
)
For mongo 3.2
Feels heaven as I have avoided $setDifference and $literal hack adjustments.
db.p1.aggregate([
{
$project:{
parent : 1,
type: {$literal : 'student'},
'dob.isodate' : 1
}
},
{
$project:{
allValues: { $setUnion: [ [{dob:"$dob", type:'$type'}], "$parent" ] }
}
},
{
$unwind : '$allValues'
}
])
In the first projection, I am adding a new field called type
In the 2nd projection, I am creating a new array with 2 different nodes of the same document.
Currently this solution works for Mongo 3.2