For MongoDB, when using $lookup to query more than one collection, is it possible to get a values-only list for a field returned in the $lookup?
What I don't want is a list of the full object with all its key/values.
Data:
failover_tool:PRIMARY> db.foo.find().pretty()
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo"
}
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo"
}
failover_tool:PRIMARY> db.bar.find().pretty()
{
"_id" : ObjectId("5ce72e0c5267960532b8df06"),
"name" : "bar1",
"foo" : "foo1"
}
{
"_id" : ObjectId("5ce72e165267960532b8df07"),
"name" : "bar2",
"foo" : "foo1"
}
{
"_id" : ObjectId("5ce72e1d5267960532b8df08"),
"name" : "bar3",
"foo" : "foo2"
}
Desired Query Output
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo",
"bars" : ["bar1", "bar2"]
},
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo",
"bars" : ["bar3"]
}
Closest
This query seems like it's almost there, but it returns too much data in the bars field:
db.foo.aggregate({
$lookup: {
from:"bar",
localField:"name",
foreignField: "foo",
as:"bars"
}
}).pretty()
Just use .dot notation with the name field
db.foo.aggregate([
{ "$lookup": {
"from": "bar",
"localField": "name",
"foreignField": "foo",
"as": "bars"
}},
{ "$addFields": { "bars": "$bars.name" }}
])
MongoPlayground
Hope below query helps :
db.foo.aggregate([{
$lookup: {
from:"bar",
localField:"name",
foreignField: "foo",
as:"bars"
}
},
{$unwind : '$bars'},
{
$group : {
_id : {
_id : '$_id',
name : '$name',
desc : '$desc'
},
bars : { $push : '$bars.name'}
}
},
{
$project : {
_id : '$_id._id',
name : '$_id.name',
desc : '$_id.desc',
bars : '$bars'
}
}
]).pretty()
output :
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo",
"bars" : [
"bar3"
]
}
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo",
"bars" : [
"bar1",
"bar2"
]
}
Related
Here I have two collections :
conso
{
"_id" : ObjectId("5684f3c454b1fd6926c322fd"),
"client" : "1",
"conso" : "4"
}
{
"_id" : ObjectId("56d82612b63f1f31cf906003"),
"client" : "1",
"conso" : "2"
}
{
"_id" : ObjectId("56d82612b63f1c31cf906004"),
"client" : "2",
"conso" : "10"
}
{
"_id" : ObjectId("59301c39028afaf3450e2890"),
"client" : "2",
"conso" : "20"
}
{
"_id" : ObjectId("59301c39029afaf4450e2885"),
"client" : "3",
"conso" : "18"
}
{
"_id" : ObjectId("59301c39030afaf4450e2885"),
"client" : "3",
"conso" : "12"
}
class
{
"_id" : ObjectId("5f6d219d345f0066299c1fd6"),
"CLIENT" : "1",
"class" : "A"
}
{
"_id" : ObjectId("5e6d219c345f0066299c1fd6"),
"CLIENT" : "2",
"class" : "A"
}
{
"_id" : ObjectId("5e6d02c97d9426227fa00401"),
"CLIENT" : "3",
"class" : "B"
}
I would like to have :
{"class" : "A", avg : 9}
{"class" : "B", avg : 15}
My code :
db.conso.aggregate([
{$group: {_id : {class: "$class"}, avg: { $avg: "$conso" }}},
{$lookup: {from: "class", localField: "client", foreignField: "CLIENT", as: "class_info"}}
]);
and I obtain :
{ "_id" : { "class" : null }, "avg" : 11, "class_info" : [ ] }
After having the below changes to your conso collection and your query, we can get the desired result.
Modification in conso collection
The attribute conso in the conso collection takes a string value which we need to change it into number, so the modified collection will look like
{"_id":"5684f3c454b1fd6926c322fd","client":"1","conso":4},
{"_id":"56d82612b63f1f31cf906003","client":"1","conso":2},
{"_id":"56d82612b63f1c31cf906004","client":"2","conso":10},
{"_id":"59301c39028afaf3450e2890","client":"2","conso":20},
{"_id":"59301c39029afaf4450e2885","client":"3","conso":18},
{"_id":"59301c39030afaf4450e2885","client":"3","conso":12}
If you are not willing to modify your collection then you can use $toInt to convert the conso string into number, this step will add another pipeline stage( $project ) to our existing query between $lookup and $group. Please note that $toInt will be giving error if the value cannot be converted to int.
Modification to the query
Do the $lookup first and then perform $group and in the group stage _id part need to changed from class: "$class" to class: "$class_info.class"
Modified query will be
db.conso.aggregate([
{ $lookup: {
from: "class",
localField: "client",
foreignField: "CLIENT",
as: "class_info"}
},
{ $group: {_id : {class: "$class_info.class"}, avg: { $avg: "$conso" }}},
]);
Hope it Helps!
For MongoDB, when using $lookup to query more than one collection, is it possible to get a values-only list for a field returned in the $lookup?
What I don't want is a list of the full object with all its key/values.
Data:
failover_tool:PRIMARY> db.foo.find().pretty()
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo"
}
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo"
}
failover_tool:PRIMARY> db.bar.find().pretty()
{
"_id" : ObjectId("5ce72e0c5267960532b8df06"),
"name" : "bar1",
"foo" : "foo1"
}
{
"_id" : ObjectId("5ce72e165267960532b8df07"),
"name" : "bar2",
"foo" : "foo1"
}
{
"_id" : ObjectId("5ce72e1d5267960532b8df08"),
"name" : "bar3",
"foo" : "foo2"
}
Desired Query Output
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo",
"bars" : ["bar1", "bar2"]
},
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo",
"bars" : ["bar3"]
}
Closest
This query seems like it's almost there, but it returns too much data in the bars field:
db.foo.aggregate({
$lookup: {
from:"bar",
localField:"name",
foreignField: "foo",
as:"bars"
}
}).pretty()
Just use .dot notation with the name field
db.foo.aggregate([
{ "$lookup": {
"from": "bar",
"localField": "name",
"foreignField": "foo",
"as": "bars"
}},
{ "$addFields": { "bars": "$bars.name" }}
])
MongoPlayground
Hope below query helps :
db.foo.aggregate([{
$lookup: {
from:"bar",
localField:"name",
foreignField: "foo",
as:"bars"
}
},
{$unwind : '$bars'},
{
$group : {
_id : {
_id : '$_id',
name : '$name',
desc : '$desc'
},
bars : { $push : '$bars.name'}
}
},
{
$project : {
_id : '$_id._id',
name : '$_id.name',
desc : '$_id.desc',
bars : '$bars'
}
}
]).pretty()
output :
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo",
"bars" : [
"bar3"
]
}
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo",
"bars" : [
"bar1",
"bar2"
]
}
I just want change a Id by thea name corresponding. But id is in a collection and name in other collection. "$lookup" Mongo doesn't work in my case...
The first collection "parameter" contains items with the "category_id":
{"_id" : ObjectId("56cc8827b9e4ed0fd42a4569"),
"data" : {
"capacity" : NumberInt(60),
"categories" : [
{
"category_id" : "5964961294ff4a37988e8f9b",
"nbMax" : NumberInt(1),
"nbRes" : NumberInt(0)
},
{
"category_id" : "596495c994ff4a37988e8f99",
"nbMax" : NumberInt(1),
"nbRes" : NumberInt(0)
},
],
},
"type" : "launcher",
"name" : "launcherp01",
"description" : "",
"_class" : "parameter"
}
....
{
"_id" : ObjectId("56cc8827b9e4ed0fd42a4847"),
"data" : {
"capacity" : NumberInt(60),
"categories" : [
{
"category_id" : "596495c994ff4a37988e8f99",
"nbMax" : NumberInt(1),
"nbRes" : NumberInt(0)
},
{
"category_id" : "8864961294ff4a37988e8f3b",
"nbMax" : NumberInt(1),
"nbRes" : NumberInt(0)
},
],
},
"type" : "launcher",
"name" : "launcherp01",
"description" : "",
"_class" : "parameter"
}
.....
The second Collection "reference" contains the description of categories with the _id (same as category_id in first collection) and the name:
{
"_id" : ObjectId("596495c994ff4a37988e8f99"),
"taskType" : "qc",
"type" : "category",
"name" : "**qcSupportNormal01**",
"_class" : "reference",
}
{
"_id" : ObjectId("5964961294ff4a37988e8f9b"),
"taskType" : "transcode",
"type" : "category",
"name" : "tsSupportNormal01",
"_class" : "reference",
}
I want something like that (item with categories names not id):
{
"_id" : ObjectId("56cc8827b9e4ed0fd42a4569"),
"type" : "launcher",
"categories" : [
"qcSupportNormal01", //--->> name from reference collection
"tsSupportNormal01", //--->> name from reference collection
}
{
"_id" : ObjectId("56cc8827b9e4ed0fd42a4847"),
"type" : "launcher",
"categories" : [
"qcSupportNormal01", //--->> name from reference collection
"qptestNormal01", //--->> name from reference collection
...
My query:
db.parameters.aggregate([
///////////////////////item filter
{$match: {
type:{ $in: [ "launcher" ] } ,
}},
///////////////////// foreign field
{$lookup: {
from: "references",
localField: "categories",
foreignField: "_id",
as: "references"
}},
/////////////////// projection
{$project:
{_id:1,type:1,categories:"$data.categories.category_id"
}},
])
but the result is always with id not the name:
{
"_id" : ObjectId("56cc8827b9e4ed0fd42a4569"),
"type" : "launcher",
"categories" : [
"5964961294ff4a37988e8f9b", //--->> id not name from reference collection !
"596495c994ff4a37988e8f99", //--->> id not name from reference collection !
...
How to have the category name not the id.
Very simple in Sql (joint with foreign key and reference table) but complex in Mongo query ...
Thanks for your help
je déteste le langage Mongo !
There are a couple of issues with your approach:
You are trying to call $lookup between an ObjectId and a string value.
For your categories field you are mapping it with the category_id from the data.categories whereas it should have come from references.name.
The field category_id is nested inside of an array and when you are calling the $lookup you are matching it with the whole array instead of the key field.
So, now the best approach would be to convert the field category_id to an ObjectId and then call the $lookup for the field(instead of modifying the actual field i have appended a new field named as cat_id).
Here is how i would do it:
db.parameter.aggregate([
{
$match: {
type: { $in: ["launcher"] },
}
},
{ $unwind: "$data.categories" },
{ $addFields: { "data.categories.cat_id": { $convert: { input: "$data.categories.category_id", to: "objectId" } } } },
{
$lookup: {
from: "references",
localField: "data.categories.cat_id",
foreignField: "_id",
as: "ref"
}
},
{
$project:
{
_id: 1, type: 1, categories: "$ref.name"
}
}
]);
Which will give you an output as:
{
"_id" : ObjectId("56cc8827b9e4ed0fd42a4569"),
"type" : "launcher",
"categories" : [
"tsSupportNormal01"
]
},
{
"_id" : ObjectId("56cc8827b9e4ed0fd42a4569"),
"type" : "launcher",
"categories" : [
"**qcSupportNormal01**"
]
},
{
"_id" : ObjectId("56cc8827b9e4ed0fd42a4847"),
"type" : "launcher",
"categories" : [
"**qcSupportNormal01**"
]
},
{
"_id" : ObjectId("56cc8827b9e4ed0fd42a4847"),
"type" : "launcher",
"categories" : [ ]
}
For MongoDB, when using $lookup to query more than one collection, is it possible to get a values-only list for a field returned in the $lookup?
What I don't want is a list of the full object with all its key/values.
Data:
failover_tool:PRIMARY> db.foo.find().pretty()
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo"
}
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo"
}
failover_tool:PRIMARY> db.bar.find().pretty()
{
"_id" : ObjectId("5ce72e0c5267960532b8df06"),
"name" : "bar1",
"foo" : "foo1"
}
{
"_id" : ObjectId("5ce72e165267960532b8df07"),
"name" : "bar2",
"foo" : "foo1"
}
{
"_id" : ObjectId("5ce72e1d5267960532b8df08"),
"name" : "bar3",
"foo" : "foo2"
}
Desired Query Output
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo",
"bars" : ["bar1", "bar2"]
},
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo",
"bars" : ["bar3"]
}
Closest
This query seems like it's almost there, but it returns too much data in the bars field:
db.foo.aggregate({
$lookup: {
from:"bar",
localField:"name",
foreignField: "foo",
as:"bars"
}
}).pretty()
Just use .dot notation with the name field
db.foo.aggregate([
{ "$lookup": {
"from": "bar",
"localField": "name",
"foreignField": "foo",
"as": "bars"
}},
{ "$addFields": { "bars": "$bars.name" }}
])
MongoPlayground
Hope below query helps :
db.foo.aggregate([{
$lookup: {
from:"bar",
localField:"name",
foreignField: "foo",
as:"bars"
}
},
{$unwind : '$bars'},
{
$group : {
_id : {
_id : '$_id',
name : '$name',
desc : '$desc'
},
bars : { $push : '$bars.name'}
}
},
{
$project : {
_id : '$_id._id',
name : '$_id.name',
desc : '$_id.desc',
bars : '$bars'
}
}
]).pretty()
output :
{
"_id" : ObjectId("5ce72e4a5267960532b8df0a"),
"name" : "foo2",
"desc" : "second foo",
"bars" : [
"bar3"
]
}
{
"_id" : ObjectId("5ce72e415267960532b8df09"),
"name" : "foo1",
"desc" : "first foo",
"bars" : [
"bar1",
"bar2"
]
}
I have three collections Subscribers,Address,Languages
Subscribers schema has following record
{
"_id" : ObjectId("5a73fae80290f7eca89e99d4"),
"FirstName" : "Rahul",
"LastName" : "Shah",
"Address" : DBRef("Address", ObjectId("5a6ec9bda3baf2b516de5769")),
"Languages" : [
DBRef("Languages", ObjectId("5a6304ffc3c3f119fc0e60c9")),
DBRef("Languages", ObjectId("5a6ee970a3baf2b516de576b"))
]
}
Address schema has following record
{
"_id" : ObjectId("5a6ec9bda3baf2b516de5769"),
"Address1" : "Vastrapur,
"Address2" : "Satellite",
"City" : "Ahmedabad",
"Country" : "India",
"State" : "Gujarat",
"ZipCode" : "380015",
"PhoneNumber" : "(987)654-3210",
"FaxNumber" : ""
}
Languages schema has following record
{
"_id" : ObjectId("5a6304ffc3c3f119fc0e60c9"),
"Name" : "English",
"LanguageCulture" : "English",
"IsDeleted" : false
}
{
"_id" : ObjectId("5a6ee970a3baf2b516de576b"),
"Name" : "Hindi",
"LanguageCulture" : "Hindi",
"IsDeleted" : false
}
I want output be like as below
{
"_id" : ObjectId("5a73fae80290f7eca89e99d4"),
"FirstName" : "Rahul",
"LastName" : "Shah",
"Address" : {
"_id" : ObjectId("5a6ec9bda3baf2b516de5769"),
"Address1" : "Vastrapur",
"Address2" : " Satellite ",
"City" : " Ahmedabad ",
"Country" : " India ",
"State" : " Gujarat ",
"ZipCode" : " 380015 ",
"PhoneNumber" : "(987)654 - 3210 ",
"FaxNumber" : " "
},
"Languages" : [{
"_id" : ObjectId(" 5a6304ffc3c3f119fc0e60c9 "),
" Name" : " English ",
"LanguageCulture" : " English ",
"IsDeleted" : false
}, {
"_id" : ObjectId(" 5a6ee970a3baf2b516de576b "),
"Name" : " Hindi ",
"LanguageCulture" : " Hindi ",
"IsDeleted" : false
}
]
}
For that Im doing aggregation as below
db.Subscribers.aggregate([{
$project : {
"FirstName" : 1,
"LastName" : 1,
Address : {
$let : {
vars : {
refParts : {
$objectToArray : "$$ROOT.Address"
}
},
in : "$$refParts"
}
}
}
}, {
$match : {
"addressRefs" : {
$exists : true
}
}
}, {
$project : {
"FirstName" : 1,
"LastName" : 1,
"addressRefs" : {
$arrayElemAt : ["$addressRefs", 1]
}
}
}, {
$lookup : {
from : "Addresses",
localField : "addressRefs",
foreignField : "_id",
as : "address_data"
}
}, {
$project : {
"FirstName" : 1,
"LastName" : 1,
"AddressUuid" : {
$arrayElemAt : ["$address_data.uuid", 0]
}
}
}
])
Here you go. It's not pretty because of your use of DBRefs but it gets the job done thanks to the following SO answer:
Mongo how to $lookup with DBRef
db.Subscribers.aggregate([{
$addFields: {
"Address": {$arrayElemAt: [{$objectToArray: "$Address"}, 1]}, // simple DBRef transformation if the field is not an array of DBRefs
"Languages": {
$map: {
input: {
$map: {
input: "$Languages",
in: {
$arrayElemAt: [{$objectToArray: "$$this"}, 1]
},
}
},
in: "$$this.v"
}
}
}
}, {
$lookup: {
from: "Address",
localField: "Address.v", // note the .v here
foreignField: "_id",
as: "Address"
}
}, {
$lookup: {
from: "Languages",
localField: "Languages",
foreignField: "_id",
as: "Languages"
}
}, {
$addFields: {
"Address": { $arrayElemAt: [ "$Address", 0 ] } // you can replace this step with an $unwind to 'flatten' the single-value array but you might want to use preserveNullAndEmptyArrays: true in order to cater for subscribers that come without valid Address references
}
}])