MongoDB Aggregation - project after lookup [duplicate] - mongodb

In mongo, after doing an aggregation with $lookup, I would like the request to return only some fields and not the whole document.
I have the following query :
db.somecollection.aggregate([{
$lookup: {
from: "campaigns",
localField: "campId",
foreignField: "_id",
as: "campaign"
}
}, {
$unwind: "$campaign"
}, {
$lookup: {
from: "entities",
localField: "campaign.clientid",
foreignField: "_id",
as: "campaign.client"
}
}]);
This request will return me this :
{
"_id" : ObjectId("56cc7cd1cc2cf62803ebfdc7"),
"campId" : ObjectId("56c740e4479f46e402efda84"),
"articleId" : ObjectId("56c742c06094640103ba3843"),
"campaign" : {
"_id" : ObjectId("56c740e4479f46e402efda84"),
"clientid" : ObjectId("56c740b8479f46e402efda83"),
"client" : [
{
"_id" : ObjectId("56c740b8479f46e402efda83"),
"username" : "someusername",
"shhh" : "somehashedpassword",
"email" : "mail#mail.com",
}
]
}
The request works well, but I would like to filter the fields in campaign.client to only get for example _id and username. Is there a way to do this in a MongoDB aggregate request?

Just to help others with this, #SiddhartAjmera has the right answer, I only needed to add double quotes for nested values like "campaign.clientid".
The final code should be:
db.somecollection.aggregate([
{
"$lookup": {
"from": "campaigns",
"localField": "campId",
"foreignField": "_id",
"as": "campaign"
}
},
{
"$unwind": "$campaign"
},
{
"$lookup": {
"from": "entities",
"localField": "campaign.clientid",
"foreignField": "_id",
"as": "campaign.client"
}
},
{
"$project": {
"_id": 1,
"campId": 1,
"articleId": 1,
"campaign._id": 1,
"campaign.clientid": 1,
"campaign.client._id": 1,
"campaign.client.username": 1
}
}
]);

Using pipeline and $project inside $lookup
db.somecollection.aggregate([{
$lookup: {
from: "campaigns",
localField: "campId",
foreignField: "_id",
as: "campaign"
}
}, {
$unwind: "$campaign"
}, {
$lookup: {
from: "entities",
let: { client_id: "$campaign.clientid" },
pipeline : [
{ $match: { $expr: { $eq: [ "$_id", "$$client_id" ] } }, },
{ $project : { _id:1, username:1 } }
],
as: "campaign.client"
}
}]);

Just to add a little thing to the previous answer: You can put a 0 to a project item that you want to ignore and the rest will be retrieved, so you don´t need to write all the list with 1:
db.somecollection.aggregate([
{
"$lookup": {
"from": "campaigns",
"localField": "campId",
"foreignField": "_id",
"as": "campaign"
}
},
{
"$unwind": "$campaign"
},
{
"$lookup": {
"from": "entities",
"localField": "campaign.clientid",
"foreignField": "_id",
"as": "campaign.client"
}
},
{
"$project": {
"campaign.client.shhh": 0
}
}
])

I know this is very late to answer to this question. But in my opinion, an update can sometimes prove to be very beneficial.
The project stage is great but you'd still be requesting for the entire dock in the $lookup stage. The fields are only filtered in the projection stage following it.
After the release of MongoDB 3.6, you can now add a pipeline to a $lookup stage, to specify multiple join conditions. Find more details in their official docs.
Specify Multiple Join Conditions with $lookup
You can transform your aggregation pipeline as follows, to get the desired result:
db.somecollection.aggregate([{
$lookup: {
from: "campaigns",
localField: "campId",
foreignField: "_id",
as: "campaign"
}
}, {
$unwind: "$campaign"
}, {
$lookup: {
from: "entities",
let: {clientid: '$campaign.clientid'},
pipeline: [
{ '$match':
{ '$expr':
{
'$eq': ['$_id', '$$clientid']
}
}
},
{ '$project':
'_id': 1,
'username': 1
}
]
as: "campaign.client"
}
}]);
This way you can filter the fields of the joined collection right inside the $lookup stage.
Notice the $$ sign inside the $match stage of inner pipeline. It is used to denote a custom field defined inside the let block.

Related

MongoDB Aggregation use value from Match Object in pipelin

i'm using the following aggregation:
const aggregate = [
{
$match: {
mainCatId: new ObjectId(catId),
},
},
{
"$lookup": {
"from": "products",
"pipeline": [
{ "$match": { "subCategory": '$_id' } },
],
"as": "products"
}
},
{ "$unwind": "$products" }
];
The problem is that i have to match the id of each doc in the pipeline section but this is not working. So the question is how can i match the id i"m getting from the match above
Is the syntax below, which uses the traditional syntax for single join conditions, what you are looking for?
const aggregate = [
{
$match: {
mainCatId: new ObjectId(catId),
},
},
{
"$lookup": {
"from": "products",
localField: "subCategory",
foreignField: "_id",
"as": "products"
}
},
{
"$unwind": "$products"
}
]
If not, please provide sample documents and further details about the ways in which your current approach (and this proposed solution) are not working.

Mongodb 3.4 version can not Join on string to an ObjectId field [duplicate]

I have two collections
User
{
"_id" : ObjectId("584aac38686860d502929b8b"),
"name" : "John"
}
Role
{
"_id" : ObjectId("584aaca6686860d502929b8d"),
"role" : "Admin",
"userId" : "584aac38686860d502929b8b"
}
I want to join these collection based on the userId (in role collection) - _id ( in user collection).
I tried the below query:
db.role.aggregate({
"$lookup": {
"from": "user",
"localField": "userId",
"foreignField": "_id",
"as": "output"
}
})
This gives me expected results as long as i store userId as a ObjectId. When my userId is a string there are no results.
Ps: I tried
foreignField: '_id'.valueOf()
and
foreignField: '_id'.toString()
. But no luck to match/join based on a ObjectId-string fields.
Any help will be appreciated.
You can use $toObjectId aggregation from mongodb 4.0 which converts String id to ObjectId
db.role.aggregate([
{ "$lookup": {
"from": "user",
"let": { "userId": "$_id" },
"pipeline": [
{ "$addFields": { "userId": { "$toObjectId": "$userId" }}},
{ "$match": { "$expr": { "$eq": [ "$userId", "$$userId" ] } } }
],
"as": "output"
}}
])
Or you can use $toString aggregation from mongodb 4.0 which converts ObjectId to String
db.role.aggregate([
{ "$addFields": { "userId": { "$toString": "$_id" }}},
{ "$lookup": {
"from": "user",
"localField": "userId",
"foreignField": "userId",
"as": "output"
}}
])
This is not possible as of MongoDB 3.4. This feature has already been requested, but hasn't been implemented yet. Here are the corresponding tickets:
SERVER-22781: Allow $lookup between ObjectId (_id.str) and
string
SERVER-24947: Need a type conversion mechanism for booleans,
ISODates, ObjectID
For now you'll have to store userId as ObjectId
EDIT
The previous tickets were fixed in MongoDB 4.0. You can now achieve this with the folowing query:
db.user.aggregate([
{
"$project": {
"_id": {
"$toString": "$_id"
}
}
},
{
"$lookup": {
"from": "role",
"localField": "_id",
"foreignField": "userId",
"as": "role"
}
}
])
result:
[
{
"_id": "584aac38686860d502929b8b",
"role": [
{
"_id": ObjectId("584aaca6686860d502929b8d"),
"role": "Admin",
"userId": "584aac38686860d502929b8b"
}
]
}
]
try it online: mongoplayground.net/p/JoLPVIb1OLS
I think the previous answer has an error on the $toObjectId case. The let statement applies to the db collection on which the function aggregate is called (i.e 'role') and not on the collection pointed by "from" (i.e 'user').
db.role.aggregate([
{ "$lookup": {
"let": { "userObjId": { "$toObjectId": "$userId" } },
"from": "user",
"pipeline": [
{ "$match": { "$expr": { "$eq": [ "$_id", "$$userObjId" ] } } }
],
"as": "userDetails"
}}
])
Or
db.role.aggregate([
{ "$project": { "userObjId": { "$toObjectId": "$userId" } } },
{ "$lookup": {
"localField": "userObjId",
"from": "user",
"foreignField": "$_id",
"as": "userDetails"
}}
])
And
db.user.aggregate([
{ "$project": { "userStrId": { "$toString": "$_id" }}},
{ "$lookup": {
"localField": "userStrId",
"from": "role",
"foreignField": "userId",
"as": "roleDetails"
}}
])
For example we have two collections:
Authors like {'_id': ObjectId(), name: 'John Smith'}
Messages like {'_id': ObjectId(), text: 'Message Text', authorId: 'stringId'}
And we need to get messages with author names and other data
Then:
db.messages.aggregate([
{
$addFields: {
'$authorObjectId': {$toObjectId: $authorId}
}
},
{
$lookup: {
from: 'authors',
localField: '$authorObjectId',
foreignField: '_id',
as: 'author'
}
}
])
Explained:
Our aggregation pipeline has two steps:
First: We add additional field to messages which contains converted string authorId to ObjectId()
Second: We use this field as localField (in messages) to be compared with foreignField '_id' (in authors)

MongoDB: Lookup - Collections with Same Fields

I've two collections Users and Notes. Both collections contain an id property and the Notes collection has an userid that is the id of some user on the Users collection.
Now, I'm trying to aggregate (join) some user information into Notes:
db.getCollection("Notes").aggregate(
{
"$lookup": {
"from": "Users",
"let": {
"idForeignField": "$id"
},
"pipeline": [
{
"$match": {
"$expr": {
"$and": [{
"$eq": ["$userid", "$$idForeignField"]
}]
}
}
}
],
"as": "Users#joined"
}
}
);
What I get in a empty Users#joined array. Why? Shouldn't my query work? Is the problem caused by the fact that both collections have an id property? If yes how can I tell let and match what is the right collection?
Update: alternatively a simpler query works just fine:
db.getCollection("Notes").aggregate(
{
$lookup:
{
from: "Users",
localField: "userid",
foreignField: "id",
as: "Users#joined"
}
}
);
However I would like to do it with let and a pipeline in order to add more match conditions.
Thank you.
Your let variable must be userid
db.getCollection("Notes").aggregate([
{ "$lookup": {
"from": "Users",
"let": { "ifForeignField": "$userid" },
"pipeline": [
{ "$match": { "$expr": { "$and": [{ "$eq": ["$id", "$$ifForeignField"] }] }}}
],
"as": "Users#joined"
}}
])

How to convert ObjectID to String in $lookup (aggregation)

I have two collections, article and comments, the articleId in comments is a foreign key of _id in article.
db.collection('article').aggregate([
{
$lookup: {
from: "comments",
localField: "_id",
foreignField: "articleId",
as: "comments"
}
},
...
])
but it doesn't work, because _id in article is an ObjectID and articleId is a string.
You can achieve this using $addFields and $toObjectId aggregations which simply converts string id to mongo objectId
db.collection('article').aggregate([
{ "$lookup": {
"from": "comments",
"let": { "article_Id": "$_id" },
"pipeline": [
{ "$addFields": { "articleId": { "$toObjectId": "$articleId" }}},
{ "$match": { "$expr": { "$eq": [ "$articleId", "$$article_Id" ] } } }
],
"as": "comments"
}}
])
Or using $toString aggregation
db.collection('article').aggregate([
{ "$addFields": { "article_id": { "$toString": "$_id" }}},
{ "$lookup": {
"from": "comments",
"localField": "article_id",
"foreignField": "articleId",
"as": "comments"
}}
])
If you are using MongoDB 4.0 or above then you can directly use $toString on the _id field:
db.collection('article').aggregate([
{
$lookup: {
from: "comments",
localField: { $toString : "_id" },
foreignField: "articleId",
as: "comments"
}
},
...
])
But MongoDB 3.6 doesn't support type conversion inside an aggregation pipeline. So $toString and $convert will only work with MongoDB 4.0.

project some fields in $lookup mongodb [duplicate]

In mongo, after doing an aggregation with $lookup, I would like the request to return only some fields and not the whole document.
I have the following query :
db.somecollection.aggregate([{
$lookup: {
from: "campaigns",
localField: "campId",
foreignField: "_id",
as: "campaign"
}
}, {
$unwind: "$campaign"
}, {
$lookup: {
from: "entities",
localField: "campaign.clientid",
foreignField: "_id",
as: "campaign.client"
}
}]);
This request will return me this :
{
"_id" : ObjectId("56cc7cd1cc2cf62803ebfdc7"),
"campId" : ObjectId("56c740e4479f46e402efda84"),
"articleId" : ObjectId("56c742c06094640103ba3843"),
"campaign" : {
"_id" : ObjectId("56c740e4479f46e402efda84"),
"clientid" : ObjectId("56c740b8479f46e402efda83"),
"client" : [
{
"_id" : ObjectId("56c740b8479f46e402efda83"),
"username" : "someusername",
"shhh" : "somehashedpassword",
"email" : "mail#mail.com",
}
]
}
The request works well, but I would like to filter the fields in campaign.client to only get for example _id and username. Is there a way to do this in a MongoDB aggregate request?
Just to help others with this, #SiddhartAjmera has the right answer, I only needed to add double quotes for nested values like "campaign.clientid".
The final code should be:
db.somecollection.aggregate([
{
"$lookup": {
"from": "campaigns",
"localField": "campId",
"foreignField": "_id",
"as": "campaign"
}
},
{
"$unwind": "$campaign"
},
{
"$lookup": {
"from": "entities",
"localField": "campaign.clientid",
"foreignField": "_id",
"as": "campaign.client"
}
},
{
"$project": {
"_id": 1,
"campId": 1,
"articleId": 1,
"campaign._id": 1,
"campaign.clientid": 1,
"campaign.client._id": 1,
"campaign.client.username": 1
}
}
]);
Using pipeline and $project inside $lookup
db.somecollection.aggregate([{
$lookup: {
from: "campaigns",
localField: "campId",
foreignField: "_id",
as: "campaign"
}
}, {
$unwind: "$campaign"
}, {
$lookup: {
from: "entities",
let: { client_id: "$campaign.clientid" },
pipeline : [
{ $match: { $expr: { $eq: [ "$_id", "$$client_id" ] } }, },
{ $project : { _id:1, username:1 } }
],
as: "campaign.client"
}
}]);
Just to add a little thing to the previous answer: You can put a 0 to a project item that you want to ignore and the rest will be retrieved, so you don´t need to write all the list with 1:
db.somecollection.aggregate([
{
"$lookup": {
"from": "campaigns",
"localField": "campId",
"foreignField": "_id",
"as": "campaign"
}
},
{
"$unwind": "$campaign"
},
{
"$lookup": {
"from": "entities",
"localField": "campaign.clientid",
"foreignField": "_id",
"as": "campaign.client"
}
},
{
"$project": {
"campaign.client.shhh": 0
}
}
])
I know this is very late to answer to this question. But in my opinion, an update can sometimes prove to be very beneficial.
The project stage is great but you'd still be requesting for the entire dock in the $lookup stage. The fields are only filtered in the projection stage following it.
After the release of MongoDB 3.6, you can now add a pipeline to a $lookup stage, to specify multiple join conditions. Find more details in their official docs.
Specify Multiple Join Conditions with $lookup
You can transform your aggregation pipeline as follows, to get the desired result:
db.somecollection.aggregate([{
$lookup: {
from: "campaigns",
localField: "campId",
foreignField: "_id",
as: "campaign"
}
}, {
$unwind: "$campaign"
}, {
$lookup: {
from: "entities",
let: {clientid: '$campaign.clientid'},
pipeline: [
{ '$match':
{ '$expr':
{
'$eq': ['$_id', '$$clientid']
}
}
},
{ '$project':
'_id': 1,
'username': 1
}
]
as: "campaign.client"
}
}]);
This way you can filter the fields of the joined collection right inside the $lookup stage.
Notice the $$ sign inside the $match stage of inner pipeline. It is used to denote a custom field defined inside the let block.