Mongodb lookup with array - mongodb

I have two collections first one is
user_profile collection
const userProfileSchema = mongoose.Schema({
phone_number: {
type: String,
required: false,
},
primary_skills: [
{
skill_id: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Skill'
},
years: Number,
}
]
});
sample data
{
"phone_number":"222",
"primary_skills":[{skill_id:1,years:12},{skill_id:2,years:13}]
}
in the primary_skills the key skill_id is mapped with another collection named skills
skills collection
const skillSchema = mongoose.Schema({
name: {
type: String,
required: true,
unique:true,
},
});
sample data
[
{
id:1,
name:'php'
},
{
id:2,
name:'java'
}
]
I want to fetch all values in the user_profile collection along with the respective skills name
expected output:
{
"phone_number":"222",
"primary_skills":[{
name:"php",skill_id:1,years:12
},{
name:"java",skill_id:2,years:13}
]
}
I found a similar thread to my question MongoDB lookup when foreign field is an array of objects but it's doing the opposite of what I want
This is the query I tried
profile.aggregate([{
$lookup:{
from:'skills',
localField:'primary_skills.skill_id',
foreignField:'_id',
'as':'primary_skills'
}
}])
This works fine but it didn't contain the years key

You need to do it with $unwind and $group,
$unwind primary_skills because its an array and we need to lookup sub document wise
db.user_profile.aggregate([
{
$unwind: "$primary_skills"
},
$lookup to join primary_skills, that you have already did
{
$lookup: {
from: "skills",
localField: "primary_skills.skill_id",
foreignField: "id",
as: "primary_skills.name"
}
},
$unwind primary_skills.name that we have stored join result, its array and we are unwinding to do object
{
$unwind: {
path: "$primary_skills.name"
}
},
$addFields replace field name that we have object and we need only name
{
$addFields: {
"primary_skills.name": "$primary_skills.name.name"
}
},
$group by _id because we have unwind and we need to combine all documents
{
$group: {
_id: "$_id",
phone_number: {
$first: "$phone_number"
},
primary_skills: {
$push: "$primary_skills"
}
}
}
])
Playground: https://mongoplayground.net/p/bDmrOwmASn5

Related

MongoDB Exclude docs from aggregate if field/reference exists in other collection

const sellerSchema = Schema(
{
name: String,
url:String
}
const productSchema = Schema(
{
title: String,
sellerUrl:String
}
Below query will return unique sellerUrl from all products:
context.Product.aggregate([
{
$group: {
_id: "$sellerUrl",
}
}
]);
But I also want to exclude from aggregation, sellers that I already saved. So if url == sellerUrl aggregation must exclude that seller.
Please help me
You can try below query :
db.product.aggregate([
{
$group: {
_id: "", /** group on no condition & push all unique `sellerUrl` to sellerUrls array */
sellerUrls: { $addToSet: "$sellerUrl" }
}
},
{
$lookup: {
from: "seller",
let: { sellerUrls: "$sellerUrls" }, // creating local variable
pipeline: [
{ $group: { _id: "", urls: { $addToSet: "$url" } } }, /** group on no condition & push all unique `url` to urls array */
{ $project: { _id: 0, uniqueAndNotInSellerColl: { $setDifference: [ "$$sellerUrls", "$urls" ] } } } // get difference between two arrays
],
as: "data" // As we're grouping will always be one doc/element in an array
}
},
/** Create a new root doc from getting first element(though it will have only one) from `data` array */
{
$replaceRoot: { newRoot: { $arrayElemAt: [ "$data", 0 ] } }
}
])
Test : mongoplayground
Update :
As you need few other fields from product collection but not just the sellerUrl field then try below query :
db.product.aggregate([
{
$group: {
_id: "$sellerUrl",
docs: { $push: { title: "$title" } } // We're only retrieving `title` field from `product` docs, if every field is needed use `$$ROOT`
}
},
/** We've used basic `lookup` stage, use this if you've only few matching docs from `seller` collection
* If you've a lot of matching docs for each `_id` (sellerUrl),
* then instead of getting entire `seller` doc (which is not needed) use `lookup` with aggregation pipeline &
* just get `_id`'s of seller docs for better performace refer previous query
*/
{
$lookup: {
from: "seller",
localField: "_id",
foreignField: "url",
as: "sellerDocs"
}
},
/** match will retain only docs which doesn't have a matching doc in seller collection */
{
$match: { sellerDocs: [] }
},
{
$project: { sellerDocs: 0 }
}
])
Test : mongoplayground

MongoDB Aggregation pipeline inner array mapping

I am trying to perform a lookup operation between two collections like shown below,
first collection records:
{
field1: "FIELD",
title: "sometitle",
secondIds: [
{
value: "nestedval1",
secondId: "234
},
{
value: "netedval2,
secondId: "342"
}
]
}
Second collection record
{
id: "234",
secvalue: "somevalue"
},
{
id: "342",
secvalue: "anothervalue"
}
I am trying to get the output in the below format for matching field1 name "FIELD" inside the first collection.
{
field1: "FIELD",
title: "sometitle",
secondIds: [
{
value: "nestedval1",
secondId: "234",
second: {
id: "234",
secvalue: "somevalue"
}
},
{
value: "nestedval2",
secondId: "342",
second: {
id: "342",
secvalue: "anothervalue"
}
}
]
}
for aggregation pipeline after matching operation, I still stuck at how to create a lookup operation for retrieving the second collection entry mapped with the first. Can it possible to do it or do have any other way to achieve it?
firstCollection.aggregate([
{ $unwind: '$secondIds' }, // Lets first separate the secondIds to as different documents
{
$lookup: {
from: 'secondCollection', // second collection name
localField: 'secondIds.secondId', // field in first collection after unwind
foreignField: 'id', // field in second collection
as: 'secondIds.second' // field to attach values matched
}
},
{ $unwind: '$secondIds.second' }, // attached value after lookup will be an array, so let unwind to make it a object
{ $project: { 'secondIds.second._id': 0 } }, // remove _id
{
$group: {
_id: { // grouper fields
field1: "$field1",
title: "$title",
},
secondIds: { $push: '$secondIds' } // group by pushing to an array
}
},
{
$project: { // projection
_id: 0,
field1: '$_id.field1',
title: "$_id.title",
secondIds: 1
}
}
]).pretty()
Explanations are in the comments

$lookup using multiple criteria mongodb java aggregation

Have 2 following collections:
user collection
{
userId:user1,
creationTimeStamp:2019-11-05T08:15:30
status:active
},
{
userId:user2,
creationTimeStamp:2019-10-05T08:15:30
status:active
}
document collection
{
userId:user1,
category:Development
published:true
},
{
userId:user2,
category:Development
published:false
}
I want to join these two collections and filter users such that documents which are of development category and are not published from active users between creationtimestamp
How can I write a mongodb java aggregation in order to get a result like this:
{
userId: user2,
status:active,
category:Development,
published:false
}
You could run below aggregation query on the document collection to get the expected result
[{$match: {
category:'development',
published: false
}}, {$lookup: {
from: 'user',
localField: 'userId',
foreignField: 'userId',
as: 'JoinedTable'
}}, {$unwind: {
path: '$JoinedTable'
}}, {$group: {
_id: '$_id',
userId: {
$first: '$userId'
},
status: {
$first: '$JoinedTable.status'
},
category: {
$first: '$category'
},
published: {
$first: '$published'
},
}}]
Explanation:
1. filter documents using match for criteria category: 'development' & published: false
2. join document collection with user collection with key userId
3. unwind the joined collection field to convert array to object
4. project the fields needed using groups.
Hope this helps!
You haven't mentioned about the duplicate of userId in User collection.
So the script is
[{
$match: {
category: "Development",
published: false
}
}, {
$lookup: {
from: 'user',
localField: 'userId',
foreignField: 'userId',
as: 'joinUser'
}
}, {
$unwind: {
path: "$joinUser",
preserveNullAndEmptyArrays: true
}
}, {
$match: {
"joinUser.status": "active"
}
}, {
$addFields: {
"status": "$joinUser.status"
}
}, {
$project: {
_id: 0,
userId: 1,
category: 1,
published: 1,
status: 1
}
}]
And the java code,
include these imports
import static org.springframework.data.mongodb.core.aggregation.Aggregation.match;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.lookup;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.unwind;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.project;
method is,
public Object findAllwithVideos() {
Aggregation aggregation=Aggregation.newAggregation(
match(Criteria.where("category").is("Development").and("published").is(false)),
lookup("user","userId","userId","joinUser"),
unwind("joinUser",true),
new AggregationOperation(){
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext){
return new Document("$addFields",
new Document("status","$joinUser.status")
);
}
},
project("userId","category","published","status")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Document.class), Object.class);
}

How to lookup a field in an array of subdocuments in mongoose?

I have an array of review objects like this :
"reviews": {
"author": "5e9167c5303a530023bcae42",
"rate": 5,
"spoiler": false,
"content": "This is a comment This is a comment This is a comment.",
"createdAt": "2020-04-12T16:08:34.966Z",
"updatedAt": "2020-04-12T16:08:34.966Z"
},
What I want to achieve is to lookup the author field and get the user data, but the problem is that the lookup I am trying to use only returns this to me:
Code :
.lookup({
from: 'users',
localField: 'reviews.author',
foreignField: '_id',
as: 'reviews.author',
})
Response :
Any way to get the author's data in that field? That's where the author's Id is.
Try to execute below query on your database :
db.reviews.aggregate([
/** unwind in general is not needed for `$lookup` for if you wanted to match lookup result with specific elem in array is needed */
{
$unwind: { path: "$reviews", preserveNullAndEmptyArrays: true },
},
{
$lookup: {
from: "users",
localField: "reviews.author",
foreignField: "_id",
as: "author", // Pull lookup result into 'author' field
},
},
/** Update 'reviews.author' field in 'reviews' object by checking if 'author' field got a match from 'users' collection.
* If Yes - As lookup returns an array get first elem & assign(As there will be only one element returned -uniques),
* If No - keep 'reviews.author' as is */
{
$addFields: {
"reviews.author": {
$cond: [
{ $ne: ["$author", []] },
{ $arrayElemAt: ["$author", 0] },
"$reviews.author",
],
},
},
},
/** Group back the documents based on '_id' field & push back all individual 'reviews' objects to 'reviews' array */
{
$group: {
_id: "$_id",
reviews: { $push: "$reviews" },
},
},
]);
Test : MongoDB-Playground
Note : Just in case if you've other fields in document along with reviews that needs to be preserved in output then starting at $group use these stages :
{
$group: {
_id: "$_id",
data: {
$first: "$$ROOT"
},
reviews: {
$push: "$reviews"
}
}
},
{
$addFields: {
"data.reviews": "$reviews"
}
},
{
$project: {
"data.author": 0
}
},
{
$replaceRoot: {
newRoot: "$data"
}
}
Test : MongoDB-Playground
Note : Try to keep queries to run on lesser datasets maybe by adding $match as first stage to filter documents & also have proper indexes.
you should use populate('author') method of mongoose on the request to the server which gets the id of that author and adds the user data to the response of mongoose
and dont forget to set your schema in a way that these two collections are connected
in your review schema you should add ref to the schema which the author user is saved
author: { type: Schema.Types.ObjectId, ref: 'users' },
You can follow this code
$lookup:{
from:'users',
localField:'reviews.author',
foreignField:'_id',
as:'reviews.author'
}
**OR**
> When You find the doc then use populate
> reviews.find().populate("author")

How can I perform a $lookup on a property value in a nested array?

I have an articles collection that contains an array of comments, and this array contains an array of sub_comments. Here is what it looks like:
let articleSchema = mongoose.Schema({
title: {type: String},
comments: [{
comment: {type: String},
creator: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
sub_comments: [{
comment: {type: String},
creator: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}
}]
}]
});
I'm trying to run an aggregate query that does a $lookup on the comments.creator and sub_comments.creator fields. Here's what I've tried so far, but it doesn't work:
this.model('Article')
.aggregate([
{
$match: {_id: article_id}
},
{
$lookup: {
from: "users",
localField: "comments.creator",
foreignField: "_id",
as: "comments.creator"
}
},
{
$unwind: {
path: "$comments.creator",
preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "users",
localField: "comments.sub_comments.creator",
foreignField: "_id",
as: "comments.sub_comments.creator"
}
},
{
$unwind: {
path: "$comments.sub_comments.creator",
preserveNullAndEmptyArrays: true
}
},
{
$project: {
_id: 1,
title: 1,
"comments.comment": 1,
"comments.creator._id": 1,
"comments.creator.name": 1,
"comments.sub_comments.comment": 1,
"comments.sub_comments.creator._id": 1,
"comments.sub_comments.creator.name": 1
}
}
])
.exec(function (err, data) {
...
});
Here's an example of the data response:
[{
"_id": "5b7e1629d00a170018e11234",
"article": "My Article",
"comments": {
"comment": "This is the comment.",
"sub_comments": {},
"creator": {
"_id": "541g2dfeab1e470b00411234",
"name": "John Doe"
}
},
....
}]
Comments should be an array, not an object. This particular comment didn't have any sub-comments, but obviously, I want that to work too. Any ideas on how to get this to work?
the as params in mongodb $lookup with your query affect your result .
as Specifies the name of the new array field to add to the input
documents. The new array field contains the matching documents from
the from collection. If the specified name already exists in the input
document, the existing field is overwritten.
it will overwrite the filed . not add to it .
if your outter is an array ,it will make it to Object and delete the current content .
so the second $lookup in aggregate will not work . in your query
if you want to keep the current data structure intac . you can as a new fieldname and use $project to change the format .