$lookup using multiple criteria mongodb java aggregation - mongodb

Have 2 following collections:
user collection
{
userId:user1,
creationTimeStamp:2019-11-05T08:15:30
status:active
},
{
userId:user2,
creationTimeStamp:2019-10-05T08:15:30
status:active
}
document collection
{
userId:user1,
category:Development
published:true
},
{
userId:user2,
category:Development
published:false
}
I want to join these two collections and filter users such that documents which are of development category and are not published from active users between creationtimestamp
How can I write a mongodb java aggregation in order to get a result like this:
{
userId: user2,
status:active,
category:Development,
published:false
}

You could run below aggregation query on the document collection to get the expected result
[{$match: {
category:'development',
published: false
}}, {$lookup: {
from: 'user',
localField: 'userId',
foreignField: 'userId',
as: 'JoinedTable'
}}, {$unwind: {
path: '$JoinedTable'
}}, {$group: {
_id: '$_id',
userId: {
$first: '$userId'
},
status: {
$first: '$JoinedTable.status'
},
category: {
$first: '$category'
},
published: {
$first: '$published'
},
}}]
Explanation:
1. filter documents using match for criteria category: 'development' & published: false
2. join document collection with user collection with key userId
3. unwind the joined collection field to convert array to object
4. project the fields needed using groups.
Hope this helps!

You haven't mentioned about the duplicate of userId in User collection.
So the script is
[{
$match: {
category: "Development",
published: false
}
}, {
$lookup: {
from: 'user',
localField: 'userId',
foreignField: 'userId',
as: 'joinUser'
}
}, {
$unwind: {
path: "$joinUser",
preserveNullAndEmptyArrays: true
}
}, {
$match: {
"joinUser.status": "active"
}
}, {
$addFields: {
"status": "$joinUser.status"
}
}, {
$project: {
_id: 0,
userId: 1,
category: 1,
published: 1,
status: 1
}
}]
And the java code,
include these imports
import static org.springframework.data.mongodb.core.aggregation.Aggregation.match;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.lookup;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.unwind;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.project;
method is,
public Object findAllwithVideos() {
Aggregation aggregation=Aggregation.newAggregation(
match(Criteria.where("category").is("Development").and("published").is(false)),
lookup("user","userId","userId","joinUser"),
unwind("joinUser",true),
new AggregationOperation(){
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext){
return new Document("$addFields",
new Document("status","$joinUser.status")
);
}
},
project("userId","category","published","status")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Document.class), Object.class);
}

Related

Applying $exists to MongoDB aggregation

I have two mongo collections structured like so:
customers
{
_id: ObjectId,
name: String,
companyId: ObjectId
}
companies
{
_id: ObjectId,
name: String,
rights: [
add: boolean,
edit: boolean,
shop: boolean
]
}
So each customer has a companyId that lets us look up the companies.rights available. I need to get a list of which companies have customers but don't have the shop property at all.
So far I have this:
db.getCollection('customers').aggregate([
{
$match: {}
},
{
$lookup: {
from: 'companies',
localField: 'companyId',
foreignField: '_id',
as: 'company'
}
},
{
$project: {
companyId: '$company._id',
companyName: '$company.name',
shopSetting: '$company.rights.shop'
}
}
])
This seems to be working ok to give me all of the companies with their shop value (true or false). But what if I only want to see the companies that don't have the shop field existing at all? How would I modify this query to accomplish that? I've tried reading up on the $exists field in mongo, but this is all pretty new to me so I'm not sure where to apply it here.
Note: I need to query the companies from the customers collection because there are some companies without customers and I need this result to only be companies that are assigned to customers but don't have the rights.shop property existing
db.customers.aggregate([
{ $match: {} },
{
$lookup: {
from: "companies",
localField: "companyId",
foreignField: "_id",
as: "company",
pipeline: [
{
$match: {
$expr: {
$eq: [
{
$filter: {
input: "$rights",
as: "r",
cond: "$$r.shop"
}
},
[]
]
}
}
}
]
}
},
{
$project: {
companyId: "$company._id",
companyName: "$company.name",
shopSetting: "$company.rights.shop"
}
}
])
mongoplayground

Best way to retrieve a reference field using $lookup in Mongoose?

I have two models Book and Author, Book has reference of Author, suppose an Author got deleted then I only want to retrieve those Books who have an author:
BookSchema with these fields
name: String,
author: {
type: Schema.Types.ObjectId,
ref: 'Author',
required: [true, 'A book must have an author']
}
AuthorSchema with these fields
name: String
I have to do it using $lookup operator. I am able to get the desired result but I don't know if it's a good way. This is my solution:
const books = await Book.aggregate([
{ $lookup: {
from: 'authors',
localField: 'author',
foreignField: '_id',
as: 'bookAuthor'
}
},
{ $match: { bookAuthor: { $not: { $size: 0 } } } },
{ $unwind: '$bookAuthor' },
{ $project: {
name: 1,
bookAuthor: { name: 1 }
}
}
]);
You have done everything correct, but you don't need to use the $match in second phase of aggregation pipeline. $unwind will automatically remove the documents with empty bookAuthor array, so if there is no author, it will be removed after $unwind stage.
Try this:
const books = await Book.aggregate([
{ $lookup: {
from: 'authors',
localField: 'author',
foreignField: '_id',
as: 'bookAuthor'
}
},
{ $unwind: '$bookAuthor' },
{ $project: {
name: 1,
bookAuthor: { name: 1 }
}
}
]);
Have a look at this Mongo Playground to see it working

Mongodb lookup with array

I have two collections first one is
user_profile collection
const userProfileSchema = mongoose.Schema({
phone_number: {
type: String,
required: false,
},
primary_skills: [
{
skill_id: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Skill'
},
years: Number,
}
]
});
sample data
{
"phone_number":"222",
"primary_skills":[{skill_id:1,years:12},{skill_id:2,years:13}]
}
in the primary_skills the key skill_id is mapped with another collection named skills
skills collection
const skillSchema = mongoose.Schema({
name: {
type: String,
required: true,
unique:true,
},
});
sample data
[
{
id:1,
name:'php'
},
{
id:2,
name:'java'
}
]
I want to fetch all values in the user_profile collection along with the respective skills name
expected output:
{
"phone_number":"222",
"primary_skills":[{
name:"php",skill_id:1,years:12
},{
name:"java",skill_id:2,years:13}
]
}
I found a similar thread to my question MongoDB lookup when foreign field is an array of objects but it's doing the opposite of what I want
This is the query I tried
profile.aggregate([{
$lookup:{
from:'skills',
localField:'primary_skills.skill_id',
foreignField:'_id',
'as':'primary_skills'
}
}])
This works fine but it didn't contain the years key
You need to do it with $unwind and $group,
$unwind primary_skills because its an array and we need to lookup sub document wise
db.user_profile.aggregate([
{
$unwind: "$primary_skills"
},
$lookup to join primary_skills, that you have already did
{
$lookup: {
from: "skills",
localField: "primary_skills.skill_id",
foreignField: "id",
as: "primary_skills.name"
}
},
$unwind primary_skills.name that we have stored join result, its array and we are unwinding to do object
{
$unwind: {
path: "$primary_skills.name"
}
},
$addFields replace field name that we have object and we need only name
{
$addFields: {
"primary_skills.name": "$primary_skills.name.name"
}
},
$group by _id because we have unwind and we need to combine all documents
{
$group: {
_id: "$_id",
phone_number: {
$first: "$phone_number"
},
primary_skills: {
$push: "$primary_skills"
}
}
}
])
Playground: https://mongoplayground.net/p/bDmrOwmASn5

How to lookup a field in an array of subdocuments in mongoose?

I have an array of review objects like this :
"reviews": {
"author": "5e9167c5303a530023bcae42",
"rate": 5,
"spoiler": false,
"content": "This is a comment This is a comment This is a comment.",
"createdAt": "2020-04-12T16:08:34.966Z",
"updatedAt": "2020-04-12T16:08:34.966Z"
},
What I want to achieve is to lookup the author field and get the user data, but the problem is that the lookup I am trying to use only returns this to me:
Code :
.lookup({
from: 'users',
localField: 'reviews.author',
foreignField: '_id',
as: 'reviews.author',
})
Response :
Any way to get the author's data in that field? That's where the author's Id is.
Try to execute below query on your database :
db.reviews.aggregate([
/** unwind in general is not needed for `$lookup` for if you wanted to match lookup result with specific elem in array is needed */
{
$unwind: { path: "$reviews", preserveNullAndEmptyArrays: true },
},
{
$lookup: {
from: "users",
localField: "reviews.author",
foreignField: "_id",
as: "author", // Pull lookup result into 'author' field
},
},
/** Update 'reviews.author' field in 'reviews' object by checking if 'author' field got a match from 'users' collection.
* If Yes - As lookup returns an array get first elem & assign(As there will be only one element returned -uniques),
* If No - keep 'reviews.author' as is */
{
$addFields: {
"reviews.author": {
$cond: [
{ $ne: ["$author", []] },
{ $arrayElemAt: ["$author", 0] },
"$reviews.author",
],
},
},
},
/** Group back the documents based on '_id' field & push back all individual 'reviews' objects to 'reviews' array */
{
$group: {
_id: "$_id",
reviews: { $push: "$reviews" },
},
},
]);
Test : MongoDB-Playground
Note : Just in case if you've other fields in document along with reviews that needs to be preserved in output then starting at $group use these stages :
{
$group: {
_id: "$_id",
data: {
$first: "$$ROOT"
},
reviews: {
$push: "$reviews"
}
}
},
{
$addFields: {
"data.reviews": "$reviews"
}
},
{
$project: {
"data.author": 0
}
},
{
$replaceRoot: {
newRoot: "$data"
}
}
Test : MongoDB-Playground
Note : Try to keep queries to run on lesser datasets maybe by adding $match as first stage to filter documents & also have proper indexes.
you should use populate('author') method of mongoose on the request to the server which gets the id of that author and adds the user data to the response of mongoose
and dont forget to set your schema in a way that these two collections are connected
in your review schema you should add ref to the schema which the author user is saved
author: { type: Schema.Types.ObjectId, ref: 'users' },
You can follow this code
$lookup:{
from:'users',
localField:'reviews.author',
foreignField:'_id',
as:'reviews.author'
}
**OR**
> When You find the doc then use populate
> reviews.find().populate("author")

MongoDB $lookup with nested object with nested array

I have 2 collections.
cases
_id: ObjectId.
name: string.
info: {
[here can be many different fields with diff types]
relatedEntities: [
{ role: string;
id: ObjectId;
} <--- here can be a lot of entities
]
}
entities
_id: ObjectId.
type: string,
name: string,
info: {
[here can be many different fields with diff types]
}
I need to retrieve all cases and for each case.info.entities object I need to have field data which will equal to entity document ( case.info.entities.id === entity_id)
Example what I need to have
_id: ObjectId.
name: string.
info: {
[here can be many different fields with diff types]
entities: [
{ role: string;
id: ObjectId;
data: {
_id: ObjectId.
type: string,
name: string,
info: {
[here can be many different fields with diff types]
}
}
} <--- here can be a lot of entities
]
}
How to do it in a proper way?
At the moment I implemented this is that way:
{ $unwind: "$info.relatedEntities" },
{ $lookup: {
"from": "entities",
"localField": "info.relatedEntities.entity",
"foreignField": "_id",
"as": "info.relatedEntities.entityObject"
}},
{ $group: {
"_id": "$_id",
"templateType":{$first: "$templateType"},
"info":{$first: "$info"},
"relatedEntities": {
$push: "$info.relatedEntities"
}
}}
It's working, but required additional parsing when data is retrieved, but I'd like to do it without workarounds..
You have done almost everything. Based on your query and model I have given below query (Field names might be different). Hope it helps.
db.cases.aggregate([
{ $unwind: '$info.relatedEntities' },
{ $lookup: {
from: 'entities',
localField: 'info.relatedEntities.entity',
foreignField: '_id',
as: 'info.relatedEntities.entityObject'
}
},
{ $group: {
_id: {
_id : '$_id',
templateType : '$templateType',
name : '$name',
info : {
address : "$info.address",
}
},
'relatedEntities': {
$push: {
role : '$info.relatedEntities.role',
entity : '$info.relatedEntities.entity',
data : { $arrayElemAt: [ '$info.relatedEntities.entityObject', 0 ] }
}
}
}
},
{
$project : {
_id : '$_id._id',
name : '$_id.name',
templateType : '$_id.templateType',
info : {
address : '$_id.info.address',
entities : '$relatedEntities'
}
}
}
]).pretty()