MongoDB $lookup with nested object with nested array - mongodb

I have 2 collections.
cases
_id: ObjectId.
name: string.
info: {
[here can be many different fields with diff types]
relatedEntities: [
{ role: string;
id: ObjectId;
} <--- here can be a lot of entities
]
}
entities
_id: ObjectId.
type: string,
name: string,
info: {
[here can be many different fields with diff types]
}
I need to retrieve all cases and for each case.info.entities object I need to have field data which will equal to entity document ( case.info.entities.id === entity_id)
Example what I need to have
_id: ObjectId.
name: string.
info: {
[here can be many different fields with diff types]
entities: [
{ role: string;
id: ObjectId;
data: {
_id: ObjectId.
type: string,
name: string,
info: {
[here can be many different fields with diff types]
}
}
} <--- here can be a lot of entities
]
}
How to do it in a proper way?
At the moment I implemented this is that way:
{ $unwind: "$info.relatedEntities" },
{ $lookup: {
"from": "entities",
"localField": "info.relatedEntities.entity",
"foreignField": "_id",
"as": "info.relatedEntities.entityObject"
}},
{ $group: {
"_id": "$_id",
"templateType":{$first: "$templateType"},
"info":{$first: "$info"},
"relatedEntities": {
$push: "$info.relatedEntities"
}
}}
It's working, but required additional parsing when data is retrieved, but I'd like to do it without workarounds..

You have done almost everything. Based on your query and model I have given below query (Field names might be different). Hope it helps.
db.cases.aggregate([
{ $unwind: '$info.relatedEntities' },
{ $lookup: {
from: 'entities',
localField: 'info.relatedEntities.entity',
foreignField: '_id',
as: 'info.relatedEntities.entityObject'
}
},
{ $group: {
_id: {
_id : '$_id',
templateType : '$templateType',
name : '$name',
info : {
address : "$info.address",
}
},
'relatedEntities': {
$push: {
role : '$info.relatedEntities.role',
entity : '$info.relatedEntities.entity',
data : { $arrayElemAt: [ '$info.relatedEntities.entityObject', 0 ] }
}
}
}
},
{
$project : {
_id : '$_id._id',
name : '$_id.name',
templateType : '$_id.templateType',
info : {
address : '$_id.info.address',
entities : '$relatedEntities'
}
}
}
]).pretty()

Related

Applying $exists to MongoDB aggregation

I have two mongo collections structured like so:
customers
{
_id: ObjectId,
name: String,
companyId: ObjectId
}
companies
{
_id: ObjectId,
name: String,
rights: [
add: boolean,
edit: boolean,
shop: boolean
]
}
So each customer has a companyId that lets us look up the companies.rights available. I need to get a list of which companies have customers but don't have the shop property at all.
So far I have this:
db.getCollection('customers').aggregate([
{
$match: {}
},
{
$lookup: {
from: 'companies',
localField: 'companyId',
foreignField: '_id',
as: 'company'
}
},
{
$project: {
companyId: '$company._id',
companyName: '$company.name',
shopSetting: '$company.rights.shop'
}
}
])
This seems to be working ok to give me all of the companies with their shop value (true or false). But what if I only want to see the companies that don't have the shop field existing at all? How would I modify this query to accomplish that? I've tried reading up on the $exists field in mongo, but this is all pretty new to me so I'm not sure where to apply it here.
Note: I need to query the companies from the customers collection because there are some companies without customers and I need this result to only be companies that are assigned to customers but don't have the rights.shop property existing
db.customers.aggregate([
{ $match: {} },
{
$lookup: {
from: "companies",
localField: "companyId",
foreignField: "_id",
as: "company",
pipeline: [
{
$match: {
$expr: {
$eq: [
{
$filter: {
input: "$rights",
as: "r",
cond: "$$r.shop"
}
},
[]
]
}
}
}
]
}
},
{
$project: {
companyId: "$company._id",
companyName: "$company.name",
shopSetting: "$company.rights.shop"
}
}
])
mongoplayground

Mongodb lookup with array

I have two collections first one is
user_profile collection
const userProfileSchema = mongoose.Schema({
phone_number: {
type: String,
required: false,
},
primary_skills: [
{
skill_id: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Skill'
},
years: Number,
}
]
});
sample data
{
"phone_number":"222",
"primary_skills":[{skill_id:1,years:12},{skill_id:2,years:13}]
}
in the primary_skills the key skill_id is mapped with another collection named skills
skills collection
const skillSchema = mongoose.Schema({
name: {
type: String,
required: true,
unique:true,
},
});
sample data
[
{
id:1,
name:'php'
},
{
id:2,
name:'java'
}
]
I want to fetch all values in the user_profile collection along with the respective skills name
expected output:
{
"phone_number":"222",
"primary_skills":[{
name:"php",skill_id:1,years:12
},{
name:"java",skill_id:2,years:13}
]
}
I found a similar thread to my question MongoDB lookup when foreign field is an array of objects but it's doing the opposite of what I want
This is the query I tried
profile.aggregate([{
$lookup:{
from:'skills',
localField:'primary_skills.skill_id',
foreignField:'_id',
'as':'primary_skills'
}
}])
This works fine but it didn't contain the years key
You need to do it with $unwind and $group,
$unwind primary_skills because its an array and we need to lookup sub document wise
db.user_profile.aggregate([
{
$unwind: "$primary_skills"
},
$lookup to join primary_skills, that you have already did
{
$lookup: {
from: "skills",
localField: "primary_skills.skill_id",
foreignField: "id",
as: "primary_skills.name"
}
},
$unwind primary_skills.name that we have stored join result, its array and we are unwinding to do object
{
$unwind: {
path: "$primary_skills.name"
}
},
$addFields replace field name that we have object and we need only name
{
$addFields: {
"primary_skills.name": "$primary_skills.name.name"
}
},
$group by _id because we have unwind and we need to combine all documents
{
$group: {
_id: "$_id",
phone_number: {
$first: "$phone_number"
},
primary_skills: {
$push: "$primary_skills"
}
}
}
])
Playground: https://mongoplayground.net/p/bDmrOwmASn5

MongoDB Exclude docs from aggregate if field/reference exists in other collection

const sellerSchema = Schema(
{
name: String,
url:String
}
const productSchema = Schema(
{
title: String,
sellerUrl:String
}
Below query will return unique sellerUrl from all products:
context.Product.aggregate([
{
$group: {
_id: "$sellerUrl",
}
}
]);
But I also want to exclude from aggregation, sellers that I already saved. So if url == sellerUrl aggregation must exclude that seller.
Please help me
You can try below query :
db.product.aggregate([
{
$group: {
_id: "", /** group on no condition & push all unique `sellerUrl` to sellerUrls array */
sellerUrls: { $addToSet: "$sellerUrl" }
}
},
{
$lookup: {
from: "seller",
let: { sellerUrls: "$sellerUrls" }, // creating local variable
pipeline: [
{ $group: { _id: "", urls: { $addToSet: "$url" } } }, /** group on no condition & push all unique `url` to urls array */
{ $project: { _id: 0, uniqueAndNotInSellerColl: { $setDifference: [ "$$sellerUrls", "$urls" ] } } } // get difference between two arrays
],
as: "data" // As we're grouping will always be one doc/element in an array
}
},
/** Create a new root doc from getting first element(though it will have only one) from `data` array */
{
$replaceRoot: { newRoot: { $arrayElemAt: [ "$data", 0 ] } }
}
])
Test : mongoplayground
Update :
As you need few other fields from product collection but not just the sellerUrl field then try below query :
db.product.aggregate([
{
$group: {
_id: "$sellerUrl",
docs: { $push: { title: "$title" } } // We're only retrieving `title` field from `product` docs, if every field is needed use `$$ROOT`
}
},
/** We've used basic `lookup` stage, use this if you've only few matching docs from `seller` collection
* If you've a lot of matching docs for each `_id` (sellerUrl),
* then instead of getting entire `seller` doc (which is not needed) use `lookup` with aggregation pipeline &
* just get `_id`'s of seller docs for better performace refer previous query
*/
{
$lookup: {
from: "seller",
localField: "_id",
foreignField: "url",
as: "sellerDocs"
}
},
/** match will retain only docs which doesn't have a matching doc in seller collection */
{
$match: { sellerDocs: [] }
},
{
$project: { sellerDocs: 0 }
}
])
Test : mongoplayground

$lookup using multiple criteria mongodb java aggregation

Have 2 following collections:
user collection
{
userId:user1,
creationTimeStamp:2019-11-05T08:15:30
status:active
},
{
userId:user2,
creationTimeStamp:2019-10-05T08:15:30
status:active
}
document collection
{
userId:user1,
category:Development
published:true
},
{
userId:user2,
category:Development
published:false
}
I want to join these two collections and filter users such that documents which are of development category and are not published from active users between creationtimestamp
How can I write a mongodb java aggregation in order to get a result like this:
{
userId: user2,
status:active,
category:Development,
published:false
}
You could run below aggregation query on the document collection to get the expected result
[{$match: {
category:'development',
published: false
}}, {$lookup: {
from: 'user',
localField: 'userId',
foreignField: 'userId',
as: 'JoinedTable'
}}, {$unwind: {
path: '$JoinedTable'
}}, {$group: {
_id: '$_id',
userId: {
$first: '$userId'
},
status: {
$first: '$JoinedTable.status'
},
category: {
$first: '$category'
},
published: {
$first: '$published'
},
}}]
Explanation:
1. filter documents using match for criteria category: 'development' & published: false
2. join document collection with user collection with key userId
3. unwind the joined collection field to convert array to object
4. project the fields needed using groups.
Hope this helps!
You haven't mentioned about the duplicate of userId in User collection.
So the script is
[{
$match: {
category: "Development",
published: false
}
}, {
$lookup: {
from: 'user',
localField: 'userId',
foreignField: 'userId',
as: 'joinUser'
}
}, {
$unwind: {
path: "$joinUser",
preserveNullAndEmptyArrays: true
}
}, {
$match: {
"joinUser.status": "active"
}
}, {
$addFields: {
"status": "$joinUser.status"
}
}, {
$project: {
_id: 0,
userId: 1,
category: 1,
published: 1,
status: 1
}
}]
And the java code,
include these imports
import static org.springframework.data.mongodb.core.aggregation.Aggregation.match;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.lookup;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.unwind;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.project;
method is,
public Object findAllwithVideos() {
Aggregation aggregation=Aggregation.newAggregation(
match(Criteria.where("category").is("Development").and("published").is(false)),
lookup("user","userId","userId","joinUser"),
unwind("joinUser",true),
new AggregationOperation(){
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext){
return new Document("$addFields",
new Document("status","$joinUser.status")
);
}
},
project("userId","category","published","status")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Document.class), Object.class);
}

Mongodb 3.2 aggregation $lookup for nested document

I have two collections: memberships and categories.
In the categories I have stored array of objects called ageGroups.
memberships : [{
_id: ObjectId,
ageCategory: ObjectId,
ageGroup: ObjectId
}]
categories: {
_id: ObjectId,
ageGroups: [{
_id: ObjectId,
name: String
}, {
_id: ObjectId,
name: String
}, {
_id: ObjectId,
name: String
}]
}
There are ageCategory and ageGroup properties in memberships collection.
Now i need to aggregate these two and get result like this
[{
id: 'membership id here',
ageGroup: 'Age Group name here'
},{
id: '2 membership id here',
ageGroup: '2 membership age Group name here'
}]
So I tried
db.memberships.aggregate([
{
$lookup: {
from: 'categories', localField: 'ageCategory', foreignField: '_id', as: 'ageCategory'
}
},
{
$unwind: '$ageCategory'
},
{
$project: {
'group': {
$filter: {
input : '$ageCategory.ageGroups',
as : 'group',
cond : 'group._id' == '$_id'
}
}
}
},
{
$unwind: '$group'
},
{
$project: {
'group' : '$group.name'
}
}
]);
This query return empty result. As much as I understand, problem in filter condition
$filter: {
input : '$ageCategory.ageGroups',
as : 'group',
cond : 'group._id' == '$_id'
}
when I tried to set true in cond, it pushed new document to result, for each group, even if ageGroup not that, which is assigned to membership. Thanks for help!
UPD
I found solution,
{
$project: {
'group': {
$filter: {
input : '$ageCategory.ageGroups',
as : 'group',
cond : { $eq:['$$group._id', '$ageGroup']}
}
}
}
},
I have added double dollar sign with $eq operator and problem has been solved. Now I search better solution and I would like have a query which get similar result
{
"_id" : // member id,
"group" : "1-3 members"
}
{
"_id" : //member id,
"group" : "17-19 members"
}
without storing ageCategory in memberships table, only with ageGroup id.