I'm trying to populate my user field with $lookup and here is my output:
[
{
...
user: [],
...
}
]
As you can see, The output is an array and user field is an empty array as well, also I want to make a query, that compare multiple fields. Lets say I have this object:
{
social: {
facebook: 'facebook.com/asd',
twitter: 'https://twitter.com/'
},
skills: [ 'asd', 'asd', 'html' ],
_id: 5eec9c9a2d6d1d46e8b05fb4,
user: {
_id: 5eec9c752d6d1d46e8b05fb3,
name: 'Simeon Lazarov',
avatar: 'uploads\\1592682240020 - background.jpg'
},
status: 'Junior Developer',
experience: [
{
current: true,
_id: 5eef06638af42217d8c5480a,
title: 'ASD',
company: 'SAD',
location: 'ASD',
from: 2020-06-08T00:00:00.000Z,
to: null
}
],
education: [
{
current: true,
_id: 5eef5f0dd06ee324a4a7b1d2,
school: 'asd',
degree: 'asd',
fieldofstudy: 'asd',
from: 2020-06-24T00:00:00.000Z,
to: null
}],
contacts: [ { _id: 5eee23bd862ab70ba0235a85, user: 5eedffa6062e6231a859433a } ],
date: 2020-06-19T11:08:10.883Z,
__v: 1,
website: 'asd',
company: 'asd'
}
And want to compare if company, skills(all skills of this object to be contained in others objects) or etc. are equal.
Here is my NOT working code:
await Profile.aggregate([{$lookup: {from:"user",localField: "user",
foreignField: "name", as: "user"}},{$match :{$or:[{"company":{$regex:profile.company}},{"skills":{ $eq: profile.skills} }]}}]).skip((req.params.page - 1) * 10).limit(11);
P.S Sorry for writing so bad, it's my first question here.
It seems you just have mismatches in your $lookup fields compare to what the schema holds, I rewrote it like this:
await Profile.aggregate([
{
$match: {
$or: [
{"company": {$regex: profile.company}},
{"skills": {$all: profile.skills}}
]
}
},
{
$skip: req.params.page ? req.params.page - 1 : 0 // make sure this can't be -1
},
{
$limit: 11
},
{
$lookup: {
from: "user",
localField: "user",
foreignField: "_id",
as: "user"
}
},
{
"$unwind": "$user"
}
])
Notice I moved the $lookup to be the last stage as it's the most expensive part of the pipeline and there's no need to do it on the entire collection.
Related
I have these 2 collections in mongodb:
the first named packages:
{
_id: "mongo id",
package_name : "some name",
color_tag : "some color",
description: "some description"
}
the second one name subscriptions:
{
_id: "mongo id"
package:"relates with package _id from collection above",
end_time:'time when subscription ends
}
I need to write a query where I can return something like this:
{
[month: 1{
{
package_name: 'A',
total_active: 3
},
{
package_name: 'B',
total_active: 4
},
[month: 2{
{
package_name: 'C',
total_active: 0
},
{
package_name: 'B',
total_active: 4
},
}
]
}
what I have done till now is this:
{
$match: {
$expr: {
{$gt: ["$end_time", new Date()]},
},
},
},
{
$lookup: {
from: 'packages',
localField: 'package',
foreignField: '_id',
as: 'package',
},
},
With this part I can get all active packages that have active subscriptions, what should I do next?
Thanks a lot for your help.
i have User document and and Category document which is connected to shop document which look like
var ShopSchema = mongoose.Schema(
{
shopName: {
type: String,
required: [true, 'A Shop must have name'],
},
phone: {
type: String,
validate: [validator.isMobilePhone,"Phone number is not a valid number"],
required: [true, 'A Shop must have hepline number'],
},
shopStatus: {
type: Boolean,
default: false,
select: false
},
managerId:{
type: mongoose.Schema.ObjectId,
ref:'User',
require: [true,'Shop must have to a manager!']
},
createdAt:{
type:Date,
default: Date.now()
},
shopCategory:{
type: mongoose.Schema.ObjectId,
ref:'Category',
require: [true,'Shop must have a Category!']
},
},
{
toJSON: { virtuals: true },
toObject: { virtuals: true }
}
);
const Shop = mongoose.model("Shop", ShopSchema);
Now i am writing a query to get managers shop(groupBy managerId) with populate of Manager and Category data. but i am getting empty array everytime.
const doc = await Shop.aggregate([
{
$match: { shopStatus: {$ne: false} }
},
{
$group: {
_id:'$managerId',
numofShop: {$sum: 1},
shopCategory: {$first: "$shopCategory"}
}
},
{
$lookup: {
from: "User",
localField : "_id",
foreignField: "_id",
as: "Manager"
}
},
{
$lookup: {
from: "Category",
localField : "shopCategory",
foreignField: "_id",
as: "Category"
}
},
])
here is how my final result look like but this is an empty array.
{
"_id": "5f467660f630e804ec07fad8",
"numofShop": 2,
"Manager": [],
"Category": []
},
{
"_id": "5f44d2f4ff04993b40684bf9",
"numofShop": 1,
"Manager": [],
"Category": []
}
I want to find All shops groupBy managerId(Who own it). there is field of managerId referencing to User document who created this shop. i want data like this
{
"_id": "5f44d2f4ff04993b40684bf9",
"Manager": {},//this is populated through User Schema
"numofShop": 1,
"Shop": [
{
"shopName":"Grocery Shop",
"phone" : "111111",
"shopCategory" :{"_id": "","name":"Electronics"}
}
]
}
.....
.....
Find Example Data here
Shops- http://txt.do/1fwi0
3 shops created by two users
User - http://txt.do/1fwio
Categories- http://txt.do/1fwij
There are 2 fixes,
in second lookup you have no shopCategory field in localField because you have not defined shopCategory in $group,
{
$group: {
_id:'$managerId',
numofShop: {$sum: 1},
shopCategory: { $first: "$shopCategory" } // define here
}
},
in first lookup with user collection, there is no managerId field and you have assigned in _id field, then you can use _id in localField,
{
$lookup: {
from: "User",
localField : "_id", // change this from managerId
foreignField: "_id",
as: "Manager"
}
},
Updated things after updated question, Playground1, Playground2
for your second edit, you can't put condition for populate in find method, how to put condition in populate, you can follow the answer1 and answer2
I'm trying to figure out how to do a lookup on an array of objects by a specific field, while retaining other fields that are at the same level of the looked up field.
My processes collection is as follows:
{
_id: 'p1',
steps: [
{
_id: 'ps1',
step: 's1',
time: 10
},
{
_id: 'ps2',
step: 's2',
time: 15
}
]
}
The steps collection looks like (for the document with _id: s1):
{
_id: 's1',
name: 'step 1'
}
My aggregation:
processes.aggregate([
{
// match stage or whatever prior stage
},
{
$lookup: {
from: 'steps',
localField: 'steps.step',
foreignField: '_id',
as: 'steps'
}
}
])
This works, but basically erases the steps._id and steps.time fields that were at the same level of the looked up field (steps.step), as seen in the output:
{
_id: 'p1',
steps: [
{
_id: 's1',
name: 'step 1'
},
{
_id: 's2',
name: 'step 2'
}
]
}
While the expected output would be:
{
_id: 'p1',
steps: [
{
_id: 'ps1',
time: 10,
stepId: 's1',
name: 'step 1',
},
{
_id: 'ps2',
time: 15,
stepId: 's2',
name: 'step 2',
}
]
}
Note: the _id field in the expected output is not very important, and can contain the _id of the step document, avoiding to have a separate stepId field.
EDIT
Additional question moved here: MongoDB lookup array of objects by field (join conditions and uncorrelated sub-queries)
After a lot of research, I could find the way to accomplish what I meant in the question.
The main idea is to merge two arrays of objects together.
processes.aggregate([
{
// match stage or whatever prior stage
},
{
$lookup: {
from: 'steps',
localField: 'steps.step',
foreignField: '_id',
as: 'steps2'
},
{
$addFields: {
stages: {
$map: {
input: '$steps',
in: {
$mergeObjects: [
'$$this',
{
$arrayElemAt: [
'$steps2',
{
$indexOfArray: ['$steps2._id', '$$this.step'],
},
],
},
],
},
},
},
},
}
}
])
Have 2 following collections:
user collection
{
userId:user1,
creationTimeStamp:2019-11-05T08:15:30
status:active
},
{
userId:user2,
creationTimeStamp:2019-10-05T08:15:30
status:active
}
document collection
{
userId:user1,
category:Development
published:true
},
{
userId:user2,
category:Development
published:false
}
I want to join these two collections and filter users such that documents which are of development category and are not published from active users between creationtimestamp
How can I write a mongodb java aggregation in order to get a result like this:
{
userId: user2,
status:active,
category:Development,
published:false
}
You could run below aggregation query on the document collection to get the expected result
[{$match: {
category:'development',
published: false
}}, {$lookup: {
from: 'user',
localField: 'userId',
foreignField: 'userId',
as: 'JoinedTable'
}}, {$unwind: {
path: '$JoinedTable'
}}, {$group: {
_id: '$_id',
userId: {
$first: '$userId'
},
status: {
$first: '$JoinedTable.status'
},
category: {
$first: '$category'
},
published: {
$first: '$published'
},
}}]
Explanation:
1. filter documents using match for criteria category: 'development' & published: false
2. join document collection with user collection with key userId
3. unwind the joined collection field to convert array to object
4. project the fields needed using groups.
Hope this helps!
You haven't mentioned about the duplicate of userId in User collection.
So the script is
[{
$match: {
category: "Development",
published: false
}
}, {
$lookup: {
from: 'user',
localField: 'userId',
foreignField: 'userId',
as: 'joinUser'
}
}, {
$unwind: {
path: "$joinUser",
preserveNullAndEmptyArrays: true
}
}, {
$match: {
"joinUser.status": "active"
}
}, {
$addFields: {
"status": "$joinUser.status"
}
}, {
$project: {
_id: 0,
userId: 1,
category: 1,
published: 1,
status: 1
}
}]
And the java code,
include these imports
import static org.springframework.data.mongodb.core.aggregation.Aggregation.match;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.lookup;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.unwind;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.project;
method is,
public Object findAllwithVideos() {
Aggregation aggregation=Aggregation.newAggregation(
match(Criteria.where("category").is("Development").and("published").is(false)),
lookup("user","userId","userId","joinUser"),
unwind("joinUser",true),
new AggregationOperation(){
#Override
public Document toDocument(AggregationOperationContext aggregationOperationContext){
return new Document("$addFields",
new Document("status","$joinUser.status")
);
}
},
project("userId","category","published","status")
).withOptions(AggregationOptions.builder().allowDiskUse(Boolean.TRUE).build());
return mongoTemplate.aggregate(aggregation, mongoTemplate.getCollectionName(Document.class), Object.class);
}
I have an articles collection that contains an array of comments, and this array contains an array of sub_comments. Here is what it looks like:
let articleSchema = mongoose.Schema({
title: {type: String},
comments: [{
comment: {type: String},
creator: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
sub_comments: [{
comment: {type: String},
creator: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}
}]
}]
});
I'm trying to run an aggregate query that does a $lookup on the comments.creator and sub_comments.creator fields. Here's what I've tried so far, but it doesn't work:
this.model('Article')
.aggregate([
{
$match: {_id: article_id}
},
{
$lookup: {
from: "users",
localField: "comments.creator",
foreignField: "_id",
as: "comments.creator"
}
},
{
$unwind: {
path: "$comments.creator",
preserveNullAndEmptyArrays: true
}
},
{
$lookup: {
from: "users",
localField: "comments.sub_comments.creator",
foreignField: "_id",
as: "comments.sub_comments.creator"
}
},
{
$unwind: {
path: "$comments.sub_comments.creator",
preserveNullAndEmptyArrays: true
}
},
{
$project: {
_id: 1,
title: 1,
"comments.comment": 1,
"comments.creator._id": 1,
"comments.creator.name": 1,
"comments.sub_comments.comment": 1,
"comments.sub_comments.creator._id": 1,
"comments.sub_comments.creator.name": 1
}
}
])
.exec(function (err, data) {
...
});
Here's an example of the data response:
[{
"_id": "5b7e1629d00a170018e11234",
"article": "My Article",
"comments": {
"comment": "This is the comment.",
"sub_comments": {},
"creator": {
"_id": "541g2dfeab1e470b00411234",
"name": "John Doe"
}
},
....
}]
Comments should be an array, not an object. This particular comment didn't have any sub-comments, but obviously, I want that to work too. Any ideas on how to get this to work?
the as params in mongodb $lookup with your query affect your result .
as Specifies the name of the new array field to add to the input
documents. The new array field contains the matching documents from
the from collection. If the specified name already exists in the input
document, the existing field is overwritten.
it will overwrite the filed . not add to it .
if your outter is an array ,it will make it to Object and delete the current content .
so the second $lookup in aggregate will not work . in your query
if you want to keep the current data structure intac . you can as a new fieldname and use $project to change the format .