I have 2 collections inside my MongoDB:
Order:
{
"date" : ISODate("2020-07-30T22:00:00.000Z"),
"item" : "banana",
"price": "4$",
"order_type" : "INTERNATIONAL", // There are 2 order types: INTERNATIONAL and MAINLAND
"user" : { // I use a dbref to the User collection
"$ref" : "user",
"$id" : "user_0"
}
}
User:
{
"_id": "user_0"
"login" : "user1",
"password" : "$2a$10$mE.qmcV0mFU5NcKh73TZx.z4ueI/.bDWbj0T1BYyqP481kGGarKLG",
"first_name" : "Henry",
"last_name" : "Windsor",
"email" : "hw#gmail.com",
}
Each order contains a DB reference to the corresponding user who made it. This is my mongo code to calculate the total number of orders that each user makes.
db.getCollection('order').aggregate([
{$group: {
_id: '$user',
totalNbOfOrders: {$sum: 1}
}},
{$addFields: {foreign_key: {$objectToArray: "$_id"}}},
{$lookup: {
from: 'user',
localField: 'foreign_key.1.v',
foreignField: '_id',
as: 'userInfo'
}
},
{ $unwind: '$userInfo'},
{ $project: {
'_id': 0,
'first_name': '$userInfo.first_name',
'last_name': '$userInfo.last_name',
'totalNbOfOrders': '$totalNbOfOrders'
}
}
])
And the result is:
/* 1 */
{
"first_name" : "John",
"last_name" : "Kennedy",
"totalNbOfOrders" : 2.0
}
/* 2 */
{
"first_name" : "Peter",
"last_name" : "Parker",
"totalNbOfOrders" : 4.0
}
/* 3 */
{
"first_name" : "Bruce",
"last_name" : "Banner",
"totalNbOfOrders" : 2.0
}
Now, what I also want to calculate is the number of international orders (and eventually of mainland orders) that each user made to have something like this:
{
"first_name" : "Tony",
"last_name" : "Stark",
"totalNbOfOrders" : 10.0,
"totalNbOfInternationalOrders": 4.0
"totalNbOfMainlandOrders": 6.0
}
I haven't figured out how to write the code.
I tried to use "$accumulator" operator (new feature in version 4.4 of MongoDB) inside "$group" but I used MongoDB 4.2.7, I have to use operators from older versions to accomplish this. Does anybody know how to solve this problem?
You can do it inside $group, using $cond and $eq,
{
$group: {
... // skipped
// INTERNATIONAL COUNT
totalNbOfInternationalOrders: {
$sum: {
$cond: {
if: {
$eq: ["$order_type", "INTERNATIONAL"]
},
then: 1,
else: 0
}
}
},
// MAINLAND COUNT
totalNbOfMainlandOrders: {
$sum: {
$cond: {
if: {
$eq: ["$order_type", "MAINLAND"]
},
then: 1,
else: 0
}
}
}
}
},
set in $project
{
$project: {
... // skipped
"totalNbOfInternationalOrders": "$totalNbOfInternationalOrders",
"totalNbOfMainlandOrders": "$totalNbOfMainlandOrders"
}
}
Playground: https://mongoplayground.net/p/_IeVcSFt_nY
Related
I have the following function that is supposed to be returning the _id value of duplicates based on the email key. However, no matter what I've tried, I can't get the function to return anything other than any empty object. What am I missing here? Or is there a better approach I should be considering?
var duplicates = [];
db.medicallists
.aggregate([
{
$group: {
_id: {
email: "$email"
},
duplicate_ids: { $addToSet: "$_id" },
count: { $sum: 1 }
}
},
{
$match: {
count: { $gt: 1 }
}
},
], { allowDiskUse: true })
.forEach(function(doc) {
doc.duplicate_ids.shift(); // First element skipped for deleting
doc.duplicate_ids.forEach(function(dupId) {
duplicates.push(dupId); // Getting all duplicate ids
});
});
printjson(duplicates);
EDIT:
Here is a sample document:
{
_id : 5a2fed0c8023cf7ea2346067,
primary_spec : "Addiction Medicine",
first_name : "John",
last_name : "Sample",
city : "Las Vegas",
state : "NV",
phone : "1111111111",
fax : "1111111111",
email : "sample#aol.com"
}
I have tested your specific query on a similar data set and it works fine. printjson(duplicates); prints out all the duplicate ids.
Also this will, in fact, remove all the duplicate entries based on email:
db.collection.aggregate([
{
$group: {
_id: {
email: "$email"
},
duplicate_ids: {
$push: "$_id"
},
count: {
$sum: 1
}
}
},
{
$match: {
count: {
$gt: 1
}
}
},
]).forEach(function(doc){
doc.duplicate_ids.shift();
db.collection.remove({
_id: {
$in: doc.duplicate_ids
}
});
})
My starting set was:
{
"_id" : ObjectId("6014331de1ef9ab1f708ddd9"),
"item" : "card",
"email" : "zzz#yahoo.com"
}
{
"_id" : ObjectId("6014331de1ef9ab1f708ddda"),
"item" : "card",
"email" : "eee#yahoo.com"
}
{
"_id" : ObjectId("6014331de1ef9ab1f708dddb"),
"item" : "card",
"email" : "zzz#yahoo.com"
}
{
"_id" : ObjectId("6014331de1ef9ab1f708dddc"),
"item" : "card",
"email" : "aaa#yahoo.com"
}
After running the query, it turned to:
{
"_id" : ObjectId("6014331de1ef9ab1f708ddd9"),
"item" : "card",
"email" : "zzz#yahoo.com"
}
{
"_id" : ObjectId("6014331de1ef9ab1f708ddda"),
"item" : "card",
"email" : "eee#yahoo.com"
}
{
"_id" : ObjectId("6014331de1ef9ab1f708dddc"),
"item" : "card",
"email" : "aaa#yahoo.com"
}
Tested on MongoDB server version: 4.2.7
I am trying to create an application for tracking employee attendances.
Using Front End : Angular , Server: Node JS(Express Framework),BackEnd : MongoDB
I succeed to record employee attendence in mongodb, considering employee first swipe is 'IN' and
second swipe 'OUT' for a day( also can have multiple IN & OUT in day,alternate IN & OUT)
Problem am facing is to generate reports on the basis of these attendence logs i stored in the collection.
I need to generate report for an employee work duration he/she is present or absent in a day or for a particular period.
Here is my collections samples.
employee collections
{
"_id" : ObjectId("5f120ba1578d051b20fb53e3"),
"emp_code" : 82140,
"emp_firstname" : "rahul",
"emp_secondname" : "narayanan unni",
"emp_gender" : "Male",
"dept_name" : "office",
"emp_designation" : "system administrator",
"avatar" : "src/assets/images/IMG-20180302-WA0025.jpg",
"createdAt" : ISODate("2020-07-17T20:35:45.546Z"),
"updatedAt" : ISODate("2020-07-19T20:09:43.164Z"),
"__v" : 0
}
{
"_id" : ObjectId("5f120c90578d051b20fb53e4"),
"emp_code" : 82141,
"emp_firstname" : "sanu",
"emp_secondname" : "prakashan",
"emp_gender" : "Female",
"dept_name" : "administraton",
"emp_designation" : "driver",
"avatar" : "src/assets/images/IMG-20180302-WA0014.jpg",
"createdAt" : ISODate("2020-07-17T20:39:44.933Z"),
"updatedAt" : ISODate("2020-07-19T20:33:02.610Z"),
"__v" : 0
}
attendancelogs collection
{
"_id" : ObjectId("5f6a18a2f16b782bfdd8c427"),
"punch_status" : true,
"owner" : ObjectId("5f120ba1578d051b20fb53e3"),
"chekedInTime" : ISODate("2020-09-22T15:30:42.547Z"),
"createdAt" : ISODate("2020-09-22T15:30:42.565Z"),
"updatedAt" : ISODate("2020-09-22T15:30:42.565Z"),
"__v" : 0
}
{
"_id" : ObjectId("5f6a190ef16b782bfdd8c429"),
"punch_status" : false,
"owner" : ObjectId("5f120ba1578d051b20fb53e3"),
"chekedOutTime" : ISODate("2020-09-22T15:32:30.277Z"),
"createdAt" : ISODate("2020-09-22T15:32:30.290Z"),
"updatedAt" : ISODate("2020-09-22T15:32:30.290Z"),
"__v" : 0
}
{
"_id" : ObjectId("5f6b5cb2a76ecf0f77729082"),
"punch_status" : true,
"owner" : ObjectId("5f120c90578d051b20fb53e4"),
"chekedInTime" : ISODate("2020-09-23T14:33:22.728Z"),
"createdAt" : ISODate("2020-09-23T14:33:22.741Z"),
"updatedAt" : ISODate("2020-09-23T14:33:22.741Z"),
"__v" : 0
}
{
"_id" : ObjectId("5f6b5cd8a76ecf0f77729086"),
"punch_status" : false,
"owner" : ObjectId("5f120c90578d051b20fb53e4"),
"chekedOutTime" : ISODate("2020-09-23T14:34:00.123Z"),
"createdAt" : ISODate("2020-09-23T14:34:00.132Z"),
"updatedAt" : ISODate("2020-09-23T14:34:00.132Z"),
"__v" : 0
}
The query i did is, (incompelte)
db.employeemodels.aggregate([
{
$lookup: {
from: "attlogsmodels",
localField: "_id",
foreignField: "owner",
as: "logs",
}
},
{
$project : {
_id:1,
emp_firstname:1,
emp_code:1,
emp_secondname:1,
dept_name:1,
avatar:1,
todayLogs: { $filter:{input:'$logs',as:"log",
cond:
{ $and: [
{ $gte: [ "$$log.createdAt",ISODate("2020-09-23T00:59:59.0Z")]},
{ $lte: [ "$$log.createdAt",ISODate("2020-09-23T24:59:59.0Z")]}
]}
}
}
}
},]).pretty()
Please suggest if any better way to collect data for effient report making if am wrong this approch
I tried different aggregation stages to obtain the result,but i failed
The query i did is (incomplete)
db.employeemodels.aggregate([
{
$lookup: {
from: "attlogsmodels",
localField: "_id",
foreignField: "owner",
as: "logs",
}
},
{
$project : {
_id:1,
emp_firstname:1,
emp_code:1,
emp_secondname:1,
dept_name:1,
avatar:1,
todayLogs: { $filter:{input:'$logs',as:"log",
cond:
{ $and: [
{ $gte: [ "$$log.createdAt",ISODate("2020-09-23T00:59:59.0Z")]},
{ $lte: [ "$$log.createdAt",ISODate("2020-09-23T24:59:59.0Z")]}
]}
}
}
}
},
]).pretty()
Required
Report 1 : EmpDayStatus
List all employees and their status in current day
Emp_Name | department | InTime | OutTime | Duration
Report 2 : Employee Report for a particular period and/or day
Emp_Name , Emp_Code, Department
How many Absences & Presents
Each Day Total Work Duration
You really need to change the structure of attendancelogs collection, because this will not produce accurate and exact report as per in and out logs, and also this is lengthy and heavy to query and processing,
Report 1: EmpDayStatus
$match to get current day logs of createdAt
$sort by createdAt ascending order
$group by user / owner id, store punch in records in pin and store punch out records in pout
db.attendancelogs.aggregate([
{
$match: {
createdAt: {
$gte: ISODate("2020-09-22T00:00:00.000Z"),
$lte: ISODate("2020-09-22T23:59:59.999Z")
}
}
},
{ $sort: { createdAt: 1 } },
{
$group: {
_id: "$owner",
pin: {
$push: { $cond: [{ $eq: ["$punch_status", true] }, "$$ROOT", "$$REMOVE"] }
},
pout: {
$push: { $cond: [{ $eq: ["$punch_status", false] }, "$$ROOT", "$$REMOVE"] }
}
}
},
$project to merge both arrays (pin, pout) element in one document object, using $mergeObjects, $map to iterate loop through 0 to size of pin elements using $range
{
$project: {
logs: {
$map: {
input: { $range: [0, { $size: "$pin" }] },
in: {
$mergeObjects: [
{ $arrayElemAt: ["$pin", "$$this"] },
{ $arrayElemAt: ["$pout", "$$this"] }
]
}
}
}
}
},
$lookup join employee collection with localfield _id and foreignfield _id
$unwind deconstruct user array
{
$lookup: {
from: "employee",
localField: "_id",
foreignField: "_id",
as: "user"
}
},
{ $unwind: "$user" },
$project to show employee name, logs to get required fields and duration of in and out date in minutes
{
$project: {
emp_firstname: "$user.emp_firstname",
logs: {
$map: {
input: "$logs",
in: {
chekedInTime: "$$this.chekedInTime",
chekedOutTime: "$$this.chekedOutTime",
duration: {
$divide: [
{
$subtract: [
"$$this.chekedOutTime",
"$$this.chekedInTime"
]
},
60000 // 1000*60 in minutes
]
}
}
}
}
}
}
])
Playground
Second report is totally lengthy it will take time, you can do it after taking reference of first report query.
I have two collections events & members :
events Schema :
{
name : String,
members: [{status : Number, memberId : {type: Schema.Types.ObjectId, ref: 'members'}]
}
events Sample Doc :
"_id" : ObjectId("5e8b0bac041a913bc608d69d")
"members" : [
{
"status" : 4,
"_id" : ObjectId("5e8b0bac041a913bc608d69e"),
"memberId" : ObjectId("5e7dbf5b257e6b18a62f2da9"),
"date" : ISODate("2020-04-06T10:59:56.997Z")
},
{
"status" : 1,
"_id" : ObjectId("5e8b0bf2041a913bc608d6a3"),
"memberId" : ObjectId("5e7e2f048f80b46d786bfd67"),
"date" : ISODate("2020-04-06T11:01:06.463Z")
}
],
members Schema :
{
firstname : String
photo : String
}
members Sample Doc :
[{
"_id" : ObjectId("5e7dbf5b257e6b18a62f2da9"),
"firstname" : "raed",
"photo" : "/users/5e7dbf5b257e6b18a62f2da9/profile/profile-02b13aef6e.png"
},
{
"_id" : ObjectId("5e7e2f048f80b46d786bfd67"),
"firstname" : "sarra",
"photo" : "/5e7e2f048f80b46d786bfd67/profile/profile-c79f91aa2e.png"
}]
I made a query with aggregate, and lookup to get populated data of members, and I want to concat the photo fields of the members by a string, but I get an error,
How can I do the concat ?
Query :
db.getCollection('events').aggregate([
{ $match: { _id: ObjectId("5e8b0bac041a913bc608d69d")}},
{
"$lookup": {
"from": "members",
"localField": "members.memberId",
"foreignField": "_id",
"as": "Members"
}
},
{
$project: {
"Members.firstname" : 1,
"Members.photo": 1,
//"Members.photo": {$concat:["http://myurl", "$Members.photo"]},
"Members._id" : 1,
},
}
])
Result without the concat :
{
"_id" : ObjectId("5e8b0bac041a913bc608d69d"),
"Members" : [
{
"_id" : ObjectId("5e7dbf5b257e6b18a62f2da9"),
"firstname" : "raed",
"photo" : "/users/5e7dbf5b257e6b18a62f2da9/profile/profile-02b13aef6e.png"
},
{
"_id" : ObjectId("5e7e2f048f80b46d786bfd67"),
"firstname" : "sarra",
"photo" : "/5e7e2f048f80b46d786bfd67/profile/profile-c79f91aa2e.png"
}
]
}
Error :
$concat only supports strings, not array
You can do that simply by adding pipeline to $lookup stage
db.events.aggregate([
{
$match: {
_id: ObjectId("5e8b0bac041a913bc608d69d"),
},
},
{
$lookup: {
from: "members",
let: { memberId: "$members.memberId" },
pipeline: [
{ $match: { $expr: { $in: ["$_id", "$$memberId"] } } },
{
$project: {
firstname: 1,
photo: { $concat: ["http://myurl", "$photo"] }
}
}
],
as: "Members",
}
},
/** Optional */
{$project : {Members: 1}}
]);
Test : MongoDB-Playground
the alternative of using a pipeline in the above answer
we may use project and group
db.events.aggregate([
{
$match: { _id: ObjectId("5e8b0bac041a913bc608d69d") }
},
{
$unwind: '$members' // to spread the members array into a stream of documents
},
{
$lookup: {
from: "members",
localField: "members.memberId",
foreignField: "_id",
as: "member"
}
},
{
$unwind: '$member' // each document will have array of only one member, so do this unwind to convert it to an object
},
{
$project: { // do the project here to be able to use the $concat operator
'member._id': 1,
'member.firstname': 1,
'member.photo': 1,
'member.photo': { $concat: ['http://myurl', '$member.photo'] } // now we can use the $concat as member is an object, then member.photo exists
}
},
{
$group: { // do that grouping stage to gather all the members belong to the same document in one array again
_id: '$_id',
Members: {
$addToSet: '$member'
}
}
}
])
I have a collection where every documents has an array named "contacts".
Sample Docs :
{
"_id" : ObjectId("5660c2dfdfdfcba2d47baa2d9"),
"name" : john,
"contacts" : [
{
"name" : "ari",
"phone" : "12341234"
},
{
"name" : "dea",
"phone" : "34234234"
}
]
},
{
"_id" : ObjectId("5660c2dfdfdfcba2d47baa2d9"),
"name" : joni,
"contacts" : [
{
"name" : "budi",
"phone" : "13341234"
},
{
"name" : "ade",
"phone" : "3242343"
},
{
"name" : "are",
"phone" : "64545345"
}
]
}
I want to know get total count of contacts for all docs.
From sample docs output should be 5 contacts
Thank you for helping me.
You can try below query :
1) If contacts array exists in every doc :
db.collection.aggregate([
/** project only needed field contacts with size of array in each doc */
{
$project: {
_id: 0,
contacts: {
$size: "$contacts"
}
}
},
/** group on empty(without any filter) & sum contacts field */
{
$group: {
_id: "",
TotalContacts: {
$sum: "$contacts"
}
}
},
/** Optional projection */
{
$project: {
_id: 0
}
}
])
2) If contacts array field might not exist in every doc, if it exists & not an array then an additional type check has to be done in below $cond :
db.collection.aggregate([
/** group on empty(without any filter) & sum contacts field's size if contacts exists else sum 0, You've can have optional projection as first stage like above query */
{
$group: {
_id: "",
TotalContacts: {
$sum: {
$cond: [
{
"$ifNull": [
"$contacts",
false
]
},
{
$size: "$contacts"
},
0
]
}
}
}
},
/** Optional projection */
{
$project: {
_id: 0
}
}
])
Test : MongoDB-Playground
I have two collection 1) profile ,2) posts find the below pic for your reference. user_prfoile collection
user_posts collection
In the lambda function, when i passed the userid, then we will get the userid revlevant data display. but i need user details in feelings array.
I tried with the below code, but i get the empty output
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=1))
Userid = event['userid']
uid = ObjectId(Userid)
dispost = list( db.user_posts.aggregate([{ "$match" : { "userid" : uid } },
{ "$unwind" : "$user_profile" },
{"$lookup":
{
"from" : 'user_profile',
"localField" : 'feelings.userid',
"foreignField" : '_id',
"as" : 'user_details'
}
},
{ "$addFields" : { "user_details.feelings.username" : "$user_profile.username", "user_details.feelings.photo" : "$user_profile.photo"}},
{ "$group" : {
"_id" : "$_id",
"user_profile" : {
"$push" : {
"$arrayElemAt" : ["$user_details", 0]
}}}},
{ "$project" : { "_id" : "$_id", "userid" : 1, "type" : 1, "feelings" : 1 }}
]))
disair = json.dumps(dispost, default=json_util.default)
return json.loads(disair)
I get an empty output.
I need output like this below.
_id :
userid :
type :
location :
feelings :
[ {userid : objectid(""),
username : "nipun",
photo : " "},
{userid : objectid(""),
username : "ramesh",
photo : " "}
]
in feelings array , i need user details from user_profile collection based on userid in feelings array.
Ok, there are couple of issues with that aggregation query, keeping most of it as is, I've modified it to work, as of now please try this and make any changes needed :
db.getCollection('user_posts').aggregate([{ "$match": { "userid": uid } },
{ "$unwind": "$feelings" },
{
"$lookup":
{
"from": 'user_profile',
"localField": 'feelings.userid',
"foreignField": '_id',
"as": 'feelings'
}
},
{
"$group": {
"_id": "$_id", userPost: { "$first": "$$CURRENT" }, "feelings": {
"$push": { "$arrayElemAt": ["$feelings", 0] }
}
}
}, { "$project": { userid: '$userPost.userid', type: '$userPost.type', "feelings": 1 } }
])