Use $ and $elemMatch to group entities - mongodb

Considering the following document in my mongo DB instance :
{
"_id": 1,
"people": [
{"id": 1, "name": "foo"},
{"id": 2, "name": "bar"},
/.../
],
"stats": [
{"peopleId": 1, "workHours": 24},
{"peopleId": 2, "workHours": 36},
/.../
}
Each element in my collection represent the work of every employee in my company, each weeks. As an important note, peopleId may change from one week to another !
I would like to get all weeks where foo worked more than 24 hours. As you can see, the format is kinda annoying since the people name and the work hours are separated in my database. A simple $and is not enough.
I wonder if, using some $ and $elemMatch I can achieve doing this query.
Can I use this to group the "people" entities with "stats" entities ?

Query to get foo worked more than 24 hours.
db.collection.aggregate([
{$unwind: { path : "$people"}},
{$unwind: { path : "$stats"}},
{$match: { "people.name" : "foo"}},
{$group: {
_id: "$_id",
peopleIdMoreThan24: { $addToSet: {
$cond : { if : { $and : [ {"$eq" : ["$people.id", "$stats.peopleId" ] },
{"$gt" : ["$stats.workHours", 24] }]} , then : "$people.id", else: "Not satisfying the condition"}}}
}
},
{$unwind: { path : "$peopleIdMoreThan24" }},
{$match: { "peopleIdMoreThan24" : {$nin : [ "Not satisfying the condition"]}}},
]);
Data in collection:-
/* 1 */
{
"_id" : 1,
"people" : [
{
"id" : 1,
"name" : "foo"
},
{
"id" : 2,
"name" : "bar"
}
],
"stats" : [
{
"peopleId" : 1,
"workHours" : 24
},
{
"peopleId" : 2,
"workHours" : 36
}
]
}
/* 2 */
{
"_id" : 2,
"people" : [
{
"id" : 1,
"name" : "foo"
},
{
"id" : 2,
"name" : "bar"
}
],
"stats" : [
{
"peopleId" : 1,
"workHours" : 25
},
{
"peopleId" : 2,
"workHours" : 36
}
]
}
/* 3 */
{
"_id" : 3,
"people" : [
{
"id" : 1,
"name" : "foo"
},
{
"id" : 2,
"name" : "bar"
}
],
"stats" : [
{
"peopleId" : 1,
"workHours" : 25
},
{
"peopleId" : 2,
"workHours" : 36
}
]
}
Output:-
The output has document id and people id of foo worked more than 24 hours.
/* 1 */
{
"_id" : 3,
"peopleIdMoreThan24" : 1
}
/* 2 */
{
"_id" : 2,
"peopleIdMoreThan24" : 1
}

Related

What is $$ROOT in MongoDB aggregate and how it works?

I am watching a tutorial I can understand how this aggregate works, What is the use of pings, $$ROOT in it.
client = pymongo.MongoClient(MY_URL)
pings = client['mflix']['watching_pings']
cursor = pings.aggregate([
{
"$sample": { "size": 50000 }
},
{
"$addFields": {
"dayOfWeek": { "$dayOfWeek": "$ts" },
"hourOfDay": { "$hour": "$ts" }
}
},
{
"$group": { "_id": "$dayOfWeek", "pings": { "$push": "$$ROOT" } }
},
{
"$sort": { "_id": 1 }
}
]);
Let's assume that our collection looks like below:
{
"_id" : ObjectId("b9"),
"key" : 1,
"value" : 20,
"history" : ISODate("2020-05-16T00:00:00Z")
},
{
"_id" : ObjectId("ba"),
"key" : 1,
"value" : 10,
"history" : ISODate("2020-05-13T00:00:00Z")
},
{
"_id" : ObjectId("bb"),
"key" : 3,
"value" : 50,
"history" : ISODate("2020-05-12T00:00:00Z")
},
{
"_id" : ObjectId("bc"),
"key" : 2,
"value" : 0,
"history" : ISODate("2020-05-13T00:00:00Z")
},
{
"_id" : ObjectId("bd"),
"key" : 2,
"value" : 10,
"history" : ISODate("2020-05-16T00:00:00Z")
}
Now based on the history field you want to group and insert the whole documents in to an array field 'items'. Here $$ROOT variable will be helpful.
So, the aggregation query to achieve the above will be:
db.collection.aggregate([{
$group: {
_id: '$history',
items: {$push: '$$ROOT'}
}
}])
It will result in following output:
{
"_id" : ISODate("2020-05-12T00:00:00Z"),
"items" : [
{
"_id" : ObjectId("bb"),
"key" : 3,
"value" : 50,
"history" : ISODate("2020-05-12T00:00:00Z")
}
]
},
{
"_id" : ISODate("2020-05-13T00:00:00Z"),
"items" : [
{
"_id" : ObjectId("ba"),
"key" : 1,
"value" : 10,
"history" : ISODate("2020-05-13T00:00:00Z")
},
{
"_id" : ObjectId("bc"),
"key" : 2,
"value" : 0,
"history" : ISODate("2020-05-13T00:00:00Z")
}
]
},
{
"_id" : ISODate("2020-05-16T00:00:00Z"),
"items" : [
{
"_id" : ObjectId("b9"),
"key" : 1,
"value" : 20,
"history" : ISODate("2020-05-16T00:00:00Z")
},
{
"_id" : ObjectId("bd"),
"key" : 2,
"value" : 10,
"history" : ISODate("2020-05-16T00:00:00Z")
}
]
}
I hope it helps.

aggregate function to project array size after removing empty value in array

My primary goal is to print titles are having number of grades greater than four, i can achieve it with below query,
db.students.aggregate({$project : { title:1 ,_id : 0, count: {$size : "$grades"}}},{$match: {"count": {$gt:4}}})
But if grades array have empty values how can i remove them, tried this but not giving correct output.
db.students.aggregate({$project : { title:1 ,_id : 0, count: {$size : "$grades"}}},{$match: {"count": {$gt:4},grades : {$ne:''}}})
You can use $filter to remove empty grades before you run $size:
db.students.aggregate([
{$project : { title:1 ,_id : 0, count: { $size : { $filter: { input: "$grades", cond: { $ne: [ "$$this", '' ] } } }}}},
{$match: {"count": {$gt:4}}}
])
Let's explain this with step by step of different different queries:
All possible values in the collection grades:
> db.grades.find()
{ "_id" : ObjectId("5cb2ff50d33f6ed856afe577"), "title" : "abc", "grades" : [ 12, 23, 1 ] }
{ "_id" : ObjectId("5cb2ff55d33f6ed856afe578"), "title" : "abc", "grades" : [ 12, 23 ] }
{ "_id" : ObjectId("5cb2ff5cd33f6ed856afe579"), "title" : "abc", "grades" : [ 12, 23, 10, 100, 34 ] }
{ "_id" : ObjectId("5cb2ff63d33f6ed856afe57a"), "title" : "abc", "grades" : "" }
{ "_id" : ObjectId("5cb2ff66d33f6ed856afe57b"), "title" : "abc", "grades" : [ ] }
{ "_id" : ObjectId("5cb2ff6bd33f6ed856afe57c"), "title" : "abc", "grades" : [ 1, 2, 3, 4, 5 ] }
Just filtered empty grades records as:
> db.grades.aggregate([{$match: {grades: {$ne:''}} }])
{ "_id" : ObjectId("5cb2ff50d33f6ed856afe577"), "title" : "abc", "grades" : [ 12, 23, 1 ] }
{ "_id" : ObjectId("5cb2ff55d33f6ed856afe578"), "title" : "abc", "grades" : [ 12, 23 ] }
{ "_id" : ObjectId("5cb2ff5cd33f6ed856afe579"), "title" : "abc", "grades" : [ 12, 23, 10, 100, 34 ] }
{ "_id" : ObjectId("5cb2ff66d33f6ed856afe57b"), "title" : "abc", "grades" : [ ] }
{ "_id" : ObjectId("5cb2ff6bd33f6ed856afe57c"), "title" : "abc", "grades" : [ 1, 2, 3, 4, 5 ] }
Now project the grades count values in a variable along with required other columns.
> db.grades.aggregate([{$match: {grades: {$ne:''}} }, {$project: {_id:0, title:1, count: {$size: "$grades"} } }])
{ "title" : "abc", "count" : 3 }
{ "title" : "abc", "count" : 2 }
{ "title" : "abc", "count" : 5 }
{ "title" : "abc", "count" : 0 }
{ "title" : "abc", "count" : 5 }
Now match required condition of grades array count greater than 4 as below:
> db.grades.aggregate([{$match: {grades: {$ne:''}} }, {$project: {_id:0, title:1, count: {$size: "$grades"} } }, {$match: {count: {$gte: 4}}} ])
{ "title" : "abc", "count" : 5 }
{ "title" : "abc", "count" : 5 }
>

Get the maximum of difference of two fields in mongodb

I'm newbie to mongoDB. Here I face with an error while I want to get the maximum value of the difference of two fields.
Here is the structure of data saved in database:
{
"_id" : ObjectId("52b3833bd3e98582d2bfb628"),
"author" : {
"name" : "Graydon Hoare",
"email" : "graydon#gmail.com"
},
"title" : "Why Rust ditched pure functions",
"body" : "sth",
"url" : "http://thread.gmane.org/gmane.comp.lang.rust.devel/3674/focus=3855",
"date" : ISODate("2013-04-30T13:23:00.000Z"),
"starred" : 105,
"ratings" : [
3,
5,
3,
2,
4,
1,
3,
3,
3,
2,
3
],
"comments" : [
{
"user" : "tr0lltherapy",
"upVotes" : 18,
"downVotes" : 2,
"text" : "sth",
"replies" : [
{
"user" : "thedeemon",
"upVotes" : 10,
"downVotes" : 0,
"text" : "sth"
},
{
"user" : "mcandre",
"upVotes" : 0,
"downVotes" : 5,
"text" : "sth"
},
{
"user" : "lacosaes0",
"upVotes" : 30,
"downVotes" : 6,
"text" : "Particular emphasis on memory safety."
}
]
},
{
"user" : "hypster",
"upVotes" : 30,
"downVotes" : 2,
"text" : "tl;dr everybody was type-fu fighting",
"replies" : [
{
"user" : "homoiconic",
"upVotes" : 15,
"downVotes" : 0,
"text" : "Here comes the Big Boss, Hu! Simon Peyton-Jones."
}
]
}
],
"tags" : [
"Rust",
"Computer",
"Programming"
],
"draft" : true,
"published" : true
}
What I want is to get the value of maximum of the subtract of upVotes and downVotes in replies and comments.
db.getCollection('links').aggregate([
{$project: {
_id: "$author",
maxVote: $max: {
$subtract: ["$comments.upVotes", "$comments.downVotes"]
}
}
}
])
I don't know how to fix it!
You can use $map to get the difference for each comment (using $subtract) and then run $max on the output from mapped comments. Additionally you need another nested $max to get the differences from replies, try:
db.col.aggregate([
{
$project: {
maxVote: {
$max: {
$map: {
input: "$comments",
as: "comment",
in: {
$max: {
$concatArrays: [
[ { $subtract: [ "$$comment.upVotes", "$$comment.downVotes" ] } ],
{
$map: {
input: "$$comment.replies",
as: "reply",
in: { $subtract: [ "$$reply.upVotes", "$$reply.downVotes" ] }
}
}
]
}
}
}
}
}
}
}
])
prints:
{ "_id" : ObjectId("..."), "maxVote" : 28 }

how to show 0 for week when no record is matching that week in $week mongodb query

My collection looks like below with details
/* 1 createdAt:6/13/2018, 5:17:07 PM*/
{ "_id" : ObjectId("5b21043b18f3bc7c0be3414c"),
"Number" : 242,
"State" : "2",
"City" : "3",
"Website" : "",
"Contact_Person_Name" : "Ajithmullassery",
"CreatedById" : "Admin",
"UpdatedById" : "Admin",
"IsActive" : true,
"UpdatedOn" : ISODate("2018-06-13T17:17:07.313+05:30"),
"CreatedOn" : ISODate("2018-06-13T17:17:07.313+05:30")
},
/* 2 createdAt:6/13/2018, 6:45:42 PM*/
{
"_id" : ObjectId("5b2118fe18f3bc7c0be3415b"),
"Number" : 243,
"State" : "1",
"City" : "143",
"Website" : "",
"Contact_Person_Name" : "sachitkumar",
"CreatedById" : "vinoth",
"UpdatedById" : "Admin",
"IsActive" : true,
"UpdatedOn" : ISODate("2018-06-13T18:45:42.590+05:30"),
"CreatedOn" : ISODate("2018-06-13T18:45:42.590+05:30")
},
/* 3 createdAt:6/18/2018, 5:34:33 PM*/
{
"_id" : ObjectId("5b279fd118f3bc7c0be34166"),
"Number" : 244,
"State" : "0",
"City" : "8",
"Website" : "",
"Contact_Person_Name" : "Akshay",
"CreatedById" : "vinoth",
"UpdatedById" : "Admin",
"IsActive" : true,
"UpdatedOn" : ISODate("2018-06-18T17:34:33.270+05:30"),
"CreatedOn" : ISODate("2018-06-18T17:34:33.270+05:30")
},
/* 4 createdAt:6/20/2018, 1:02:21 PM*/
{
"_id" : ObjectId("5b2a030518f3bc7c0be3416d"),
"Number" : 245,
"State" : "5",
"City" : "6",
"Website" : "",
"Contact_Person_Name" : "Dr DS Mithra",
"CreatedById" : "vinoth",
"UpdatedById" : "Admin",
"FacilityID" : "594387f5e2de7be83be5d5f1",
"IsActive" : true,
"UpdatedOn" : ISODate("2018-06-20T13:02:21.887+05:30"),
"CreatedOn" : ISODate("2018-06-20T13:02:21.887+05:30")
},
/* 5 createdAt:6/20/2018, 1:08:58 PM*/
{
"_id" : ObjectId("5b2a049218f3bc7c0be3416e"),
"Number" : 245,
"State" : "5",
"City" : "6",
"Website" : "",
"Contact_Person_Name" : "Ramaswamy Manickam",
"CreatedById" : "vinoth",
"UpdatedById" : "Admin",
"IsActive" : true,
"UpdatedOn" : ISODate("2018-06-20T13:08:58.040+05:30"),
"CreatedOn" : ISODate("2018-06-20T13:08:58.040+05:30")
}
I have the query like below
db.collectionName.aggregate([
//where query
{ "$match": { $and:[{CreatedOn:{$lte:ISODate("2018-07-14T13:59:08.266+05:30")}},{CreatedOn:{$gte:ISODate("2018-06-10T13:59:08.266+05:30")}}] } },
//distinct column
{
"$group": {
_id: {$week: '$CreatedOn'},
documentCount: {$sum: 1}
}
}
])
The query will return the weeknumber and number of documents created as below
/* 1 */
{
"_id" : 26,
"documentCount" : 1
},
/* 2 */
{
"_id" : 25,
"documentCount" : 1
},
/* 3 */
{
"_id" : 24,
"documentCount" : 9
},
/* 4 */
{
"_id" : 23,
"documentCount" : 2
}
In above _id is the weeknumber. If in case in above results weekNumber : 23 no records are created then the query gives only 3 records removing the "_id":23.
How to get the records with documentcount as zero when there is no records created.
Like in above example when no records for _id: 23 should get like below
/* 4 */
{
"_id" : 23,
"documentCount" : 0
}
As $week can return a value between 0 and 53 I assume you expect 54 documents as a result with 0 or non-zero values for documentCount. To achieve that you should collect all your documents into one ($group-ing by null) and then generate the output.
To generate a range of numbers you can use $range operator and then you can generate the output using $map. To transform an array of documents into multiple docs you can use $unwind.
db.collectionName.aggregate([
//where query
{ "$match": { $and:[{CreatedOn:{$lte:ISODate("2018-07-14T13:59:08.266+05:30")}},{CreatedOn:{$gte:ISODate("2018-06-10T13:59:08.266+05:30")}}] } },
//distinct column
{
"$group": {
_id: {$week: '$CreatedOn'},
documentCount: {$sum: 1}
}
},
{
$group: {
_id: null,
docs: { $push: "$$ROOT" }
}
},
{
$project: {
docs: {
$map: {
input: { $range: [ {$week:ISODate("2018-06-10T13:59:08.266+05:30")}, {$week:ISODate("2018-07-14T13:59:08.266+05:30")}]},
as: "weekNumber",
in: {
$let: {
vars: { index: { $indexOfArray: [ "$docs._id", "$$weekNumber" ] } },
in: {
$cond: {
if: { $eq: [ "$$index", -1 ] },
then: { _id: "$$weekNumber", documentCount: 0 },
else: { $arrayElemAt: [ "$docs", "$$index" ] }
}
}
}
}
}
}
}
},
{
$unwind: "$docs"
},
{
$replaceRoot: {
newRoot: "$docs"
}
}
])
Using $indexOfArray to check if array of current docs contains the document (-1 otherwise) and $arrayElemAt to get existing document from docs. Last step ($replaceRoot) is just to get rid of one level of nesting (docs). Outputs:
{ "_id" : 0, "documentCount" : 0 }
{ "_id" : 1, "documentCount" : 0 }
{ "_id" : 2, "documentCount" : 0 }
...
{ "_id" : 22, "documentCount" : 0 }
{ "_id" : 23, "documentCount" : 2 }
{ "_id" : 24, "documentCount" : 9 }
{ "_id" : 25, "documentCount" : 1 }
{ "_id" : 26, "documentCount" : 1 }
{ "_id" : 27, "documentCount" : 0 }
...
{ "_id" : 52, "documentCount" : 0 }
{ "_id" : 53, "documentCount" : 0 }
You can easily customize returned results modifying the input of $map stage. For instance you can pass an array of consts like input: [21, 22, 23, 24] as well.
EDIT: To get the weeks between specified dates you can use $week for start and end date to get the numbers.

aggregation and display details in mongodb

I have been learning MongoDB, while doing so I tried to implement the aggregation property for my database collection. I grouped the details of the employee based on their age and by using match function, my question is it possible to display the other key-value once they pass the age criteria?
db.employee.aggregate([
{ $match: { age: { $gte: 23 } } },
{
$group: {
_id:'$age',
total: { $sum: 1 },
name: { $addToSet: '$name' }
}
}
])
and the output was like this
{ "_id" : 27, "total" : 2, "name" : [ "indhu", "logesh" ] }
{ "_id" : 26, "total" : 1, "name" : [ "keerthana" ] }
{ "_id" : 25, "total" : 1, "name" : [ "sneha" ] }
{ "_id" : 24, "total" : 1, "name" : [ "dhiva" ] }
{ "_id" : 23, "total" : 1, "name" : [ "elango" ] }
where _id denotes their age.