group collection data by multiple fields mongodb - mongodb

Collection Structure
Order = new Schema
index: { type: Number, unique: true }
number: Date
status: { type: String, enum: ['success', 'failure'] }
created_at: { type: Date, default: Date.now }
updated_at: { type: Date, default: Date.now }
I need help with a query that returns me an array of objects having data as success count and failure count grouped by date.
Ex-
orders = {
28-10-2016:{
success_count: 10,
failure_count: 10
},
29-10-2016: {
success_count: 10,
failure_count: 10
}
}

With the aggregation framework, the result will be slightly different from your "desired" output as instead of having hash keys, you get an array of objects with the _id key having a value that represents you group by field. For instance, instead of
{
"28-10-2016":{
"success_count": 10,
"failure_count": 10
},
"29-10-2016": {
"success_count": 10,
"failure_count": 10
}
}
you'd have a better structure like
[
{
"_id": "28-10-2016",
"success_count": 10,
"failure_count": 10
},
"_id": "29-10-2016",
"success_count": 10,
"failure_count": 10
}
]
Accomplishing the above result would require using the $cond operator in the $sum accumulator operator. The $cond operator will evaluate a logical condition based on its first argument (if) and then returns the second argument where the evaluation is true (then) or the third argument where false (else). This converts the true/false logic into 1 and 0 numerical values that feed into $sum respectively:
"success_count": {
"$sum": {
"$cond": [ { "$eq": [ "$status", "success" ] }, 1, 0 ]
}
}
As a resulting pipeline, one needs to run the aggregation operation which uses the $dateToString operator in the _id key expression for the $group pipeline:
Orders.aggregate([
{
"$group": {
"_id": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$created_at"
}
},
"success_count": {
"$sum": {
"$cond": [ { "$eq": [ "$status", "success" ] }, 1, 0 ]
}
},
"failure_count": {
"$sum": {
"$cond": [ { "$eq": [ "$status", "failure" ] }, 1, 0 ]
}
}
}
}
], function (err, orders){
if (err) throw err;
console.log(orders);
})
However, there is a more flexible and better performant approach which executes much faster than the above, where the most efficient data structure for your aggregation result follows the schema for example:
orders = [
{
"_id": "28-10-2016",
"counts": [
{ "status": "success", "count": 10 },
{ "status": "failure", "count": 10 }
]
},
{
"_id": "29-10-2016",
"counts": [
{ "status": "success", "count": 10 },
{ "status": "failure", "count": 10 }
]
}
]
Then consider running an alternative pipeline as follows
Orders.aggregate([
{
"$group": {
"_id": {
"date": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$created_at"
}
},
"status": { "$toLower": "$status" }
},
"count": { "$sum": 1 }
}
},
{
"$group": {
"_id": "$_id.date",
"counts": {
"$push": {
"status": "$_id.status",
"count": "$count"
}
}
}
}
], function (err, orders){
if (err) throw err;
console.log(orders);
})

Related

How can I group boolean fields in mongodb?

Here's a very simple data example.
[
{
"aaa": true,
"bbb": 111,
},
{
"aaa": false,
"bbb": 111,
}
]
Then, what query should be executed so that I can get the result like this?
[
{
"_id": "0",
"bbb_sum": 222,
"aaa_and": false,
"aaa_or": true
}
]
Actually, I've tried with a query like this
db.collection.aggregate([
{
"$group": {
"_id": "0",
"bbb_sum": {
"$sum": "$bbb"
},
"aaa_and": {
"$and": ["$aaa", true]
},
"aaa_or": {
"$or": ["$aaa", false]
}
}
}
])
But the Mongo Playground complains query failed: (Location40237) The $and accumulator is a unary operator, that's quite confusing.
You can also find this simple test case here https://mongoplayground.net/p/8dqtXJ93vIx
Also, I've searched for similar questions on both Google and Stackoverflow, but I can't find one.
Thanks in advance!
Not like "$sum","$and" and "$or" are not aggregation operators that can be used in "$group". You can temporary save all the "aaa" field into an array and then use "$project" operator to process the data.
db.collection.aggregate([
{
"$group": {
"_id": "0",
"sum": {
"$sum": "$bbb"
},
"aaa_all": {
"$push": "$aaa"
}
}
},
{
"$project": {
"sum": 1,
"aaa_and": {
"$allElementsTrue": "$aaa_all"
},
"aaa_or": {
"$anyElementTrue": "$aaa_all"
}
}
}
])
Here is the case: https://mongoplayground.net/p/Y-Fs_Ch9lwk
$min and $max operators actually work with booleans too, false being considered smaller than true
thinking of them as 0 and 1 might be easier to understand :
$min: [a,…,n] will return 1/true only if all elements are 1/true => this is a AND
$max: [a,…,n] will return 0/false only if all elements are 0/false => this is a OR
(the operators will return booleans if input booleans, the analogy with numbers is only for the sake of comprehension)
So your request can simply become :
db.collection.aggregate([
{
"$group": {
"_id": "0",
"bbb_sum": {
"$sum": "$bbb"
},
"aaa_and": {
"$min": "$aaa"
},
"aaa_or": {
"$max": "$aaa"
}
}
}
])
You can do some logics as below
db.collection.aggregate([
{
"$group": {//Group by desired id
"_id": null,
"sum": {//Sum the value
"$sum": "$bbb"
},
"aaa_and": {
"$sum": {
"$cond": {
"if": {
"$eq": [
"$aaa",
true
]
},
"then": 1, //If true returns 1
"else": 0 // else 0
}
}
},
"total": { //helper to do the logic
$sum: 1
}
}
},
{
$project: {
aaa_and: {
"$eq": [//If total matches with number of true, all are true
"$total",
"$aaa_and"
]
},
aaa_or: {
"$ne": [//if value greater than 0, then there is at least one true
"$aaa_and",
"0"
]
},
sum: 1
}
}
])
playground

MongoDB: Assign document objects to field in '$project' stage

I have a user collection:
[
{"_id": 1,"name": "John", "age": 25, "valid_user": true}
{"_id": 2, "name": "Bob", "age": 40, "valid_user": false}
{"_id": 3, "name": "Jacob","age": 27,"valid_user": null}
{"_id": 4, "name": "Amelia","age": 29,"valid_user": true}
]
I run a '$facet' stage on this collection. Checkout this MongoPlayground.
I want to talk about the first output from the facet stage. The following is the response currently:
{
"user_by_valid_status": [
{
"_id": false,
"count": 1
},
{
"_id": true,
"count": 2
},
{
"_id": null,
"count": 1
}
]
}
However, I want to restructure the output in this way:
"analytics": {
"invalid_user": {
"_id": false
"count": 1
},
"valid_user": {
"_id": true
"count": 2
},
"user_with_unknown_status": {
"_id": null
"count": 1
}
}
The problem with using a '$project' stage along with 'arrayElemAt' is that the order may not be definite for me to associate an index with an attribute like 'valid_users' or others. Also, it gets further complicated because unlike the sample documents that I have shared, my collection may not always contain all the three categories of users.
Is there some way I can do this?
You can use $switch conditional operator,
$project to show value part in v with _id and count field as object, k to put $switch condition
db.collection.aggregate([
{
"$facet": {
"user_by_valid_status": [
{
"$group": {
"_id": "$valid_user",
"count": { "$sum": 1 }
}
},
{
$project: {
_id: 0,
v: { _id: "$_id", count: "$count" },
k: {
$switch: {
branches: [
{ case: { $eq: ["$_id", null] }, then: "user_with_unknown_status" },
{ case: { $eq: ["$_id", false] }, then: "invalid_user" },
{ case: { $eq: ["$_id", true] }, then: "valid_user" }
]
}
}
}
}
],
"users_above_30": [{ "$match": { "age": { "$gt": 30 } } }]
}
},
$project stage in root, convert user_by_valid_status array to object using $arrayToObject
{
$project: {
analytics: { $arrayToObject: "$user_by_valid_status" },
users_above_30: 1
}
}
])
Playground

Active and Total User Count Mongodb

I want to write a group by query to written active user and total count(both active and inactive) grouped by a date column in mongodb. I am able to run them as two separate scripts but how to retrieve the same information in one script
db.user.aggregate(
{
"$match": { 'phoneInfo.verifiedFlag': true}
},
{
"$project": {
yearMonthDayUTC: { $dateToString: { format: "%Y-%m-%d", date: "$createdOn" } }
}
},
{
"$group": {
"_id": {day: "$yearMonthDayUTC"},
count: {
"$sum": 1
}
}
},
{
$sort: {
"_id.day": 1,
}
})
You can use the $cond operator in your group to create a conditional count as follows (assuming the inactive/active values are in a field called status):
db.user.aggregate([
{ "$match": { 'phoneInfo.verifiedFlag': true} },
{
"$group": {
"_id": { "$dateToString": { "format": "%Y-%m-%d", "date": "$createdOn" } },
"total": { "$sum": 1 },
"active_count": {
"$sum": {
"$cond": [ { "$eq": [ "$status", "active" ] }, 1, 0 ]
}
},
"inactive_count": {
"$sum": {
"$cond": [ { "$eq": [ "$status", "inactive" ] }, 1, 0 ]
}
}
}
},
{ "$sort": { "_id": 1 } }
])
For different values you can adapt the following pipeline:
db.user.aggregate([
{ "$match": { 'phoneInfo.verifiedFlag': true} },
{
"$group": {
"_id": {
"day": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$createdOn"
}
},
"status": { "$toLower": "$status" }
},
"count": { "$sum": 1 }
}
},
{
"$group": {
"_id": "$_id.day",
"counts": {
"$push": {
"status": "$_id.status",
"count": "$count"
}
}
}
},
{ "$sort": { "_id": 1 } }
])

Group and count in Mongo DB

I have many tweets object like this:
{
"_id" : ObjectId("5a2f4a381cb29b482553e2c9"),
"user_id" : 21898942,
"created_at" : ISODate("2009-03-09T19:48:50Z"),
"id" : 1301923516,
"place" : "",
"retweet_count" : 0,
"tweet" : "Save the Date! March 28th Vietnamese Cooking Class! Call to Reserve 312.255.0088",
"favorite_count" : 0
"type": A
}
I'm using this code to qroup the tweets by date and by type:
pipeline = [
{
"$group": {
"_id": {
"date": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$created_at"
}
},
"type": "$type"
},
"count": {
"$sum": 1
}
}
}
]
results = mongo.db.tweets.aggregate(pipeline)
Here is the result I get:
{
"_id": {
"date": "2009-03-17",
"type": A
},
"count": 4
,
{
"_id": {
"date": "2009-03-17",
"type": B
},
"count": 6
}
But now I want to have the result in this format:
{date: "2009-03-17", A: 4, B: 6, C: 9}
Is there anyway I can achieve this through aggregate directly?
Note: I'm using MongoDB and PyMongo
You can try the below aggregation query in 3.6 version.
Added the second group to create array of type and count value pairs followed by $mergeObjects to merge date key value with $arrayToObject, which produces create a type value key and count value pairs, to generate the expected response.
$replaceRoot to promote the document to the top level.
pipeline = [
{
"$group": {
"_id": {
"date": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$created_at"
}
},
"type": "$type"
},
"count": {
"$sum": 1
}
}
},
{
"$group": {
"_id": "$_id.date",
"typeandcount": {
"$push": {
"k": "$_id.type",
"v": "$count"
}
}
}
},
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [
{
"date": "$_id"
},
{
"$arrayToObject": "$typeandcount"
}
]
}
}
}
]
Mongo 3.4 version:
Replace the last stage with below
{
"$replaceRoot": {
"newRoot": {
"$arrayToObject": {
"$concatArrays": [
[
{
"k": "date",
"v": "$_id"
}
],
"$typeandcount"
]
}
}
}
}

Mongo Aggregation : $group and $project array to object for counts

I have documents like:
{
"platform":"android",
"install_date":20151029
}
platform - can have one value from [android|ios|kindle|facebook ] .
install_date - there are many install_dates
There are also many fields.
Aim : I am calculating installs per platform on particular date.
So I am using group by in aggregation framework and make counts by platform. Document should look like like:
{
"install_date":20151029,
"platform" : {
"android":1000,
"ios": 2000,
"facebook":1500
}
}
I have done like:
db.collection.aggregate([
{
$group: {
_id: { platform: "$platform",install_date:"$install_date"},
count: { "$sum": 1 }
}
},
{
$group: {
_id: { install_date:"$_id.install_date"},
platform: { $push : {platform :"$_id.platform", count:"$count" } }
}
},
{
$project : { _id: 0, install_date: "$_id.install_date", platform: 1 }
}
])
which Gives document like:
{
"platform": [
{
"platform": "facebook",
"count": 1500
},
{
"platform": "ios",
"count": 2000
},
{
"platform": "android",
"count": 1000
}
],
"install_date": 20151027
}
Problem:
Projecting array to single object as "platform"
With MongoDb 3.4 and newer, you can leverage the use of $arrayToObject operator to get the desired result. You would need to run the following aggregate pipeline:
db.collection.aggregate([
{ "$group": {
"_id": {
"date": "$install_date",
"platform": { "$toLower": "$platform" }
},
"count": { "$sum": 1 }
} },
{ "$group": {
"_id": "$_id.date",
"counts": {
"$push": {
"k": "$_id.platform",
"v": "$count"
}
}
} },
{ "$addFields": {
"install_date": "$_id",
"platform": { "$arrayToObject": "$counts" }
} },
{ "$project": { "counts": 0, "_id": 0 } }
])
For older versions, take advantage of the $cond operator in the $group pipeline step to evaluate the counts based on the platform field value, something like the following:
db.collection.aggregate([
{ "$group": {
"_id": "$install_date",
"android_count": {
"$sum": {
"$cond": [ { "$eq": [ "$platform", "android" ] }, 1, 0 ]
}
},
"ios_count": {
"$sum": {
"$cond": [ { "$eq": [ "$platform", "ios" ] }, 1, 0 ]
}
},
"facebook_count": {
"$sum": {
"$cond": [ { "$eq": [ "$platform", "facebook" ] }, 1, 0 ]
}
},
"kindle_count": {
"$sum": {
"$cond": [ { "$eq": [ "$platform", "kindle" ] }, 1, 0 ]
}
}
} },
{ "$project": {
"_id": 0, "install_date": "$_id",
"platform": {
"android": "$android_count",
"ios": "$ios_count",
"facebook": "$facebook_count",
"kindle": "$kindle_count"
}
} }
])
In the above, $cond takes a logical condition as it's first argument (if) and then returns the second argument where the evaluation is true (then) or the third argument where false (else). This makes true/false returns into 1 and 0 to feed to $sum respectively.
So for example, if { "$eq": [ "$platform", "facebook" ] }, is true then the expression will evaluate to { $sum: 1 } else it will be { $sum: 0 }