Let's say I have documents in my MongoDB collection that look like this:
{ name: "X", ...}
{ name: "Y", ...}
{ name: "X", ...}
{ name: "X", ...}
I can create a pipeline view using aggregation that shows sub-totals i.e.
$group: {
_id: '$name',
count: {
$sum: 1
}
}
which results in:
{ _id: "X",
count: 3 },
{ _id: "Y",
count: 1}
but how do I add a total in this view i.e.
{ _id: "X",
count: 3 },
{ _id: "Y",
count: 1},
{_id: "ALL",
count: 4}
Query1
group to count
union with the same collection, with pipeline to add the total count, in one extra document
Test code here
coll1.aggregate(
[{"$group":{"_id":"$name", "count":{"$sum":1}}},
{"$unionWith":
{"coll":"coll1",
"pipeline":[{"$group":{"_id":"ALL", "count":{"$sum":1}}}]}}])
Query2
without $union for MongoDB < 4.4
group and count
group by null and collect the documents, and total count
add to docs array the extra document with the total count
unwind and replace root to restore the structure
Test code here
aggregate(
[{"$group":{"_id":"$name", "count":{"$sum":1}}},
{"$group":
{"_id":null, "docs":{"$push":"$$ROOT"}, "total":{"$sum":"$count"}}},
{"$project":
{"docs":
{"$concatArrays":["$docs", [{"_id":"ALL", "count":"$total"}]]}}},
{"$unwind":"$docs"},
{"$replaceRoot":{"newRoot":"$docs"}}])
Try this one:
db.collection.aggregate([
{
$group: {
_id: "$name",
count: { $count: {} }
}
},
{
$unionWith: {
coll: "collection",
pipeline: [
{
$group: {
_id: "ALL",
count: { $count: {} }
}
}
]
}
}
])
Mongo Playground
Related
Here is my sample data:
[
{ name: "bob",number:20 },
{ name: "bob",number:10 },
{ name: "kol",number:20 },
{ name: "bob",number:20 }
{ name: "kol",number:10 },
{ name: "kol",number:10 }
]
I want to get a document per name with an array of unique numbers sorted.
expected output:
[
{
name:"bob",
number: [10,20] // with sort, not 20,10
},
{
name:"kol",
number: [10,20]
}
]
I tried something like this but not working as I want. I did not get the array of numbers sorted.
user.aggregate([
{
$group: {
_id: '$name',
name: { $first: '$name' },
number: {$addToSet: '$number'}
}
},
{$sort: {name: 1}}
])
You just need to $sort before you $group by the name and use $push in order to keep all numbers, instead of $addToSet. For keeping the numbers a set just use $setIntersection to remove duplicates:
db.collection.aggregate([
{$sort: {number: 1}},
{
$group: {
_id: "$name",
number: {$push: "$number"}
}
},
{
$project: {
name: "$_id",
number: {$setIntersection: ["$number"]},
_id: 0
}
},
{$sort: {name: 1}}
])
See how it works on the playground example
so I have this problem
I have this duplicate collection that goes like:
{name: "a", otherField: 1, _id: "id1"},
{name: "a", otherField: 2, _id: "id2"},
{name: "a", otherField: 3, _id: "id3"},
{name: "b", otherField: 1, _id: "id4"}
{name: "b", otherField: 2, _id: "id5"}
My goal is to get id of with less otherField that will look like:
{"name": "a", _id: "id1"},
{"name": "a", _id: "id2"},
{"name": "b", _id: "id4"}
Since highest otherField from a and b is "id3" and "id5", I want id other than the highest otherField
How to achieve this through query in mongodb?
Thank you
You can try below query :
db.collection.aggregate([
/** group all docs based on name & push docs to data field & find max value for otherField field */
{
$group: {
_id: "$name",
data: {
$push: "$$ROOT"
},
maxOtherField: {
$max: "$otherField"
}
}
},
/** Recreate data field array with removing doc which has max otherField value */
{
$addFields: {
data: {
$filter: {
input: "$data",
cond: {
$ne: [
"$$this.otherField",
"$maxOtherField"
]
}
}
}
}
},
/** unwind data array */
{
$unwind: "$data"
},
/** Replace data field as new root for each doc in coll */
{
$replaceRoot: {
newRoot: "$data"
}
}
])
Test : MongoDB-Playground
Note : We might lean towards sorting docs on field otherField, but it's not preferable on huge datasets.
Given the following data, I'm trying to get an average of all their ages, at the same time I want to return an array of their names. Ideally, I want to do this in just one query but can't seem to figure it out.
Data:
users:[
{user:{
id: 1,
name: “Bob”,
age: 23
}},
{user:{
id: 1,
name: “Susan”,
age: 32
}},
{user:{
id: 2,
name: “Jeff”,
age: 45
}
}]
Query:
var dbmatch = db.users.aggregate([
{$match: {"id" : 1}},
{$group: {_id: null, avg_age: { $avg: "$age" }}},
{$group: {_id : { name: "$name"}}}
)]
Running the above groups one at a time outputs the results I expect, either an _id of null and an average of 27.5, or an array of the names.
When I combine them as you see above using a comma, I get:
Issue Generated Code:
[ { _id: {name: null } } ]
Expected Generated Code:
[
{name:"Bob"},
{name:"Susan"},
avg_age: 27.5
]
Any help would be greatly appreciated!
Not sure if this is exactly what you want, but this query
db.users.aggregate([
{
$match: {
id: 1
}
},
{
$group: {
_id: "$id",
avg_age: {
$avg: "$age"
},
names: {
$push: {
name: "$name"
}
}
}
},
{
$project: {
_id: 0
}
}
])
Results in this result:
[
{
"avg_age": 27.5,
"names": [
{
"name": "Bob"
},
{
"name": "Susan"
}
]
}
]
This will duplicate names, so if there are two documents with the name Bob, it will be two times in the array. If you don't want duplicates, change $push to $addToSet.
Also, if you want names to be just an array of names instead of objects, change names query to
names: {
$push: "$name"
}
This will result in
[
{
"avg_age": 27.5,
"names": ["Bob", "Susan"]
}
]
Hope it helps,
Tomas :)
You can use $facet aggregation to run the multiple queries at once
db.collection.aggregate([
{ "$facet": {
"firstQuery": [
{ "$match": { "id": 1 }},
{ "$group": {
"_id": null,
"avg_age": { "$avg": "$age" }
}}
],
"secondQuery": [
{ "$match": { "id": 1 }},
{ "$group": { "_id": "$name" }}
]
}}
])
Say, I have following documents:
{name: 'A', fav_fruits: ['apple', 'mango', 'orange'], 'type':'test'}
{name: 'B', fav_fruits: ['apple', 'orange'], 'type':'test'}
{name: 'C', fav_fruits: ['cherry'], 'type':'test'}
I am trying to query to find the total count of fav_fruits field on overall documents returned by :
cursor = db.collection.find({'type': 'test'})
I am expecting output like:
cursor.count() = 3 // Getting
Without much idea of aggregate, can mongodb aggregation framework help me achieve this in any way:
1. sum up the lengths of all 'fav_fruits' field: 6
and/or
2. unique 'fav_fruit' field values = ['apple', 'mango', 'orange', 'cherry']
You need to $project your document after the $match stage and use the $size operator which return the number of items in each array. Then in the $group stage you use the $sum accumulator operator to return the total count.
db.collection.aggregate([
{ "$match": { "type": "test" } },
{ "$project": { "count": { "$size": "$fav_fruits" } } },
{ "$group": { "_id": null, "total": { "$sum": "$count" } } }
])
Which returns:
{ "_id" : null, "total" : 6 }
To get unique fav_fruits simply use .distinct()
> db.collection.distinct("fav_fruits", { "type": "test" } )
[ "apple", "mango", "orange", "cherry" ]
Do this to get just the number of fruits in the fav_fruits array:
db.fruits.aggregate([
{ $match: { type: 'test' } },
{ $unwind: "$fav_fruits" },
{ $group: { _id: "$type", count: { $sum: 1 } } }
]);
This will return the total number of fruits.
But if you want to get the array of unique fav_fruits along with the total number of elements in the fav_fruits field of each document, do this:
db.fruits.aggregate([
{ $match: { type: 'test' } },
{ $unwind: "$fav_fruits" },
{ $group: { _id: "$type", count: { $sum: 1 }, fav_fruits: { $addToSet: "$fav_fruits" } } }
])
You can try this. It may helpful to you.
db.collection.aggregate([{ $match : { type: "test" } }, {$group : { _id : null, count:{$sum:1} } }])
Here's my problem:
Model:
{ application: "abc", date: Time.now, status: "1" user_id: [ id1, id2,
id4] }
{ application: "abc", date: Time.yesterday, status: "1", user_id: [
id1, id3, id5] }
{ application: "abc", date: Time.yesterday-1, status: "1", user_id: [
id1, id3, id5] }
I need to count the unique number of user_ids in a period of time.
Expected result:
{ application: "abc", status: "1", unique_id_count: 5 }
I'm currently using the aggregation framework and counting the ids outside mongodb.
{ $match: { application: "abc" } }, { $unwind: "$users" }, { $group:
{ _id: { status: "$status"},
users: { $addToSet: "$users" } } }
My arrays of users ids are very large, so I have to iterate the dates or I'll get the maximum document limit (16mb).
I could also $group by
{ year: { $year: "$date" }, month: { $month: "$date" }, day: {
$dayOfMonth: "$date" }
but I also get the document size limitation.
Is it possible to count the set size in mongodb?
thanks
The following will return number of uniqueUsers per application. This will apply an group operation to a result of a group operation by using pipeline feature of mongodb.
{ $match: { application: "abc" } },
{ $unwind: "$users" },
{ $group: { _id: "$status", users: { $addToSet: "$users" } } },
{ $unwind:"$users" },
{ $group : {_id : "$_id", count : {$sum : 1} } }
Hopefully this will be done in an easier way in the following releases of mongo by a command which gives the size of an array under a projection. {$project: {id: "$_id", count: {$size: "$uniqueUsers"}}}
https://jira.mongodb.org/browse/SERVER-4899
Cheers
Sorry I'm a little late to the party. Simply grouping on the 'user_id' and counting the result with a trivial group works just fine and doesn't run into doc size limits.
[
{$match: {application: 'abc', date: {$gte: startDate, $lte: endDate}}},
{$unwind: '$user_id'},
{$group: {_id: '$user_id'}},
{$group: {_id: 'singleton', count: {$sum: 1}}}
];
Use $size to get the size of set.
[
{
$match: {"application": "abc"}
},
{
$unwind: "$user_id"
},
{
$group: {
"_id": "$status",
"application": "$application",
"unique_user_id": {$addToSet: "$user_id"}
}
},
{
$project:{
"_id": "$_id",
"application": "$application",
"count": {$size: "$unique_user_id"}
}
}
]