MongoDB, Grouping by Multiple Fields - mongodb

pretty new to Mongo and am finding some simple things that i would do in SQL frustratingly difficult in Mongo.
I have an object similar to this below
[{
"_id" : ObjectId("5870fb29a1fe030e1a2909db"),
"updatedAt" : ISODate("2017-01-07T14:28:57.224Z"),
"createdAt" : ISODate("2017-01-07T14:28:57.224Z"),
"state" : "Available",
},
{
"_id" : ObjectId("5870fb29a1fe030e1a2909dc"),
"updatedAt" : ISODate("2017-01-07T14:28:57.224Z"),
"createdAt" : ISODate("2017-01-07T14:28:57.224Z"),
"state" : "notReady",
},
{
"_id" : ObjectId("5870fb29a1fe030e1a2909d9"),
"updatedAt" : ISODate("2017-01-07T14:28:57.224Z"),
"createdAt" : ISODate("2017-01-07T14:28:57.224Z"),
"state" : "Disconnected",
}]
What i'm looking to do it group the data by the Maximum date and the state.
Ideally the result i would be looking for would be something like the following.
{
latestDate: "2017-01-07T14:28:57",
states : {
available : 10,
disconnected : 5,
notReady : 2
}}
Basically i'm looking for the SQL equivalent of this:
SELECT createdAt, state, COUNT(rowid)
FROM db
WHERE date = (SELECT MAX(createdAt) FROM db)
GROUP BY 1,2
I've searched around here and have found some good info but am probably missing something straight forward. Ive only managed to get here so far
db.collection.aggregate([
{$project: {"_id" : 0,"state": 1, "date" : "$createdAt"}},
{$group : {"_id" : {"date":"$date", "state": "actual"}, "count":{"$sum":1}}}
])
Any help would be appreciated :)

db.collection.aggregate([
{
$group : {
_id : {
date : "$createdAt",
state : "$state"
},
count : {$sum : 1}
}
},
{
$group : {
_id : "$_id.date",
states : {
$addToSet : {
state : "$_id.state",
count : "$count"
}
}
}
},
{
$sort : {_id : -1}
},
{
$limit : 1
},
{
$project : {
_id : 0,
latestDate : "$_id",
states : "$states"
}
}
])
output :
{
"latestDate" : ISODate("2017-01-07T14:28:57.224Z"),
"states" : [
{
"state" : "Available",
"count" : 1
},
{
"state" : "notReady",
"count" : 1
},
{
"state" : "Disconnected",
"count" : 1
}
]
}

Related

Mongo aggregation groups and subgroup

Hi I have a Mongo aggregation:
[
{
"$match" : {
"dateTime" : {
"$gte" : ISODate("2017-01-01T00:00:00.000+0000"),
"$lt" : ISODate("2018-01-01T00:00:00.000+0000")
}
}
},
{
"$group" : {
"_id" : "dateTime",
"totals" : {
"$sum" : "$payment.totalAmount"
},
"count" : {
"$sum" : 1.0
}
}
}
],
{
"allowDiskUse" : false
}
);
This works fine. It aggregates, and sums by date range I supplied and I get an output as follows.
{
"_id" : "dateTime",
"totals" : 2625293.825017198,
"count" : 12038.0
}
However, I also want to further refine the groupings.
I have a field called 'companyId' and I want to calculate the sum and count by each company Id for the given time range.
I would like to get an output similar to this, where I get a sum and count for each company ID in the date range I queried, not just a sum/count of all the data:
[
{
"companyId" : "Acme Co",
"totals" : 2625293.825017198,
"count" : 12038.0
},
{
"companyId" : "Beta Co",
"totals" : 162593.82198,
"count" : 138.0
},
{
"companyId" : "Cel Co",
"totals" : 593.82,
"count" : 38.0
}
]
How do I do this? I have not been able to find a good example online.
Thanks

Find sum of fields inside array in MongoDB

I have a data as follows:
> db.PQRCorp.find().pretty()
{
"_id" : 0,
"name" : "Ancy",
"results" : [
{
"evaluation" : "term1",
"score" : 1.463179736705023
},
{
"evaluation" : "term2",
"score" : 11.78273309957772
},
{
"evaluation" : "term3",
"score" : 6.676176060654615
}
]
}
{
"_id" : 1,
"name" : "Mark",
"results" : [
{
"evaluation" : "term1",
"score" : 5.89772766299929
},
{
"evaluation" : "term2",
"score" : 12.7726680028769
},
{
"evaluation" : "term3",
"score" : 2.78092882672992
}
]
}
{
"_id" : 2,
"name" : "Jeff",
"results" : [
{
"evaluation" : "term1",
"score" : 36.78917882992872
},
{
"evaluation" : "term2",
"score" : 2.883687879200287
},
{
"evaluation" : "term3",
"score" : 9.882668212003763
}
]
}
What I want to achieve is ::Find employees who failed in aggregate (term1 + term2 + term3)
What I am doing and eventually getting is:
db.PQRCorp.aggregate([
{$unwind:"$results"},
{ $group: {_id: "$id",
'totalTermScore':{ $sum:"$results.score" }
}
}])
OUTPUT:{ "_id" : null, "totalTermScore" : 90.92894831067625 }
Simply I am getting a output of a flat sum of all scores. What I want is, to sum terms 1 , 2 and 3 separately for separate employees.
Please can someone help me. I am new to MongoDB (quite evident though).
You do not need to use $unwind and $group here... A simple $project query can $sum your entire score...
db.PQRCorp.aggregate([
{ "$project": {
"name": 1,
"totalTermScore": {
"$sum": "$results.score"
}
}}
])

Mongodb find data and groupby for another column

{
"_id" : ObjectId("5763e4d6c0140edcb8731485"),
"_class" : "net.microservice.product.domain.Product",,
"createdAt" : ISODate("2016-06-17T11:53:58.228Z"),
"createdBy" : "user-0",
"modifiedAt" : ISODate("2016-06-21T06:21:47.524Z"),
"modifiedBy" : "user-0",
"merchant" : "a746f24safa5-e96f-4281-9759-a4a02b306d77",
"type" : DBRef("productTypes", ObjectId("575fd99236623f70c959247f")),
"fields" : {
"Image4" : {
"value" : "http://i.hizliresim.com/ZdELXa.jpg",
"detail" : {
"revisedBy" : "CTA",
"revisionDate" : ISODate("2016-06-21T06:21:47.204Z")
}
},
"Image3" : {
"value" : "http://i.hizliresim.com/l1WkqX.jpg",
"detail" : {
"revisedBy" : "CTA",
"revisionDate" : ISODate("2016-06-21T06:21:47.204Z")
}
},
"Image2" : {
"value" : "http://i.hizliresim.com/VYMl9n.jpg",
"detail" : {
"revisedBy" : "CTA",
"revisionDate" : ISODate("2016-06-21T06:21:47.204Z")
}
},
"Kur" : {
"value" : "TL",
"detail" : {
"revisedBy" : "CTA",
"revisionDate" : ISODate("2016-06-21T06:21:47.204Z")
}
},
"Image1" : {
"value" : "http://i.hizliresim.com/nrWAQ0.jpg",
"detail" : {
"revisedBy" : "CTA",
"revisionDate" : ISODate("2016-06-21T06:21:47.204Z")
}
},
"uploadDate" : ISODate("2016-06-17T11:53:00Z"),
"tasks" : [ ]
}
this is sample of database. I want to get data in which:
- modifiedAt is before "modifiedAt" : ISODate("2016-07-21T06:21:47.524Z"),
so i do this and this works:
db.products.find({
'modifiedAt':
{$lte: ISODate("2016-10-18T13:05:18.961Z"
)} }).
count()
14999
But i need to find for each merchant. Beause 14999 result is not true because a merchant have lots of product so 14999 includes multiple products.
I need to group by merchant and distinct. I couldnot do it.
i do this but
db.products.
aggregate([ {
$group: {
_id: '$merchant', } }, {
$match: {
modifiedAt:
{$lte: ISODate("2016-06-18T13:05:18.961Z")} }} ])
brings nothing and no error.
you can try something like this. This gives you the number of products by merchant.
db.products.aggregate([
{$match: {modifiedAt:{$lte: ISODate("2016-06-21T06:21:47.524Z")}}},
{$group: { _id: "$merchant",count: { $sum: 1 }}}
])
Output:
{ "_id" : "a89846f24safa5-e96f-4281-9759-a4a02b306d77", "count" : 1 }
Always place the $match as early in the aggregation pipeline as possible. Because $match limits the total number of documents in the aggregation pipeline, earlier $match operations minimize the amount of processing down the pipe.
So your query would be like
db.products.aggregate([
{
$match: {
modifiedAt: {
$lte: ISODate("2016-06-18T13:05:18.961Z")
}
}
},
{
$group: {
_id: '$merchant'
}
}
])

Mongodb sort by sum of keys

I have a json document
{
{
"_id" : ObjectId("5715c4bbac530eb3018b456a"),
"content_id" : "5715c4bbac530eb3018b4569",
"views" : NumberLong(200),
"likes" : NumberLong(100),
"comments" : NumberLong(0)
},
{
"_id" : ObjectId("5715c4bbac530eb3018b4568"),
"content_id" : "5715c4bbac530eb3018b4567",
"views" : NumberLong(300),
"likes" : NumberLong(200),
"comments" : NumberLong(0)
},
{
"_id" : ObjectId("5715c502ac530ee5018b4956"),
"content_id" : "5715c502ac530ee5018b4955",
"views" : NumberLong(500),
"likes" : NumberLong(0),
"comments" : NumberLong(200)
}
}
How can we sort the document order by SUM("views", "likes", "comments")
something like in mysql
SELECT SUM(key1, key2, key3) AS key
FROM document
ORDER BY key
Thanks in advance.
First do a projection to obtain the sum of all the likes, views and comments, then sort based on that sum. I am considering group by content_id if is needed in the second snippet
db.test.aggregate([
{ $project : { "_id" : "$content_id", "total" : { $add : [ "$likes", "$views", "$comments"]}}},
{ $sort : { "total" : 1 }}
])
If you need a group operation if content_id can be duplicated
db.test.aggregate([
{ $project : { "_id" : "$content_id", "total" : { $add : [ "$likes", "$views", "$comments"]}}},
{ $group : { "_id" : "$_id" , totalPerId : { $sum : "$total" }}},
{ $sort : { "total" : 1 }}
])
Based on your test data, you will get:
{ "_id" : "5715c502ac530ee5018b4955", "totalPerId" : NumberLong(700) }
{ "_id" : "5715c4bbac530eb3018b4567", "totalPerId" : NumberLong(500) }
{ "_id" : "5715c4bbac530eb3018b4569", "totalPerId" : NumberLong(300) }

MongoDB: Sort in combination with Aggregation group

I have a collection called transaction with below documents,
/* 0 */
{
"_id" : ObjectId("5603fad216e90d53d6795131"),
"statusId" : "65c719e6727d",
"relatedWith" : "65c719e67267",
"status" : "A",
"userId" : "100",
"createdTs" : ISODate("2015-09-24T13:15:36.609Z")
}
/* 1 */
{
"_id" : ObjectId("5603fad216e90d53d6795134"),
"statusId" : "65c719e6727d",
"relatedWith" : "65c719e6726d",
"status" : "B",
"userId" : "100",
"createdTs" : ISODate("2015-09-24T13:14:31.609Z")
}
/* 2 */
{
"_id" : ObjectId("5603fad216e90d53d679512e"),
"statusId" : "65c719e6727d",
"relatedWith" : "65c719e6726d",
"status" : "C",
"userId" : "100",
"createdTs" : ISODate("2015-09-24T13:13:36.609Z")
}
/* 3 */
{
"_id" : ObjectId("5603fad216e90d53d6795132"),
"statusId" : "65c719e6727d",
"relatedWith" : "65c719e6726d",
"status" : "D",
"userId" : "100",
"createdTs" : ISODate("2015-09-24T13:16:36.609Z")
}
When I run the below Aggregation query without $group,
db.transaction.aggregate([
{
"$match": {
"userId": "100",
"statusId": "65c719e6727d"
}
},
{
"$sort": {
"createdTs": -1
}
}
])
I get the result in expected sorting order. i.e Sort createdTs in descending order (Minimal result)
/* 0 */
{
"result" : [
{
"_id" : ObjectId("5603fad216e90d53d6795132"),
"createdTs" : ISODate("2015-09-24T13:16:36.609Z")
},
{
"_id" : ObjectId("5603fad216e90d53d6795131"),
"createdTs" : ISODate("2015-09-24T13:15:36.609Z")
},
{
"_id" : ObjectId("5603fad216e90d53d6795134"),
"createdTs" : ISODate("2015-09-24T13:14:31.609Z")
},
{
"_id" : ObjectId("5603fad216e90d53d679512e"),
"createdTs" : ISODate("2015-09-24T13:13:36.609Z")
}
],
"ok" : 1
}
If I apply the below aggregation with $group, the resultant is inversely sorted(i.e Ascending sort)
db.transaction.aggregate([
{
"$match": {
"userId": "100",
"statusId": "65c719e6727d"
}
},
{
"$sort": {
"createdTs": -1
}
},
{
$group: {
"_id": {
"statusId": "$statusId",
"relatedWith": "$relatedWith",
"status": "$status"
},
"status": {$first: "$status"},
"statusId": {$first: "$statusId"},
"relatedWith": {$first: "$relatedWith"},
"createdTs": {$first: "$createdTs"}
}
}
]);
I get the result in inverse Order i.e. ** Sort createdTs in Ascending order**
/* 0 */
{
"result" : [
{
"_id" : ObjectId("5603fad216e90d53d679512e"),
"createdTs" : ISODate("2015-09-24T13:13:36.609Z")
},
{
"_id" : ObjectId("5603fad216e90d53d6795134"),
"createdTs" : ISODate("2015-09-24T13:14:31.609Z")
},
{
"_id" : ObjectId("5603fad216e90d53d6795131"),
"createdTs" : ISODate("2015-09-24T13:15:36.609Z")
},
{
"_id" : ObjectId("5603fad216e90d53d6795132"),
"createdTs" : ISODate("2015-09-24T13:16:36.609Z")
}
],
"ok" : 1
}
Where am I wrong ?
The $group stage doesn't insure the ordering of the results. See here the first paragraph.
If you want the results to be sorted after a $group, you need to add a $sort after the $group stage.
In your case, you should move the $sort after the $group and before you ask the question : No, the $sort won't be able to use an index after the $group like it does before the $group :-).
The internal algorithm of $group seems to keep some sort of ordering (reversed apparently), but I would not count on that and add a $sort.
You are not doing anything wrong here, Its a $group behavior in Mongodb
Lets have a look in this example
Suppose you have following doc in collection
{ "_id" : 1, "item" : "abc", "price" : 10, "quantity" : 2, "date" : ISODate("2014-01-01T08:00:00Z") }
{ "_id" : 2, "item" : "jkl", "price" : 20, "quantity" : 1, "date" : ISODate("2014-02-03T09:00:00Z") }
{ "_id" : 3, "item" : "xyz", "price" : 5, "quantity" : 5, "date" : ISODate("2014-02-03T09:05:00Z") }
{ "_id" : 4, "item" : "abc", "price" : 10, "quantity" : 10, "date" : ISODate("2014-02-15T08:00:00Z") }
{ "_id" : 5, "item" : "xyz", "price" : 5, "quantity" : 10, "date" : ISODate("2014-02-15T09:05:00Z") }
{ "_id" : 6, "item" : "xyz", "price" : 5, "quantity" : 5, "date" : ISODate("2014-02-15T12:05:10Z") }
{ "_id" : 7, "item" : "xyz", "price" : 5, "quantity" : 10, "date" : ISODate("2014-02-15T14:12:12Z") }
Now if you run this
db.collection.aggregate([{ $sort: { item: 1,date:1}} ] )
the output will be in ascending order of item and date.
Now if you add group stage in aggregation pipeline it will reverse the order.
db.collection.aggregate([{ $sort: { item: 1,date:1}},{$group:{_id:"$item"}} ] )
Output will be
{ "_id" : "xyz" }
{ "_id" : "jkl" }
{ "_id" : "abc" }
Now the solution for your problem
change "createdTs": -1 to "createdTs": 1 for group