MongoDB calculate count on date grouping - mongodb

I want to calculate the allotment count of each day, by using day grouping on assigned_on I am able to get assignment count of each day, but those are unique count of each day.
In that count I also want that bed to be included which was assigned yesterday or few days before but not yet released.
For example, I have following records
{
"assigned_on":ISODate("2015-12-01T00:00:00Z"),
"released_on":ISODate("2015-12-01T14:01:23Z"),
"bed_id":1
},
{
"assigned_on":ISODate("2015-12-01T00:00:00Z"),
"released_on":ISODate("2015-12-04T14:01:23Z"),
"bed_id":2
},
{
"assigned_on":ISODate("2015-12-01T00:00:00Z"),
"released_on":ISODate("2015-12-01T14:01:23Z"),
"bed_id":3
},
{
"assigned_on":ISODate("2015-12-02T00:00:00Z"),
"released_on":ISODate("2015-12-02T14:01:23Z"),
"bed_id":1
},
{
"assigned_on":ISODate("2015-12-02T00:00:00Z"),
"released_on":ISODate("2015-12-02T14:01:23Z"),
"bed_id":3
},
{
"assigned_on":ISODate("2015-12-03T00:00:00Z"),
"released_on":ISODate("2015-12-03T14:01:23Z"),
"bed_id":1
},
{
"assigned_on":ISODate("2015-12-03T00:00:00Z"),
"released_on":ISODate("2015-12-03T14:01:23Z"),
"bed_id":3
}
Current query
db.test.aggregate([
{
"$match": {
"assigned_on": {
"$gte": ISODate("2015-12-01T00:00:00Z"),
"$lt": ISODate("2015-12-03T23:59:59Z")
}
}
},
{
"$group": {
"_id": {
"$dayOfMonth": "$assigned_on"
},
"Count": {
"$sum": 1
}
}
}
])
As by day grouping on assigned_on I get above result for day 1, 2 and 3, but I want to the Count for day 1, 2 and 3 as 3 for each in result because in second record the released_on date is 4th december which means that bed 2 was occupied on day 1, 2, 3 and 4.
Current output :
{ "_id" : 3, "Count" : 2 }
{ "_id" : 2, "Count" : 2 }
{ "_id" : 1, "Count" : 3 }
Expected output :
{ "_id" : 3, "Count" : 3 }
{ "_id" : 2, "Count" : 3 }
{ "_id" : 1, "Count" : 3 }
Edit : The _id are the dates that is 1st December, 2nd December and 3rd December and count are the number of beds allotted on respective days
An help or pointer will be very helpful

You can do it with the mongodb mapReduce:
var map = function(){
var startDate = new Date(this.assigned_on.getTime());
//set time to midnight
startDate.setHours(0,0,0,0);
//foreach date in date range [assigned_on, released_on) emit date with value(count) 1
for (var date = startDate; date < this.released_on; date.setDate(date.getDate() + 1)) {
if(this.bed_id) {
emit(date, 1);
}
}
};
//calculate total count foreach emitted date(key)
var reduce = function(key, values){
return Array.sum(values)
};
db.collection.mapReduce(map, reduce, {out : {inline : 1}}, callback);
For your data I got such result:
[ { _id: Tue Dec 01 2015 02:00:00 GMT+0200 (EET), value: 3 },
{ _id: Wed Dec 02 2015 02:00:00 GMT+0200 (EET), value: 3 },
{ _id: Thu Dec 03 2015 02:00:00 GMT+0200 (EET), value: 3 },
{ _id: Fri Dec 04 2015 02:00:00 GMT+0200 (EET), value: 1 } ]

Related

Convert month from number to string question in Mongodb query

I am trying to get some avg number per month in the financial year. The collection is called test and the month data comes from CreateDate field. I want to get the avg price per month. The collection data is like below:
{
"_id" : ObjectId("5fd289a93f7cf02c36837ca7"),
"ClientName" : "John",
"OrderNumber" : "12345A",
"Price" : 10,
"CreateDate" : ISODate("2020-09-20T06:00:00.000Z"),
}
{
"_id" : ObjectId("5fd289a93f7cf02c36837cc7"),
"ClientName" : "John",
"OrderNumber" : "12345",
"Price" : 20,
"CreateDate" : ISODate("2020-09-12T06:00:00.000Z"),
}
So I am writing the query to get the avg number per month by the following within the financial year (from Sep to Aug):
db.test.aggregate([
{
$match: {
"CreateDate": {
$lt: ISODate("2021-08-31T00:00:00.000Z"),
$gte: ISODate("2020-09-01T00:00:00.000Z")
}
}
},
{
$group: {
_id: {$month: "$CreateDate"},
"AvgPrice": {
"$avg": "$Price",
}
}
},
{ $project:{ _id : 0 , Month: '$_id' , "AvgPrice ": '$AvgPrice' } }
])
The result I am getting is with the following format:
{
"Month" : 9,
"AvgPrice " : 15.0
}
{
"Month" : 10,
"AvgPrice " : 18.6666666666667
}
How can I display of the month converting to a string instead of the number. For example, the following is the ideal return:
{
"Month" : Sep,
"AvgPrice" : 15.0
}
{
"Month" : Oct,
"AvgPrice" : 18.6666666666667
}
I also have two more questions:
I am using the Mongodb 3.6 version, is there any way to round up the avg price to two digit after the decimal point? For example, above will be "18.67" instead of "18.66666". Mongo 4.2 has something called $round but 3.6 seems doesn't have this function.
If I want to break down by client, has the returning result like below:
{
"ClientName": "John",
"Month" : Sep,
"AvgPrice" : 15.0
}
{
"ClientName" : "Mary"
"Month" : Oct,
"AvgPrice" : 18.6666666666667
}
How do I put another level of the group to breakdown to the client level and then month level?
Any help will be appreciated!
If I want to break down by client
You can add ClientName field in _id,
{
$group: {
_id: {
ClientName: "$ClientName",
month: { $month: "$CreateDate" }
},
AvgPrice: { $avg: "$Price" }
}
},
How can I display of the month converting to a string instead of the number.
There is no any straight way to get month name in mongodb, but if you prepare array of months in string and access it by index,
$arrayElemAt to select month by its number
{
$project: {
_id: 0,
ClientName: "$_id.ClientName",
Month: {
$arrayElemAt: [
["","Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"],
"$_id.month"
]
},
AvgPrice: 1
}
}
Playground
I am using the Mongodb 3.6 version, is there any way to round up the avg price to two digit after the decimal point?
There is no any option in mongodb 3.6 or below, you already know there is a option $round in mongodb 4.2.
You can refer this question Rounding to 2 decimal places using MongoDB aggregation framework
, there are many tricks.

Mongodb $gte date query not working as expected

I have collection with Below data. While I am using
db.collection.find({ endDate: { $gte: new Date() } })
it's not showing result of current date which is ISODate("2018-07-06T14:59:08.794+0000").
{
"_id" : "GMDJcQMfs8j8EP9EE",
"endDate" : ISODate("2018-07-06T14:59:08.794+0000")
}
{
"_id" : "GMDJcQMfs12233",
"endDate" : ISODate("2020-02-21T00:00:00.000+0000")
}
{
"_id" : "GMDJerrr8j8EP9EE",
"endDate" : ISODate("2020-02-21T00:00:00.000+0000")
}
{
"_id" : "rrrJcQMfs8j8EP9EE",
"endDate" : ISODate("2020-02-21T00:00:00.000+0000")
}
You have to consider the time of the day. You can set the hours, minutes, seconds and milliseconds of the requested date back to zero.
var date = new Date()
date.setHours(0, 0, 0, 0)
db.collection.find({ endDate: { $gte: date } })
Now all records ending today and later will be returned.
The condition you have quoted is $gte and the date you are passing for to the condition is newDate().
print(new Date()) - Execute this command in Mongo shell it Should be giving you the current date
My Console output
Fri Jul 06 2018 14:28:54 GMT+0530 (India Standard Time)
If your date is also Jul 06, 2018 then you also wont get the Jul 05, 2018 in your results
Ok, i get back on my answer to your previous question and explain it :
db.test1.aggregate(
[
{
$project: {
endDate:1,
endDateFormatted:{$dateToString: {date:"$endDate",format:"%Y-%m-%d"}},
current:{$dateToString: {date:new Date(),format:"%Y-%m-%d"}}
}
},
{
$project: { ab: {$cmp: ['$endDateFormatted','$current']},endDate:1}
},
{
$match: {ab:{$eq:1}} // <= note de difference with your previous question
},
{
$project: {
endDate:1
}
},
]
);
Explanations :
$dateToString transforms provided ISOdate in provided format, ie
{$dateToString: {date:ISODate("2020-02-21T00:00:00.000+0000"),format:"%Y-%m-%d"}} outputs "2020-02-01"
$cmp compare first value A with second B, with following result :
A -1
A=B => 0
A>B => 1
just adapt ab criteria to what you need ({$eq:1} for future dates, {$eq:-1} for past dates, {$eq:0} for current date, {$ne:0} for not today dates, etc... )

Aggregate trunc date function?

I worked with a bunch of SQL databases before; like Postgres and BigQuery and they have date truncation function (for instance: date_trunc or TIMESTAMP_TRUNC ).
I wonder if mongodb has a DATE_TRUNC function?
I have found the $trunc operator but it works for numbers only.
I want a DATE_TRUNC function to truncate a given Date (the timestamp type in other SQL databases) to a particular boundary, like beginning of year, beginning of month, beginning of hour, may be ok to compose a new Date by getting its year, month, date, hour.
Does someone have some kinds of workaround? Especially for beginning moment of WEEK, and beginning of ISOWEEK, does anyone have a good workaround?
Its possible to get the start of ISO week by doing arithmetic on date or timestamp field, here the start of week is Monday (1) and end of week is Sunday (7)
db.dd.aggregate(
[
{
$addFields : {
startOfWeek : 1, // Monday
currentDayOfWeek : {$dayOfWeek : "$date"},
daysToMinus : { $subtract : [{$dayOfWeek : "$date"} , 1] },
startOfThisWeek : { $subtract : [ "$date", {$multiply : [{ $subtract : [{$dayOfWeek : "$date"} , 1 ] }, 24, 60, 60, 1000 ] } ] }
}
}
]
).pretty()
document
> db.dd.find()
{ "_id" : ObjectId("5a62e2697702c6be61d672f4"), "date" : ISODate("2018-01-20T06:32:09.157Z") }
start of week
{
"_id" : ObjectId("5a62e2697702c6be61d672f4"),
"date" : ISODate("2018-01-20T06:32:09.157Z"),
"startOfWeek" : 1,
"currentDayOfWeek" : 7,
"daysToMinus" : 6,
"startOfThisWeek" : ISODate("2018-01-14T06:32:09.157Z")
}
>
It's possible to truncate date to iso week with $dateFromParts function:
For example
db.dd.aggregate(
{
$dateFromParts: {
isoWeekYear: { $isoWeekYear: "$date" },
isoWeek: { $isoWeek: "$date" }
}
}
)
For Fri, 22 Jun 2018 20:46:50 UTC +00:00 it returns Fri, 18 Jun 2018 00:00:00 UTC +00:00.
To truncate to hour, day, month, etc. it's easier to use $dateFromString and $dateToString. The following example truncated date to hour:
db.dd.aggregate(
{
$dateFromString: {
dateString: {
$dateToString: {
format: '%Y-%m-%dT%H:00:00+00:00',
date: '$date'
}
}
}
}
)
Can be combined $dateToParts and $dateFromParts
For year, month, day, hour, minute:
db.getCollection("data").aggregate([
{"$addFields": {
"dateVarFull": {"$dateToParts": {date: {"$toDate" : "2020-08-27T13:00:00Z"}}}
}},
{"$addFields": {
"dateVarTrunc": { "$dateFromParts": {
'year': "$dateVarFull.year",
'month': "$dateVarFull.month",
'day': "$dateVarFull.day"
}}
}}
])
Result:
{
"dateVarFull" : {
"year" : NumberInt(2020),
"month" : NumberInt(8),
"day" : NumberInt(27),
"hour" : NumberInt(13),
"minute" : NumberInt(0),
"second" : NumberInt(0),
"millisecond" : NumberInt(0)
},
"dateVarTrunc" : ISODate("2020-08-27T00:00:00.000+0000")
}
For week trunc use iso8601: true parameter:
db.getCollection("data").aggregate([
{"$addFields": {
"dateVarFull": {
"$dateToParts": {
date: {"$toDate" : "2020-08-27T13:00:00Z"},
iso8601: true
}
}
}},
{"$addFields": {
"dateVarTrunc": { "$dateFromParts": {
'isoWeekYear': "$dateVarFull.isoWeekYear",
'isoWeek': "$dateVarFull.isoWeek",
'isoDayOfWeek': 1
}}
}}
])
Result:
{
"dateVarFull" : {
"isoWeekYear" : NumberInt(2020),
"isoWeek" : NumberInt(35),
"isoDayOfWeek" : NumberInt(4),
"hour" : NumberInt(13),
"minute" : NumberInt(0),
"second" : NumberInt(0),
"millisecond" : NumberInt(0)
},
"dateVarTrunc" : ISODate("2020-08-24T00:00:00.000+0000")
}
Starting in Mongo 5, your wish has been granted with the $dateTrunc operator.
For instance, to truncate dates to their year:
// { date: ISODate("2021-12-05T13:20:56Z") }
// { date: ISODate("2019-04-27T05:00:32Z") }
db.collection.aggregate([
{ $project: { year: { $dateTrunc: { date: "$date", unit: "year" } } } }
])
// { year: ISODate("2021-01-01T00:00:00Z") }
// { year: ISODate("2019-01-01T00:00:00Z") }
You can truncate at different levels of units (year, months, day, hours, ... even quarters) using the unit parameter. And for a given unit at different multiples of units (for instance 3 years, 6 months, ...) using the binSize parameter.
And you can also specify the day at which weeks start:
// { date: ISODate("2021-12-05T13:20:56Z") } <= Sunday
// { date: ISODate("2021-12-06T05:00:32Z") } <= Monday
db.collection.aggregate([
{ $project: {
week: { $dateTrunc: { date: "$date", unit: "week", startOfWeek: "monday" } }
}}
])
// { week: ISODate("2021-11-29T00:00:00Z") }
// { week: ISODate("2021-12-06T00:00:00Z") }

MongoDB - aggregate by date, right-aligned boundaries

I have some data in MongoDB that contains 10-minutely period-to-date sums:
db.test.insert({perEnd: ISODate('2013-06-05T18:00:00'), val: 7.3})
db.test.insert({perEnd: ISODate('2013-06-05T18:10:00'), val: 6.23})
db.test.insert({perEnd: ISODate('2013-06-05T18:20:00'), val: 4.1})
db.test.insert({perEnd: ISODate('2013-06-05T18:30:00'), val: 0.21})
db.test.insert({perEnd: ISODate('2013-06-05T18:40:00'), val: 12.1})
db.test.insert({perEnd: ISODate('2013-06-05T18:50:00'), val: 6.0})
db.test.insert({perEnd: ISODate('2013-06-05T19:00:00'), val: 8.9})
db.test.insert({perEnd: ISODate('2013-06-05T19:10:00'), val: .98})
db.test.insert({perEnd: ISODate('2013-06-05T19:20:00'), val: 14.7})
I would like to aggregate to find sums for each hour-ending period, so I should get the following values:
ending 2013-06-05 18:00:00 - 7.3
ending 2013-06-05 19:00:00 - 37.54
ending 2013-06-05 20:00:00 - 15.68
Using the built-in date operators doesn't work, because they round (truncate) all dates down to the nearest boundary, and I need to round up:
> db.test.aggregate({$group: {_id: {Year: {$year: "$perEnd"},
Day: {$dayOfYear: "$perEnd"},
Hour: {$hour: "$perEnd"}},
sum: {$sum: "$val"}}})
{
"result" : [
{ "_id" : { "Year" : 2013,
"Day" : 156,
"Hour" : 19 },
"sum" : 24.58 },
{ "_id" : { "Year" : 2013,
"Day" : 156,
"Hour" : 18 },
"sum" : 35.940000000000005 }
],
"ok" : 1
}
Anyone see a way to achieve this with decent performance?
You can do it using mongodb map-reduce:
var map = function(){
var date = new Date(this.perEnd.getTime());
if(date.getMinutes() > 0){
date.setHours(date.getHours() + 1, 0, 0, 0);
} else {
date.setHours(date.getHours(), 0, 0, 0);
}
emit(date, this.val);
};
var reduce = function(key, values){
return Array.sum(values)
};
db.collection.mapReduce(map, reduce, {out : {inline : 1}}, callback);
For your data I got the following result:
[ { _id: Wed Jun 05 2013 21:00:00 GMT+0300 (EEST), value: 7.3 },
{ _id: Wed Jun 05 2013 22:00:00 GMT+0300 (EEST), value: 37.54 },
{ _id: Wed Jun 05 2013 23:00:00 GMT+0300 (EEST), value: 15.68 } ]

Find all documents within last n days

My daily collection has documents like:
..
{ "date" : ISODate("2013-01-03T00:00:00Z"), "vid" : "ED", "san" : 7046.25, "izm" : 1243.96 }
{ "date" : ISODate("2013-01-03T00:00:00Z"), "vid" : "UA", "san" : 0, "izm" : 0 }
{ "date" : ISODate("2013-01-03T00:00:00Z"), "vid" : "PAL", "san" : 0, "izm" : 169.9 }
{ "date" : ISODate("2013-01-03T00:00:00Z"), "vid" : "PAL", "san" : 0, "izm" : 0 }
{ "date" : ISODate("2013-01-03T00:00:00Z"), "vid" : "CTA_TR", "san" : 0, "izm" : 0 }
{ "date" : ISODate("2013-01-04T00:00:00Z"), "vid" : "CAD", "san" : 0, "izm" : 169.9 }
{ "date" : ISODate("2013-01-04T00:00:00Z"), "vid" : "INT", "san" : 0, "izm" : 169.9 }
...
I left off _id field to spare the space here.
My task is to "fetch all documents within last 15 days". As you can see I need somehow to:
Get 15 unique dates. The newest one should be taken as the newest document in collection (what I mean that it isn't necessary the today's date, it's just the latest one in collection based on date field), and the oldest.. well, maybe it's not necessary to strictly define the oldest day in query, what I need is some kind of top15 starting from the newest day, if you know what I mean. Like 15 unique days.
db.daily.find() all documents, that have date field in that range of 15 days.
In the result, I should see all documents within 15 days starting from the newest in collection.
I just tested the following query against your data sample and it worked perfectly:
db.datecol.find(
{
"date":
{
$gte: new Date((new Date().getTime() - (15 * 24 * 60 * 60 * 1000)))
}
}
).sort({ "date": -1 })
Starting in Mongo 5, it's a nice use case for the $dateSubtract operator:
// { date: ISODate("2021-12-05") }
// { date: ISODate("2021-12-02") }
// { date: ISODate("2021-12-02") }
// { date: ISODate("2021-11-28") } <= older than 5 days
db.collection.aggregate([
{ $match: {
$expr: {
$gt: [
"$date",
{ $dateSubtract: { startDate: "$$NOW", unit: "day", amount: 5 } }
]
}
}}
])
// { date: ISODate("2021-12-05") }
// { date: ISODate("2021-12-02") }
// { date: ISODate("2021-12-02") }
With $dateSubtract, we create the oldest date after which we keep documents, by subtracting 5 (amount) "days" (unit) out of the current date $$NOW (startDate).
And you can obviously add a $sort stage to sort documents by date.
You need to run the distinct command to get all the unique dates. Below is the example. The "values" array has all the unique dates of the collection from which you need to retrieve the most recent 15 days on the client side
db.runCommand ( { distinct: 'datecol', key: 'date' } )
{
"values" : [
ISODate("2013-01-03T00:00:00Z"),
ISODate("2013-01-04T00:00:00Z")
],
"stats" : {
"n" : 2,
"nscanned" : 2,
"nscannedObjects" : 2,
"timems" : 0,
"cursor" : "BasicCursor"
},
"ok" : 1
}
You then use the $in operator with the most recent 15 dates from step 1. Below is an example that finds all documents that belong to one of the mentioned two dates.
db.datecol.find({
"date":{
"$in":[
new ISODate("2013-01-03T00:00:00Z"),
new ISODate("2013-01-04T00:00:00Z")
]
}
})