I am trying to group by DayHours in a mongo aggregate function to get the past 24 hours of data.
For example: if the time of an event was 6:00 Friday the "DayHour" would be 6-5.
I'm easily able to group by hour with the following query:
db.api_log.aggregate([
{ '$group': {
'_id': {
'$hour': '$time'
},
'count': {
'$sum':1
}
}
},
{ '$sort' : { '_id': -1 } }
])
I feel like there is a better way to do this. I've tried concatenation in the $project statement, however you can only concatenate strings in mongo(apparently).
I effectively just need to end up grouping by day and hour, however it gets done. Thank You.
I assume that time field contains ISODate.
If you want only last 24 hours you can use this:
var yesterday = new Date((new Date).setDate(new Date().getDate() - 1));
db.api_log.aggregate(
{$match: {time: {$gt: yesterday}}},
{$group: {
_id: {
hour: {$hour: "$time"},
day: {$dayOfMonth: "$time"},
},
count: {$sum: 1}
}}
)
If you want general grouping by day-hour you can use this:
db.api_log.aggregate(
{$group: {
_id: {
hour: {$hour: "$time"},
day: {$dayOfMonth: "$time"},
month: {$month: "$time"},
year: {$year: "$time"}
},
count: {$sum: 1}
}}
)
Also this is not an answer per se (I do not have mongodb now to come up with the answer), but I think that you can not do this just with aggregation framework (I might be wrong, so I will explain myself).
You can obtain date and time information from mongoId using .getTimestamp method. The problem that you can not output this information in mongo query (something like db.find({},{_id.getTimestamp}) does not work). You also can not search by this field (except of using $where clause).
So if it is possible to achieve, it can be done only using mapreduce, where in reduce function you group based on the output of getTimestamp.
If this is the query you are going to do quite often I would recommend actually adding date field to your document, because using this field you will be able properly aggregate your data and also you can use indeces not to scan all your collection (like you are doing with $sort -1, but to $match only the part which is bigger then current date - 24 hours).
I hope this can help even without a code. If no one will be able to answer this, I will try to play with it tomorrow.
Related
I have a collection, ledger, with the following document format:
{
_id: ###,
month: 202112,
name: 'XXXXXXXXXXXX',
gross_revenue: 482.28
}
The month actually contains both the year and month, YYYYMM. And there are multiple entries per 'month'. What I'm wanting to do is sum the gross_revenue values across the years. So take a $substr of month to get the year and then sum up gross_revenue.The result would ideally look like this:
2019: 99999.99,
2020: 88888.88,
.
.
I can aggregate for a given month, and I can get the substr, but can't figure out how to do combine them to aggregate by year.
db.ledger.aggregate([ { $match: {month: 202111} },{ $group: { _id: null, total: { $sum: "$gross_revenue" } } } ] )
db.ledger.aggregate([{$project: { year: {$substr:["$month", 0,4]}}}])
Any help is appreciated.
Query
group by year (we can use any aggregation operator to group)
replace root to make the expected output
Test code here
aggregate(
[{"$group":
{"_id":{"$substrCP":["$month", 0, 4]},
"sum":{"$sum":"$gross_revenue"}}},
{"$replaceRoot":
{"newRoot":{"$arrayToObject":[[{"k":"$_id", "v":"$sum"}]]}}}])
I have a collection in MongoDB. Model is:
{
currency: String,
price: Number,
time: Date
}
Documents are recorded to that collection any time the official rate for currency changes.
I am given a timestamp, and I need to fetch rates for all available currencies to that time. So first I need to filter all documents whose time $lte then required, then I need to fetch only those with max timestamps. For each currency.
after seeing your requirement , I think you want max number of price and time , use max operator
db.collection.aggregate(
[
{
$group:
{
_id: "$currency",
time: { $max: "$time"},
price: { $max: "$price" }
}
}
]
)
You can use mongo aggregate function to do so. Please find the example below:
db.<collection_name>.aggregate([
// First sort all the docs by time in descending
{$sort: {time: -1}},
// Take the first 3 of those
{$limit: 3}
])
Hope this helps !!
I want to be able to set a date range and then only retrieve the last record of each month.
I have an idea of how to do it but need to run it past you guys who are much wiser than I at this stuff.
I think I would do do a $group which includes the month and year number from the 'local_date_time'. I would then $sort by year, month and then local_date_time and then in the next stage, $project only the $last record for each.
Is that the best way to do this or is there a cleaner more effective way of doing this? Thanks, Matt
db.users.aggregate({
$match: {
'createdDate': {
$gte: new Date('2015'+'-01-01T00:00:00.000+0530'),
$lte: new Date('2015'+'-12-31T23:59:59.000+0530')
}
},
},{
$group:{
_id : { month: { $month: '$createdDate' }},
lastUserEmail: { $last: "$email" }
}
});
$last : Returns a value from the last document for each group, check This..
I have just checked with createdDate,
You can change the fields acordingly..!
hope, this will help u in some way..!
I have this huge dataset for which every entry has a datetime field. The data was inserted irregularly. For example:
2015-04-20 : 500 entries,
2015-04-23 : 300 entries,
2015-05-01 : 600 entries
The thing is, I do not know when these active days are. What I would like is a mongodb query which returns some sort of array containing all days which occur in the database, like so:
['2015-04-20,
'2015-04-23,
'2015-04-23,
'2015-04-25,
'2015-05-01,
'2015-05-05,
'2015-05-09]
Is this possible, and if so: how can I achieve this?
There is a "distinct" command that has shell wrapper, which can be used something like:
db.collection.distinct(dateFieldName, query)
If you are not running it from shell, check whether your driver wraps this command, if not you can use the command directly:
{ distinct: "<collection>", key: "<field>", query: <query> }
http://docs.mongodb.org/manual/reference/command/distinct/#dbcmd.distinct
If your time stamp field needs some additinal processing, you can use aggregation framework.
db.collection.aggregate([{$group: {_id: $substr: ["$timestamp", 0, 10]}}]
http://docs.mongodb.org/v2.6/core/aggregation-introduction/
Assuming a field named dateField that contains Date values, you can use the aggregation date operators with $group to do this.
It's easiest if you're using Mongo 3.x where the $dateToString operator is available:
db.dates.aggregate([
{$group: {
_id: {$dateToString: {format: '%Y-%m-%d', date: '$dateField'}},
count: {$sum: 1}
}},
{$sort: {count: -1}}
])
Prior to 3.0 you need to use multiple date operators to piece together the date into the _id when grouping:
db.dates.aggregate([
{$group: {
_id: {
year: {$year: '$dateField'},
month: {$month: '$dateField'},
day: {$dayOfMonth: '$dateField'}
},
count: {$sum: 1}
}},
{$sort: {count: -1}}
])
In both cases, note the use of $sort to order the results by the number of docs on each day, descending.
Consider a collection which contains documents with a date and a count field :
{ _id: ObjectId("..."), date: ISODate("..."), count: 3}
I would like to query the count by week, so I have to group the data by a date truncated to the beginning of the week.
But it seems there is no way to achieve that with the mongodb aggregation framework.
I was expecting to be able to do something like this ($dateOfWeek is a date operator I imagined to truncate the date at the beginning of the week) :
db.data.aggregate( [ {$project : { date: {$dateOfWeek: '$date'}, count:1},
{ $group: {_id:'$date', count: {$sum: '$count'}} ])
But I didn't find a suitable date operator to do it.
I know I can do it with mapreduce but it would be so much more elegant to have a date operator rather than writing javascript code.