I have time series data stored in a mongodb database, where one of the fields is an ISODate object. I'm trying to retrieve all items for which the ISODate object has a zero value for minutes and seconds. That is, all the objects that have a timestamp at a round hour.
Is there any way to do that, or do I need to create separate fields for hour, min, second, and query for them directly by doing, e.g., find({"minute":0, "second":0})?
Thanks!
You could do this as #Devesh says or if it fits better you could use the aggregation framework:
db.col.aggregate([
{$project: {_id:1, date: {mins: {$minute: '$dateField'}, secs: {$second: '$dateField'}}}},
{$match: {mins: 0, secs: 0}}
]);
Like so.
Use the $expr operator along with the date aggregate operators $minute and $second in your find query as:
db.collection.find({
'$expr': {
'$and': [
{ '$eq': [ { '$minute': '$dateField' }, 0 ] },
{ '$eq': [ { '$second': '$dateField' }, 0 ] },
]
}
})
Can you have one more column added in the collection only containing the datetime without minutes and seconds . It will make your query faster and easy to use. It will be datetime column with no minutes and seconds parts
Related
I'm trying to filter a collection on a date type field using a long number (number of ms since epoch) and can't seem to get it to work in a regular find() or an aggregation. What am I missing ?
TIA!
db.testColl.insertOne({_id:"test", createDate: ISODate("2022-12-26T01:00:00.000+0000")})
db.testColl.find( {createDate: {$gte : {$toDate:1608947820333} }} )
I think you can't use aggregate expression $toDate in find/match. Instead, try using it with $expr.
db.testColl.find({
$expr: {
$gte: [
"$createDate",
{
$toDate: 1608947820333
}
]
}
})
First occasion experimenting with dates and times in MongoDB.
Currently, I have correctly inserted UTC dates into my documents:
{
"Value": 10
"DateTime": {
"$date": "2013-08-23T08:00:00.000Z"
},
}
I want to query the data to return all of the 'value' and datetime fields for all documents in which the hour is 08. I have gotten as far as:
db.readings.aggregate([{$project:{hour:{$hour:"$DateTime"}}},{$match:{hour:{"$in":[08]}}}])
Which returns the _id and "hour": 8 for all matching entries but I'm confused at how to proceed. It seems an unnecessarily complicated way to search and so I wonder if I am barking up the wrong tree here? Admittedly, I am somewhat out of my depth so some guidance would be appreciated.
You can try $expr expression operator to use aggregation operators ($hour) in query,
$expr to use aggregation operators
$hour will return an hour from the date
$eq to match hour and input hour 8
db.collection.find({
$expr: {
$eq: [
{ $hour: "$DateTime" },
8
]
}
})
Playground
I have a collection in MongoDB. Model is:
{
currency: String,
price: Number,
time: Date
}
Documents are recorded to that collection any time the official rate for currency changes.
I am given a timestamp, and I need to fetch rates for all available currencies to that time. So first I need to filter all documents whose time $lte then required, then I need to fetch only those with max timestamps. For each currency.
after seeing your requirement , I think you want max number of price and time , use max operator
db.collection.aggregate(
[
{
$group:
{
_id: "$currency",
time: { $max: "$time"},
price: { $max: "$price" }
}
}
]
)
You can use mongo aggregate function to do so. Please find the example below:
db.<collection_name>.aggregate([
// First sort all the docs by time in descending
{$sort: {time: -1}},
// Take the first 3 of those
{$limit: 3}
])
Hope this helps !!
If I have documents in the following schema saved in my mongoDB:
{
createdDate: Date,
lastUpdate: Date
}
is it possible to query for documents where the period of time between creation and the last update is e.g. greater than one day?
Best option is to use the $redact aggregation pipeline stage:
db.collection.aggregate([
{ "$redact": {
"$cond": {
"if": {
"$gt": [
{ "$subtract": [ "$lastUpdate", "$createdDate" ] },
1000 * 60 * 60 * 24
]
},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
So you are looking at the milliseconds value from the difference being greater than the milliseconds value for one day. The $subtract does the math for the difference, and when two dates are subtracted the difference in miliseconds is returned.
The $redact operator takes a logical expression as "if", and where that condition is true it takes the action in "then" which is to $$KEEP the document. Where it is false then the document is removed from results with $$PRUNE.
Note that since this is a logical condition and not a set value or a range of values, then an "index" is not used.
Since the operations in the aggregation pipeline are natively coded, this is the fastest execution of such a statement that you can get though.
The alternate is JavaScript evaluation with $where. This takes a JavaScript function expression that needs to similarly return a true or false value. In the shell you can shorthand like this:
db.collection.find(function() {
return ( this.lastUpdate.valueOf() - this.createdDate.valueOf() )
> ( 1000 * 60 * 60 * 24 );
})
Same thing, except that JavaScript evalution requires interpretation and will run much slower than the .aggregate() equivalent. By the same token, this type of expression cannot use an index to optimize performance.
For the best results, store the difference in the document. Then you can simply query directly on that property, and of course you can index it as well.
You can use $expr ( 3.6 mongo version operator ) to use aggregation functions in regular query.
Compare query operators vs aggregation comparison operators.
db.col.find({$expr:{$gt:[{"$subtract":["$lastUpdate","$createdDate"]},1000*60*60*24]}})
Starting in Mongo 5, it's a perfect use case for the new $dateDiff aggregation operator:
// { created: ISODate("2021-12-05T13:20"), lastUpdate: ISODate("2021-12-06T05:00") }
// { created: ISODate("2021-12-04T09:20"), lastUpdate: ISODate("2021-12-05T18:00") }
db.collection.aggregate([
{ $match: {
$expr: {
$gt: [
{ $dateDiff: { startDate: "$created", endDate: "$lastUpdate", unit: "hour" } },
24
]
}
}}
])
// { created: ISODate("2021-12-04T09:20"), lastUpdate: ISODate("2021-12-05T18:00") }
This computes the number of hours of difference between the created and lastUpdate dates and checks if it's more than 24 hours.
I am trying to group by DayHours in a mongo aggregate function to get the past 24 hours of data.
For example: if the time of an event was 6:00 Friday the "DayHour" would be 6-5.
I'm easily able to group by hour with the following query:
db.api_log.aggregate([
{ '$group': {
'_id': {
'$hour': '$time'
},
'count': {
'$sum':1
}
}
},
{ '$sort' : { '_id': -1 } }
])
I feel like there is a better way to do this. I've tried concatenation in the $project statement, however you can only concatenate strings in mongo(apparently).
I effectively just need to end up grouping by day and hour, however it gets done. Thank You.
I assume that time field contains ISODate.
If you want only last 24 hours you can use this:
var yesterday = new Date((new Date).setDate(new Date().getDate() - 1));
db.api_log.aggregate(
{$match: {time: {$gt: yesterday}}},
{$group: {
_id: {
hour: {$hour: "$time"},
day: {$dayOfMonth: "$time"},
},
count: {$sum: 1}
}}
)
If you want general grouping by day-hour you can use this:
db.api_log.aggregate(
{$group: {
_id: {
hour: {$hour: "$time"},
day: {$dayOfMonth: "$time"},
month: {$month: "$time"},
year: {$year: "$time"}
},
count: {$sum: 1}
}}
)
Also this is not an answer per se (I do not have mongodb now to come up with the answer), but I think that you can not do this just with aggregation framework (I might be wrong, so I will explain myself).
You can obtain date and time information from mongoId using .getTimestamp method. The problem that you can not output this information in mongo query (something like db.find({},{_id.getTimestamp}) does not work). You also can not search by this field (except of using $where clause).
So if it is possible to achieve, it can be done only using mapreduce, where in reduce function you group based on the output of getTimestamp.
If this is the query you are going to do quite often I would recommend actually adding date field to your document, because using this field you will be able properly aggregate your data and also you can use indeces not to scan all your collection (like you are doing with $sort -1, but to $match only the part which is bigger then current date - 24 hours).
I hope this can help even without a code. If no one will be able to answer this, I will try to play with it tomorrow.