First occasion experimenting with dates and times in MongoDB.
Currently, I have correctly inserted UTC dates into my documents:
{
"Value": 10
"DateTime": {
"$date": "2013-08-23T08:00:00.000Z"
},
}
I want to query the data to return all of the 'value' and datetime fields for all documents in which the hour is 08. I have gotten as far as:
db.readings.aggregate([{$project:{hour:{$hour:"$DateTime"}}},{$match:{hour:{"$in":[08]}}}])
Which returns the _id and "hour": 8 for all matching entries but I'm confused at how to proceed. It seems an unnecessarily complicated way to search and so I wonder if I am barking up the wrong tree here? Admittedly, I am somewhat out of my depth so some guidance would be appreciated.
You can try $expr expression operator to use aggregation operators ($hour) in query,
$expr to use aggregation operators
$hour will return an hour from the date
$eq to match hour and input hour 8
db.collection.find({
$expr: {
$eq: [
{ $hour: "$DateTime" },
8
]
}
})
Playground
Related
In my mongodb table, I have 2 (relevant for this Q) columns: service, timestamp.
I want to query only rows with service=liveness and that those with timestamp of 12th Novermber 2020.
How can I do it, if timestamp field is of type Number (UNIX epoch number)..?
This is my query currently:
{ service: "liveness" }.
This is how the timestamp column looks like:
To query by two fields you only need this syntax:
db.collection.find({
"field1": yourField1Value,
"field2": yourField2Value
So, if your date is a Number instead of a Date you can try this query:
db.collection.find({
"service": "liveness",
"timestamp": 1600768437934
})
And should works. Example here.
Now, if the problem is parse 12th November 2020 to UNIX timestamp, then the easiest way is convert first the date in your app language.
Edit:
Also, I don't know if I've missunderstood your question but, here is another query.
db.collection.aggregate([
{
"$match": {
"service": "liveness",
}
},
{
"$project": {
"timestamp": {
"$toDate": "$timestamp"
}
}
},
{
"$match": {
"timestamp": {
"$gt": ISODate("1990-01-01"),
"$lt": ISODate("2060-01-01")
}
}
}
])
This query first match all documents with service as liveness, so the next stage is faster. Into $project the timestamp is parsed to Date so you can match again with your date.
Using $gt and $lt you can search by a whole day.
And also, if you can get the days into UNIX timestamp you can do this:
db.collection.find({
"service": "liveness",
"timestamp": {
"$gte": yourDay,
"$lt": nextrDay
}
})
Using $gte and $lt you ensure the query will find all values in the day.
I am trying to query my database on the date_recorded field, but the query give zero results. I'm using the MongoShell and have also tried it in Compass.
I've tried variations of $eq with ISODate and Date:
db.my_collection.find({ "date_recorded": { "$eq": new ISODate("2017-06-09T01:27:33.967Z") }}).count()
I have also tried variations of $gte with ISODate and Date:
db.my_collection.find({ "date_recorded": { "$gte": new Date("2017-06-09T01:27:33.967Z") }}).count()
Record is in the db, notice the highlighted field
Like #JohnnyHK said, the date_recorded is a string. To make a comparison, we either need to convert date_recorded to a date or compare is with a string.
The following queries can get us the expected output:
db.my_collection.find({
"date_recorded": {
$eq:"2017-06-09T01:27:33.967Z"
}
}).count()
db.my_collection.find({
$expr:{
$eq:[
{
$toDate:"$date_recorded"
},
new ISODate("2017-06-09T01:27:33.967Z")
]
}
}).count()
I was trying to apply a regex filter(year) to a date type field but the query returned no results. So was trying to understand the behavior of this query?
db.collection.find({"ReportedDate":/2016/i})
Note: I can filter it by date type without regex. But I am just trying to understand the behavior of the regex filter against date type field?
From MongoDB version >= 4.4 we can write custom filters using $function operator so try this:
let yearRegex = /2016/i;
db.collection.find({
$expr: {
$function: {
body: function(reportedDate, yearRegex) {
return yearRegex.test(reportedDate);
},
args: [{ $toString: "$ReportedDate" }, yearRegex],
lang: "js"
}
}
});
Note: Instead of $toString we can also use $dateToString with timezone to cover edge cases.
As mentioned, the $regex query operator provides regular expression capabilities for pattern matching strings in queries.
An alternative to using regex filter to query the year value, you can utilise MongoDB Aggregation Pipeline to project the year portion of a date object using $year aggregation operator.
For example:
db.collection.aggregate([
{"$addFields": {"year": {"$year":"$ReportedDate" } } },
{"$match": {"year": 2016 } }
], {cursor:{batchSize:10}})
There are other useful date related operators such as: $dayOfYear, $dayOfWeek, $dayOfMonth, etc.
I did the same thing but instead of find method, I use aggregation pipeline
db.collection.aggregate(
[
{
$project: {
"d": { $dateToString: {format: "%Y-%m-%d", date: "$mydate"}},key:1
}
} ,
{
$match: {
d: /2012-/i
}
}
]
)
By this I am able to apply the regex pattern in ISO Date too.
If I have documents in the following schema saved in my mongoDB:
{
createdDate: Date,
lastUpdate: Date
}
is it possible to query for documents where the period of time between creation and the last update is e.g. greater than one day?
Best option is to use the $redact aggregation pipeline stage:
db.collection.aggregate([
{ "$redact": {
"$cond": {
"if": {
"$gt": [
{ "$subtract": [ "$lastUpdate", "$createdDate" ] },
1000 * 60 * 60 * 24
]
},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}}
])
So you are looking at the milliseconds value from the difference being greater than the milliseconds value for one day. The $subtract does the math for the difference, and when two dates are subtracted the difference in miliseconds is returned.
The $redact operator takes a logical expression as "if", and where that condition is true it takes the action in "then" which is to $$KEEP the document. Where it is false then the document is removed from results with $$PRUNE.
Note that since this is a logical condition and not a set value or a range of values, then an "index" is not used.
Since the operations in the aggregation pipeline are natively coded, this is the fastest execution of such a statement that you can get though.
The alternate is JavaScript evaluation with $where. This takes a JavaScript function expression that needs to similarly return a true or false value. In the shell you can shorthand like this:
db.collection.find(function() {
return ( this.lastUpdate.valueOf() - this.createdDate.valueOf() )
> ( 1000 * 60 * 60 * 24 );
})
Same thing, except that JavaScript evalution requires interpretation and will run much slower than the .aggregate() equivalent. By the same token, this type of expression cannot use an index to optimize performance.
For the best results, store the difference in the document. Then you can simply query directly on that property, and of course you can index it as well.
You can use $expr ( 3.6 mongo version operator ) to use aggregation functions in regular query.
Compare query operators vs aggregation comparison operators.
db.col.find({$expr:{$gt:[{"$subtract":["$lastUpdate","$createdDate"]},1000*60*60*24]}})
Starting in Mongo 5, it's a perfect use case for the new $dateDiff aggregation operator:
// { created: ISODate("2021-12-05T13:20"), lastUpdate: ISODate("2021-12-06T05:00") }
// { created: ISODate("2021-12-04T09:20"), lastUpdate: ISODate("2021-12-05T18:00") }
db.collection.aggregate([
{ $match: {
$expr: {
$gt: [
{ $dateDiff: { startDate: "$created", endDate: "$lastUpdate", unit: "hour" } },
24
]
}
}}
])
// { created: ISODate("2021-12-04T09:20"), lastUpdate: ISODate("2021-12-05T18:00") }
This computes the number of hours of difference between the created and lastUpdate dates and checks if it's more than 24 hours.
I have time series data stored in a mongodb database, where one of the fields is an ISODate object. I'm trying to retrieve all items for which the ISODate object has a zero value for minutes and seconds. That is, all the objects that have a timestamp at a round hour.
Is there any way to do that, or do I need to create separate fields for hour, min, second, and query for them directly by doing, e.g., find({"minute":0, "second":0})?
Thanks!
You could do this as #Devesh says or if it fits better you could use the aggregation framework:
db.col.aggregate([
{$project: {_id:1, date: {mins: {$minute: '$dateField'}, secs: {$second: '$dateField'}}}},
{$match: {mins: 0, secs: 0}}
]);
Like so.
Use the $expr operator along with the date aggregate operators $minute and $second in your find query as:
db.collection.find({
'$expr': {
'$and': [
{ '$eq': [ { '$minute': '$dateField' }, 0 ] },
{ '$eq': [ { '$second': '$dateField' }, 0 ] },
]
}
})
Can you have one more column added in the collection only containing the datetime without minutes and seconds . It will make your query faster and easy to use. It will be datetime column with no minutes and seconds parts