This question already has answers here:
MongoDB / Pymongo Query with Datetime
(2 answers)
Closed 7 years ago.
I have documents in a MongoDB collection that have a filed 'date'. In the Mongo Shell they appear as ISO Date.
For example:
"date" : ISODate("2015-12-21T09:00:05.396Z")
I would like to find documents within a certain range date.
The documents were inserted with PyMongo if it makes any difference.
How can I do this?
Thanks!
Operate regular Python datetime objects (documentation sample):
from datetime import datetime
start = datetime(2015, 12, 20, 7, 51, 04)
end = datetime(2015, 12, 21, 7, 52, 04)
col.find_one({'date': {'$lt': end, '$gt': start}})
Related
I have this function which parses the dates in my database
def parserQuestions():
res=db.questionsActual.find({'date_created':{'$type':'string'}})
for doc in res:
db.questionsActual.update_one({'_id': doc.get('_id')}, {'$set': { 'date_created': parse(doc.get('date_created'))}})
My input format is:
2020-12-11T13:23:58.677-04:00
but when I apply the function, the output is:
2020-12-11T13:23:58.677.000Z
How can I keep the timezone? or even better, can I set my own timezone?
the desired output is:
2020-12-11T13:23:58.677-04:00
MongoDB's BSON date type always stores dates as UTC date/times, so you cannot store an offset directly in a BSON date field. You can choose to store your offset in a separate field with something like:
from pymongo import MongoClient
from dateutil import parser
db = MongoClient()['mydatabase']
dt = parser.parse('2020-12-11T13:23:58.677-04:00')
db.questionsActual.insert_one({'date': dt, 'offsetSeconds': dt.tzinfo.utcoffset(dt).total_seconds()})
print(db.questionsActual.find_one({}, {'_id': 0}))
gives:
{'date': datetime.datetime(2020, 12, 11, 17, 23, 58, 677000), 'offsetSeconds': -14400.0}
I am having a MongoDb collection of 70M items (200GB) and I am trying to get those in the range [from_date, to_date] using the command:
from_date = datetime.datetime(2015, 1, 14, 9, 46, 23)
to_date = datetime.datetime(2015, 1, 14, 9, 46, 24)
db.collection.find({"datetime": {"$gte": from_date, "$lte": to_date}})
However it takes a lot of time to retrieve those items even for a single query. Is there any more efficient way to do this?
from the mongodb shell, I'm having an issue with a date query. This is a test database and the problem is better explained by example:
Example field in a collection
"createdOn" : ISODate("2015-11-23T00:49:01.800Z")
When I enter a date of this month for testing at the shell such as:
new Date(2015, 11, 23)
ISODate("2015-12-23T05:00:00Z")
You can see it's a month a head, meaning It's messing my queries. Where I have to drop back a month in order to get what I need for this month:
db.collection.find({'createdOn': {'$lte': new Date(2015, 10, 23)}}).count()
306
db.collection.find({'createdOn': {'$gte': new Date(2015, 10, 23)}}).count()
10
All entries above were created in Nov, but when using the correct month:
db.collection.find({'createdOn': {'$gte': new Date(2015, 11, 23)}}).count()
0
How does this even happen?
You should use Date like this: new Date("2015-11-23"). See Mongo Reference
What you are doing is almost right but you have missed on a point that:
Javascript counts month from 0 to 11 instead of 1 to 12
January is 0. December is 11. So to see for entries created in november you have to apply query like:
db.collection.find({'createdOn': {'$gte': new Date(2015, 10, 23)}})
In mongodb,
if I have a date, and I want to query for records that have a date later than the provided date I can do this:
collection.find({datefield:{$gt:somedate}})
And if I want to find records between a date range, I can do an $and with lt and gt.
But say that my datefield is actually datefields, a list of dates. And say that there are the following records for datefield
datefields = [june 1, 2015 hhmmss, june 2, 2015 hhmmss, june 14, 2015 hhmmss]
datefields = [june 1, 2015 hhmmss, june 3, 2015 hhmmss, june 8, 2016 hhmmss, june 17, 2015 hhmmss]
How would I construct the search to fetch for all records whose dates are between say june 3 and june 7, so that I only get the second record. And if ranges are not possible, can I do just a single search for june 3, 2015 while disregarding the hhmmss?
I think this will solve what you're asking. Use an aggregate operation to unwind your arrays and treat each element as a separate value in a document.
db.collection.aggregate( [
{ $unwind : { "$datefields" } },
{ $match : { datefields : { $gt : ISODate("2015-06-02T00:00:01Z"), $lte : ISODate("2015-06-03T23:59:59Z")} } }
])
That alone will give you an "exploded view" of your array results, with one document per array element that matches. If you want to collapse it down you can add a $group stage.
If this isn't what you're asking could you give a bit more detail?
For the second half of your question regarding "disregarding the hhmmss", there are a number of questions already answered on SO about that. Check out Query Mongodb on month, day, year... of a datetime
On the mongo shell this returns a document just fine:
> db.orderbook_log.findOne({'time': { '$gte': new Date(2014, 9, 24, 17, 38, 20, 546000), '$lt': new Date(2014, 10, 24, 17, 39, 20, 546000)}})
//... returns document with this time stamp:
"time" : ISODate("2014-10-25T00:47:30.819Z")
Notice I used "9" for October because JavaScript's months are 0-11.
And I also tested with "23" as the day because it looks like JS days are also 0-indexed, and that also returned a document: "time" : ISODate("2014-10-24T17:32:13.595Z")
atime = datetime.datetime(2014, 10, 24, 17, 38, 20, 546000)
btime = datetime.datetime(2014, 10, 24, 17, 39, 20, 546000)
future_book = log.find_one({"time": {"$gte": atime, "$lt": btime}})
But when I execute find_one in pymongofuture_book is None
All I'm really trying to do is loop though the first 100 records or so and get a record that occurred a relative minute later.
Javascript days are not zero-indexed, alas. Only months are.
I see in your Javascript you're adding 546,000 ms to the first date, so that results in 2014-10-24 at 17:48:26. Javascript then converts to your local timezone, so in my case it adds 5 hours:
> new Date(2014, 10, 24, 17, 39, 20, 546000)
ISODate("2014-11-24T22:48:26Z")
This is then compared (ignoring timezones) with the "time" field in your documents.
Better to remove the final milliseconds argument, and use the MongoDB shell's ISODate function, which is designed to be more rational than Javascript Dates:
> ISODate('2014-10-24T17:38:20')
ISODate("2014-10-24T17:38:20Z")
That will then compare to your documents in the way you expect, and it should match the PyMongo behavior. I suggest you drop the milliseconds argument from your Python datetime constructors, too, to clarify things.