I have various timestamps stored in Mongo collections, some as floats and some as ints.
They are all stored in BST and the server will be switched to UTC soon. How do I convert them within Mongo to be UTC timestamps?
In MySQL I can do this:
UPDATE `table` SET `field` = CONVERT_TZ(`field`, 'Europe/London', 'UTC');
Is there a Mongo equivalent?
You'll have to use your language of choice and update them one at a time. It should be as simple as a for loop that loads the data and rewrites it.
Just double-check how your language of choice handles timestamps across timezones. Making such a data change can have all kinds of unexpected effects on production code.
Timestamps are generally in UTC and not in a specific timezone. All date/time libraries I've worked with return timestamps that are the number of seconds (or milliseconds) since Jan 1st 1970 UTC. Check the documentation for the library you used to create the timestamp to be sure.
This means that you should be ok, unless you have used a date/time library that does not follow this convention, or somehow calculated the timestamps yourself and accounted for the timezone.
For example in JavaScript if you store the value returned from new Date().getTime() and later pass that value to new Date(...) on a different system you will end up with the same absolute date/time, regardless of the timezones of the two systems. The same goes for Ruby, do Time.new.to_i on one machine, and run Time.at(...) on another and you will get the same absolute date/time. When I say "absolute date/time" I mean that it's UTC time will be the same, most likely the system will display it in the local time zone, but that is what you want.
Some points to consider about date in Mongo:
All dates are stored in UTC in MongoDB
MongoDB stores dates internally as a 64 bit integer representing
milliseconds since 1970-01-01T00:00:00Z
If the date value you provide is not already in UTC it
will be converted to UTC before being stored in MongoDB by the driver
The recommendation is not to use DateTime.Parse.
You will get into all sorts timezone issues regarding how DateTimes are formatted.
Instead just use one of the DateTime constructors with UTC flavor.
Example:
var dateTime = new DateTime(2011, 5, 5, 0, 0, 0, DateTimeKind.Utc);
Hope you find it useful.
Related
I'm using flutter. In which way a DateTime should be stored in firestore so that I can make a filter based on the DateTime. Should it be converted to String or something else? What is the perfect way here?
Firestore has a native Timestamp type that can be used to store timestamps, which:
A Timestamp represents a point in time independent of any time zone or calendar, represented as seconds and fractions of seconds at nanosecond resolution in UTC Epoch time
This type is perfectly suited for filtering on date/time ranges, for example when you want to get all documents with a field between January 7, 2021 at 7:41:00AM and January 20, 2021 at 2:00:42PM.
In cases where I'm purely interested in a date and not in a time, I often store that as a string in "20210107" format (so yyyyMMdd). I find this format slightly easier when I want to get documents for a specific date, as I can use an equality filter. But the trade-off is that I love the precision of time. As you can probably guess from my phrasing this is a personal preference, not necessarily a best way for all cases or everyone.
It must be a string, you can split or format it as the day / month / year you want after the string value
I save my date as a local date, but when I read it back, it treats it as if it was a UTC date so it slips by several hours.
The dates are passed in as strings in the form '2020-03-05 09:05:23' as query parameters but when they are retrieved they might look like '2020-03-04 10:05:23' because I am 13 hours ahead of Greenwich.
For MariaDB (or MySQL):
Use DATETIME as a picture of the clock on the wall.
Use TIMESTAMP to adjust to the system's timezone.
Set the system's timezone according to where it lives in the world.
I will begin using MongoDB in a project and made the following observation:
A ‘Date’, e.g.: ISODate("2000-07-24T00:00:00.000Z") is stored in as UTC 0 (or Zulu).
Let’s assume that an end user submits a plain date, such as “2000-07-24” and his or her local timezone is -5.
Because of the ‘Z’ character in the time (indicates UTC 0), storing the following would be incorrect because the user’s 5 hour difference:
ISODate("2000-07-24T00:00:00.000Z")
I am assuming that it would be more proper to consider the end user’s local time zone into account and store the date as follows:
ISODate(“2000-07-23T19:00:00.000Z”)
By doing this, I can see that any queries would need a timezone adjustment for 'Date'.
By storing all ‘Date’ fields as UTC 0, we can guarantee that exact points in time from people posting from different continents is properly handled.
Does this seem like a good way to handle timezones in MongoDB?
I have a table that I am using to store iso dates with timezones. I realize that dates should "always" be stored as utc but I have an exception to that rule. The timestamps aren't in any way related to the server they are running on. I want to be able to store an iso date like this:
2016-03-06T01:15:52-06:00
And regardless of the time zone of the server or anything else I want the timestamp returned as:
2016-03-06T01:15:52-06:00
Currently if I insert an iso date it automatically converts it to whatever the server timezone is. My above date gets converted to:
2016-03-06 07:15:52+00 (server is utc)
The only thing I can think of is storing the timezone offset in a separate column, storing my date as utc and then converting using the offset column, horribly messy. Surely there is a way to store my date in one column and get it out the way it was originally created?
Your proposed solution is correct. Or more precisely, it is one of several correct implementations. Any of the following would work:
Store the UTC timestamp in one field, store the offset in another.
Store the local timestamp in one field, store the offset in another.
Store the local date in one field, and store a time with time zone in another. (though time with time zone is generally discouraged...)
Store the UTC timestamps in one field and the local timestamp in another.
The easiest by far is the first one, which you already proposed.
I'd avoid against storing timestamps in text fields, as they tend not to be very efficiently searchable.
Also note - if you're coming from a SQL Server background, you might recall its datetimeoffset type, which stores the local datetime and offset in the field, and uses the UTC equivalent during indexing. It's common to think that Postgres and MySQL's timestamp with time zone would have the same behavior, but they don't. They simply use the session time zone to convert to/from UTC. SQL Server has no concept of a session time zone, and thus the discrepancy.
Be sure to read this part of the Postgres docs.
When storing a timestamp in mongodb should I store the 'unix-time' millis since some date or should I store an actual date/time?
What are the benefits of either?
Edit
To be more specific should I store a long that is seconds since Jan 1. 1970, or should I store a Javascript date object.
I recommend using the MongoDB native date format. The advantage of that is that you can then easily use the date operators:
http://docs.mongodb.org/manual/reference/operator/aggregation-date/