Is there a standard for encoding a date as a timestamp? My thoughts:
This should be 12:00pm UTC in local time, eg 9:00am at T-3, therefore anyone consuming the timestamp, regardless of their -12/+12 offset, will recognize the same date, regardless of whether they parse at the UTC timezone
It could be 12:00pm at UTC
It could be the start of the day (12:00am) at UTC
It could be start of the day (12:00am UTC) in local time eg 9:00pm at T-3
Is there an official spec or standard to adhere to?
It would be easy to point to this document and say 'this is the standard' as opposed to being unaware and having to change our logic down the line.
There isn't a standard for this, because a date and a timestamp are logically two very different concepts.
A date covers the entire range of time on that day, not a specific point in time.
It may be a different date for a person in another time zone at any given point in time, but dates themselves do not have any association with time zones. Visualize a date as just a square on a calendar, not a point on a timeline.
Many APIs will use midnight (00:00) as the default time when a date-only value is assigned to a date+time value. However:
Whether it is UTC based or local-time based is very dependent on that particular API. There is no standard for this, nor is one answer necessarily better than the other.
Assigning a local-time midnight can be problematic for time zones with transitions near midnight. For example, in Santiago, Chile on 2019-09-08, the day started at 01:00 due to the start of DST. There was no 00:00 on that day.
Also, you tagged your question with momentjs. Since a Moment object is basically a timestamp (not a date), then Moment.js will generally assign the start of the day if provided a date-only value. The time zone involved is key to deciding which moment that actually is, which illustrates my prior points.
For example:
// Parsing as UTC
moment.utc('2019-09-08').format() //=> "2019-09-08T00:00:00Z"
// Parsing as Local Time (my local time zone is US Pacific Time)
moment('2019-09-08').format() //=> "2019-09-08T00:00:00-07:00"
// Parsing in a specific time zone (on a day without midnight)
moment.tz('2019-09-08', 'America/Santiago').format() //=> "2019-09-08T01:00:00-03:00"
Also keep in mind that sometimes APIs can be misnamed. The JavaScript Date object is not a date-only value, but actually a timestamp, just like a moment.
Related
I'm trying to figure out how Postgres handles DST in combination with intervals. Specifically I want to allow my users to create events with recurring dates - e.g. everyday at 16:00 local time.
For what I'm doing I need to store the first date in the user's local time, and then add a number of days to it, without changing the time of day in the user's local time. I was hoping that timestamptz with a full timezone name (so it knows when to apply DST?) combined with simple 1 day intervals would do the job - but it fails at my simple example:
Germany uses CET (+1:00) and switches to CEST (+2:00) at 2:00 in the morning on March 28.
Tunesia uses CET all year.
Thus I expected, that using a timestamptz on March 27 and adding 1 day to it, I'd see a different utc-offset in Berlin, and no change in Tunis - but they both changed the offset equally, as if Tunis was using DST:
select
'2021-03-27 16:00:00 Africa/Tunis'::timestamptz as "tunis_before_dst",
'2021-03-27 16:00:00 Africa/Tunis'::timestamptz + INTERVAL '1 day' as "tunis_after_dst",
'2021-03-27 16:00:00 Europe/Berlin'::timestamptz as "berlin_before_dst",
'2021-03-27 16:00:00 Europe/Berlin'::timestamptz + INTERVAL '1 day' as "berlin_after_dst"
results in:
tunis_before_dst: '2021-03-27 16:00:00+01'
tunis_after_dst: '2021-03-28 16:00:00+02'
berlin_before_dst: '2021-03-27 16:00:00+01'
berlin_after_dst: '2021-03-28 16:00:00+02'
Looking through pg_timezone_names, I can see that my Postgres instance is aware of Africa/Tunis not having DST - so I'm wondering why it's changing the UTC offset for it then.
I guess it's obvious that timezones and DST are very confusing to me, but am I doing something wrong handling them or is timezonetz not what I think it is?
Rant first. The concept of DST is breathtaking nonsense. Even the name is obvious BS. "Daylight Saving Time". No daylight has been saved. I can't believe the EU still did not manage to get rid of it, even though the overwhelming majority wants it gone, and it's been consensus to scrap it for a while now.
With that out of my system, the primary misunderstanding is this: you assume that the data type timestamp with time zone would store time zone information. It does not. Becomes obvious here:
Thus I expected, [...] I'd see a different utc-offset in Berlin, and no change in
Tunis - but they both changed the offset equally
The time zone offset you see in the output is the offset determined by the current timezone setting of your session. Time zone serves as input / output modifier / decorator. Postgres always stores UTC time internally. And no time zone information whatsoever.
The type name is a bit deceiving there. It has been known to fool the best:
Time zone storage in data type "timestamp with time zone"
Once you've grasped that concept, the rest should become obvious.
To preserve the local time (wall clock time of day), use the data type timestamp without time zone (timestamp), or even just time (never use the broken timetz), and store time zone information additionally - ideally the time zone name ('Europe/Berlin' like you have it), not a time zone abbreviation or a numeric offset.
timestamp with time zone (timestamptz) is the right choice to store unique points in time, independent of any time zones. The time zone offset is just an input modifier. Both of the following literals result in the same timestamptz value exactly, because both time zones happen to apply the same offset at this time of the year:
'2021-03-27 16:00:00 Africa/Tunis'::timestamptz
'2021-03-27 16:00:00 Europe/Berlin'::timestamptz
But these differ by one hour, because the German offset has changed according to the local DST regime:
'2021-03-28 16:00:00 Africa/Tunis'::timestamptz
'2021-03-28 16:00:00 Europe/Berlin'::timestamptz
Related:
Ignoring time zones altogether in Rails and PostgreSQL
Preserve timezone in PostgreSQL timestamptz type
How do I get date function to return date according to current system date?
Right now, with the code snippet below, it always returns UK time, not the current system date.
<calculate>
<script>$ = concat( num2date(date(), DateFmt()), " ", num2Time(Time(), TimeFmt()) )</script>
Any help is appreciated!
It's probably not UK time exactly, but rather GMT (or UTC, to use a more precise term). The UK happens to be aligned to GMT in the winter, but in the summer it advances one hour to BST for daylight saving time.
Now, I've never used LiveCycle myself, but nonetheless, I've read through the somewhat minimal docs for LiveCycle FormCalc Date and Time Functions, and the spec, and it appears to me that a few critical mistakes were made.
The date and time functions return UTC-based values, but only the time-related functions have been made aware of the local time zone. That is, there are separate Num2Time and Num2GMTime functions, but there is only one Num2Date function.
The Num2Date function works in terms of whole integer days, and thus they are simply days since 1900-01-01. Therefore, the number being passed in to the function must already be representative of the desired time zone. However, the Date function only gets the current GMT date. There does not appear to be a function to get the current local date.
It's different on the time side, because of the millisecond precision involved. However, there's yet another flaw here. Despite the docs saying that the Time function returning "the number of milliseconds since the epoch", its actually returning only the number of milliseconds since midnight GMT. There is no day-over-day accumulation of milliseconds from the date part. The docs here are even lying in their sample code which says:
Returns the current system time as the number of milliseconds since the epoch.
Time() => 71533235 at precisely 3:52:15 P.M. on September 15th, 2003 to a user in the Eastern Standard Time (EST) zone.
If that was indeed the case (and ensuring to use their 1900-01-01 epoch), the value would actually include an additional 3272572800000 milliseconds representing the days between 1900-01-01 and 2003-09-15, bringing the total timestamp to 3272644333235. Additionally, there's a typo there, because the timestamp they give is 3:52:13, not 3:52:15. Clearly nobody paid close attention to these docs!
The real problem is that one cannot be certain that the number of milliseconds since midnight of the current day in the local time zone is the same on every day. If instead of getting the current time, you were working with past stored time values, you might be an hour off (+ or -) if the current offset is different due to daylight saving time. Example: Eastern time might be UTC-5 or UTC-4, but only the offset currently in effect will be used by the Num2Time function, even if the date you're working with is using the other offset.
So, in summary, the Date function is insufficient, leading to your observations, and the date/time functionality in general is poorly designed. Given the constraints of this API, I can't even recommend a workaround. There would have to be a LocalDate function of some kind to be used instead of the Date function, but it doesn't exist.
The only advice I can offer is that it appears (from my research, not experience) that LiveCycle can use either FormCalc or JavaScript. So - use JavaScript instead.
I've been going through a lot of pain dealing with Timestamps lately with JPA. I have found that a lot of my issues have been cleared up by using TIMESTAMPTZ for my fields instead of TIMESTAMP. My server is in UTC while my JVM is in PST. It seems almost impossible with JPA to normalize on UTC values in the database when using TIMESTAMP WITHOUT TIMEZONE.
For me I use these fields for stuff like "when was the user created", "when did they last use their device", "when was the last time they got an alert", etc. These are typically events so they are instance in time sorts of values. And because they will now by TIMESTAMPTZ I can always query them for a particular zone if I don't want them UTC.
So my question is, for a Java/JPA/PostgreSQL server, when WOULD I want to use TIMESTAMP over TIMESTAMPTZ? What are the use cases for it? Right now I have a hard time seeing why I'd ever want to use TIMESTAMP and because of that I'm concerned that I'm not grasping its value.
Generally use TIMESTAMPTZ
Here's advice from David E. Wheeler, a Postgres expert, in a blog post whose title says it all:Always Use TIMESTAMP WITH TIME ZONE (TIMESTAMPTZ)
If you are tracking actual moments, specific points on the timeline, use TIMESTAMP WITH TIME ZONE.
One Exception: Partitioning
Wheeler’s sole exception is when partitioning on timestamps, because of technical limitations. A rare exception for most of us.
For information about partitioning, see doc and see the Wiki.
Misnomer
The data types names timestamp with time zone and timestamp without time zone are misnomers. In both cases the date-time value is stored in UTC (no time zone offset). Read that previous sentence again. UTC, always. The "with time zone" phrase means "with attention paid to time zone", not "store the time zone alongside this value". The difference between the types is whether any time zone should be applied either during storage (INSERT or UPDATE) or retrieval (SELECT query). (This behavior is described for Postgres -- Other databases vary widely in this regard.)
More precisely, one should say that TIMESTAMP WITHOUT TIME ZONE stores date-time values with no time zone. But without any time frame reference, anyone looking at that data would have to assume (hope, pray?) that the values are UTC. But again, moot as you should almost never use this type.
Read the doc carefully, and experiment a bit to clarify your understanding.
Unzoned
If you want to store the general idea of a possible time rather than a specific moment, use the other type, TIMESTAMP WITHOUT TIME ZONE.
For example, Christmas starts this year at the first moment of December 25th, 2017. That would be 2017-12-25T
00:00:00 with no indicator of time zone nor offset-from-UTC. This value is only a vague idea about possible moments. It has no meaning until we apply a time zone (or offset). So we store this using TIMESTAMP WITHOUT TIME ZONE.
The elves staffing Santa’s Special Events Logistics Department apply the time zones as part of their planning process. The earliest time zone is currently Pacific/Kiribati, 14 hours ahead of UTC. The elves schedule Santa’s first arrival there. The elves schedule a flight plan taking the reindeer on to other time zones where midnight comes shortly after, such as Pacific/Auckland. They continue going westward as each zone’s midnight arrives. Hours later in Asia/Kolkata, still later in Europe/Paris, still more hours later in America/Montreal and so on.
Each of these specific delivery moments would be recorded by the elves using WITH TIME ZONE, while that general idea of Christmas would by stored as WITHOUT TIME ZONE.
Another use in business apps for WITHOUT TIME ZONE is scheduling appointments farther out than several weeks. Politicians around the world have an inexplicable predilection for messing with the clock and redefining time zone rules. They join Daylight Saving Time (DST), leave DST, start DST on a different date, or end DST on a different date, or shift their clocks by 15 minutes or half-hour. All of these have been done in last several years by Turkey, United States, Russia, Venezuela, and others.
The politicians often make these changes with little forewarning. So if you are scheduling a dental appointment for six months out at 13:00, that should probably be stored as TIMESTAMP WITHOUT TIME ZONE or otherwise the politicians may effectively be changing you appointment to noon, or 2 PM, or 13:30.
You could use it to represent what Joda-Time and the new Java 8 time APIs call a LocalDateTime. A LocalDateTime doesn't represent a precise point on the timeline. It's just a set of fields, from year to nanoseconds. It is "a description of the date, as used for birthdays, combined with the local time as seen on a wall clock".
You could use it to represent, for example, the fact that your precise birth date is 1975-07-19 at 6 PM. Or that, all across the world, the next new year is celebrated on 2015-01-01 at 00:00.
To represent precise moments, like the moment Armstrong walked on the moon, a timestamp with timezone is indeed more appropriate. Regardless of the timezone of the JVM and the timezone of the database, it should return you the correct moment.
Update for the answers above: partitioning is no longer an exceptional case in PG11 thanks to pruning.
https://www.postgresql.org/docs/11/ddl-partitioning.html#DDL-PARTITION-PRUNING
Personally successfully tested queries against PG11 AWS RDS. Also the official PG wiki states the use of timestamp without timezone is a bad idea:
https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_use_timestamp_.28without_time_zone.29_to_store_UTC_times
With the Java 8 date & time API I wouldn't blindly jump into a timestamptz camp.
If you map timestamp <=> LocalDateTime you always get the same value regardless default Java application timezone. Regardless how many calls TimeZone.setDefault(TimeZone.getTimeZone("TZ")) mixing different TZ you put in between SELECT/INSERT you will get the same LocalDateTime in Java at any time and date/time components will be the same as in Postgresql TO_CHAR(ts, 'YYYY-MM-DD HH24:MI:SS').
If you map timestamptz <=> LocalDateTime Postgresql JDBC driver (supporting JDBC 4.2 spec) converts LocalDateTime to UTC using default Java timezone when saving value to DB. If you save it in one default TZ and read in another you get different "local" results.
Airplane departure time is local to an airport. If you don't need to compare departure time between different cities timestamptz & UTC doesn't make sense, you just print exact city local time in a ticket. With timestamp it is possible to keep date/time as is, avoiding double TZ correction due to Java app default TZ + city specific TZ (business logic).
timestamptz is useful when you heavily convert TZ in SQL. With only timestamp you write:
date_trunc('day', x.datecol AT TIME ZONE 'UTC' AT TIME ZONE x.timezone)
AT TIME ZONE x.timezone AT TIME ZONE 'UTC'
while with timestamptz there is no need to mention that time is in UTC (if you follow such convention, probably you should xD):
date_trunc('day', x.datecol AT TIME ZONE x.timezone)
AT TIME ZONE x.timezone
Operator AT TIME ZONE is overloaded:
timestamp AT TIME ZONE 'X' => timestamptz
timestamptz AT TIME ZONE 'X' => timestamp
Postgresql JDBC + Java 8 date&time API spec.
We're deploying our own stream gauges (a lot like this USGS gauge: http://waterdata.usgs.gov/usa/nwis/uv?site_no=03539600) so us kayakers know whether or not there's enough water to paddle the stream and don't waste time and gas to drive out there. We hope install a few of these across the southeast whitewater region which spans the eastern and central time zones.
I'm storing the time a record is inserted using the default value of current_time for the record. I'd like to later display the data using the MM/DD/YYYY HH12:MI AM TZ format, which outputs reading like 03/12/2012 01:00 AM CDT. I'd also like for the output to be aware of changes in day light savings time, so the last part of the previous sentence would change between CST and CDT when we 'spring forward' and 'fall back'. This change occurred on 3/11/2012 this year and I've included dates on both sides of this DST line below. I'm using my Windows 7 laptop for development and we will later be deploying on a Unix box. Postgres has apparently detected that my Windows computer is set to eastern US time zone. I'm trying this with a 'timestamp without time zone' field and a 'timestamp with time zone' field but can't get it to work.
I've tried using 'at time zone' in my selects and every thing is working until it's time to display the time zone. The actual hour is part of the time stamp is correctly subtracted by an hour when I ask for the time in CDT. But EDT is displayed in the output.
SELECT reading_time as raw,
reading_time at time zone 'CDT',
to_char(reading_time at time zone 'CDT',
'MM/DD/YYYY HH12:MI AM TZ') as formatted_time
FROM readings2;
"2012-04-29 17:59:35.65";"2012-04-29 18:59:35.65-04";"04/29/2012 06:59 PM EDT"
"2012-04-29 17:59:40.19";"2012-04-29 18:59:40.19-04";"04/29/2012 06:59 PM EDT"
"2012-03-10 00:00:00";"2012-03-10 00:00:00-05";"03/10/2012 12:00 AM EST"
"2012-03-11 00:00:00";"2012-03-11 00:00:00-05";"03/11/2012 12:00 AM EST"
"2012-03-12 00:00:00";"2012-03-12 01:00:00-04";"03/12/2012 01:00 AM EDT"
I'm storing the time zone that each of our gauges is located in a character varying field a separate table. I considered just appending this value to the end of the time output, but I want it to change from from CST to CDT without my intervention.
Thanks for your help.
Instead of using time zone abbreviations like CDT or CST, you could consider using full Olson-style time zone names. In the case of central time, you could choose a time zone. Either one that matches your location, such as America/Chicago, or just US/Central. This ensures PostgreSQL uses the Olson tz database to automatically figure out whether daylight saving time applies at any given date.
You definitely want a TIMESTAMP WITH TIME ZONE column (which is also known as timestamptz in PostgreSQL). That will store the timestamp in UTC, so that it represents a particular moment in time. Contrary to what the name suggests, it does not save a time zone in the column -- you can view the retrieved timestamp in the time zone of your choosing with the AT TIME ZONE phrase.
The semantics of TIMESTAMP WITHOUT TIME ZONE are confusing and nearly useless. I strongly recommend you don't use that type at all for what you are describing.
I'm really confused by the part of the question which talks about storing the timestamp in a CHARACTER VARYING column. That seems as though it might be part of the problem. If you can store it in timestamptz right from the start I suspect that you will have fewer problems. Barring that, it would be safest to use the -04 notation for offset from UTC; but that seems like more work to me for no benefit.
You can create a table of known timezones in the format suggested in Guan Yang's answer, and then use a foreign key column to this table. Valid timezones can be obtained from pg_timezone_names I've gone into more detail in this related answer.
I just want to clarify if I am understanding how dates & time zones work.
Basically, I have a date string #"2008-07-06 12:08:49" that I want to convert to an NSDate. I want this date and time to be in whatever the current user's time zone is set in. So if they are in GMT or HST, it's still 12:08:49.
If I have date in unix form 1215382129 (UTC) and my time zone is set to London (GMT), the outputted date from NSLog() is:
2008-07-06 12:08:49 +0100
If I then change my time zone to Hawaii (HST) and output the same date, I get:
2008-07-06 12:08:49 -1000
This seems to work fine, but I was under the impression to get the time in Hawaiian, I'd have to physically add the time difference (-10hrs) to the unix time stamp. Is this not required then?
Does that mean, whatever date and time a unix time is pointing to, it always points to the same date and time in whatever time zone a user is in?
Hope this makes sense!
Edit
I've just realised (thanks to Kevin Conner!) that in fact NSDateFormatter is creating different unix timestamps for that date string depending on the current timezone! So I was totally wrong!! :-)
Disclaimer, I'm mostly a Java guy. But Cocoa seems to work like the Java library in this regard: Dates are zoneless timestamps. Time zones are in the domain of formatting dates for display. In other words, the internal format doesn't consider time zones, it's all in UTC. Time zones are relatively a convenience for humans, so they are in the display/parsing side.
I noticed there is a setTimeZone: method on NSDateFormatter. Try calling that on your formatter before performing the format.