Seeking help understanding Postgres timezone conversion and comparison
query
http://sqlfiddle.com/#!17/9eecb/79009
SELECT
now()
, ( now() at time zone 'AEDT' ) AS now_AEDT
, ( ( now() at time zone 'AEDT' ) at time zone 'AEDT' ) AS now_AEDT_AEDT
, ( now() at time zone 'UTC' ) + interval '1' second
> ( now() at time zone 'AEDT' ) AS should_be_true
Result
now : 2021-07-30T01:03:10.707834Z (i assume system is UTC?)
now_AEDT : 2021-07-30T12:03:10.707834Z
now_AEDT_AEDT : 2021-07-30T01:03:10.707834Z (why is this different?)
should_be_true : false (after adding a second; this should be true, no?)
questions / confusions
why does double-setting the timezone change the time? (now_AEDT_AEDT)
how are timestamps with timezones handled in comparison? It seems the zone is ignored and just the values are compared?
"at time zone" does opposite things when applied to timestamptz versus a timestamp. So applying it twice in a row just gives you back the original. First it does something, then it undoes it.
When applied to timestamptz, it converts the time to look like what it would be expressed as in the indicated time zone, and datatypes it as a timestamp without timezone (except in sqlfiddle, where it seems to do something slightly different, but without changing the overall effect). When applied to a timestamp without timezone, it assumes that that time expressed was already in the indicated time zone, and converts it back to the system time with it datatyped as timestamptz.
how are timestamps with timezones handled in comparison? It seems the zone is ignored and just the values are compared?
You aren't comparing timestamps with timezones. You are comparing timestamps without timezones.
select pg_typeof(now() at time zone 'AEDT');
pg_typeof
-----------------------------
timestamp without time zone
So yes, it ignores the timezones in your comparison, because they are no longer there anymore.
Related
The behavior of the at time zone operator on the date type is not as expected in PostgreSQL.
set time zone 'UTC';
select
'2021-01-03'::date at time zone 'America/Chicago' as "value",
pg_typeof('2021-01-03'::date at time zone 'America/Chicago') as "type";
x
value
type
Expected
2021-01-03 06:00:00+00
timestamp with time zone
Actual
2021-01-02 18:00:00
timestamp without time zone
DB Fiddle
Tested on PostgreSQL server versions 10, 11, 12, and 13.
This behavior is very strange to me, and I have not been able to find anywhere it is described or discussed. It seems PSQL is implicitly casting date to timestamptz (using the system time zone) and then applying the at time zone operator. I know that the at time zone operator is only defined (in the docs) for the timestamp(tz) and time(tz) types. However, I would have assumed PSQL would implicitly cast date to timestamp and not timestamptz because:
It is a "safe" conversion (no need to assume a time zone). Simply append 00:00:00.
It results in a timestamptz which one would expect when using the at time zone operator on a type that does not have a time zone.
The current behavior simply doesn't make sense to me and I can't think of a use case that would benefit from it. You are asking for the timestamp with time zone for a date at a particular time zone and instead you get a timestamp of the date at the system time zone localized to the given time zone.
Of course, a simple workaround is to cast to timestamp first:
set time zone 'UTC';
select '2021-01-03'::date::timestamp at time zone 'America/Chicago';
I was hoping someone could shed some light on this. What am I missing? Perhaps some overarching rule about how date is implicitly cast to timestamp(tz)? Is this behavior documented somewhere?
Thanks!
Mike
You ask "why". The reason is that AT TIME ZONE does not operate on date, so PostgreSQL invokes an implicit type cast to convert it to a different data type. Lacking an explicit directive, PostgreSQL opts for the preferred data type for datetime, which happens to be timestamp with time zone.
The data type resolution rules for operators are documented here, and the preferred type for a type category can be found in the typispreferred and typcategory columns of the pg_type system catalog.
As per the Postgresql documentation:
The AT TIME ZONE converts time stamp without time zone to/from time stamp with time zone, and time values to different time zones
Here in first query, you are trying to apply at time zone to date type, not to time stamp
set time zone 'UTC';
select
'2021-01-03'::date at time zone 'America/Chicago' as "value",
pg_typeof('2021-01-03'::date at time zone 'America/Chicago') as "type";
which is eventually gets ignored, and you will get a timestamp without timezone
But in second query,
set time zone 'UTC';
select '2021-01-03'::date::timestamp at time zone 'America/Chicago';
You are casting the date to time stamp first and then converting it to desired time zone,
which gives you the expected result of time stamp with time zone.
Hope this clears your doubt.
We have a table that is filled with data from a legacy report of another system. The columns of that table reflect the same structure of the report.
Here are a abbreviated structure of the table:
CREATE TABLE IF NOT EXISTS LEGACY_TABLE (
REPORT_DATE DATE NOT NULL,
EVENT_ID BIGINT PRIMARY KEY NOT NULL,
START_HOUR TIMESTAMP WITHOUT TIME ZONE,
END_HOUR TIME WITHOUT TIME ZONE,
EXPECTED_HOUR TIME WITHOUT TIME ZONE
);
We are refactoring this table to deal with different time zones of different clients. The new structure would be something like:
CREATE TABLE IF NOT EXISTS LEGACY_TABLE (
REPORT_DATE DATE NOT NULL,
EVENT_ID BIGINT PRIMARY KEY NOT NULL,
START_HOUR TIMESTAMP WITH TIME ZONE,
END_HOUR TIME WITH TIME ZONE,
EXPECTED_HOUR TIME WITH TIME ZONE
);
These hour fields represents a specific point in time during the day represented by the REPORT_DATE column. What I mean by that is that every TIME column represents a moment during the day specified in REPORT_DATE.
Some other points to consider:
We don't know why the START_HOUR is in TIMESTAMP format in the report we receive from the legacy system. But we import the data the way it comes to us.
The fields in the report are formatted according to the timezone of the client, so to refactor this table we need to combine the timezone of the client (we have this info) to properly insert the timestamps/times in UTC.
But now to the problem. The value of these columns are used to compute another values multiple times in our system, something like the following:
START_HOUR - END_HOUR (the result of this operation is currently being casted to TIME WITHOUT TIME ZONE)
START_HOUR < END_HOUR
START_HOUR + EXPECTED_HOUR
EXPECTED_HOUR - END_HOUR
EXPECTED_HOUR < '05:00'
After some research I found that is not recommended to use the type TIME WITH TIME ZONE (Postgres time with time zone equality) and now I'm a bit confused about what is the best way to refactor this table to deal with different time zones and handle the different column operations that we need to.
Besides that, I already know that is safe to subtract two columns of type TIMESTAMP WITH TIME ZONE. This subtraction operation is taking into account DST changes (Subtracting two columns of type timestamp with time zone) but how about the others? And the one subtracting a TIME from a TIMESTAMP?.
And about the table refactoring, should we use TIME WITH TIME ZONE anyways? Should we continue using TIME WITHOUT TIME ZONE? Or is better to forget the type TIME altogether and combine the DATE with the TIME and change the columns to TIMESTAMP WITH TIME ZONE?
I think these questions are related because the new column types we choose to use, will define how we operate with the columns.
You asserted that:
every TIME column represents a moment during the day specified in REPORT_DATE.
So you never cross the a dateline within the same row. I suggest to save 1x date 3x time and the time zone (as text or FK column):
CREATE TABLE legacy_table (
event_id bigint PRIMARY KEY NOT NULL
, report_date date NOT NULL
, start_hour time
, end_hour time
, expected_hour time
, tz text -- time zone
);
Like you already found, timetz (time with time zone) should generally be avoided. It cannot deal with DST rules properly (daylight saving time).
So basically what you already had. Just drop the date component from start_hour, that's dead freight. Cast timestamp to time to cut off the date. Like: (timestamp '2018-03-25 1:00:00')::time
tz can be any string accepted by the AT TIME ZONE construct, but to deal with different time zones reliably, it's best to use time zone names exclusively. Any name you find in the system catalog pg_timezone_names.
To optimize storage, you could collect allowed time zone names in a small lookup table and replace tz text with tz_id int REFERENCES my_tz_table.
Two example rows with and without DST:
INSERT INTO legacy_table VALUES
(1, '2018-03-25', '1:00', '3:00', '2:00', 'Europe/Vienna') -- sadly, with DST
, (2, '2018-03-25', '1:00', '3:00', '2:00', 'Europe/Moscow'); -- Russians got rid of DST
For representation purposes or calculations you can do things like:
SELECT (report_date + start_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS start_utc
, (report_date + end_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS end_utc
, (report_date + expected_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS expected_utc
-- START_HOUR - END_HOUR
, (report_date + start_hour) AT TIME ZONE tz
- (report_date + end_hour) AT TIME ZONE tz AS start_minus_end
FROM legacy_table;
You might create one or more views to readily display strings as needed. The table is for storing the information you need.
Note the parentheses! Else the operator + would bind before AT TIME ZONE due to operator precedence.
And behold the results:
db<>fiddle here
Since the time is manipulated in Vienna (like any place where silly DST rules apply), you get "surprising" results.
Related:
Accounting for DST in Postgres, when selecting scheduled items
Ignoring time zones altogether in Rails and PostgreSQL
I don't know What is difference between 4 ways convert timezone in postgresql:
SELECT (timestamp '2018-01-20 00:00:00' at time zone 'Asia/Saigon') at time zone 'UTC';
SELECT CAST('2018-01-20 00:00:00' as timestamp without time zone) at time zone 'Asia/Saigon' at time zone 'UTC'
SELECT (TO_TIMESTAMP('2018-01-20 00:00:00', 'YYYY-MM-DD HH24:MI:SS') at time zone 'Asia/Saigon') at time zone 'UTC'
SELECT ('2018-01-20 00:00:00' at time zone 'Asia/Saigon') at time zone 'UTC';
The results are different. Why?
The first two statements do the same thing.
The difference is the way in which a constant of type timestamp without time zone is created, but the result is the same in both cases.
The third statement creates a timestamp with time zone using to_timestamp, where the string is interpreted in your session time zone. This is then converted to a timestamp without time zone as the wall clock in Saigon would show, and then converted to a timestamp with time zone imagining the wall clock were teleported to UTC.
The fourth statement does the same as the third, because the string is implicitly cast to timestamp with time zone. There is an ambiguity here because AT TIME ZONE can also be applied to timestamp without time zone, but in case of doubt the preferred type of its category is used, which is timestamp with time zone.
The SQL standard differentiates timestamp without time zone and timestamp with time zone literals by the presence of a "+" or "-" symbol and time zone offset after the time. Hence, according to the standard
Also you can see below articles:
Section 8.5.1.3. Time Stamps
Time zone
I am trying to process input data putting timestamp when save it
tsSrc timestamp with time zone;
...
tsSrc := strTelegram.rte_data[ iPos ];-- this input data datetime
-- string e.g.'2015/12/13 21:35:26.000'
...
insert into telegram(
tld_id,
ddt_num, tld_src_timestamp,
tld_dst_timestamp, tld_year, tld_month,
tld_day, tld_hour, tld_min,
tld_sec, tld_data
) values(
uuId,
strTelegram.rte_type,
tsSrc,
strTelegram.rte_dst_timestamp,
extract(year from tsSrc), extract(month from tsSrc),
extract(day from tsSrc), extract(hour from tsSrc),
extract(minute from tsSrc), extract(second from tsSrc),
strTelegram.rte_data
);
But I have got unexpected result, tsSrc saved as 2015-12-13 20:35:26+03 i.e. has hour -1 shift, at the same time extract(hour from tsSrc) returns right value and saved as 21. What I am doing wrong?
Timezone is set as 'MSK-3' in postgresql.conf, select now() returns right datetime, postgresql 9.3.
You need to understand handling of timestamp (timestamp without time zone) and timestamptz (timestamp with time zone) and how each interacts with the timezone setting of your current session.
To explain the "difference" you observe we would need to know the exact table definition and the timezone setting of the session saving the row, as well as the timezone setting of the session displaying the row.
For example, if you take the timestamp literal '2015-12-13 21:35:26' (use ISO format to avoid additional complication with the input format!) and save it to a timestamptz column in a session with time zone offset +2 and later select the same row in a session with with time zone offset +3, then you get what you see:
SELECT '2015-12-13 21:35:26'::timestamp AT TIME ZONE '+2' AT TIME ZONE '+3';
Result:
'2015-12-13 20:35:26'
In other words: the timestamptz value '2015-12-13 20:35:26+03' is exactly the same (same point in time) as '2015-12-13 21:35:26+02', only the display has been adapted to your time zone setting. When you treat the timestamptz value according to the clock on the wall in your corner of the world (like you do with extract(hour from tsSrc)), you get different results depending where you are currently (the timezone setting of your session).
Detailed explanation:
Ignoring timezones altogether in Rails and PostgreSQL
I have a column added_at of type timestamp without time zone. I want it's default value to be the current date-time but without time zone. The function now() returns a timezone as well.
How do I solve that problem?
SELECT now()::timestamp;
The cast converts the timestamptz returned by now() to the corresponding timestamp in your time zone - defined by the timezone setting of the session. That's also how the standard SQL function LOCALTIMESTAMP is implemented in Postgres.
If you don't operate in multiple time zones, that works just fine. Else switch to timestamptz for added_at. The difference?
Ignoring time zones altogether in Rails and PostgreSQL
BTW, this does exactly the same, just more noisy and expensive:
SELECT now() AT TIME ZONE current_setting('timezone');
Well you can do something like:
SELECT now() AT TIME ZONE current_setting('TimeZone');
SELECT now() AT TIME ZONE 'Europe/Paris';
SELECT now() AT TIME ZONE 'UTC';
Not sure how that makes any sense for a column "added_at". You almost always want an absolute timestamp (timestamp with time zone) not a floating one.
Edit responding to points below:
Yes, should use timestamp with time zone (absolute time) unless you have a good reason not to.
The client timezone is given by SHOW TimeZone or current_setting(...) as shown above.
Do take some time to skim the manuals - they cover all this quite well.
"Current Date/Time":
CURRENT_TIME and CURRENT_TIMESTAMP deliver values with time zone; LOCALTIME and LOCALTIMESTAMP deliver values without time zone.
New, and Native Answer in 2020
In PostgreSQL, If you only want the current date-time by calling CURRENT_TIMESTAMP() without time zone, and fractional digits in the seconds field which come after the decimal point of the seconds field?
(Tested on PostgreSQL v12.4)
Then use this:
SELECT CURRENT_TIMESTAMP(0)::TIMESTAMP WITHOUT TIME ZONE;
If you define your column's data type as timestamp (not as timestamptz), then you can store the timestamp without time zone, in that case you don't neet to add TIMESTAMP WITHOUT TIME ZONE
Like this:
CREATE TABLE foo (created timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP(0))
In the above function, 0 is passed to get rid of the fractional digits in the seconds field.
If your application doesn't care about timezone, you can use SELECT LOCALTIMESTAMP for it.
Ex:
SELECT LOCALTIMESTAMP
-- Result: 2023-01-30 17:43:33.628952