Postgres timestamp conversion is backwards - postgresql

The server is in EST, I'm storing a timestamptz as lastlogin.
It seems like the conversion from UTC to EST is happening backwards.
select lastlogin at time zone 'EST' as lastlogin from users where id = 1;
-- > 2021-01-13 18:56:28
select lastlogin at time zone 'UTC' as lastlogin from users where id = 1;
-- > 2021-01-13 13:56:28
If to convert from UTC to EST I subtract 5 hours, why is EST 5 hours in the future? It's as if converting to UTC actually gives me the EST time, and converting to EST gives me the UTC time.
But this works:
select NOW() at time zone 'EST', NOW() at time zone 'UTC';
-- > 2021-01-13 14:23:21 2021-01-13 19:23:21
EDIT:
I found out that doing two conversions seems to work:
select lastlogin at time zone 'UTC' at time zone 'EST' as lastlogin from users where id = 1;
-- > 2021-01-13 13:56:28
This is shown on Popsql How to Convert UTC to Local Time Zone in PostgreSQL. Is it that the first at time zone will set the time zone for the time, while the second will actually do the conversion?

I figured it out.
First off, apparently that timestamp may not be a timestamptz. DBeaver says it is when I look at the table columns, but reports timestamp if I query it. Pgcli also says it is a timestamp.
That said, I found in the postgresql docs 9.9.3 AT TIME ZONE that:
The AT TIME ZONE operator converts time stamp without time zone to/from time stamp with time zone, and time with time zone values to different time zones.
The explanation in my question's edit was correct. AT TIME ZONE on a timestamp will set the time's time zone to whatever you specify, then the second AT TIME ZONE will convert it.

What you are seeing and why it is important to store timestamp values in timestamptz:
set timezone = 'US/Eastern';
test(5432)=> \d ts_test
Table "public.ts_test"
Column | Type | Collation | Nullable | Default
----------+-----------------------------+-----------+----------+---------
ts_tz | timestamp with time zone | | |
ts | timestamp without time zone | | |
ts_txt | character varying | | |
time_fld | time without time zone | | |
-- A timestamptz field always stores the value as UTC. A timestamp field is stored -- as a naive value. In below the timestamptz value is rotated to EST as that what -- the timezone is set to. The timestamp is also displayed in EST but with no time -- zone value, so naive.
select ts_tz, ts from ts_test ;
ts_tz | ts
--------------------------------+----------------------------
01/14/2021 11:37:13.217229 EST | 01/14/2021 11:37:13.217229
-- Here the time zone is being explicitly set and both values are the same as
-- timezone = 'EST'. The formally naive value does pick up a time zone designation -- per the docs "timestamp without time zone AT TIME ZONE zone Return Type --timestamp with time -- zone"
select ts_tz AT TIME ZONE 'EST', ts AT TIME ZONE 'EST' from ts_test;
timezone | timezone
----------------------------+--------------------------------
01/14/2021 11:37:13.217229 | 01/14/2021 11:37:13.217229 EST
-- Here the the timestamptz value correctly gets rotated to UTC. The timestamp
-- value gets set to UTC then gets rotated back to EST
select ts_tz AT TIME ZONE 'UTC', ts AT TIME ZONE 'UTC' from ts_test;
timezone | timezone
----------------------------+--------------------------------
01/14/2021 16:37:13.217229 | 01/14/2021 06:37:13.217229 EST

Related

Query Postgres Timestamp with Timezone wrong result

I have a table with a time column defined as
time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
my database and the server have the timezone UTC
Tue Apr 20 18:22:50 UTC 2021
When I ran the following query
SELECT id, time AT TIME ZONE 'utc' AS "TZ: UTC", time AT TIME ZONE 'Europe/Berlin' AS "TZ: Berlin" FROM orders ORDER BY id DESC;
results to
id | TZ: UTC | TZ: Berlin
-----+------------------------------+------------------------------
600 | 2021-04-20 18:18:07.11372+00 | 2021-04-20 16:18:07.11372+00
I would have expected in the TZ: Berlin column is
2021-04-20 20:18:07.11372+00.
When I run the query
SELECT id, time AT TIME ZONE 'utc' AS "TZ: UTC", time AT TIME ZONE 'UTC+2' AS "TZ: UTC+2" FROM orders ORDER BY id DESC
which results to the expected
id | TZ: UTC | TZ: UTC+2
-----+------------------------------+------------------------------
600 | 2021-04-20 18:18:07.11372+00 | 2021-04-20 20:18:07.11372+00
At this time of the year (summer) the timezones UTC+2 and Europe/Berlin are identical.
So what am I missing?
Kind regards
PS: The Version is
$ postgres -V postgres
postgres (PostgreSQL) 12.6
When I run
SELECT NOW() at time zone 'Europe/Berlin';
I get
timezone
----------------------------
2021-04-20 20:41:57.155641
Adding AT TIME ZONE to a timestamp without time zone merely adds the time zone to the timestamp. In your case, the time was stored as:
2021-04-20 18:18:07.11372
Adding the UTC time zone didn't change the hour portion of the time because it matches your session time zone.
Adding the Europe/Berlin time zone had the following effect:
2021-04-20 18:18:07.11372 = the time in Berlin, so we subtract two hours to get the time in UTC.
The case of UTC+2 is even more confusing. I suggest reading more about the posix time specifications.
Note this important bit:
The positive sign is used for zones west of Greenwich.
It worked as you expected, but for the wrong reason.
This might make it more clear:
SELECT t,
t at time zone 'UTC' as utc,
t at time zone 'Europe/Berlin' as berlin,
t at time zone 'UTC+2' as somewhere_west_of_utc,
t at time zone 'UTC-2' as berlinish
FROM (
VALUES ('2021-04-20T12:00:00'::timestamp)
) as v(t);
t | utc | berlin | somewhere_west_of_utc | berlinish
---------------------+------------------------+------------------------+------------------------+------------------------
2021-04-20 12:00:00 | 2021-04-20 12:00:00+00 | 2021-04-20 10:00:00+00 | 2021-04-20 14:00:00+00 | 2021-04-20 10:00:00+00
(1 row)
Again, I would strongly advise you to use timestamp with time zone where all the timestamps will be stored as UTC.
If you execute AT TIME ZONE on a timestamp without time zone, you get the absolute time when a clock in that time zone shows that time.
When a clock in Berlin shows 18:00, it is 16:00 UTC.

postgresql `at time zone` incorrect behaviour?

I have a simple postgresql table with the following data:
create table ti (
l text,
t timestamp without time zone
);
insert into ti(l, t) values ('now', now());
I do a select using at time zone, I expect UTC + 5.30 hours
select now(), t, t at time zone 'Asia/Kolkata' from ti;
But this is what I get:
now| t| timezone
2019-06-06T12:11:42.576388Z| 2019-06-06T11:50:48.178689Z| 2019-06-06T06:20:48.178689Z
It subtracted 5.30 hours instead of adding them.
Sqlfiddle here
now() returns a timestamp with time zone. The time zone info will be stripped off when it is cast to a timestamp without time zone when it is saved in your table, but the actual value saved in there will depend on the session time zone.
You can see this behavior pretty easily:
postgres=# begin;
BEGIN
postgres=# set time zone utc;
SET
postgres=# insert into test select now();
INSERT 0 1
postgres=# set time zone 'US/Eastern';
SET
postgres=# insert into test select now();
INSERT 0 1
postgres=# select * from test;
a
----------------------------
2019-06-06 12:46:10.475424
2019-06-06 08:46:10.475424
(2 rows)
A little more explanation after your comment. I think the issue you are having is with converting between timestamp and timestamptz. Timestamps are much less confusing if you stick with just timestamptz. Let's remove now() from the discussion, since that adds an additional layer of complexity because converting from the result of now() to a timestamp without time zone depends on the session time zone.
select '2019-06-06 12:00:00UTC'::timestamp with time zone,
('2019-06-06 12:00:00UTC'::timestamp with time zone) at time zone 'Asia/Kolkata';
timestamptz | timezone
------------------------+---------------------
2019-06-06 12:00:00+00 | 2019-06-06 17:30:00
I believe this is what you expect? We convert from a timestamp with time zone to a timestamp without time zone at a specific time zone.
What you are doing is similar to this, though:
select '2019-06-06 12:00:00UTC'::timestamp with time zone,
(('2019-06-06 12:00:00UTC'::timestamp with time zone)::timestamp without time zone) at time zone 'Asia/Kolkata';
timestamptz | timezone
------------------------+------------------------
2019-06-06 12:00:00+00 | 2019-06-06 06:30:00+00
(1 row)
I think you will find this much less confusing if you can store timestamp with time zone instead of timestamp without time zone.

Inserting current_timestamp at time zone GMT gives local time instead

I got a PostgreSQL database in which I have a date field in the invoices table:
Column | Type |
payment_date | timestamp with time zone |
The server is located at GMT-5, as you can see:
$date
Tue Jan 22 17:33:01 EST 2019
Getting the GMT time via a PostgreSQL command gives me:
select current_timestamp at time zone 'GMT';
timezone
----------------------------
2019-01-22 22:33:01.087354
The problem is when I do an insertion/update:
update invoices set payment_date = current_timestamp at time zone 'GMT'
Then, when I get the query result...
select payment_date from invoices
it gives me:
2019-01-22 22:33:01.087354-05
Wrong! should be giving me 2019-01-22 22:33:01.087354-00
What am I doing wrong?
Your field payment_date is of the (correct) type timestamp with time zone, so the timestamp without time zone that is the result of the AT TIME ZONE operation is converted using your session time zone.
PostgreSQL's timestamp with time zone data type does not store time zone information, it stores an UTC timestamp that is converted to the session time zone on display.
You are best advised to store the timestamp as timestamp with time zone and convert it to the desired time zone when you display it.

Why datetime without time zone foormatted as it have timezone?

I am experimenting with time zones.
My postgres table created as:
Table "public.xx"
Column | Type | Collation | Nullable | Default
--------+-----------------------------+-----------+----------+---------
dtz | timestamp with time zone | | |
dt | timestamp without time zone | | |
The server timezone is 'UTC'. I know this from show timezone;
Then I insert data:
insert into xx values( TIMESTAMP WITH TIME ZONE '2018-08-01 13:00:00+3', TIMESTAMP WITH TIME ZONE '2018-08-01 13:00:00+3' );
INSERT 0 1
tucha=> select * from xx;
dtz | dt
------------------------+---------------------
2018-08-01 10:00:00+00 | 2018-08-01 10:00:00
(1 row)
Results are easy to understand: the dates are stored in UTC thus 3 is subtracted.
Also notice that dtz have +00
Now when I use at time zone
tucha=> select dtz at time zone 'UTC', dt at time zone 'UTC' from xx;
timezone | timezone
---------------------+------------------------
2018-08-01 10:00:00 | 2018-08-01 10:00:00+00
(1 row)
The +00 is added for field which has no timezone and viceversa: no +00 for dtz field.
Why this happened?
a short answer is - because it is desined this way:
https://www.postgresql.org/docs/current/static/functions-datetime.html#FUNCTIONS-DATETIME-ZONECONVERT
timestamp without time zone AT TIME ZONE zone returns timestamp
with time zone
timestamp with time zone AT TIME ZONE zone returns
timestamp without time zone
Longer:
as you said your client works in UTC timezone and server always stores timestamptz in UTC, thus select times match (you use UTC everywhere).
first select timestamptz is shown with +00 because this is timezone aware field - and it shows you the time zone. dt knows nothing of time zones, thus shows none...
Now when you use AT TIME ZONE you ask to show timestamps at SOME SPECIFIC time zone, thus time zone aware data type shows you the time for specific zone (adding hours and so), but after "moving" time it can't be reused as time zone aware (because it's not server UTC time anymore), thus it hides TZ. Opposite the time that was not aware of time zone, when displayed at some specific time zone, gains that "awareness" and deducting hours it shows +00 so you would know you operated on TZ not aware time stamp. I think that is the logic here.
I have found good article how to work with timezones (RU):
So – timestamp at time zone gives timestamptz which represents the moment in time when local time at given zone was as given.
and timestamptz at time zone gives timestamp, which shows what the time was in given timezone at given point in time.

Converting timestamp to timestamp in a specific time zone in Postgres

I have a table that has a column whose data type is timestamp without time zone , I'm trying to convert it to timestamp without time zone at a given time zone.
These are my Postgres 9.3 settings
select current_setting('TIMEZONE');
current_setting
-----------------
Hongkong
select * from pg_timezone_names where name = 'Hongkong';
name | abbrev | utc_offset | is_dst
----------+--------+------------+--------
Hongkong | HKT | 08:00:00 | f
Here is what I did to convert it to HKT:
-- this SQL gives me what I expected
select '2015-01-05 12:00:00'::timestamp without time zone
at time zone 'UTC'
at time zone 'HKT';
---------------------
2015-01-05 20:00:00
-- Shouldn't this produce the same result with the above one?
-- How do I make this work?
-- Don't tell me to change it to UTC-08:00 ...
select '2015-01-05 12:00:00'::timestamp without time zone
at time zone 'UTC'
at time zone 'UTC+08:00';
---------------------
2015-01-05 04:00:00 -- WHY?
The reason behind this is why you really shouldn't use time-zone offsets in PostgreSQL (unless you are know exactly what you do).
The time zone 'UTC+08:00' is a POSIX-style time zone specification, 'Hongkong' is the exact time zone name, and 'HKT' is (one of) its abbreviation(s).
The pg_timezone_names system view's utc_offset column is defined to be the Offset from UTC (positive means east of Greenwich).
But in the POSIX-style time zone specification, the offset part is different:
... Another issue to keep in mind is that in POSIX time zone names, positive offsets are used for locations west of Greenwich. Everywhere else, PostgreSQL follows the ISO-8601 convention that positive timezone offsets are east of Greenwich.
So, instead of using offsets (as intervals), or POSIX-style time zone specifications, you should use:
Time zone abbreviations, if you want to deal with daylight saving rules on your own,
Exact time zone names, everywhere else (preferred).
In short, this is the difference between abbreviations and full names: abbreviations represent a specific offset from UTC, whereas many of the full names imply a local daylight-savings time rule, and so have two possible UTC offsets.
To complicate matters, some jurisdictions have used the same timezone abbreviation to mean different UTC offsets at different times; for example, in Moscow MSK has meant UTC+3 in some years and UTC+4 in others. PostgreSQL interprets such abbreviations according to whatever they meant (or had most recently meant) on the specified date; but, as with the EST example above, this is not necessarily the same as local civil time on that date.
But the most simple solution is to use timestamp with time zone: you already set your TimeZone setting to 'Hongkong', so timestamp with time zone values will be displayed in that time zone to your (PostgreSQL) client.
set time zone 'Hongkong';
select current_setting('TimeZone') TimeZone,
dt original,
dt AT TIME ZONE 'UTC' AT TIME ZONE 'UTC+08:00' "UTC+08:00",
dt AT TIME ZONE 'UTC' AT TIME ZONE 'UTC+8' "UTC+8",
dt AT TIME ZONE 'UTC' AT TIME ZONE 'UTC-8' "UTC-8",
dt AT TIME ZONE 'UTC' AT TIME ZONE INTERVAL '+08:00' "INTERVAL +08:00",
dt AT TIME ZONE 'UTC' AT TIME ZONE 'HKT' "HKT",
dt AT TIME ZONE 'UTC' AT TIME ZONE 'Hongkong' "Hongkong",
dt AT TIME ZONE 'UTC' "with time zone"
from (values (timestamp '2015-01-05 12:00:00')) v(dt);
-- TIMEZONE | ORIGINAL | UTC+08:00
-- ---------+---------------------+--------------------
-- Hongkong | 2015-01-05 12:00:00 | 2015-01-05 04:00:00
-- UTC+8 | UTC-8 | INTERVAL +08:00
-- --------------------+---------------------+--------------------
-- 2015-01-05 04:00:00 | 2015-01-05 20:00:00 | 2015-01-05 20:00:00
-- HKT | HONGKONG | WITH TIME ZONE
-- --------------------+---------------------+-----------------------
-- 2015-01-05 20:00:00 | 2015-01-05 20:00:00 | 2015-01-05 20:00:00+08
SQLFiddle