postgresql `at time zone` incorrect behaviour? - postgresql

I have a simple postgresql table with the following data:
create table ti (
l text,
t timestamp without time zone
);
insert into ti(l, t) values ('now', now());
I do a select using at time zone, I expect UTC + 5.30 hours
select now(), t, t at time zone 'Asia/Kolkata' from ti;
But this is what I get:
now| t| timezone
2019-06-06T12:11:42.576388Z| 2019-06-06T11:50:48.178689Z| 2019-06-06T06:20:48.178689Z
It subtracted 5.30 hours instead of adding them.
Sqlfiddle here

now() returns a timestamp with time zone. The time zone info will be stripped off when it is cast to a timestamp without time zone when it is saved in your table, but the actual value saved in there will depend on the session time zone.
You can see this behavior pretty easily:
postgres=# begin;
BEGIN
postgres=# set time zone utc;
SET
postgres=# insert into test select now();
INSERT 0 1
postgres=# set time zone 'US/Eastern';
SET
postgres=# insert into test select now();
INSERT 0 1
postgres=# select * from test;
a
----------------------------
2019-06-06 12:46:10.475424
2019-06-06 08:46:10.475424
(2 rows)
A little more explanation after your comment. I think the issue you are having is with converting between timestamp and timestamptz. Timestamps are much less confusing if you stick with just timestamptz. Let's remove now() from the discussion, since that adds an additional layer of complexity because converting from the result of now() to a timestamp without time zone depends on the session time zone.
select '2019-06-06 12:00:00UTC'::timestamp with time zone,
('2019-06-06 12:00:00UTC'::timestamp with time zone) at time zone 'Asia/Kolkata';
timestamptz | timezone
------------------------+---------------------
2019-06-06 12:00:00+00 | 2019-06-06 17:30:00
I believe this is what you expect? We convert from a timestamp with time zone to a timestamp without time zone at a specific time zone.
What you are doing is similar to this, though:
select '2019-06-06 12:00:00UTC'::timestamp with time zone,
(('2019-06-06 12:00:00UTC'::timestamp with time zone)::timestamp without time zone) at time zone 'Asia/Kolkata';
timestamptz | timezone
------------------------+------------------------
2019-06-06 12:00:00+00 | 2019-06-06 06:30:00+00
(1 row)
I think you will find this much less confusing if you can store timestamp with time zone instead of timestamp without time zone.

Related

Why does PostgreSQL apparently coerce a string with timestamp to `TIMESTAMP WITH TIME ZONE` even without the time zone offset?

I am on PostgreSQL v11.10, and have the TimeZone set to UTC.
When I say select '2021-02-16 17:45+00' at time zone 'America/New_York';, I get 2021-02-16 12:45:00, which is correct and expected.
However, when I say select '2021-02-16 17:45' at time zone 'America/New_York';, I get the same result.
It seems like both strings are coerced to timestamp with time zone, which seems a bit counterintuitive to me in the case of the latter one. Why does PostgreSQL behave like that? Is it documented in the manual (I looked at the following places: AT TIME ZONE, then Date/Time Types and also Date/Time Input Interpretation and Handling of Invalid or Ambiguous Timestamps, all to no avail).
Because your TimeZone is set to UTC, any timestamp without a specified timezone will be interpreted as local to the UTC timezone. Since you as asking for a timestamp to be interpreted as America/New_York, Postgres will first interpret the timestamp as a UTC timestamp, then do the math to convert it to America/New_York.
Note that if you actually look at the types being sent to Postgres, it is not coercing into timestamp with timezone unless you specify it:
edb=# select pg_typeof('2021-02-16 17:45' at time zone 'America/New_York');
pg_typeof
-----------------------------
timestamp without time zone
(1 row)
edb=# select pg_typeof('2021-02-16 17:45+00' at time zone 'America/New_York');
pg_typeof
-----------------------------
timestamp without time zone
(1 row)
edb=# select pg_typeof('2021-02-16 17:45+00'::timestamptz at time zone 'America/New_York');
pg_typeof
-----------------------------
timestamp without time zone
(1 row)
edb=# select pg_typeof('2021-02-16 17:45+00'::timestamptz);
pg_typeof
--------------------------
timestamp with time zone
(1 row)
edb=# select pg_typeof('2021-02-16 17:45+00'::timestamp);
pg_typeof
-----------------------------
timestamp without time zone
(1 row)

Postgres cast to TIMESTAMPTZ

What is the behavior of PostgreSQL when we cast a DATE to TIMESTAMP to TIMESTAMPTZ
What time zone is used?
PostgreSQL server
Client that run the query (Current session)
If you cast a date to a timestamp, time zones don't play a role, because both data types are without a time zone. The resulting timestamp will be the beginning of the day.
If you cast date to timestamp with time zone, resulting timestamp will be the beginning of the date in the time zone defined by the parameter timezone in your current session.
SHOW timezone;
TimeZone
---------------
Europe/Vienna
(1 row)
SELECT CAST (DATE '2021-01-15' AS timestamp);
timestamp
---------------------
2021-01-15 00:00:00
(1 row)
SELECT CAST (DATE '2021-01-15' AS timestamp with time zone);
timestamptz
------------------------
2021-01-15 00:00:00+01
(1 row)
Casting date to timestamp will append time 00:00:00.0 to the date.
The time zone of the current server session will be used.
By default this is the time zone setting of Postgresql server.
You can change the time zone of the current server session like this:
set time zone 'Europe/Sofia';
I do not think that the time zone of the client has any effect. More on this issue here.
Edit
As Adrian Klaver noticed "When dealing with timestamps it's best to assume the worst". Therefore better set the session time zone explicitly.
Given the situation where one needs to change the column type from integer (as epoc seconds) to timzezonetz ->
casting directly from integer to timezonetz is not possible.
but you can do this:
ALTER TABLE table_name ALTER COLUMN column_name TYPE timestamptz
USING column_name::abstime::timestamptz

Postgres timestamp conversion is backwards

The server is in EST, I'm storing a timestamptz as lastlogin.
It seems like the conversion from UTC to EST is happening backwards.
select lastlogin at time zone 'EST' as lastlogin from users where id = 1;
-- > 2021-01-13 18:56:28
select lastlogin at time zone 'UTC' as lastlogin from users where id = 1;
-- > 2021-01-13 13:56:28
If to convert from UTC to EST I subtract 5 hours, why is EST 5 hours in the future? It's as if converting to UTC actually gives me the EST time, and converting to EST gives me the UTC time.
But this works:
select NOW() at time zone 'EST', NOW() at time zone 'UTC';
-- > 2021-01-13 14:23:21 2021-01-13 19:23:21
EDIT:
I found out that doing two conversions seems to work:
select lastlogin at time zone 'UTC' at time zone 'EST' as lastlogin from users where id = 1;
-- > 2021-01-13 13:56:28
This is shown on Popsql How to Convert UTC to Local Time Zone in PostgreSQL. Is it that the first at time zone will set the time zone for the time, while the second will actually do the conversion?
I figured it out.
First off, apparently that timestamp may not be a timestamptz. DBeaver says it is when I look at the table columns, but reports timestamp if I query it. Pgcli also says it is a timestamp.
That said, I found in the postgresql docs 9.9.3 AT TIME ZONE that:
The AT TIME ZONE operator converts time stamp without time zone to/from time stamp with time zone, and time with time zone values to different time zones.
The explanation in my question's edit was correct. AT TIME ZONE on a timestamp will set the time's time zone to whatever you specify, then the second AT TIME ZONE will convert it.
What you are seeing and why it is important to store timestamp values in timestamptz:
set timezone = 'US/Eastern';
test(5432)=> \d ts_test
Table "public.ts_test"
Column | Type | Collation | Nullable | Default
----------+-----------------------------+-----------+----------+---------
ts_tz | timestamp with time zone | | |
ts | timestamp without time zone | | |
ts_txt | character varying | | |
time_fld | time without time zone | | |
-- A timestamptz field always stores the value as UTC. A timestamp field is stored -- as a naive value. In below the timestamptz value is rotated to EST as that what -- the timezone is set to. The timestamp is also displayed in EST but with no time -- zone value, so naive.
select ts_tz, ts from ts_test ;
ts_tz | ts
--------------------------------+----------------------------
01/14/2021 11:37:13.217229 EST | 01/14/2021 11:37:13.217229
-- Here the time zone is being explicitly set and both values are the same as
-- timezone = 'EST'. The formally naive value does pick up a time zone designation -- per the docs "timestamp without time zone AT TIME ZONE zone Return Type --timestamp with time -- zone"
select ts_tz AT TIME ZONE 'EST', ts AT TIME ZONE 'EST' from ts_test;
timezone | timezone
----------------------------+--------------------------------
01/14/2021 11:37:13.217229 | 01/14/2021 11:37:13.217229 EST
-- Here the the timestamptz value correctly gets rotated to UTC. The timestamp
-- value gets set to UTC then gets rotated back to EST
select ts_tz AT TIME ZONE 'UTC', ts AT TIME ZONE 'UTC' from ts_test;
timezone | timezone
----------------------------+--------------------------------
01/14/2021 16:37:13.217229 | 01/14/2021 06:37:13.217229 EST

Why datetime without time zone foormatted as it have timezone?

I am experimenting with time zones.
My postgres table created as:
Table "public.xx"
Column | Type | Collation | Nullable | Default
--------+-----------------------------+-----------+----------+---------
dtz | timestamp with time zone | | |
dt | timestamp without time zone | | |
The server timezone is 'UTC'. I know this from show timezone;
Then I insert data:
insert into xx values( TIMESTAMP WITH TIME ZONE '2018-08-01 13:00:00+3', TIMESTAMP WITH TIME ZONE '2018-08-01 13:00:00+3' );
INSERT 0 1
tucha=> select * from xx;
dtz | dt
------------------------+---------------------
2018-08-01 10:00:00+00 | 2018-08-01 10:00:00
(1 row)
Results are easy to understand: the dates are stored in UTC thus 3 is subtracted.
Also notice that dtz have +00
Now when I use at time zone
tucha=> select dtz at time zone 'UTC', dt at time zone 'UTC' from xx;
timezone | timezone
---------------------+------------------------
2018-08-01 10:00:00 | 2018-08-01 10:00:00+00
(1 row)
The +00 is added for field which has no timezone and viceversa: no +00 for dtz field.
Why this happened?
a short answer is - because it is desined this way:
https://www.postgresql.org/docs/current/static/functions-datetime.html#FUNCTIONS-DATETIME-ZONECONVERT
timestamp without time zone AT TIME ZONE zone returns timestamp
with time zone
timestamp with time zone AT TIME ZONE zone returns
timestamp without time zone
Longer:
as you said your client works in UTC timezone and server always stores timestamptz in UTC, thus select times match (you use UTC everywhere).
first select timestamptz is shown with +00 because this is timezone aware field - and it shows you the time zone. dt knows nothing of time zones, thus shows none...
Now when you use AT TIME ZONE you ask to show timestamps at SOME SPECIFIC time zone, thus time zone aware data type shows you the time for specific zone (adding hours and so), but after "moving" time it can't be reused as time zone aware (because it's not server UTC time anymore), thus it hides TZ. Opposite the time that was not aware of time zone, when displayed at some specific time zone, gains that "awareness" and deducting hours it shows +00 so you would know you operated on TZ not aware time stamp. I think that is the logic here.
I have found good article how to work with timezones (RU):
So – timestamp at time zone gives timestamptz which represents the moment in time when local time at given zone was as given.
and timestamptz at time zone gives timestamp, which shows what the time was in given timezone at given point in time.

Using current time in UTC as default value in PostgreSQL

I have a column of the TIMESTAMP WITHOUT TIME ZONE type and would like to have that default to the current time in UTC. Getting the current time in UTC is easy:
postgres=# select now() at time zone 'utc';
timezone
----------------------------
2013-05-17 12:52:51.337466
(1 row)
As is using the current timestamp for a column:
postgres=# create temporary table test(id int, ts timestamp without time zone default current_timestamp);
CREATE TABLE
postgres=# insert into test values (1) returning ts;
ts
----------------------------
2013-05-17 14:54:33.072725
(1 row)
But that uses local time. Trying to force that to UTC results in a syntax error:
postgres=# create temporary table test(id int, ts timestamp without time zone default now() at time zone 'utc');
ERROR: syntax error at or near "at"
LINE 1: ...int, ts timestamp without time zone default now() at time zo...
A function is not even needed. Just put parentheses around the default expression:
create temporary table test(
id int,
ts timestamp without time zone default (now() at time zone 'utc')
);
Still another solution:
timezone('utc', now())
Wrap it in a function:
create function now_utc() returns timestamp as $$
select now() at time zone 'utc';
$$ language sql;
create temporary table test(
id int,
ts timestamp without time zone default now_utc()
);
What about
now()::timestamp
If your other timestamp are without time zone then this cast will yield the matching type "timestamp without time zone" for the current time.
I would like to read what others think about that option, though. I still don't trust in my understanding of this "with/without" time zone stuff.
EDIT:
Adding Michael Ekoka's comment here because it clarifies an important point:
Caveat. The question is about generating default timestamp in UTC for
a timestamp column that happens to not store the time zone (perhaps
because there's no need to store the time zone if you know that all
your timestamps share the same). What your solution does is to
generate a local timestamp (which for most people will not necessarily
be set to UTC) and store it as a naive timestamp (one that does not
specify its time zone).
These are 2 equivalent solutions:
(in the following code, you should substitute 'UTC' for zone and now() for timestamp)
timestamp AT TIME ZONE zone - SQL-standard-conforming
timezone(zone, timestamp) - arguably more readable
The function timezone(zone, timestamp) is equivalent to the SQL-conforming construct timestamp AT TIME ZONE zone.
Explanation:
zone can be specified either as a text string (e.g., 'UTC') or as an interval (e.g., INTERVAL '-08:00') - here is a list of all available time zones
timestamp can be any value of type timestamp
now() returns a value of type timestamp (just what we need) with your database's default time zone attached (e.g. 2018-11-11T12:07:22.3+05:00).
timezone('UTC', now()) turns our current time (of type timestamp with time zone) into the timezonless equivalent in UTC.
E.g., SELECT timestamp with time zone '2020-03-16 15:00:00-05' AT TIME ZONE 'UTC' will return 2020-03-16T20:00:00Z.
Docs: timezone()
Function already exists:
timezone('UTC'::text, now())