I have a table with a time column defined as
time TIMESTAMP DEFAULT CURRENT_TIMESTAMP
my database and the server have the timezone UTC
Tue Apr 20 18:22:50 UTC 2021
When I ran the following query
SELECT id, time AT TIME ZONE 'utc' AS "TZ: UTC", time AT TIME ZONE 'Europe/Berlin' AS "TZ: Berlin" FROM orders ORDER BY id DESC;
results to
id | TZ: UTC | TZ: Berlin
-----+------------------------------+------------------------------
600 | 2021-04-20 18:18:07.11372+00 | 2021-04-20 16:18:07.11372+00
I would have expected in the TZ: Berlin column is
2021-04-20 20:18:07.11372+00.
When I run the query
SELECT id, time AT TIME ZONE 'utc' AS "TZ: UTC", time AT TIME ZONE 'UTC+2' AS "TZ: UTC+2" FROM orders ORDER BY id DESC
which results to the expected
id | TZ: UTC | TZ: UTC+2
-----+------------------------------+------------------------------
600 | 2021-04-20 18:18:07.11372+00 | 2021-04-20 20:18:07.11372+00
At this time of the year (summer) the timezones UTC+2 and Europe/Berlin are identical.
So what am I missing?
Kind regards
PS: The Version is
$ postgres -V postgres
postgres (PostgreSQL) 12.6
When I run
SELECT NOW() at time zone 'Europe/Berlin';
I get
timezone
----------------------------
2021-04-20 20:41:57.155641
Adding AT TIME ZONE to a timestamp without time zone merely adds the time zone to the timestamp. In your case, the time was stored as:
2021-04-20 18:18:07.11372
Adding the UTC time zone didn't change the hour portion of the time because it matches your session time zone.
Adding the Europe/Berlin time zone had the following effect:
2021-04-20 18:18:07.11372 = the time in Berlin, so we subtract two hours to get the time in UTC.
The case of UTC+2 is even more confusing. I suggest reading more about the posix time specifications.
Note this important bit:
The positive sign is used for zones west of Greenwich.
It worked as you expected, but for the wrong reason.
This might make it more clear:
SELECT t,
t at time zone 'UTC' as utc,
t at time zone 'Europe/Berlin' as berlin,
t at time zone 'UTC+2' as somewhere_west_of_utc,
t at time zone 'UTC-2' as berlinish
FROM (
VALUES ('2021-04-20T12:00:00'::timestamp)
) as v(t);
t | utc | berlin | somewhere_west_of_utc | berlinish
---------------------+------------------------+------------------------+------------------------+------------------------
2021-04-20 12:00:00 | 2021-04-20 12:00:00+00 | 2021-04-20 10:00:00+00 | 2021-04-20 14:00:00+00 | 2021-04-20 10:00:00+00
(1 row)
Again, I would strongly advise you to use timestamp with time zone where all the timestamps will be stored as UTC.
If you execute AT TIME ZONE on a timestamp without time zone, you get the absolute time when a clock in that time zone shows that time.
When a clock in Berlin shows 18:00, it is 16:00 UTC.
Related
The server is in EST, I'm storing a timestamptz as lastlogin.
It seems like the conversion from UTC to EST is happening backwards.
select lastlogin at time zone 'EST' as lastlogin from users where id = 1;
-- > 2021-01-13 18:56:28
select lastlogin at time zone 'UTC' as lastlogin from users where id = 1;
-- > 2021-01-13 13:56:28
If to convert from UTC to EST I subtract 5 hours, why is EST 5 hours in the future? It's as if converting to UTC actually gives me the EST time, and converting to EST gives me the UTC time.
But this works:
select NOW() at time zone 'EST', NOW() at time zone 'UTC';
-- > 2021-01-13 14:23:21 2021-01-13 19:23:21
EDIT:
I found out that doing two conversions seems to work:
select lastlogin at time zone 'UTC' at time zone 'EST' as lastlogin from users where id = 1;
-- > 2021-01-13 13:56:28
This is shown on Popsql How to Convert UTC to Local Time Zone in PostgreSQL. Is it that the first at time zone will set the time zone for the time, while the second will actually do the conversion?
I figured it out.
First off, apparently that timestamp may not be a timestamptz. DBeaver says it is when I look at the table columns, but reports timestamp if I query it. Pgcli also says it is a timestamp.
That said, I found in the postgresql docs 9.9.3 AT TIME ZONE that:
The AT TIME ZONE operator converts time stamp without time zone to/from time stamp with time zone, and time with time zone values to different time zones.
The explanation in my question's edit was correct. AT TIME ZONE on a timestamp will set the time's time zone to whatever you specify, then the second AT TIME ZONE will convert it.
What you are seeing and why it is important to store timestamp values in timestamptz:
set timezone = 'US/Eastern';
test(5432)=> \d ts_test
Table "public.ts_test"
Column | Type | Collation | Nullable | Default
----------+-----------------------------+-----------+----------+---------
ts_tz | timestamp with time zone | | |
ts | timestamp without time zone | | |
ts_txt | character varying | | |
time_fld | time without time zone | | |
-- A timestamptz field always stores the value as UTC. A timestamp field is stored -- as a naive value. In below the timestamptz value is rotated to EST as that what -- the timezone is set to. The timestamp is also displayed in EST but with no time -- zone value, so naive.
select ts_tz, ts from ts_test ;
ts_tz | ts
--------------------------------+----------------------------
01/14/2021 11:37:13.217229 EST | 01/14/2021 11:37:13.217229
-- Here the time zone is being explicitly set and both values are the same as
-- timezone = 'EST'. The formally naive value does pick up a time zone designation -- per the docs "timestamp without time zone AT TIME ZONE zone Return Type --timestamp with time -- zone"
select ts_tz AT TIME ZONE 'EST', ts AT TIME ZONE 'EST' from ts_test;
timezone | timezone
----------------------------+--------------------------------
01/14/2021 11:37:13.217229 | 01/14/2021 11:37:13.217229 EST
-- Here the the timestamptz value correctly gets rotated to UTC. The timestamp
-- value gets set to UTC then gets rotated back to EST
select ts_tz AT TIME ZONE 'UTC', ts AT TIME ZONE 'UTC' from ts_test;
timezone | timezone
----------------------------+--------------------------------
01/14/2021 16:37:13.217229 | 01/14/2021 06:37:13.217229 EST
I’ve been conditioned to store past dates as UTC in a database since that is in fact when the event occurred. For future dates, I would store it with a specific timezone, to avoid changes such as leap seconds or timezone rule changes.
Postgres has timestamp with timezone, but under the covers, it stores it as UTC, inferring that the specified timezone is an offset of UTC. If the timezone rules were to change, that would not be reflected in the column.
What is recommended in this case?
Think [of it] like a calendar event. UTC doesn’t make sense for that
It sounds like you want to store a localtime with respect to a certain timezone.
In that case, store a timestamp (without timezone) and the timezone in a separate column.
For example, suppose you want to record an event which will occur at 10 am on Feb 26, 2030 in Chicago
and it must be at 10 am localtime regardless of the timezone rule in effect on that date.
If the database stores the timestamp without timezone:
unutbu=# select '2030-02-26 10:00:00'::timestamp as localtime, 'America/Chicago' AS tzone;
+---------------------+-----------------+
| localtime | tzone |
+---------------------+-----------------+
| 2030-02-26 10:00:00 | America/Chicago |
+---------------------+-----------------+
Then later, you can find the UTC datetime of the event using
unutbu=# select '2030-02-26 10:00:00'::timestamp AT TIME ZONE 'America/Chicago' AT TIME ZONE 'UTC';
+---------------------+
| timezone |
+---------------------+
| 2030-02-26 16:00:00 |
+---------------------+
The query returns the UTC datetime, 2030-02-26 16:00:00, which corresponds to 2030-02-26 10:00:00 localtime in Chicago.
Using AT TIME ZONE delays the application of the timezone rules to when the query is made instead of when the timestamptz was inserted.
Using AT TIME ZONE on a timestamp localizes the datetime to the given timezone, but reports the datetime in the user's timezone.
Using AT TIME ZONE on a timestamptz converts the datetime to the given timezone, then drops the offset, thus returning a timestamp.
Above, AT TIME ZONE is used twice: first to localize a timestamp and next to convert the returned timestamptz to a new timezone (UTC). The result is a timestamp in UTC.
Here is an example, demonstrating AT TIME ZONE's behavior on timestamps:
unutbu=# SET timezone = 'America/Chicago';
unutbu=# SELECT '2030-02-26 10:00:00'::timestamp AT TIME ZONE 'America/Chicago';
+------------------------+
| timezone |
+------------------------+
| 2030-02-26 10:00:00-06 |
+------------------------+
unutbu=# SET timezone = 'America/Los_Angeles';
unutbu=# SELECT '2030-02-26 10:00:00'::timestamp AT TIME ZONE 'America/Chicago';
+------------------------+
| timezone |
+------------------------+
| 2030-02-26 08:00:00-08 |
+------------------------+
2030-02-26 10:00:00-06 and 2030-02-26 08:00:00-08 are the same datetimes but reported in different user timezones. This shows 10am in Chicago is 8am in Los Angeles (using current timezone definitions):
unutbu=# SELECT '2030-02-26 10:00:00-06'::timestamptz AT TIME ZONE 'America/Los_Angeles';
+---------------------+
| timezone |
+---------------------+
| 2030-02-26 08:00:00 |
+---------------------+
An alternative to using AT TIME ZONE twice is to set the user timezone to UTC. Then you could use
select localtime AT TIME ZONE tzone
Note that when done this way, a timestamptz is returned instead of a timestamp.
Beware that storing localtimes can be problematic because there can be nonexistent times and ambiguous times.
For example, 2018-03-11 02:30:00 is a nonexistent localtime in America/Chicago. Postgresql normalizes nonexistent localtimes by assuming it refers to the corresponding time after Daylight Savings Time (DST) has begun (as though someone forgot to set their clock forward):
unutbu=# select '2018-03-11 02:30:00'::timestamp AT TIME ZONE 'America/Chicago' AT TIME ZONE 'UTC';
+---------------------+
| timezone |
+---------------------+
| 2018-03-11 08:30:00 |
+---------------------+
(1 row)
unutbu=# select '2018-03-11 03:30:00'::timestamp AT TIME ZONE 'America/Chicago' AT TIME ZONE 'UTC';
+---------------------+
| timezone |
+---------------------+
| 2018-03-11 08:30:00 |
+---------------------+
(1 row)
An example of an ambiguous localtime is 2018-11-04 01:00:00 in America/Chicago. It occurs twice due to DST. Postgresql resolves this ambiguity by choosing the later time, after DST has ended:
unutbu=# select '2018-11-04 01:00:00'::timestamp AT TIME ZONE 'America/Chicago' AT TIME ZONE 'UTC';
+---------------------+
| timezone |
+---------------------+
| 2018-11-04 07:00:00 |
+---------------------+
Notice that this means there is no way to refer to 2018-11-04 06:00:00 UTC by storing localtimes in the America/Chicago timezone:
unutbu=# select '2018-11-04 00:59:59'::timestamp AT TIME ZONE 'America/Chicago' AT TIME ZONE 'UTC';
+---------------------+
| timezone |
+---------------------+
| 2018-11-04 05:59:59 |
+---------------------+
Particularly if you want to protect yourself from future time zune changes you should use timestamp with time zone.
PostgreSQL internally stores the number of microseconds since 2000-01-01 00:00:00, which is safe from time zone changes. If you keep updating your PostgreSQL, it will always display that absolute value correctly for your session time zone.
There is no provision for leap seconds in PostgreSQL.
For future dates, I would store it with a specific timezone, to avoid changes such as leap seconds or timezone rule changes.
That seems backward. The main advantage of UTC over other time zones is that it is less prone to unexpected future changes: UTC only introduces leap seconds at known, limited points of the calendar year; and has none of the frequent politically-mandated offset changes.
Storing values in some locally-managed timezone leaves those values more prone (compared with UTC) to arbitrary, unpredictable future changes in meaning.
So, the general recommendation is: Store all time values (whether date, or date+time) as UTC in the database, process them internally as UTC values; and convert to/from a local timezone only at external interfaces.
For PostgreSQL, that means prefer TIMESTAMP WITH TIME ZONE.
I got a PostgreSQL database in which I have a date field in the invoices table:
Column | Type |
payment_date | timestamp with time zone |
The server is located at GMT-5, as you can see:
$date
Tue Jan 22 17:33:01 EST 2019
Getting the GMT time via a PostgreSQL command gives me:
select current_timestamp at time zone 'GMT';
timezone
----------------------------
2019-01-22 22:33:01.087354
The problem is when I do an insertion/update:
update invoices set payment_date = current_timestamp at time zone 'GMT'
Then, when I get the query result...
select payment_date from invoices
it gives me:
2019-01-22 22:33:01.087354-05
Wrong! should be giving me 2019-01-22 22:33:01.087354-00
What am I doing wrong?
Your field payment_date is of the (correct) type timestamp with time zone, so the timestamp without time zone that is the result of the AT TIME ZONE operation is converted using your session time zone.
PostgreSQL's timestamp with time zone data type does not store time zone information, it stores an UTC timestamp that is converted to the session time zone on display.
You are best advised to store the timestamp as timestamp with time zone and convert it to the desired time zone when you display it.
I am experimenting with time zones.
My postgres table created as:
Table "public.xx"
Column | Type | Collation | Nullable | Default
--------+-----------------------------+-----------+----------+---------
dtz | timestamp with time zone | | |
dt | timestamp without time zone | | |
The server timezone is 'UTC'. I know this from show timezone;
Then I insert data:
insert into xx values( TIMESTAMP WITH TIME ZONE '2018-08-01 13:00:00+3', TIMESTAMP WITH TIME ZONE '2018-08-01 13:00:00+3' );
INSERT 0 1
tucha=> select * from xx;
dtz | dt
------------------------+---------------------
2018-08-01 10:00:00+00 | 2018-08-01 10:00:00
(1 row)
Results are easy to understand: the dates are stored in UTC thus 3 is subtracted.
Also notice that dtz have +00
Now when I use at time zone
tucha=> select dtz at time zone 'UTC', dt at time zone 'UTC' from xx;
timezone | timezone
---------------------+------------------------
2018-08-01 10:00:00 | 2018-08-01 10:00:00+00
(1 row)
The +00 is added for field which has no timezone and viceversa: no +00 for dtz field.
Why this happened?
a short answer is - because it is desined this way:
https://www.postgresql.org/docs/current/static/functions-datetime.html#FUNCTIONS-DATETIME-ZONECONVERT
timestamp without time zone AT TIME ZONE zone returns timestamp
with time zone
timestamp with time zone AT TIME ZONE zone returns
timestamp without time zone
Longer:
as you said your client works in UTC timezone and server always stores timestamptz in UTC, thus select times match (you use UTC everywhere).
first select timestamptz is shown with +00 because this is timezone aware field - and it shows you the time zone. dt knows nothing of time zones, thus shows none...
Now when you use AT TIME ZONE you ask to show timestamps at SOME SPECIFIC time zone, thus time zone aware data type shows you the time for specific zone (adding hours and so), but after "moving" time it can't be reused as time zone aware (because it's not server UTC time anymore), thus it hides TZ. Opposite the time that was not aware of time zone, when displayed at some specific time zone, gains that "awareness" and deducting hours it shows +00 so you would know you operated on TZ not aware time stamp. I think that is the logic here.
I have found good article how to work with timezones (RU):
So – timestamp at time zone gives timestamptz which represents the moment in time when local time at given zone was as given.
and timestamptz at time zone gives timestamp, which shows what the time was in given timezone at given point in time.
I have a table that has a column whose data type is timestamp without time zone , I'm trying to convert it to timestamp without time zone at a given time zone.
These are my Postgres 9.3 settings
select current_setting('TIMEZONE');
current_setting
-----------------
Hongkong
select * from pg_timezone_names where name = 'Hongkong';
name | abbrev | utc_offset | is_dst
----------+--------+------------+--------
Hongkong | HKT | 08:00:00 | f
Here is what I did to convert it to HKT:
-- this SQL gives me what I expected
select '2015-01-05 12:00:00'::timestamp without time zone
at time zone 'UTC'
at time zone 'HKT';
---------------------
2015-01-05 20:00:00
-- Shouldn't this produce the same result with the above one?
-- How do I make this work?
-- Don't tell me to change it to UTC-08:00 ...
select '2015-01-05 12:00:00'::timestamp without time zone
at time zone 'UTC'
at time zone 'UTC+08:00';
---------------------
2015-01-05 04:00:00 -- WHY?
The reason behind this is why you really shouldn't use time-zone offsets in PostgreSQL (unless you are know exactly what you do).
The time zone 'UTC+08:00' is a POSIX-style time zone specification, 'Hongkong' is the exact time zone name, and 'HKT' is (one of) its abbreviation(s).
The pg_timezone_names system view's utc_offset column is defined to be the Offset from UTC (positive means east of Greenwich).
But in the POSIX-style time zone specification, the offset part is different:
... Another issue to keep in mind is that in POSIX time zone names, positive offsets are used for locations west of Greenwich. Everywhere else, PostgreSQL follows the ISO-8601 convention that positive timezone offsets are east of Greenwich.
So, instead of using offsets (as intervals), or POSIX-style time zone specifications, you should use:
Time zone abbreviations, if you want to deal with daylight saving rules on your own,
Exact time zone names, everywhere else (preferred).
In short, this is the difference between abbreviations and full names: abbreviations represent a specific offset from UTC, whereas many of the full names imply a local daylight-savings time rule, and so have two possible UTC offsets.
To complicate matters, some jurisdictions have used the same timezone abbreviation to mean different UTC offsets at different times; for example, in Moscow MSK has meant UTC+3 in some years and UTC+4 in others. PostgreSQL interprets such abbreviations according to whatever they meant (or had most recently meant) on the specified date; but, as with the EST example above, this is not necessarily the same as local civil time on that date.
But the most simple solution is to use timestamp with time zone: you already set your TimeZone setting to 'Hongkong', so timestamp with time zone values will be displayed in that time zone to your (PostgreSQL) client.
set time zone 'Hongkong';
select current_setting('TimeZone') TimeZone,
dt original,
dt AT TIME ZONE 'UTC' AT TIME ZONE 'UTC+08:00' "UTC+08:00",
dt AT TIME ZONE 'UTC' AT TIME ZONE 'UTC+8' "UTC+8",
dt AT TIME ZONE 'UTC' AT TIME ZONE 'UTC-8' "UTC-8",
dt AT TIME ZONE 'UTC' AT TIME ZONE INTERVAL '+08:00' "INTERVAL +08:00",
dt AT TIME ZONE 'UTC' AT TIME ZONE 'HKT' "HKT",
dt AT TIME ZONE 'UTC' AT TIME ZONE 'Hongkong' "Hongkong",
dt AT TIME ZONE 'UTC' "with time zone"
from (values (timestamp '2015-01-05 12:00:00')) v(dt);
-- TIMEZONE | ORIGINAL | UTC+08:00
-- ---------+---------------------+--------------------
-- Hongkong | 2015-01-05 12:00:00 | 2015-01-05 04:00:00
-- UTC+8 | UTC-8 | INTERVAL +08:00
-- --------------------+---------------------+--------------------
-- 2015-01-05 04:00:00 | 2015-01-05 20:00:00 | 2015-01-05 20:00:00
-- HKT | HONGKONG | WITH TIME ZONE
-- --------------------+---------------------+-----------------------
-- 2015-01-05 20:00:00 | 2015-01-05 20:00:00 | 2015-01-05 20:00:00+08
SQLFiddle