I am struggling in understanding how PostgreSQL is working with TIMESTAMP and TIME ZONE. I have a Debian Server where timezone is set to UTC running PostgreSQL 9.3 where cluster's postgresql.conf file is also set to UTC. My client is PgAdmin and my computer locale is also set to UTC. Therefore, all timezones are equals, there are no offsets and no DST in storage or display.
I have read many posts over the internet and I cannot understand the following output of my queries.
I assume as true the following affirmations:
Default data-type TIMESTAMP is WITHOUT TIME ZONE;
PostgreSQL stores all TIMESTAMP and TIMESTAMPTZ in UTC;
When using TIMESTAMP type, timezone conversion is not performed;
When using TIMESTAMPTZ type timezone conversion is performed;
POSIX and ISO standards define offset in a opposite fashion;
PostgreSQL display in ISO format but abbreviation are POSIX compliant.
First question: are all those statements right?
Then I checked it out, issuing:
SELECT '2000-01-01 00:00:00' AT TIME ZONE 'UTC-1';
-- Result: 2000-01-01 01:00:00
-- UTC --> LT
-- TIMESTAMP
SELECT '2000-01-01 00:00:00'::TIMESTAMP AT TIME ZONE 'UTC-1';
-- Result: 1999-12-31 23:00:00+00
-- LT --> UTC
-- TIMESTAMPTZ
SELECT '2000-01-01 00:00:00'::TIMESTAMPTZ AT TIME ZONE 'UTC-1';
-- Result: 2000-01-01 01:00:00
-- UTC --> LT
-- TIMESTAMP
This is very confusing to me because:
it looks like default data-type infered from string without cast is TIMESTAMPTZ instead of TIMESTAMP;
conversion is performed in different way when using different data-type;
when specifying data-type with casting ::operator it is the opposite type that is retured.
Second question: How must I interpret the result of those three queries?
Bonus question: I have read there that it is better to always store data with TIMESTAMPTZ. Is this a good practice?
Thank you further for your help!
Related
I am trying to understand the behavior of AT TIME ZONE and I get some strange behavioral differences between SQL Server and PostgreSQL in this regard.
For demonstration purposes I produced two queries, one for SQL Server and one for PostgreSQL.
The SQL Server query:
SELECT
[LocalTime],
[LocalTime] AT TIME ZONE 'W. Europe Standard Time' AS [LocalTimeWithOffset],
([LocalTime] AT TIME ZONE 'W. Europe Standard Time') AT TIME ZONE 'UTC' AS [UniversalTime]
FROM
(VALUES
(CAST('2020-06-20 12:34:56.789' AS DATETIME2)),
(CAST('2020-12-20 12:34:56.789' AS DATETIME2))) AS [X] ([LocalTime])
produces this result (in SQL Server Management Studio):
LocalTime LocalTimeWithOffset UniversalTime
--------- ------------------- ----------------
2020-06-20 12:34:56.7890000 2020-06-20 12:34:56.7890000 +02:00 2020-06-20 10:34:56.7890000 +00:00
2020-12-20 12:34:56.7890000 2020-12-20 12:34:56.7890000 +01:00 2020-12-20 11:34:56.7890000 +00:00
which seems fine to me, since I live in the Netherlands: in the summertime, we are 2 hours ahead of UTC (due to daylight saving) and in the wintertime we are only 1 hour ahead of UTC.
BUT... the PostgreSQL query:
SELECT
"LocalTime",
"LocalTime" AT TIME ZONE 'WET' AS "LocalTimeWithOffset",
("LocalTime" AT TIME ZONE 'WET') AT TIME ZONE 'UTC' AS "UniversalTime"
FROM
(VALUES
(timestamp '2020-06-20 12:34:56.789'),
(timestamp '2020-12-20 12:34:56.789')) AS "X" ("LocalTime")
produces the following result (in pgAdmin):
LocalTime LocalTimeWithOffset UniversalTime
timestamp without time zone timestamp with time zone timestamp without time zone
--------------------------- ------------------------ ---------------------------
2020-06-20 12:34:56.789 2020-06-20 14:34:56.789+02 2020-06-20 12:34:56.789
2020-12-20 12:34:56.789 2020-12-20 13:34:56.789+01 2020-12-20 12:34:56.789
which seems incorrect to me.
After struggling with this for a little while, I came up with another PostgreSQL query, which seems to yield the same results as the SQL Server query:
SELECT
"LocalTime",
"LocalTime"::timestamptz AS "LocalTimeWithOffset",
"LocalTime"::timestamptz AT TIME ZONE 'UTC' AS "UniversalTime"
FROM
(VALUES
(timestamp '2020-06-20 12:34:56.789'),
(timestamp '2020-12-20 12:34:56.789')) AS "X" ("LocalTime")
But this query seems to be dependent on the PostgreSQL server's time zone. :-(
Is there any (preferably elegant) way to mimic the SQL Server logic/behavior in PostgreSQL without a dependency on the server's time zone (and use an explicitly specified time zone in the query)?
Edit
I am aware that it is a good practice to store UTC date/time values in the database and convert them (preferably in/by the client) to local date/time values when presented to end users. I am intending to do so as well, of course, but my current scenario is a database migration from SQL Server to PostgreSQL (as part of a big application overhaul), where the DATETIME2 table columns in the SQL Server database contain local date/time values, which I want to convert to UTC date/time values in corresponding timestamp columns in the PostgreSQL tables. My migration script will work fine with the queries above, so actually I do not have a blocking issue. The reason for this question is that I am just curious about the apparent differences between SQL Server and PostgreSQL regarding the AT TIME ZONE behavior and the ways to overcome those behavioral differences.
In postgresql, AT TIME ZONE allows time zones to be added to date/time values that lack them (TIMESTAMP WITHOUT TIME ZONE, ::timestamp), and allows TIMESTAMP WITH TIME ZONE values (::timestamptz) to be shifted to non-local time zones and the time zone designation removed.
So, in the first query you've written for postgres, you're trying to cast '2020-06-20 12:34:56.789' to timestamp, and you've applied AT TIME ZONE on it for LocalTimeWithOffset. This is the case where postgres considered the input as timestamp in UTC and converted to the corresponding time zone, in your case WET, which is UTC+2:00 in Summer and UTC+1:00 in winter.
Now, in the third case, you're first applying the AT TIME ZONE 'WET' to timestamp '2020-06-20 12:34:56.789', hence the input is considered as UTC and converted to WET, the output is timstamptz, and this output is then fed again to AT TIME ZONE 'UTC', so as per behavior of AT TIME ZONE, it gets converted to UTC and the timezone tag is removed, which is, the input we given.
So, AT TIME ZONE did not behaved weird in postgresql rather did exactly what it was supposed to do, according to the documentation.
Hope your confusion is cleared.
What is the behavior of PostgreSQL when we cast a DATE to TIMESTAMP to TIMESTAMPTZ
What time zone is used?
PostgreSQL server
Client that run the query (Current session)
If you cast a date to a timestamp, time zones don't play a role, because both data types are without a time zone. The resulting timestamp will be the beginning of the day.
If you cast date to timestamp with time zone, resulting timestamp will be the beginning of the date in the time zone defined by the parameter timezone in your current session.
SHOW timezone;
TimeZone
---------------
Europe/Vienna
(1 row)
SELECT CAST (DATE '2021-01-15' AS timestamp);
timestamp
---------------------
2021-01-15 00:00:00
(1 row)
SELECT CAST (DATE '2021-01-15' AS timestamp with time zone);
timestamptz
------------------------
2021-01-15 00:00:00+01
(1 row)
Casting date to timestamp will append time 00:00:00.0 to the date.
The time zone of the current server session will be used.
By default this is the time zone setting of Postgresql server.
You can change the time zone of the current server session like this:
set time zone 'Europe/Sofia';
I do not think that the time zone of the client has any effect. More on this issue here.
Edit
As Adrian Klaver noticed "When dealing with timestamps it's best to assume the worst". Therefore better set the session time zone explicitly.
Given the situation where one needs to change the column type from integer (as epoc seconds) to timzezonetz ->
casting directly from integer to timezonetz is not possible.
but you can do this:
ALTER TABLE table_name ALTER COLUMN column_name TYPE timestamptz
USING column_name::abstime::timestamptz
I'm struggling to understand how "AT TIME ZONE 'localtime'" exactly work? By playing with it, I found out that it acts exactly as "AT TIME ZONE 'UTC'"... But why? Is "localtime" a synonym of "UTC" in postgres? Or it comes from some setting (environment? connection timezone? although checked both, seems they are not related)...
There's "localtime" function but I think it is not involved here.
Sample SQLs:
# date
Thu Dec 8 12:00:05 AEDT 2016
# SELECT LOCALTIMESTAMP;
----------------------------
2016-12-08 01:13:29.444725
# SELECT LOCALTIMESTAMP AT TIME ZONE 'America/New_York';
-------------------------------
2016-12-08 06:08:31.183103+00
# SELECT LOCALTIMESTAMP AT TIME ZONE'localtime';
------------------------------
2016-12-08 01:09:25.294063+00
# SELECT LOCALTIMESTAMP AT TIME ZONE 'utc';
-------------------------------
2016-12-08 01:09:44.32587+00 -- SAME AS ABOVE
# SET TIME ZONE 'America/New_York';
# SELECT LOCALTIMESTAMP;
----------------------------
2016-12-07 20:13:34.924647
# SELECT LOCALTIMESTAMP AT TIME ZONE 'localtime';
------------------------------
2016-12-07 15:10:08.188197-05
# SELECT LOCALTIMESTAMP AT TIME ZONE 'utc';
------------------------------
2016-12-07 15:10:44.88332-05 -- SAME AS ABOVE
Any hint? Is it documented somewhere?
A timestamp in Postgres does not actually store any timezone information. Rather, this information comes from the timezone which is set by the server. Internally, all timestamp information is recorded in UTC time. So, for example, if you stored timestamp information from a timezone other than UTC, Postgres would first convert that timestamp to UTC before storing it.
From the documentation:
For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's timezone parameter, and is converted to UTC using the offset for the timezone zone.
To your actual question, localtime is just the timezone of the server which is always UTC.
Furthermore, it appears that Postgres' localtime simply wraps the C library function localtime(), which attempts to find the local system time (which is in default UTC time). Again, from the documentation:
If timezone is not specified in postgresql.conf or as a server command-line option, the server attempts to use the value of the TZ environment variable as the default time zone. If TZ is not defined or is not any of the time zone names known to PostgreSQL, the server attempts to determine the operating system's default time zone by checking the behavior of the C library function localtime(). The default time zone is selected as the closest match among PostgreSQL's known time zones. (These rules are also used to choose the default value of log_timezone, if not specified.)
I run a Postgres 8.3 database where times seem to be stored in UTC without time zone.
I am trying to display in local time but not with '+01' suffix :
With select scheduled_start_ts I get :
2014-01-20 05:01:35.663
With select scheduled_start_ts at time zone 'MET' :
2014-01-20 05:01:35.663+01
I would like to get "2014-01-20 06:01:35.663" which is in local time.
The database I am using cannot be modified and I am not allowed to modify how data are stored.
If you want to format times, use the to_char function. See formatting functions in the docs.
regress=> SELECT to_char(
(TIMESTAMP '2014-01-20 05:01:35.663' AT TIME ZONE 'UTC')
AT TIME ZONE 'MET',
'YYYY-MM-DD HH:MI:SS'
);
to_char
---------------------
2014-01-20 06:01:35
(1 row)
The (TIMESTAMP 'xxx' AT TIME ZONE 'UTC') gets me a timestamptz with the correct time, by re-interpreting the TIMESTAMP as being in UTC. The second AT TIME STAMP instead converts the timestamptz into a timestamp in timezone MET. This then gets formatted.
Whatever the SQL standards committe were smoking when they designed this, I never, ever, ever want to be anywhere near it.
Are timestamp values stored differently in PostgreSQL when the data type is WITH TIME ZONE versus WITHOUT TIME ZONE? Can the differences be illustrated with simple test cases?
The differences are covered at the PostgreSQL documentation for date/time types. Yes, the treatment of TIME or TIMESTAMP differs between one WITH TIME ZONE or WITHOUT TIME ZONE. It doesn't affect how the values are stored; it affects how they are interpreted.
The effects of time zones on these data types is covered specifically in the docs. The difference arises from what the system can reasonably know about the value:
With a time zone as part of the value, the value can be rendered as a local time in the client.
Without a time zone as part of the value, the obvious default time zone is UTC, so it is rendered for that time zone.
The behaviour differs depending on at least three factors:
The timezone setting in the client.
The data type (i.e. WITH TIME ZONE or WITHOUT TIME ZONE) of the value.
Whether the value is specified with a particular time zone.
Here are examples covering the combinations of those factors:
foo=> SET TIMEZONE TO 'Japan';
SET
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 00:00:00+09
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 06:00:00+09
(1 row)
foo=> SET TIMEZONE TO 'Australia/Melbourne';
SET
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 00:00:00+11
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 08:00:00+11
(1 row)
I try to explain it more understandably than the referred PostgreSQL documentation.
Neither TIMESTAMP variants store a time zone (or an offset), despite what the names suggest. The difference is in the interpretation of the stored data (and in the intended application), not in the storage format itself:
TIMESTAMP WITHOUT TIME ZONE stores local date-time (aka. wall calendar date and wall clock time). Its time zone is unspecified as far as PostgreSQL can tell (though your application may knows what it is). Hence, PostgreSQL does no time zone related conversion on input or output. If the value was entered into the database as '2011-07-01 06:30:30', then no mater in what time zone you display it later, it will still say year 2011, month 07, day 01, 06 hours, 30 minutes, and 30 seconds (in some format). Also, any offset or time zone you specify in the input is ignored by PostgreSQL, so '2011-07-01 06:30:30+00' and '2011-07-01 06:30:30+05' are the same as just '2011-07-01 06:30:30'.
For Java developers: it's analogous to java.time.LocalDateTime.
TIMESTAMP WITH TIME ZONE stores a point on the UTC time line. How it looks (how many hours, minutes, etc.) depends on your time zone, but it always refers to the same "physical" instant (like the moment of an actual physical event). The
input is internally converted to UTC, and that's how it's stored. For that, the offset of the input must be known, so when the input contains no explicit offset or time zone (like '2011-07-01 06:30:30') it's assumed to be in the current time zone of the PostgreSQL session, otherwise the explicitly specified offset or time zone is used (as in '2011-07-01 06:30:30+05'). The output is displayed converted to the current time zone of the PostgreSQL session.
For Java developers: It's analogous to java.time.Instant (with lower resolution though), but with JDBC and JPA 2.2 you are supposed to map it to java.time.OffsetDateTime (or to java.util.Date or java.sql.Timestamp of course).
Some say that both TIMESTAMP variations store UTC date-time. Kind of, but it's confusing to put it that way in my opinion. TIMESTAMP WITHOUT TIME ZONE is stored like a TIMESTAMP WITH TIME ZONE, which rendered with UTC time zone happens to give the same year, month, day, hours, minutes, seconds, and microseconds as they are in the local date-time. But it's not meant to represent the point on the time line that the UTC interpretation says, it's just the way the local date-time fields are encoded. (It's some cluster of dots on the time line, as the real time zone is not UTC; we don't know what it is.)
Here is an example that should help. If you have a timestamp with a timezone, you can convert that timestamp into any other timezone. If you haven't got a base timezone it won't be converted correctly.
SELECT now(),
now()::timestamp,
now() AT TIME ZONE 'CST',
now()::timestamp AT TIME ZONE 'CST'
Output:
-[ RECORD 1 ]---------------------------
now | 2018-09-15 17:01:36.399357+03
now | 2018-09-15 17:01:36.399357
timezone | 2018-09-15 08:01:36.399357
timezone | 2018-09-16 02:01:36.399357+03
Timestamptz vs Timestamp
The timestamptz field in Postgres is basically just the timestamp field where Postgres actually just stores the “normalised” UTC time, even if the timestamp given in the input string has a timezone.
If your input string is: 2018-08-28T12:30:00+05:30 , when this timestamp is stored in the database, it will be stored as 2018-08-28T07:00:00.
The advantage of this over the simple timestamp field is that your input to the database will be timezone independent, and will not be inaccurate when apps from different timezones insert timestamps, or when you move your database server location to a different timezone.
To quote from the docs:
For timestamp with time zone, the internally stored value is always in
UTC (Universal Coordinated Time, traditionally known as Greenwich Mean
Time, GMT). An input value that has an explicit time zone specified is
converted to UTC using the appropriate offset for that time zone. If
no time zone is stated in the input string, then it is assumed to be
in the time zone indicated by the system’s TimeZone parameter, and is
converted to UTC using the offset for the timezone zone. To give a
simple analogy, a timestamptz value represents an instant in time, the
same instant for anyone viewing it. But a timestamp value just
represents a particular orientation of a clock, which will represent
different instances of time based on your timezone.
For pretty much any use case, timestamptz is almost always a better choice. This choice is made easier with the fact that both timestamptz and timestamp take up the same 8 bytes of data.
source:
https://hasura.io/blog/postgres-date-time-data-types-on-graphql-fd926e86ee87/
The diffrences are shown in PostgreSQL official docs. Please refer the docs for deep digging.
In a nutshell TIMESTAMP WITHOUT TIME ZONE doesn't save any timezone related informations if you give date time with timezone info,it takes date & time only and ignores timezone
For example
When I save this 12:13, 11 June 2021 IST to PostgreSQL TIMESTAMP WITHOUT TIME ZONE will reject the timezone information and saves the date time 12:13,11 June 2021
But the the case of TIMESTAMP WITH TIME ZONE it saves the timezone info in UTC format.
For example
When I save this 12:13, 11 June 2021 IST to PostgreSQL TIMESTAMP WITH TIME ZONE type variable it will interpret this time to UTC value and
stored as shown in below 6:43,11 June 2021 UTC
NB : UTC + 5.30 is IST
During the time conversion time returned by TIMESTAMP WITH TIME ZONE will be stored in UTC format and we can convert it to the required timezone like IST or PST etc.
So the recommented timestamp type in PostgreSQL is TIMESTAMP WITH TIME ZONE or TIMESTAMPZ
Run the following to see diff in pgAdmin:
create table public.testts (tz timestamp with time zone, tnz timestamp without time zone);
insert into public.testts values(now(), now());
select * from public.testts;
If you have similar issues I had of timestamp precision in Angular / Typescript / Node API / PostgreSql environment, hope my complete answer and solution will help you out.