I'm struggling to understand how "AT TIME ZONE 'localtime'" exactly work? By playing with it, I found out that it acts exactly as "AT TIME ZONE 'UTC'"... But why? Is "localtime" a synonym of "UTC" in postgres? Or it comes from some setting (environment? connection timezone? although checked both, seems they are not related)...
There's "localtime" function but I think it is not involved here.
Sample SQLs:
# date
Thu Dec 8 12:00:05 AEDT 2016
# SELECT LOCALTIMESTAMP;
----------------------------
2016-12-08 01:13:29.444725
# SELECT LOCALTIMESTAMP AT TIME ZONE 'America/New_York';
-------------------------------
2016-12-08 06:08:31.183103+00
# SELECT LOCALTIMESTAMP AT TIME ZONE'localtime';
------------------------------
2016-12-08 01:09:25.294063+00
# SELECT LOCALTIMESTAMP AT TIME ZONE 'utc';
-------------------------------
2016-12-08 01:09:44.32587+00 -- SAME AS ABOVE
# SET TIME ZONE 'America/New_York';
# SELECT LOCALTIMESTAMP;
----------------------------
2016-12-07 20:13:34.924647
# SELECT LOCALTIMESTAMP AT TIME ZONE 'localtime';
------------------------------
2016-12-07 15:10:08.188197-05
# SELECT LOCALTIMESTAMP AT TIME ZONE 'utc';
------------------------------
2016-12-07 15:10:44.88332-05 -- SAME AS ABOVE
Any hint? Is it documented somewhere?
A timestamp in Postgres does not actually store any timezone information. Rather, this information comes from the timezone which is set by the server. Internally, all timestamp information is recorded in UTC time. So, for example, if you stored timestamp information from a timezone other than UTC, Postgres would first convert that timestamp to UTC before storing it.
From the documentation:
For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's timezone parameter, and is converted to UTC using the offset for the timezone zone.
To your actual question, localtime is just the timezone of the server which is always UTC.
Furthermore, it appears that Postgres' localtime simply wraps the C library function localtime(), which attempts to find the local system time (which is in default UTC time). Again, from the documentation:
If timezone is not specified in postgresql.conf or as a server command-line option, the server attempts to use the value of the TZ environment variable as the default time zone. If TZ is not defined or is not any of the time zone names known to PostgreSQL, the server attempts to determine the operating system's default time zone by checking the behavior of the C library function localtime(). The default time zone is selected as the closest match among PostgreSQL's known time zones. (These rules are also used to choose the default value of log_timezone, if not specified.)
Related
I have one server which is on UTC time and one which is on EST. Now I dumped the database from the UTC one and imported it to the EST one. As timestamps are always saved as UTC I cannot find a reason the reason I get two different results for the same query:
select reported_on at time zone 'UTC', temperature from data order by reported_on desc;
UTC Server:
temperature | device | timezone
-------------+--------------------------------------+----------------------------
17.2 | ------------------------------------ | 2020-05-05 12:13:16.256+00
EST Server:
temperature | device | timezone
-------------+--------------------------------------+----------------------------
17.2 | ------------------------------------ | 2020-05-05 14:13:16.256+02
What am I missing here? How can I query the data without the timezones, I need the UTC time, not the +02 time? How can I achieve this?
Edit:
I added the lines through nodejs:
INSERT INTO data(device, reported_on, temperature, humidity) VALUES($1, to_timestamp(' + Date.now() /1000.0 + '), $2, $3) RETURNING *
The Typ of the column is:
reported_on TIMESTAMP,
Update:
Altering the timezone fixed the issue!
ALTER DATABASE sensors SET timezone TO 'UTC';
SELECT pg_reload_conf();
This is about your local database configuration. I created a dummy database locally and the result was this:
test=# CREATE TABLE timestamptest (timezone TIMESTAMPTZ);`
Showed my timezone pattern:
test=# SHOW TIMEZONE;
TimeZone
----------
UTC
(1 row)
And inserted some values inside:
test=# SELECT * FROM timestamptest;
timezone
-------------------------------
2020-05-05 15:26:27.377549+00
2020-05-05 15:28:14.014597+00
(2 rows)
Now, I changed the local timezone variable:
test=# SET TIME ZONE 'America/Montreal';
SET
And selected the info again:
test=# INSERT INTO timestamptest VALUES (now());
INSERT 0 1
test=# SELECT * FROM timestamptest ;
timezone
-------------------------------
2020-05-05 11:26:27.377549-04
2020-05-05 11:28:14.014597-04
(2 rows)
And my timezone is different because the SET command:
test=# SHOW timezone;
TimeZone
------------------
America/Montreal
(1 row)
You can change your local configuration and, about your selects showed in your question, the import seems to be correct, just the local timestamp configuration needed to be set equals from one to another.
Note that -04 on the end of the timestamp shows that your time has changed 4 hours in relation of -00 originally. Just a formatting ouptut.
More information here: here on postgresql docs
The statement "timestamps are always saved as UTC" is incorrect. Timestamp without time (timestamp) is stored with local time, timestamp with time zone (timestamptz) is always stored in UTC. From the documentation:
timestamp without time zone, PostgreSQL will silently ignore any time
zone indication. That is, the resulting value is derived from the
date/time fields in the input value, and is not adjusted for time
zone.
For timestamp with time zone, the internally stored value is always in
UTC (Universal Coordinated Time, traditionally known as Greenwich Mean
Time, GMT). An input value that has an explicit time zone specified is
converted to UTC using the appropriate offset for that time zone. If
no time zone is stated in the input string, then it is assumed to be
in the time zone indicated by the system's TimeZone parameter, and is
converted to UTC using the offset for the timezone zone.
Additional be very careful using 'EST', or any of the other timezone abbreviations, instead use the full timezone name. The abbreviations indicate a fixed UTC offset (i.e. EST is always UTC-5, and EDT is always UTC-4) and do not recognize Daylight Savings Time (DST). On the other hand the full names (i.e. are US/Central' or 'America/Montreal' or any other full timezone name) are DST aware and adjust UTC offset accordingly.
Unlike the abbreviations shown in pg_timezone_abbrevs, many of these
names imply a set of daylight-savings transition date rules.
You can observe this with:
select * from pg_timezone_abbrevs where abbrev in ('EDT', 'EST')
select * from pg_timezone_names where name in ('US/Eastern','America/New_York','America/Montreal')
The type of the column is TIMESTAMP*"
I guess there's your problem. You'll want to use TIMESTAMP WITH TIME ZONE instead. Alternatively, do not use AT TIME ZONE 'UTC' in your SELECT statement, so that you just get back the same timestamp that was stored, regardless of timezone.
What is happening in SELECT reported_on AT TIME ZONE 'UTC' is that the stored date is converted from the database's local timezone to UTC. In your second database with a different local timezone, that's a different conversion, leading to the offset in the result.
I have a date in UTC (timestamp with time zone):
test=# select ('2018-05-31T21:00:00+00'::timestamptz);
timestamptz
------------------------
2018-05-31 21:00:00+00
(1 row)
I want to convert it to another timezone, for example UTC+2. The documentation states that at time zone should do it for me:
Experession
<timestamp with time zone> AT TIME ZONE <zone>
Return type
timestamp without time zone
Description
Convert the given time stamp with time zone to the new time zone, with no time zone designation
See here.
But when I try to do it, I get strange results:
test# select ('2018-05-31T21:00:00+00'::timestamptz) at time zone 'utc+2';
timezone
---------------------
2018-05-31 19:00:00
(1 row)
I am pretty sure that 21 hours in UTC is 23 hours in UTC+2 (that's the reason for +2, after all).
What am I doing wrong here?
Your problem is the POSIX standard that according to the documentation decrees that
Another issue to keep in mind is that in POSIX time zone names, positive offsets are used for locations west of Greenwich. Everywhere else, PostgreSQL follows the ISO-8601 convention that positive timezone offsets are east of Greenwich.
How do I convert a timestamp that is in UTC to timestamptz?
If my local time zone is GMT-1 and I run:
select '2017-01-01 00:00:00'::timestamptz
I get:
2017-01-01 00:00:00-01
but I want:
2017-01-01 01:00:00-01
Better:
SELECT timestamp '2017-01-01 00:00:00' AT TIME ZONE 'UTC';
No additional cast after the AT TIME ZONE construct. It returns timestamp with time zone (= timestamptz) for timestamp without time zone (= timestamp) input and vice versa.
For the given example, the shortest, most efficient way to provide a timestamp constant is timestamp '2017-01-01'. Or use a cast, almost as good: '2017-01-01'::timestamp. The time component 00:00:00 is assumed when missing.
There is no such thing as a "timestamp that is in UTC". A timestamp carries no time zone information. Only you know that it's supposed to be located in the UTC time zone.
The type name "timestamp with time zone" is a bit misleading. timestamptz does not carry any time zone information, either. The given time zone name, abbreviation or offset is used to compute the corresponding UTC time. Text output (display) is adapted to the current time zone setting of your session. Only the bare value of the corresponding UTC time is stored. The time zone itself is never stored. If you need it, store it additionally in another column. In your particular case, UTC happens to be the time zone used for input as well.
Detailed explanation:
Time zone storage in data type "timestamp with time zone"
Ignoring time zones altogether in Rails and PostgreSQL
Solution: select ('2017-01-01 00:00:00' at time zone 'utc')::timestamptz
My database and client are currently in America/New_York, but could be moved anywhere.
SHOW TIMEZONE; -- America/New_York
CREATE TABLE times (
t TIMESTAMP WITHOUT TIME ZONE
);
INSERT INTO times VALUES
(NOW()),
(NOW() AT TIME ZONE 'UTC');
SELECT t FROM times;
-- 2017-06-13 14:53:17.766969
-- 2017-06-13 18:53:17.766969
This is unexpected. I thought my SELECT would return the same value for both records.
When I insert the current time into column t, I want it to mean the current time in any time zone (whether the database stores the underlying value as current time UTC or not). This way, regardless of the time zone the database is operating in, or the time zone the client is operating in, everybody can agree on the universally coordinated fixed point in time.
What is the proper way to INSERT a record so that everyone in the world knows I'm referencing a 'UTC' time?
First, it seems like SELECT assumes that timestamp without time zone fields are stored in the local time zone. Try SET TIME ZONE 'UTC+<n>. The output of SELECT does not change (i.e., it's not as if timestamp means UTC if no timezone is stored!)
When a field it of type timestamp with time zone, SELECT correctly converts the output to the current time zone.
So when you say AT TIME ZONE 'UTC' you convert the time to UTC, then discard the timezone making Postgres think you are dealing with local time.
You can re-introduce the UTC time zone by adding another AT TIME ZONE 'UTC' clause:
# select pg_typeof(now() at time zone 'utc');
timestamp without time zone
# select pg_typeof(now() at time zone 'utc' at time zone 'utc');
timestamp with time zone
Bottom line: I would either always use a TIME WITH TIME ZONE or always have the database time zone set to UTC if you insist on working without time zones. The storage size of both types seems identical (64 bit).
Are timestamp values stored differently in PostgreSQL when the data type is WITH TIME ZONE versus WITHOUT TIME ZONE? Can the differences be illustrated with simple test cases?
The differences are covered at the PostgreSQL documentation for date/time types. Yes, the treatment of TIME or TIMESTAMP differs between one WITH TIME ZONE or WITHOUT TIME ZONE. It doesn't affect how the values are stored; it affects how they are interpreted.
The effects of time zones on these data types is covered specifically in the docs. The difference arises from what the system can reasonably know about the value:
With a time zone as part of the value, the value can be rendered as a local time in the client.
Without a time zone as part of the value, the obvious default time zone is UTC, so it is rendered for that time zone.
The behaviour differs depending on at least three factors:
The timezone setting in the client.
The data type (i.e. WITH TIME ZONE or WITHOUT TIME ZONE) of the value.
Whether the value is specified with a particular time zone.
Here are examples covering the combinations of those factors:
foo=> SET TIMEZONE TO 'Japan';
SET
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 00:00:00+09
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 06:00:00+09
(1 row)
foo=> SET TIMEZONE TO 'Australia/Melbourne';
SET
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 00:00:00+11
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 08:00:00+11
(1 row)
I try to explain it more understandably than the referred PostgreSQL documentation.
Neither TIMESTAMP variants store a time zone (or an offset), despite what the names suggest. The difference is in the interpretation of the stored data (and in the intended application), not in the storage format itself:
TIMESTAMP WITHOUT TIME ZONE stores local date-time (aka. wall calendar date and wall clock time). Its time zone is unspecified as far as PostgreSQL can tell (though your application may knows what it is). Hence, PostgreSQL does no time zone related conversion on input or output. If the value was entered into the database as '2011-07-01 06:30:30', then no mater in what time zone you display it later, it will still say year 2011, month 07, day 01, 06 hours, 30 minutes, and 30 seconds (in some format). Also, any offset or time zone you specify in the input is ignored by PostgreSQL, so '2011-07-01 06:30:30+00' and '2011-07-01 06:30:30+05' are the same as just '2011-07-01 06:30:30'.
For Java developers: it's analogous to java.time.LocalDateTime.
TIMESTAMP WITH TIME ZONE stores a point on the UTC time line. How it looks (how many hours, minutes, etc.) depends on your time zone, but it always refers to the same "physical" instant (like the moment of an actual physical event). The
input is internally converted to UTC, and that's how it's stored. For that, the offset of the input must be known, so when the input contains no explicit offset or time zone (like '2011-07-01 06:30:30') it's assumed to be in the current time zone of the PostgreSQL session, otherwise the explicitly specified offset or time zone is used (as in '2011-07-01 06:30:30+05'). The output is displayed converted to the current time zone of the PostgreSQL session.
For Java developers: It's analogous to java.time.Instant (with lower resolution though), but with JDBC and JPA 2.2 you are supposed to map it to java.time.OffsetDateTime (or to java.util.Date or java.sql.Timestamp of course).
Some say that both TIMESTAMP variations store UTC date-time. Kind of, but it's confusing to put it that way in my opinion. TIMESTAMP WITHOUT TIME ZONE is stored like a TIMESTAMP WITH TIME ZONE, which rendered with UTC time zone happens to give the same year, month, day, hours, minutes, seconds, and microseconds as they are in the local date-time. But it's not meant to represent the point on the time line that the UTC interpretation says, it's just the way the local date-time fields are encoded. (It's some cluster of dots on the time line, as the real time zone is not UTC; we don't know what it is.)
Here is an example that should help. If you have a timestamp with a timezone, you can convert that timestamp into any other timezone. If you haven't got a base timezone it won't be converted correctly.
SELECT now(),
now()::timestamp,
now() AT TIME ZONE 'CST',
now()::timestamp AT TIME ZONE 'CST'
Output:
-[ RECORD 1 ]---------------------------
now | 2018-09-15 17:01:36.399357+03
now | 2018-09-15 17:01:36.399357
timezone | 2018-09-15 08:01:36.399357
timezone | 2018-09-16 02:01:36.399357+03
Timestamptz vs Timestamp
The timestamptz field in Postgres is basically just the timestamp field where Postgres actually just stores the “normalised” UTC time, even if the timestamp given in the input string has a timezone.
If your input string is: 2018-08-28T12:30:00+05:30 , when this timestamp is stored in the database, it will be stored as 2018-08-28T07:00:00.
The advantage of this over the simple timestamp field is that your input to the database will be timezone independent, and will not be inaccurate when apps from different timezones insert timestamps, or when you move your database server location to a different timezone.
To quote from the docs:
For timestamp with time zone, the internally stored value is always in
UTC (Universal Coordinated Time, traditionally known as Greenwich Mean
Time, GMT). An input value that has an explicit time zone specified is
converted to UTC using the appropriate offset for that time zone. If
no time zone is stated in the input string, then it is assumed to be
in the time zone indicated by the system’s TimeZone parameter, and is
converted to UTC using the offset for the timezone zone. To give a
simple analogy, a timestamptz value represents an instant in time, the
same instant for anyone viewing it. But a timestamp value just
represents a particular orientation of a clock, which will represent
different instances of time based on your timezone.
For pretty much any use case, timestamptz is almost always a better choice. This choice is made easier with the fact that both timestamptz and timestamp take up the same 8 bytes of data.
source:
https://hasura.io/blog/postgres-date-time-data-types-on-graphql-fd926e86ee87/
The diffrences are shown in PostgreSQL official docs. Please refer the docs for deep digging.
In a nutshell TIMESTAMP WITHOUT TIME ZONE doesn't save any timezone related informations if you give date time with timezone info,it takes date & time only and ignores timezone
For example
When I save this 12:13, 11 June 2021 IST to PostgreSQL TIMESTAMP WITHOUT TIME ZONE will reject the timezone information and saves the date time 12:13,11 June 2021
But the the case of TIMESTAMP WITH TIME ZONE it saves the timezone info in UTC format.
For example
When I save this 12:13, 11 June 2021 IST to PostgreSQL TIMESTAMP WITH TIME ZONE type variable it will interpret this time to UTC value and
stored as shown in below 6:43,11 June 2021 UTC
NB : UTC + 5.30 is IST
During the time conversion time returned by TIMESTAMP WITH TIME ZONE will be stored in UTC format and we can convert it to the required timezone like IST or PST etc.
So the recommented timestamp type in PostgreSQL is TIMESTAMP WITH TIME ZONE or TIMESTAMPZ
Run the following to see diff in pgAdmin:
create table public.testts (tz timestamp with time zone, tnz timestamp without time zone);
insert into public.testts values(now(), now());
select * from public.testts;
If you have similar issues I had of timestamp precision in Angular / Typescript / Node API / PostgreSql environment, hope my complete answer and solution will help you out.