I try to deploy a JS application using TypeORM and Postgres on a host. Locally I had a Postgress DB so running in my own TZ. The remote host happens to have its system time set to UTC:
$ date
Mon Oct 7 15:45:00 UTC 2019
$ psql
> select localtimestamp;
2019-10-07 15:45:00.123456
I have a table with an automatically updated date in it (meaning it updates when record is updated).
// my.entity.ts
#UpdateDateColumn({type: 'timestamp', name: 'lastUpdate', default: () => 'LOCALTIMESTAMP' })
lastUpdate: Date;
A row is inserted at 12:00 CEST:
> select "lastUpdate" from myTable;
2019-10-07 10:00:00.000000+00
I would like to get the date out in my timezone (CEST) regardless of the server time, so it should return me 2019-10-07 12:00. I prefer to not hardcode any tricks because it should also work on my CEST machine.
Postgress has all the info:
> show timezone;
UCT
it knows it runs on UTC time
I can tell it in which timezone I want the date
So I would expect it to be easy to convert this to my requested format. However, the following examples I found don't seem to work:
> select ("lastUpdate" at time zone 'CEST') from myTable;
2019-10-07 08:00:00.000000+00
> select timezone('CEST', "lastUpdate") from myTable;
2019-10-07 08:00:00.000000+00
There is one way I can get it right, and thats by specifying the current timezone:
> select ("lastUpdate" at time zone 'UTC' at time zone 'CEST') from myTable;
2019-10-07 12:00:00.000000
However, like I said I don't want to hardcode this (as other DB's run at other TZs) and Postgres knows it's own timezone.
Is there another syntax to do this correctly?
All timestamps in Postgres are stored in UTC regardless of the server configuration. This is important to realize, unlike other databases it does not interpret your input to be in its local time zone. However it will present it back in its time zone.
timestamp stores just the time without a time zone. timestamptz stores UTC plus a time zone.
Demonstrating these together on a Postgres server with its time zone set to UTC...
=> create table demo ( time timestamp, timetz timestamptz );
CREATE TABLE
=> insert into demo values ('2019-10-07 10:00', '2019-10-07 12:00 +0200');
INSERT 0 1
=> select * from demo;
time | timetz
---------------------+------------------------
2019-10-07 10:00:00 | 2019-10-07 10:00:00+00
timestamp ignores the offset and simply stores October 7th, 2019 10:00 with no offset. While timestamptz stores the offset of +0200, but presents the time back to me in its own UTC time zone. They're the same point in time.
You could change the database time zone for your session to CEST. Then Postgres will format timestamptz in CEST.
=> set time zone 'Antarctica/Troll';
=> select * from demo;
time | timetz
---------------------+------------------------
2019-10-07 10:00:00 | 2019-10-07 12:00:00+02
But this is brittle. You have to make sure it happens on every connection. Your application is relying on a specific configuration of your database connection. It's an action-at-a-distance anti-pattern. It's not obvious what's doing the formatting, and if it's removed lots of seemingly unrelated things break.
Instead, you should reformat your times as you like after receiving them from the database. You can do this manually...
=> select timetz at time zone 'CEST' from demo;
timezone
---------------------
2019-10-07 12:00:00
But generally this is something your ORM takes care of for you. I don't know typeorm, but most have a means of translating database types consistently. You should be able to set up typeorm to automatically translate Postgres timestamptz into your local application time zone. Better yet, instead of returning a string it can translate it into a proper Time object which you then have full control over.
select created_at at time zone 'utc' at time zone 'america/los_angeles'
from users;
select ts_tz AT TIME ZONE 'UTC' from test;
Related
Using vscode in debug mode, when I hover over a date field, it show as the below image.
but when I logged it out, it get converted to
"execution_date":"2021-12-02T20:23:48.322Z"
which is minus 7 hour.
The field is stored in postgres database on AWS RDS as timestamp, running show timezone; returns UTC and I am using VSCode in GMT+7 time. How can I fix this because the date get changed and used to call api, so the returned result would be incorrect.
This is not a complete answer as that would depend on more information. Instead it is an explanation of what is going on that may help you troubleshoot:
set TimeZone = UTC;
show timezone;
TimeZone
----------
UTC
--Show that timestamp is taken at UTC
select now();
now
-------------------------------
2021-12-05 18:23:38.604681+00
--Table with timestamp and timestamptz to show different behavior.
create table dt_test(id integer, ts_fld timestamp, tsz_fld timestamptz);
--Insert local time 'ICT'
insert into dt_test values (1, '2021-12-03 03:23:48.322+07', '2021-12-03 03:23:48.322+07');
--The timestamp entry ignores the time zone offset, while the timestamptz uses it to rotate to UTC as '2021-12-03 03:23:48.322+07' is same as '2021-12-02 20:23:48.322+00'
select * from dt_test ;
id | ts_fld | tsz_fld
----+-------------------------+----------------------------
1 | 2021-12-03 03:23:48.322 | 2021-12-02 20:23:48.322+00
--timestamp takes the value as at 'ICT' and then rotates it to the current 'TimeZone' UTC. The timestamptz takes the value at UTC at rotates it to 'ICT'
select ts_fld AT TIME ZONE 'ICT', tsz_fld AT TIME ZONE 'ICT' from dt_test ;
timezone | timezone
----------------------------+-------------------------
2021-12-02 20:23:48.322+00 | 2021-12-03 03:23:48.322
I am guessing at some point in the process to get the value for the API the code is taking the timestamp value and applying AT TIME ZONE 'ICT' either in the database or downstream using some equivalent procedure.
I have found the problem, it's is related to how Sequelize and postgres deal with timestamp without timezone. If you have the same problem like me, please refer to the following link: https://github.com/sequelize/sequelize/issues/3000
I am trying to understand the behavior of AT TIME ZONE and I get some strange behavioral differences between SQL Server and PostgreSQL in this regard.
For demonstration purposes I produced two queries, one for SQL Server and one for PostgreSQL.
The SQL Server query:
SELECT
[LocalTime],
[LocalTime] AT TIME ZONE 'W. Europe Standard Time' AS [LocalTimeWithOffset],
([LocalTime] AT TIME ZONE 'W. Europe Standard Time') AT TIME ZONE 'UTC' AS [UniversalTime]
FROM
(VALUES
(CAST('2020-06-20 12:34:56.789' AS DATETIME2)),
(CAST('2020-12-20 12:34:56.789' AS DATETIME2))) AS [X] ([LocalTime])
produces this result (in SQL Server Management Studio):
LocalTime LocalTimeWithOffset UniversalTime
--------- ------------------- ----------------
2020-06-20 12:34:56.7890000 2020-06-20 12:34:56.7890000 +02:00 2020-06-20 10:34:56.7890000 +00:00
2020-12-20 12:34:56.7890000 2020-12-20 12:34:56.7890000 +01:00 2020-12-20 11:34:56.7890000 +00:00
which seems fine to me, since I live in the Netherlands: in the summertime, we are 2 hours ahead of UTC (due to daylight saving) and in the wintertime we are only 1 hour ahead of UTC.
BUT... the PostgreSQL query:
SELECT
"LocalTime",
"LocalTime" AT TIME ZONE 'WET' AS "LocalTimeWithOffset",
("LocalTime" AT TIME ZONE 'WET') AT TIME ZONE 'UTC' AS "UniversalTime"
FROM
(VALUES
(timestamp '2020-06-20 12:34:56.789'),
(timestamp '2020-12-20 12:34:56.789')) AS "X" ("LocalTime")
produces the following result (in pgAdmin):
LocalTime LocalTimeWithOffset UniversalTime
timestamp without time zone timestamp with time zone timestamp without time zone
--------------------------- ------------------------ ---------------------------
2020-06-20 12:34:56.789 2020-06-20 14:34:56.789+02 2020-06-20 12:34:56.789
2020-12-20 12:34:56.789 2020-12-20 13:34:56.789+01 2020-12-20 12:34:56.789
which seems incorrect to me.
After struggling with this for a little while, I came up with another PostgreSQL query, which seems to yield the same results as the SQL Server query:
SELECT
"LocalTime",
"LocalTime"::timestamptz AS "LocalTimeWithOffset",
"LocalTime"::timestamptz AT TIME ZONE 'UTC' AS "UniversalTime"
FROM
(VALUES
(timestamp '2020-06-20 12:34:56.789'),
(timestamp '2020-12-20 12:34:56.789')) AS "X" ("LocalTime")
But this query seems to be dependent on the PostgreSQL server's time zone. :-(
Is there any (preferably elegant) way to mimic the SQL Server logic/behavior in PostgreSQL without a dependency on the server's time zone (and use an explicitly specified time zone in the query)?
Edit
I am aware that it is a good practice to store UTC date/time values in the database and convert them (preferably in/by the client) to local date/time values when presented to end users. I am intending to do so as well, of course, but my current scenario is a database migration from SQL Server to PostgreSQL (as part of a big application overhaul), where the DATETIME2 table columns in the SQL Server database contain local date/time values, which I want to convert to UTC date/time values in corresponding timestamp columns in the PostgreSQL tables. My migration script will work fine with the queries above, so actually I do not have a blocking issue. The reason for this question is that I am just curious about the apparent differences between SQL Server and PostgreSQL regarding the AT TIME ZONE behavior and the ways to overcome those behavioral differences.
In postgresql, AT TIME ZONE allows time zones to be added to date/time values that lack them (TIMESTAMP WITHOUT TIME ZONE, ::timestamp), and allows TIMESTAMP WITH TIME ZONE values (::timestamptz) to be shifted to non-local time zones and the time zone designation removed.
So, in the first query you've written for postgres, you're trying to cast '2020-06-20 12:34:56.789' to timestamp, and you've applied AT TIME ZONE on it for LocalTimeWithOffset. This is the case where postgres considered the input as timestamp in UTC and converted to the corresponding time zone, in your case WET, which is UTC+2:00 in Summer and UTC+1:00 in winter.
Now, in the third case, you're first applying the AT TIME ZONE 'WET' to timestamp '2020-06-20 12:34:56.789', hence the input is considered as UTC and converted to WET, the output is timstamptz, and this output is then fed again to AT TIME ZONE 'UTC', so as per behavior of AT TIME ZONE, it gets converted to UTC and the timezone tag is removed, which is, the input we given.
So, AT TIME ZONE did not behaved weird in postgresql rather did exactly what it was supposed to do, according to the documentation.
Hope your confusion is cleared.
I have one server which is on UTC time and one which is on EST. Now I dumped the database from the UTC one and imported it to the EST one. As timestamps are always saved as UTC I cannot find a reason the reason I get two different results for the same query:
select reported_on at time zone 'UTC', temperature from data order by reported_on desc;
UTC Server:
temperature | device | timezone
-------------+--------------------------------------+----------------------------
17.2 | ------------------------------------ | 2020-05-05 12:13:16.256+00
EST Server:
temperature | device | timezone
-------------+--------------------------------------+----------------------------
17.2 | ------------------------------------ | 2020-05-05 14:13:16.256+02
What am I missing here? How can I query the data without the timezones, I need the UTC time, not the +02 time? How can I achieve this?
Edit:
I added the lines through nodejs:
INSERT INTO data(device, reported_on, temperature, humidity) VALUES($1, to_timestamp(' + Date.now() /1000.0 + '), $2, $3) RETURNING *
The Typ of the column is:
reported_on TIMESTAMP,
Update:
Altering the timezone fixed the issue!
ALTER DATABASE sensors SET timezone TO 'UTC';
SELECT pg_reload_conf();
This is about your local database configuration. I created a dummy database locally and the result was this:
test=# CREATE TABLE timestamptest (timezone TIMESTAMPTZ);`
Showed my timezone pattern:
test=# SHOW TIMEZONE;
TimeZone
----------
UTC
(1 row)
And inserted some values inside:
test=# SELECT * FROM timestamptest;
timezone
-------------------------------
2020-05-05 15:26:27.377549+00
2020-05-05 15:28:14.014597+00
(2 rows)
Now, I changed the local timezone variable:
test=# SET TIME ZONE 'America/Montreal';
SET
And selected the info again:
test=# INSERT INTO timestamptest VALUES (now());
INSERT 0 1
test=# SELECT * FROM timestamptest ;
timezone
-------------------------------
2020-05-05 11:26:27.377549-04
2020-05-05 11:28:14.014597-04
(2 rows)
And my timezone is different because the SET command:
test=# SHOW timezone;
TimeZone
------------------
America/Montreal
(1 row)
You can change your local configuration and, about your selects showed in your question, the import seems to be correct, just the local timestamp configuration needed to be set equals from one to another.
Note that -04 on the end of the timestamp shows that your time has changed 4 hours in relation of -00 originally. Just a formatting ouptut.
More information here: here on postgresql docs
The statement "timestamps are always saved as UTC" is incorrect. Timestamp without time (timestamp) is stored with local time, timestamp with time zone (timestamptz) is always stored in UTC. From the documentation:
timestamp without time zone, PostgreSQL will silently ignore any time
zone indication. That is, the resulting value is derived from the
date/time fields in the input value, and is not adjusted for time
zone.
For timestamp with time zone, the internally stored value is always in
UTC (Universal Coordinated Time, traditionally known as Greenwich Mean
Time, GMT). An input value that has an explicit time zone specified is
converted to UTC using the appropriate offset for that time zone. If
no time zone is stated in the input string, then it is assumed to be
in the time zone indicated by the system's TimeZone parameter, and is
converted to UTC using the offset for the timezone zone.
Additional be very careful using 'EST', or any of the other timezone abbreviations, instead use the full timezone name. The abbreviations indicate a fixed UTC offset (i.e. EST is always UTC-5, and EDT is always UTC-4) and do not recognize Daylight Savings Time (DST). On the other hand the full names (i.e. are US/Central' or 'America/Montreal' or any other full timezone name) are DST aware and adjust UTC offset accordingly.
Unlike the abbreviations shown in pg_timezone_abbrevs, many of these
names imply a set of daylight-savings transition date rules.
You can observe this with:
select * from pg_timezone_abbrevs where abbrev in ('EDT', 'EST')
select * from pg_timezone_names where name in ('US/Eastern','America/New_York','America/Montreal')
The type of the column is TIMESTAMP*"
I guess there's your problem. You'll want to use TIMESTAMP WITH TIME ZONE instead. Alternatively, do not use AT TIME ZONE 'UTC' in your SELECT statement, so that you just get back the same timestamp that was stored, regardless of timezone.
What is happening in SELECT reported_on AT TIME ZONE 'UTC' is that the stored date is converted from the database's local timezone to UTC. In your second database with a different local timezone, that's a different conversion, leading to the offset in the result.
I store timestamps with time zone in my postgres database. The server time zone is UTC. All the timestamptzs are stored as UTC datetimes.
Now, I'd like to retrieve those timestamps, but for a specific time zone (e.g. US/Pacific). Because of daylight savings time, the correct time zone is sometimes PDT and sometimes PST. So I can't just run a query like select t at time zone 'pdt' because this will be wrong for the pst dates.
Is there a way to pull the dates from the database in the correct time zone?
According to the documentation, together with TIME ZONE code you can also specify locales. For your case you can use something like that:
ds=# SELECT current_setting('TIMEZONE');
current_setting
-----------------
UTC
(1 row)
ds=# SELECT now();
now
-------------------------------
2015-11-05 00:35:03.126317+00
(1 row)
pm7=# SELECT now() AT TIME ZONE 'America/Los_Angeles';
timezone
----------------------------
2015-11-04 16:35:06.344367
(1 row)
I run a Postgres 8.3 database where times seem to be stored in UTC without time zone.
I am trying to display in local time but not with '+01' suffix :
With select scheduled_start_ts I get :
2014-01-20 05:01:35.663
With select scheduled_start_ts at time zone 'MET' :
2014-01-20 05:01:35.663+01
I would like to get "2014-01-20 06:01:35.663" which is in local time.
The database I am using cannot be modified and I am not allowed to modify how data are stored.
If you want to format times, use the to_char function. See formatting functions in the docs.
regress=> SELECT to_char(
(TIMESTAMP '2014-01-20 05:01:35.663' AT TIME ZONE 'UTC')
AT TIME ZONE 'MET',
'YYYY-MM-DD HH:MI:SS'
);
to_char
---------------------
2014-01-20 06:01:35
(1 row)
The (TIMESTAMP 'xxx' AT TIME ZONE 'UTC') gets me a timestamptz with the correct time, by re-interpreting the TIMESTAMP as being in UTC. The second AT TIME STAMP instead converts the timestamptz into a timestamp in timezone MET. This then gets formatted.
Whatever the SQL standards committe were smoking when they designed this, I never, ever, ever want to be anywhere near it.