Using vscode in debug mode, when I hover over a date field, it show as the below image.
but when I logged it out, it get converted to
"execution_date":"2021-12-02T20:23:48.322Z"
which is minus 7 hour.
The field is stored in postgres database on AWS RDS as timestamp, running show timezone; returns UTC and I am using VSCode in GMT+7 time. How can I fix this because the date get changed and used to call api, so the returned result would be incorrect.
This is not a complete answer as that would depend on more information. Instead it is an explanation of what is going on that may help you troubleshoot:
set TimeZone = UTC;
show timezone;
TimeZone
----------
UTC
--Show that timestamp is taken at UTC
select now();
now
-------------------------------
2021-12-05 18:23:38.604681+00
--Table with timestamp and timestamptz to show different behavior.
create table dt_test(id integer, ts_fld timestamp, tsz_fld timestamptz);
--Insert local time 'ICT'
insert into dt_test values (1, '2021-12-03 03:23:48.322+07', '2021-12-03 03:23:48.322+07');
--The timestamp entry ignores the time zone offset, while the timestamptz uses it to rotate to UTC as '2021-12-03 03:23:48.322+07' is same as '2021-12-02 20:23:48.322+00'
select * from dt_test ;
id | ts_fld | tsz_fld
----+-------------------------+----------------------------
1 | 2021-12-03 03:23:48.322 | 2021-12-02 20:23:48.322+00
--timestamp takes the value as at 'ICT' and then rotates it to the current 'TimeZone' UTC. The timestamptz takes the value at UTC at rotates it to 'ICT'
select ts_fld AT TIME ZONE 'ICT', tsz_fld AT TIME ZONE 'ICT' from dt_test ;
timezone | timezone
----------------------------+-------------------------
2021-12-02 20:23:48.322+00 | 2021-12-03 03:23:48.322
I am guessing at some point in the process to get the value for the API the code is taking the timestamp value and applying AT TIME ZONE 'ICT' either in the database or downstream using some equivalent procedure.
I have found the problem, it's is related to how Sequelize and postgres deal with timestamp without timezone. If you have the same problem like me, please refer to the following link: https://github.com/sequelize/sequelize/issues/3000
Related
What is the behavior of PostgreSQL when we cast a DATE to TIMESTAMP to TIMESTAMPTZ
What time zone is used?
PostgreSQL server
Client that run the query (Current session)
If you cast a date to a timestamp, time zones don't play a role, because both data types are without a time zone. The resulting timestamp will be the beginning of the day.
If you cast date to timestamp with time zone, resulting timestamp will be the beginning of the date in the time zone defined by the parameter timezone in your current session.
SHOW timezone;
TimeZone
---------------
Europe/Vienna
(1 row)
SELECT CAST (DATE '2021-01-15' AS timestamp);
timestamp
---------------------
2021-01-15 00:00:00
(1 row)
SELECT CAST (DATE '2021-01-15' AS timestamp with time zone);
timestamptz
------------------------
2021-01-15 00:00:00+01
(1 row)
Casting date to timestamp will append time 00:00:00.0 to the date.
The time zone of the current server session will be used.
By default this is the time zone setting of Postgresql server.
You can change the time zone of the current server session like this:
set time zone 'Europe/Sofia';
I do not think that the time zone of the client has any effect. More on this issue here.
Edit
As Adrian Klaver noticed "When dealing with timestamps it's best to assume the worst". Therefore better set the session time zone explicitly.
Given the situation where one needs to change the column type from integer (as epoc seconds) to timzezonetz ->
casting directly from integer to timezonetz is not possible.
but you can do this:
ALTER TABLE table_name ALTER COLUMN column_name TYPE timestamptz
USING column_name::abstime::timestamptz
I have one server which is on UTC time and one which is on EST. Now I dumped the database from the UTC one and imported it to the EST one. As timestamps are always saved as UTC I cannot find a reason the reason I get two different results for the same query:
select reported_on at time zone 'UTC', temperature from data order by reported_on desc;
UTC Server:
temperature | device | timezone
-------------+--------------------------------------+----------------------------
17.2 | ------------------------------------ | 2020-05-05 12:13:16.256+00
EST Server:
temperature | device | timezone
-------------+--------------------------------------+----------------------------
17.2 | ------------------------------------ | 2020-05-05 14:13:16.256+02
What am I missing here? How can I query the data without the timezones, I need the UTC time, not the +02 time? How can I achieve this?
Edit:
I added the lines through nodejs:
INSERT INTO data(device, reported_on, temperature, humidity) VALUES($1, to_timestamp(' + Date.now() /1000.0 + '), $2, $3) RETURNING *
The Typ of the column is:
reported_on TIMESTAMP,
Update:
Altering the timezone fixed the issue!
ALTER DATABASE sensors SET timezone TO 'UTC';
SELECT pg_reload_conf();
This is about your local database configuration. I created a dummy database locally and the result was this:
test=# CREATE TABLE timestamptest (timezone TIMESTAMPTZ);`
Showed my timezone pattern:
test=# SHOW TIMEZONE;
TimeZone
----------
UTC
(1 row)
And inserted some values inside:
test=# SELECT * FROM timestamptest;
timezone
-------------------------------
2020-05-05 15:26:27.377549+00
2020-05-05 15:28:14.014597+00
(2 rows)
Now, I changed the local timezone variable:
test=# SET TIME ZONE 'America/Montreal';
SET
And selected the info again:
test=# INSERT INTO timestamptest VALUES (now());
INSERT 0 1
test=# SELECT * FROM timestamptest ;
timezone
-------------------------------
2020-05-05 11:26:27.377549-04
2020-05-05 11:28:14.014597-04
(2 rows)
And my timezone is different because the SET command:
test=# SHOW timezone;
TimeZone
------------------
America/Montreal
(1 row)
You can change your local configuration and, about your selects showed in your question, the import seems to be correct, just the local timestamp configuration needed to be set equals from one to another.
Note that -04 on the end of the timestamp shows that your time has changed 4 hours in relation of -00 originally. Just a formatting ouptut.
More information here: here on postgresql docs
The statement "timestamps are always saved as UTC" is incorrect. Timestamp without time (timestamp) is stored with local time, timestamp with time zone (timestamptz) is always stored in UTC. From the documentation:
timestamp without time zone, PostgreSQL will silently ignore any time
zone indication. That is, the resulting value is derived from the
date/time fields in the input value, and is not adjusted for time
zone.
For timestamp with time zone, the internally stored value is always in
UTC (Universal Coordinated Time, traditionally known as Greenwich Mean
Time, GMT). An input value that has an explicit time zone specified is
converted to UTC using the appropriate offset for that time zone. If
no time zone is stated in the input string, then it is assumed to be
in the time zone indicated by the system's TimeZone parameter, and is
converted to UTC using the offset for the timezone zone.
Additional be very careful using 'EST', or any of the other timezone abbreviations, instead use the full timezone name. The abbreviations indicate a fixed UTC offset (i.e. EST is always UTC-5, and EDT is always UTC-4) and do not recognize Daylight Savings Time (DST). On the other hand the full names (i.e. are US/Central' or 'America/Montreal' or any other full timezone name) are DST aware and adjust UTC offset accordingly.
Unlike the abbreviations shown in pg_timezone_abbrevs, many of these
names imply a set of daylight-savings transition date rules.
You can observe this with:
select * from pg_timezone_abbrevs where abbrev in ('EDT', 'EST')
select * from pg_timezone_names where name in ('US/Eastern','America/New_York','America/Montreal')
The type of the column is TIMESTAMP*"
I guess there's your problem. You'll want to use TIMESTAMP WITH TIME ZONE instead. Alternatively, do not use AT TIME ZONE 'UTC' in your SELECT statement, so that you just get back the same timestamp that was stored, regardless of timezone.
What is happening in SELECT reported_on AT TIME ZONE 'UTC' is that the stored date is converted from the database's local timezone to UTC. In your second database with a different local timezone, that's a different conversion, leading to the offset in the result.
I try to deploy a JS application using TypeORM and Postgres on a host. Locally I had a Postgress DB so running in my own TZ. The remote host happens to have its system time set to UTC:
$ date
Mon Oct 7 15:45:00 UTC 2019
$ psql
> select localtimestamp;
2019-10-07 15:45:00.123456
I have a table with an automatically updated date in it (meaning it updates when record is updated).
// my.entity.ts
#UpdateDateColumn({type: 'timestamp', name: 'lastUpdate', default: () => 'LOCALTIMESTAMP' })
lastUpdate: Date;
A row is inserted at 12:00 CEST:
> select "lastUpdate" from myTable;
2019-10-07 10:00:00.000000+00
I would like to get the date out in my timezone (CEST) regardless of the server time, so it should return me 2019-10-07 12:00. I prefer to not hardcode any tricks because it should also work on my CEST machine.
Postgress has all the info:
> show timezone;
UCT
it knows it runs on UTC time
I can tell it in which timezone I want the date
So I would expect it to be easy to convert this to my requested format. However, the following examples I found don't seem to work:
> select ("lastUpdate" at time zone 'CEST') from myTable;
2019-10-07 08:00:00.000000+00
> select timezone('CEST', "lastUpdate") from myTable;
2019-10-07 08:00:00.000000+00
There is one way I can get it right, and thats by specifying the current timezone:
> select ("lastUpdate" at time zone 'UTC' at time zone 'CEST') from myTable;
2019-10-07 12:00:00.000000
However, like I said I don't want to hardcode this (as other DB's run at other TZs) and Postgres knows it's own timezone.
Is there another syntax to do this correctly?
All timestamps in Postgres are stored in UTC regardless of the server configuration. This is important to realize, unlike other databases it does not interpret your input to be in its local time zone. However it will present it back in its time zone.
timestamp stores just the time without a time zone. timestamptz stores UTC plus a time zone.
Demonstrating these together on a Postgres server with its time zone set to UTC...
=> create table demo ( time timestamp, timetz timestamptz );
CREATE TABLE
=> insert into demo values ('2019-10-07 10:00', '2019-10-07 12:00 +0200');
INSERT 0 1
=> select * from demo;
time | timetz
---------------------+------------------------
2019-10-07 10:00:00 | 2019-10-07 10:00:00+00
timestamp ignores the offset and simply stores October 7th, 2019 10:00 with no offset. While timestamptz stores the offset of +0200, but presents the time back to me in its own UTC time zone. They're the same point in time.
You could change the database time zone for your session to CEST. Then Postgres will format timestamptz in CEST.
=> set time zone 'Antarctica/Troll';
=> select * from demo;
time | timetz
---------------------+------------------------
2019-10-07 10:00:00 | 2019-10-07 12:00:00+02
But this is brittle. You have to make sure it happens on every connection. Your application is relying on a specific configuration of your database connection. It's an action-at-a-distance anti-pattern. It's not obvious what's doing the formatting, and if it's removed lots of seemingly unrelated things break.
Instead, you should reformat your times as you like after receiving them from the database. You can do this manually...
=> select timetz at time zone 'CEST' from demo;
timezone
---------------------
2019-10-07 12:00:00
But generally this is something your ORM takes care of for you. I don't know typeorm, but most have a means of translating database types consistently. You should be able to set up typeorm to automatically translate Postgres timestamptz into your local application time zone. Better yet, instead of returning a string it can translate it into a proper Time object which you then have full control over.
select created_at at time zone 'utc' at time zone 'america/los_angeles'
from users;
select ts_tz AT TIME ZONE 'UTC' from test;
I'm trying to understand how the session time zone works in PostgreSQL.
When I set session time zone to '01:00', returned date is in '-01:00'. Sign is always inverted and I do not understand this behavior.
How can this be explained?
postgres=# set time zone '00:00';
SET
postgres=# select timestamp with time zone '2017-11-29 15:00:00.000000+00';
timestamptz
------------------------
2017-11-29 15:00:00+00
(1 row)
postgres=# set time zone '01:00';
SET
postgres=# select timestamp with time zone '2017-11-29 15:00:00.000000+00';
timestamptz
------------------------
2017-11-29 14:00:00-01
(1 row)
postgres=# set time zone '-01:00';
SET
postgres=# select timestamp with time zone '2017-11-29 15:00:00.000000+00';
timestamptz
------------------------
2017-11-29 16:00:00+01
(1 row)
example, here
ts is time (text) - just time as we say it, :
not_aware the time if we don't know its timezone (local to the saying person)
aware the time, but knowing the timezone of it(if server time zone is UTC, then of utc zone)
, so:
s=# with s(city,tz) as (values('Moscow','UTC+3'),('New York','UTC-5'))
, ts as (select '2017-01-01 00:00:00'::text ts)
select *,ts::timestamp at time zone tz not_aware, ts::timestamptz at time zone tz aware
from s
join ts on true;
city | tz | ts | not_aware | aware
----------+-------+---------------------+------------------------+---------------------
Moscow | UTC+3 | 2017-01-01 00:00:00 | 2017-01-01 03:00:00+00 | 2016-12-31 21:00:00
New York | UTC-5 | 2017-01-01 00:00:00 | 2016-12-31 19:00:00+00 | 2017-01-01 05:00:00
(2 rows)
lets say you you come from Lodon and have have watches that show UTC0 time
- local for London (this is your server with timezone UTC0). Now are fly to Moscow, looking at the Kremlin Clock and you see the ts, midnight. Your watches are aware of time difference and they show you that to Ney Year is in three hours (this is timestamptz - aware column). This is why UTC+3 requires to minus the time. Because for your server Moscow time is in future, thus when you see local 2017, for server it is still 2016 and gonna be like it for +3 hours...
Now the behaviour that you expected (not_aware column) is more tricky. The server runs at UTC0, but has to pretend it know nothing about the "real time". As timestamp without time zone is such type. So it behaves as if you were watching new year celebration on Red Square on TV and would just scroll the bar to adjust your watches to Kremlin Clock, so in fact the time you see is your London time +3 (not_aware) column.
I came across this while looking for answers to other questions.
In case you never figured it out:
I think this is a quirk of the timezone parsing code trying to follow POSIX syntax, as noted above. You get the same behavior by e.g. SET TIME ZONE 'CET1:00'; or SET TIME ZONE 'CET1'.
I think the "actual" SQL way of saying what you were trying to say is SET TIME ZONE 1.
You probably don't want to have this be set as your timezone anyhow because it doesn't track DST changes. You probably want to choose a timezone by name from the tz database, for example America/New_York or (going with the GMT+1 example) Europe/Madrid.
I run a Postgres 8.3 database where times seem to be stored in UTC without time zone.
I am trying to display in local time but not with '+01' suffix :
With select scheduled_start_ts I get :
2014-01-20 05:01:35.663
With select scheduled_start_ts at time zone 'MET' :
2014-01-20 05:01:35.663+01
I would like to get "2014-01-20 06:01:35.663" which is in local time.
The database I am using cannot be modified and I am not allowed to modify how data are stored.
If you want to format times, use the to_char function. See formatting functions in the docs.
regress=> SELECT to_char(
(TIMESTAMP '2014-01-20 05:01:35.663' AT TIME ZONE 'UTC')
AT TIME ZONE 'MET',
'YYYY-MM-DD HH:MI:SS'
);
to_char
---------------------
2014-01-20 06:01:35
(1 row)
The (TIMESTAMP 'xxx' AT TIME ZONE 'UTC') gets me a timestamptz with the correct time, by re-interpreting the TIMESTAMP as being in UTC. The second AT TIME STAMP instead converts the timestamptz into a timestamp in timezone MET. This then gets formatted.
Whatever the SQL standards committe were smoking when they designed this, I never, ever, ever want to be anywhere near it.