When running Redshift queries using Razor SQL, UTC dates appear to be treated as being in the local timezone, complete with daylight saving times.
For example, running
SELECT 'first',CONVERT_TIMEZONE('UTC', 'America/New_York', '2016-03-27 06:00:00')
UNION
SELECT 'second', CONVERT_TIMEZONE('UTC', 'America/New_York', '2016-03-27 07:00:00')
returns the same time for each, 2016-03-27 03:00
New York actually changed to daylight saving time on the 13th March and this does work:
SELECT 'first',CONVERT_TIMEZONE('UTC', 'America/New_York', '2016-03-13 06:00:00')
UNION
SELECT 'second', CONVERT_TIMEZONE('UTC', 'America/New_York', '2016-03-13 07:00:00')
So this turned out to be a bug in the amazon redshift JDBCS driver. If you run the same query in the standard postgres JDBC drivers it works just fine.
I haven't had any feedback from Amazon on a possible fix.
Related
I am trying to understand the behavior of AT TIME ZONE and I get some strange behavioral differences between SQL Server and PostgreSQL in this regard.
For demonstration purposes I produced two queries, one for SQL Server and one for PostgreSQL.
The SQL Server query:
SELECT
[LocalTime],
[LocalTime] AT TIME ZONE 'W. Europe Standard Time' AS [LocalTimeWithOffset],
([LocalTime] AT TIME ZONE 'W. Europe Standard Time') AT TIME ZONE 'UTC' AS [UniversalTime]
FROM
(VALUES
(CAST('2020-06-20 12:34:56.789' AS DATETIME2)),
(CAST('2020-12-20 12:34:56.789' AS DATETIME2))) AS [X] ([LocalTime])
produces this result (in SQL Server Management Studio):
LocalTime LocalTimeWithOffset UniversalTime
--------- ------------------- ----------------
2020-06-20 12:34:56.7890000 2020-06-20 12:34:56.7890000 +02:00 2020-06-20 10:34:56.7890000 +00:00
2020-12-20 12:34:56.7890000 2020-12-20 12:34:56.7890000 +01:00 2020-12-20 11:34:56.7890000 +00:00
which seems fine to me, since I live in the Netherlands: in the summertime, we are 2 hours ahead of UTC (due to daylight saving) and in the wintertime we are only 1 hour ahead of UTC.
BUT... the PostgreSQL query:
SELECT
"LocalTime",
"LocalTime" AT TIME ZONE 'WET' AS "LocalTimeWithOffset",
("LocalTime" AT TIME ZONE 'WET') AT TIME ZONE 'UTC' AS "UniversalTime"
FROM
(VALUES
(timestamp '2020-06-20 12:34:56.789'),
(timestamp '2020-12-20 12:34:56.789')) AS "X" ("LocalTime")
produces the following result (in pgAdmin):
LocalTime LocalTimeWithOffset UniversalTime
timestamp without time zone timestamp with time zone timestamp without time zone
--------------------------- ------------------------ ---------------------------
2020-06-20 12:34:56.789 2020-06-20 14:34:56.789+02 2020-06-20 12:34:56.789
2020-12-20 12:34:56.789 2020-12-20 13:34:56.789+01 2020-12-20 12:34:56.789
which seems incorrect to me.
After struggling with this for a little while, I came up with another PostgreSQL query, which seems to yield the same results as the SQL Server query:
SELECT
"LocalTime",
"LocalTime"::timestamptz AS "LocalTimeWithOffset",
"LocalTime"::timestamptz AT TIME ZONE 'UTC' AS "UniversalTime"
FROM
(VALUES
(timestamp '2020-06-20 12:34:56.789'),
(timestamp '2020-12-20 12:34:56.789')) AS "X" ("LocalTime")
But this query seems to be dependent on the PostgreSQL server's time zone. :-(
Is there any (preferably elegant) way to mimic the SQL Server logic/behavior in PostgreSQL without a dependency on the server's time zone (and use an explicitly specified time zone in the query)?
Edit
I am aware that it is a good practice to store UTC date/time values in the database and convert them (preferably in/by the client) to local date/time values when presented to end users. I am intending to do so as well, of course, but my current scenario is a database migration from SQL Server to PostgreSQL (as part of a big application overhaul), where the DATETIME2 table columns in the SQL Server database contain local date/time values, which I want to convert to UTC date/time values in corresponding timestamp columns in the PostgreSQL tables. My migration script will work fine with the queries above, so actually I do not have a blocking issue. The reason for this question is that I am just curious about the apparent differences between SQL Server and PostgreSQL regarding the AT TIME ZONE behavior and the ways to overcome those behavioral differences.
In postgresql, AT TIME ZONE allows time zones to be added to date/time values that lack them (TIMESTAMP WITHOUT TIME ZONE, ::timestamp), and allows TIMESTAMP WITH TIME ZONE values (::timestamptz) to be shifted to non-local time zones and the time zone designation removed.
So, in the first query you've written for postgres, you're trying to cast '2020-06-20 12:34:56.789' to timestamp, and you've applied AT TIME ZONE on it for LocalTimeWithOffset. This is the case where postgres considered the input as timestamp in UTC and converted to the corresponding time zone, in your case WET, which is UTC+2:00 in Summer and UTC+1:00 in winter.
Now, in the third case, you're first applying the AT TIME ZONE 'WET' to timestamp '2020-06-20 12:34:56.789', hence the input is considered as UTC and converted to WET, the output is timstamptz, and this output is then fed again to AT TIME ZONE 'UTC', so as per behavior of AT TIME ZONE, it gets converted to UTC and the timezone tag is removed, which is, the input we given.
So, AT TIME ZONE did not behaved weird in postgresql rather did exactly what it was supposed to do, according to the documentation.
Hope your confusion is cleared.
I'm quite new to Grafana and Postgres and could use some help with this. I have a dataset in PostgreSQL with temperature forecasts. Mutiple forecasts are published at various points throughout the day (indicated by dump_date) for the same reference date. Say: at 06:00 today and at 12:00 today a forecast is published for tomorrow (where the time is indicated by start_time). Now I want to visualize the temperature forecast as a time series using Grafana. However, I only want to visualize the latest published forecast (12:00) and not both forecasts. I thought I would use DISTINCT ON() to select only the latest published forecast from this dataset, but somehow with Grafana this is not responding. My code in Grafana is as follows:
SELECT
$__time(distinct on(t_ID.start_time)),
concat('Forecast')::text as metric,
t_ID.value
FROM
forecast_table t_ID
WHERE
$__timeFilter(t_ID.start_time)
and t_ID.start_time >= (current_timestamp - interval '30 minute')
and t_ID.dump_date >= (current_timestamp - interval '30 minute')
ORDER BY
t_ID.start_time asc,
t_ID.dump_date desc
This is not working however since I get the message: 'syntax error at or near AS'. What should I do?
You are using Grafana macro $__time, so your query in the editor:
SELECT
$__time(distinct on(t_ID.start_time)),
generates SQL:
SELECT
distinct on(t_ID.start_time AS "time"),
which is incorrect SQL syntax.
I wouldn't use macro. I would write correct SQL directly, e.g.
SELECT
distinct_on(t_ID.start_time) AS "time",
Also use Generated SQL and Query inspector Grafana features for debugging and query development. Make sure that Grafana generates correct SQL for Postgres.
I try to deploy a JS application using TypeORM and Postgres on a host. Locally I had a Postgress DB so running in my own TZ. The remote host happens to have its system time set to UTC:
$ date
Mon Oct 7 15:45:00 UTC 2019
$ psql
> select localtimestamp;
2019-10-07 15:45:00.123456
I have a table with an automatically updated date in it (meaning it updates when record is updated).
// my.entity.ts
#UpdateDateColumn({type: 'timestamp', name: 'lastUpdate', default: () => 'LOCALTIMESTAMP' })
lastUpdate: Date;
A row is inserted at 12:00 CEST:
> select "lastUpdate" from myTable;
2019-10-07 10:00:00.000000+00
I would like to get the date out in my timezone (CEST) regardless of the server time, so it should return me 2019-10-07 12:00. I prefer to not hardcode any tricks because it should also work on my CEST machine.
Postgress has all the info:
> show timezone;
UCT
it knows it runs on UTC time
I can tell it in which timezone I want the date
So I would expect it to be easy to convert this to my requested format. However, the following examples I found don't seem to work:
> select ("lastUpdate" at time zone 'CEST') from myTable;
2019-10-07 08:00:00.000000+00
> select timezone('CEST', "lastUpdate") from myTable;
2019-10-07 08:00:00.000000+00
There is one way I can get it right, and thats by specifying the current timezone:
> select ("lastUpdate" at time zone 'UTC' at time zone 'CEST') from myTable;
2019-10-07 12:00:00.000000
However, like I said I don't want to hardcode this (as other DB's run at other TZs) and Postgres knows it's own timezone.
Is there another syntax to do this correctly?
All timestamps in Postgres are stored in UTC regardless of the server configuration. This is important to realize, unlike other databases it does not interpret your input to be in its local time zone. However it will present it back in its time zone.
timestamp stores just the time without a time zone. timestamptz stores UTC plus a time zone.
Demonstrating these together on a Postgres server with its time zone set to UTC...
=> create table demo ( time timestamp, timetz timestamptz );
CREATE TABLE
=> insert into demo values ('2019-10-07 10:00', '2019-10-07 12:00 +0200');
INSERT 0 1
=> select * from demo;
time | timetz
---------------------+------------------------
2019-10-07 10:00:00 | 2019-10-07 10:00:00+00
timestamp ignores the offset and simply stores October 7th, 2019 10:00 with no offset. While timestamptz stores the offset of +0200, but presents the time back to me in its own UTC time zone. They're the same point in time.
You could change the database time zone for your session to CEST. Then Postgres will format timestamptz in CEST.
=> set time zone 'Antarctica/Troll';
=> select * from demo;
time | timetz
---------------------+------------------------
2019-10-07 10:00:00 | 2019-10-07 12:00:00+02
But this is brittle. You have to make sure it happens on every connection. Your application is relying on a specific configuration of your database connection. It's an action-at-a-distance anti-pattern. It's not obvious what's doing the formatting, and if it's removed lots of seemingly unrelated things break.
Instead, you should reformat your times as you like after receiving them from the database. You can do this manually...
=> select timetz at time zone 'CEST' from demo;
timezone
---------------------
2019-10-07 12:00:00
But generally this is something your ORM takes care of for you. I don't know typeorm, but most have a means of translating database types consistently. You should be able to set up typeorm to automatically translate Postgres timestamptz into your local application time zone. Better yet, instead of returning a string it can translate it into a proper Time object which you then have full control over.
select created_at at time zone 'utc' at time zone 'america/los_angeles'
from users;
select ts_tz AT TIME ZONE 'UTC' from test;
Is there a better way to get local time to the second in PostgreSQL than this?
Set timezone= 'America/New_York';
Select to_char(current_timestamp, 'MM/DD/YYYY HH24:MI:SS') As EasternCurrentTime;
We do a lot of these in different places for logging within our ETL and would like to do it on one line.
I'd suggest
SELECT to_char(current_timestamp AT TIME ZONE 'America/New_York',
'MM/DD/YYYY HH24:MI:SS');
Hi I have a db2 DB on mainframe that has data being written to by a web app and phone voice telephony ivr app. They were all located in the Central us time zone. The web app is being rewritten and moved to Eastern US time. The insert records all use sysdate for records timestamps like when the record was created or last updated. Since the db queries are "select ...... order by create_time" or "select .... order by update_time" The different time zones cause order problems.
I want a way to display all the records in one TZ (probably eastern).
select some_time_util(*,easternTZ) from table where condition=easternTz order by some_time_util( create_date, easten_tz)
union
select some_time_util(*,centralTZ) from table where condition=centralTz order by some_time_util( create_date, centeral_tz)
how is db2 supporting the concept of timestamsp and timezones?
Yes it is supported by DB2 - check out Time zone specific expressions
http://www.ibm.com/support/knowledgecenter/SSEPEK_10.0.0/com.ibm.db2z10.doc.sqlref/src/tpc/db2z_tzspecificexpression.dita