I wanted to convert (UPDATE in Postgres with SQL) a DateTime field without timezone from Middle European Time to UTC.
There is a problem here that the DateTime may be daylight time or winter time so that the DateTime is UTC+1 or UTC+2. The sticking point is to find the the DST boundary with Postgres with bord means.
I tried to write a stored procedure that could find the above mentioned DST boundary but I didn't find any simply suitable solution but this one:
https://www.keithf4.com/postgresql_dst/
Is there maybe a simpler solution?
That is simple. Convert it to an absolute timestamp and back. If you specify the time zone correctly, daylight savings time will be considered automatically.
For Austria, it would look like this:
SELECT TIMESTAMP '...' AT TIME ZONE 'Vienna/Europe' AT TIME ZONE 'UTC';
Related
I faced with the following issue this morning:
select '2011-12-30 00:30:00'::timestamp without time zone AT TIME ZONE 'EST5EDT';
returns me 2011-12-30 05:30:00+00 witch is wrong.
But next queries below:
select '2011-12-30 00:30:00'::timestamp without time zone AT TIME ZONE 'UTC-5';
select '2011-12-30 00:30:00' AT TIME ZONE 'EST5EDT';
i see right date 2011-12-29 19:30:00
Preventing your question about my local timezone:
SELECT current_setting('TIMEZONE');
current_setting
-----------------
UTC
(1 row)
Do anyone have answer why postgresql converts timestamp without time zone some weird way and instead taking away 5 hours it adds instead?
Key things to understand
timestamp without time zone AT TIME ZONE re-interprets a timestamp as being in that time zone for the purpose of converting it to UTC.
timestamp with time zone AT TIME ZONE converts a timestamptz into a timestamp at the specified timezone.
PostgreSQL uses ISO-8601 timezones, which specify that east of Greenwich is positive ... unless you use a POSIX timezone specifier, in which case it follows POSIX. Insanity ensues.
Why the first one produces an unexpected result
Timestamps and timezones in SQL are horrible. This:
select '2011-12-30 00:30:00'::timestamp without time zone AT TIME ZONE 'EST5EDT';
inteprets the unknown-typed literal '2011-12-30 00:30:00' as timestamp without time zone, which Pg assumes is in the local TimeZone unless told otherwise. When you use AT TIME ZONE, it is (per the spec) re-interpreted as a timestamp with time zone in the time zone EST5EDT then stored as an absolute time in UTC - so it's converted from EST5EDT to UTC, i.e the timezone offset gets subtracted. x - (-5) is x + 5.
This timestamp, adjusted to UTC storage, is then adjusted for your server TimeZone setting for display so that it gets displayed in local time.
If you instead wish to say "I have this timestamp in UTC time, and wish to see what the equivalent local time in EST5EDT is", if you want to be independent of the server TimeZone setting, you need to write something like:
select TIMESTAMP '2011-12-30 00:30:00' AT TIME ZONE 'UTC'
AT TIME ZONE 'EST5EDT';
This says "Given timestamp 2011-12-30 00:30:00, treat it as a timestamp in UTC when converting to timestamptz, then convert that timestamptz to a local time in EST5EDT".
Horrible, isn't it? I want to give a firm talking to whoever decided on the crazy semantics of AT TIME ZONE - it should really be something like timestamp CONVERT FROM TIME ZONE '-5' and timestamptz CONVERT TO TIME ZONE '+5'. Also, timestamp with time zone should actually carry its timezone with it, not be stored in UTC and auto-converted to localtime.
Why the second works (so long as TimeZone = UTC)
Your original "works" version:
select '2011-12-30 00:30:00' AT TIME ZONE 'EST5EDT';
will only be correct if TimeZone is set to UTC, because the text-to-timestamptz cast assumes TimeZone when one isn't specified.
Why the third one works
Two problems cancel each other out.
The other version that appears to work is TimeZone independent, but it only works because two problems cancel themselves out. First, as explained above, timestamp without time zone AT TIME ZONE re-interprets the timestamp as being in that time zone for conversion to a UTC timestamptz; this effectively subtracts the timezone offset.
However, for reasons I beyond my ken, PostgreSQL uses timestamps with the reverse sign to what I'm used to seeing most places. See the documentation:
Another issue to keep in mind is that in POSIX time zone names, positive offsets are used for locations west of Greenwich. Everywhere else, PostgreSQL follows the ISO-8601 convention that positive timezone offsets are east of Greenwich.
This means that EST5EDT is the same as +5, not -5. Which is why it works: because you're subtracting the tz offset not adding it, but you're subtracting a negated offset!
What you'd need to get it correct is instead:
select TIMESTAMP '2011-12-30 00:30:00' AT TIME ZONE 'UTC'
AT TIME ZONE '+5';
I need to convert local time to UTC using a function. The inputs I have is the local time, timezone of the local time (e.g. 'Pacific/Auckland'). What I need to get from the procedure is the UTC time for the local time based on given timezone.
Can someone please help?
I am using version 8.3
This is covered in the manual, but it's not always obvious how to actually work with dates/times. The SQL spec is a bit bizarre.
In the case of your question it isn't clear whether you want to store the time in UTC but display it in the server's local time (TimeZone), or whether you wan to ignore TimeZone and always display it as UTC. I'll assume the latter.
For timestamps you'd just use AT TIME ZONE twice, like:
SELECT TIMESTAMP '2013-08-13 00:00:00' AT TIME ZONE 'Australia/Sydney' AT TIME ZONE 'UTC';
You need to use AT TIME ZONE twice. Once to convert the input timestamp to timestamptz according to the argument timezone, then another to convert that to a timestamp at UTC.
Unfortunately because of the (IMO insane) way the SQL spec defines AT TIME ZONE for TIME, you can't do the same thing for TIME. You'll have to manipulate the TimeZone variable instead:
SET TimeZone = 'UTC';
SELECT TIME '07:00' AT TIME ZONE 'Australia/Sydney';
This still leaves you with a timetz not a time. So its display value changes with the timezone setting.
You can use select now() at time zone 'UTC' for postgresql.
I'm having an issue selecting dates properly from Postgres - they are being stored in UTC, but
not converting with the Date() function properly.
Converting the timestamp to a date gives me the wrong date if it's past 4pm PST.
2012-06-21 should be 2012-06-20 in this case.
The starts_at column datatype is timestamp without time zone. Here are my queries:
Without converting to PST timezone:
Select starts_at from schedules where id = 40;
starts_at
---------------------
2012-06-21 01:00:00
Converting gives this:
Select (starts_at at time zone 'pst') from schedules where id = 40;
timezone
------------------------
2012-06-21 02:00:00-07
But neither convert to the correct date in the timezone.
Basically what you want is:
$ select starts_at AT TIME ZONE 'UTC' AT TIME ZONE 'US/Pacific' from schedules where id = 40
I got the solution from this article is below, which is straight GOLD!!! It explains this non-trivial issue very clearly, give it a read if you wish to understand pstgrsql TZ management better.
Expressing PostgreSQL timestamps without zones in local time
Here is what is going on. First you should know that 'PST timezone is 8 hours behind UTC timezone so for instance Jan 1st 2014, 4:30 PM PST (Wed, 01 Jan 2014 16:00:30 -0800) is equivalent to Jan 2nd 2014, 00:30 AM UTC (Thu, 02 Jan 2014 00:00:30 +0000). Any time after 4:00pm in PST slips over to the next day, interpreted as UTC.
Also, as Erwin Brandstetter mentioned above, postresql has two type of timestamps data type, one with a timezone and one without.
If your timestamps include a timezone, then a simple:
$ select starts_at AT TIME ZONE 'US/Pacific' from schedules where id = 40
will work. However if your timestamp is timezoneless, executing the above command will not work, and you must FIRST convert your timezoneless timestamp to a timestamp with a timezone, namely a UTC timezone, and ONLY THEN convert it to your desired 'PST' or 'US/Pacific' (which are the same up to some daylight saving time issues. I think you should be fine with either).
Let me demonstrate with an example where I create a timezoneless timestamp. Let's assume for convenience that our local timezone is indeed 'PST' (if it weren't then it gets a tiny bit more complicated which is unnecessary for the purpose of this explanation).
Say I have:
$ select timestamp '2014-01-2 00:30:00' AS a, timestamp '2014-01-2 00:30:00' AT TIME ZONE 'UTC' AS b, timestamp '2014-01-2 00:30:00' AT TIME ZONE 'UTC' AT TIME ZONE 'PST' AS c, timestamp '2014-01-2 00:30:00' AT TIME ZONE 'PST' AS d
This will yield:
"a"=>"2014-01-02 00:30:00" (This is the timezoneless timestamp)
"b"=>"2014-01-02 00:30:00+00" (This is the UTC TZ timestamp, note that up to a timezone, it is equivalent to the timezoneless one)
"c"=>"2014-01-01 16:30:00" (This is the correct 'PST' TZ conversion of the UTC timezone, if you read the documentation postgresql will not print the actual TZ for this conversion)
"d"=>"2014-01-02 08:30:00+00"
The last timestamp is the reason for all the confusion regarding converting timezoneless timestamp from UTC to 'PST' in postgresql. When we write:
timestamp '2014-01-2 00:30:00' AT TIME ZONE 'PST' AS d
We are taking a timezoneless timestamp and try to convert it to 'PST TZ (we indirectly assume that postgresql will understand that we want it to convert the timestamp from a UTC TZ, but postresql has plans of its own!). In practice, what postgresql does is it takes the timezoneless timestamp ('2014-01-2 00:30:00) and treats it as if it WERE ALREADY a 'PST' TZ timestamp (i.e: 2014-01-2 00:30:00 -0800) and converts that to UTC timezone!!! So it actually pushes it 8 hours ahead instead of back! Thus we get (2014-01-02 08:30:00+00).
Anyway, this last (un-intuitive) behavior is the cause of all confusion. Read the article if you want a more thorough explanation, I actually got results which are a bit different then their on this last part, but the general idea is the same.
I don't see the exact type of starts_at in your question. You really should include this information, it is the key to the solution. I'll have to guess.
PostgreSQL always stores UTC time for the type timestamp with time zone internally. Input and output (display) are adjusted to the current timezone setting or to the given time zone. The effect of AT TIME ZONE also changes with the underlying data type. See:
Ignoring time zones altogether in Rails and PostgreSQL
If you extract a date from type timestamp [without time zone], you get the date for the current time zone. The day in the output will be the same as in the display of the timestamp value.
If you extract a date from type timestamp with time zone (timestamptz for short), the time zone offset is "applied" first. You still get the date for the current time zone, which agrees with the display of the timestamp. The same point in time translates to the next day in parts of Europe, when it is past 4 p.m. in California for instance. To get the date for a certain time zone, apply AT TIME ZONE first.
Therefore, what you describe at the top of the question contradicts your example.
Given that starts_at is a timestamp [without time zone] and the time on your server is set to the local time. Test with:
SELECT now();
Does it display the same time as a clock on your wall? If yes (and the db server is running with correct time), the timezone setting of your current session agrees with your local time zone. If no, you may want to visit the setting of timezone in your postgresql.conf or your client for the session. Details in the manual.
Be aware that the timezone offset used the opposite sign of what's displayed in timestamp literals. See:
Peculiar time zone handling in a Postgres database
To get your local date from starts_at just
SELECT starts_at::date
Tantamount to:
SELECT date(starts_at)
BTW, your local time is at UTC-7 right now, not UTC-8, because daylight savings time is in effect (not among the brighter ideas of the human race).
Pacific Standard TIME (PST) is normally 8 hours "earlier" (bigger timestamp value) than UTC (Universal Time Zone), but during daylight saving periods (like now) it can be 7 hours. That's why timestamptz is displayed as 2012-06-21 02:00:00-07 in your example. The construct AT TIME ZONE 'PST' takes daylight saving time into account. These two expressions yield different results (one in winter, one in summer) and may result in different dates when cast:
SELECT '2012-06-21 01:00:00'::timestamp AT TIME ZONE 'PST'
, '2012-12-21 01:00:00'::timestamp AT TIME ZONE 'PST'
I know this is an old one but You may want to consider using AT TIME ZONE "US/Pacific" when casting to avoid any PST/PDT issues. So
SELECT starts_at::TIMESTAMPTZ AT TIME ZONE "US/Pacific"
FROM schedules
WHERE ID = '40';
We're deploying our own stream gauges (a lot like this USGS gauge: http://waterdata.usgs.gov/usa/nwis/uv?site_no=03539600) so us kayakers know whether or not there's enough water to paddle the stream and don't waste time and gas to drive out there. We hope install a few of these across the southeast whitewater region which spans the eastern and central time zones.
I'm storing the time a record is inserted using the default value of current_time for the record. I'd like to later display the data using the MM/DD/YYYY HH12:MI AM TZ format, which outputs reading like 03/12/2012 01:00 AM CDT. I'd also like for the output to be aware of changes in day light savings time, so the last part of the previous sentence would change between CST and CDT when we 'spring forward' and 'fall back'. This change occurred on 3/11/2012 this year and I've included dates on both sides of this DST line below. I'm using my Windows 7 laptop for development and we will later be deploying on a Unix box. Postgres has apparently detected that my Windows computer is set to eastern US time zone. I'm trying this with a 'timestamp without time zone' field and a 'timestamp with time zone' field but can't get it to work.
I've tried using 'at time zone' in my selects and every thing is working until it's time to display the time zone. The actual hour is part of the time stamp is correctly subtracted by an hour when I ask for the time in CDT. But EDT is displayed in the output.
SELECT reading_time as raw,
reading_time at time zone 'CDT',
to_char(reading_time at time zone 'CDT',
'MM/DD/YYYY HH12:MI AM TZ') as formatted_time
FROM readings2;
"2012-04-29 17:59:35.65";"2012-04-29 18:59:35.65-04";"04/29/2012 06:59 PM EDT"
"2012-04-29 17:59:40.19";"2012-04-29 18:59:40.19-04";"04/29/2012 06:59 PM EDT"
"2012-03-10 00:00:00";"2012-03-10 00:00:00-05";"03/10/2012 12:00 AM EST"
"2012-03-11 00:00:00";"2012-03-11 00:00:00-05";"03/11/2012 12:00 AM EST"
"2012-03-12 00:00:00";"2012-03-12 01:00:00-04";"03/12/2012 01:00 AM EDT"
I'm storing the time zone that each of our gauges is located in a character varying field a separate table. I considered just appending this value to the end of the time output, but I want it to change from from CST to CDT without my intervention.
Thanks for your help.
Instead of using time zone abbreviations like CDT or CST, you could consider using full Olson-style time zone names. In the case of central time, you could choose a time zone. Either one that matches your location, such as America/Chicago, or just US/Central. This ensures PostgreSQL uses the Olson tz database to automatically figure out whether daylight saving time applies at any given date.
You definitely want a TIMESTAMP WITH TIME ZONE column (which is also known as timestamptz in PostgreSQL). That will store the timestamp in UTC, so that it represents a particular moment in time. Contrary to what the name suggests, it does not save a time zone in the column -- you can view the retrieved timestamp in the time zone of your choosing with the AT TIME ZONE phrase.
The semantics of TIMESTAMP WITHOUT TIME ZONE are confusing and nearly useless. I strongly recommend you don't use that type at all for what you are describing.
I'm really confused by the part of the question which talks about storing the timestamp in a CHARACTER VARYING column. That seems as though it might be part of the problem. If you can store it in timestamptz right from the start I suspect that you will have fewer problems. Barring that, it would be safest to use the -04 notation for offset from UTC; but that seems like more work to me for no benefit.
You can create a table of known timezones in the format suggested in Guan Yang's answer, and then use a foreign key column to this table. Valid timezones can be obtained from pg_timezone_names I've gone into more detail in this related answer.
I have an app that stores times in GMT, but associated with airports which have Olsen tz names like 'America/New York'.
I need to determine if a given date is within the local daylight savings period.
The closest I could find is TZ_OFFSET('Tzname'), but no simple way to get the next DST start/end time relative to some date.
This seems like a step in the right direction...
select cast (mydate as timestamp) at time zone 'GMT' as mydateZ,
from_tz(cast (mydate as timestamp),'GMT') at time zone myzoneName as mydateLocal
Thinking about a similar problem I wrote the following query to your question:
select decode(to_char(from_tz(cast (sysdate as timestamp),'CET'),'TZR'),
to_char(from_tz(cast (sysdate as timestamp),'CET'),'TZD'),'N','Y') is_daylight
from dual
A short explanation:
I convert the current date (sysdate) in my timezone ('CET') to a timestamp and compare the the normal name ('TZR') and daylight savings time name ('TZD') of the timezone. If they match then sysdate is not within daylight saving and if they do not match (for example sysdate-100 is 'CEST' for 'CET' timezone) then the date is within daylight saving.
Date and timezone can be a parameter of the query.