I have column 'event_date_time' in my 'events' table with type 'timestamp with timezone'. My python flask application is saving date like '2014-08-30 02:17:02+00:00' but postgres automatically converts it to '2014-08-30 07:17:02+05'. It converts the timestamp to my local timezone i-e Pakistan. I want to save it without converting.
I have tried
set timezone='UTC'
and it does change timezone to 'UTC' but pgadmin3 is still saving the converted time.
I am using MAC OS and Postgresql 9.3.
The reason pgadmin is displaying hours +5 is because your system timezone is set to this.
When you save a "timestamp with time zone" value at GMT + or - any value, the system offsets whatever timezone your input was to GMT (or UTC), so that when you go to retrieve it, you can specify the timezone you want it displayed in.
For example let's establish a current time for say... New York.
select now()::timestamp with time zone at time zone 'America/New_York';
At the time of asking it returned '2014-08-23 08:50:57.136817'. 8:50 Saturday morning, or 8:51 if you're being pedantic.
Now if we take that same time and display it in GMT we will see a different result:
select '2014-08-23 08:50:57.136817 America/New_York'::timestamp with time zone at time zone 'GMT';
Now have a new time of '2014-08-23 12:50:57.136817'... 5 hours into the "future"!
Finally let's get the original timestamp and display it in what I believe is the Pakistan time zone (PKT) and see what it shows
select '2014-08-23 08:50:57.136817 America/New_York'::timestamp with time zone at time zone 'PKT';
The result? '2014-08-23 17:50:57.136817' further into the future still!
Again I must stress the reason it can do this is because it is always converting the input time offset to UTC or GMT. Postgres processes all of its "timestamp with time zone" data types in this way. It is designed to avoid time zone problems such as daylight savings and so on.
Your issue appears to be that python is inserting the time at an offset of +00, and if this was supposed to be a local time then you will be 5 hours off as far as postgres is concerned. Without knowing exactly what queries python is making, I would assume you may want to look at that to make sure it is giving you the correct time, presumably set timezone='PKT' should be a fix. Either way, when you are viewing timestamp with time zone using a browser such as pgadmin, the timestamp is being converted to your local timezone and this is why you see +5.
Alternatively if you do wish to see those times at +00 then you must specify that you want this in your SELECT queries.
Related
I am using django models. TIMEZONE in django settings is UTC.
and constructing a timestamp by doing some arithmetic.
return queryset.annotate(
**{f"server_time":RawSQL(
f'''
SELECT TO_TIMESTAMP((FLOOR((EXTRACT(EPOCH FROM "{gmdts_table}"."date_key" AT TIME ZONE %s) + %s) / %s)::INTEGER * %s) - %s)::TIMESTAMP
''',
params=[str(requested_time_zone), bucket_offset, time_interval, time_interval, bucket_offset]
)}
)
I have timestamp being returned as 2021-07-26 00:00:00 when I am using ::timestamp
If I use ::TIMESTAMPTZ, it becomes 2021-07-26 00:00:00+00:00 even though requested_time_zone is 'America/New_York'
I want the output to be 2021-07-26 00:00:00-04:00 ie. show the same time with offset of 'America/New_York appended'. Essentially I just want to append the offset.
I have derived this timestamp by making some calculations and it actually belongs to the timezone 'America/New_York' (it can be any time zone say calculated_time_zone).
How can I update the timezone of this timestamp without actually changing the time for the output, which is:
2021-07-26 00:00:00-04:00
When I use AS TIME ZONE 'America/New_York', I get the output as 2021-07-26 04:00:00+00 which is not what I want.
In python, it can be easily done with a <datetime_object>.replace(tzinfo=new_time_zone). I am looking for the same thing in postgresql.
Any manner in which it can be done with django database functions would be helpful too (I wasn't able to find any)
It's important to understand that the timestamptz type does not store a timezone! It simply stores the UTC timestamp. Which means that your timestamp isn't "in the wrong timezone" because it isn't in any timezone. It is, however, displayed to you in a certain timezone, defined by your session settings.
If you need to shift your timestamp by four hours, as the case may be, then you can add an interval of 4 hours to it. But if you need to "reinterpret" the UTC timestamp to be a timestamp of a different timezone (which means you probably did something wrong when building the timestamp initially), then you can do so by switching to a timezone-naive timestamp and back again:
SELECT timestamp AT TIME ZONE 'UTC' AT TIME ZONE 'America/New_York'
But don't be surprised if this new timestamp is displayed in the same timezone as before, because that's defined by your session settings, not by the timestamp itself.
Strip off the timezone, then add back the new timezone you want. This can be done with:
(myfield::timestamp || '+00:00')::timestamptz
I'm gonna to save date and time in my PostgreSQL database and fetch and show it to the user in an appropriate format.
Suppose that application users are located in Iran and use Jalali date/time system. In the first half of year, Iran time is UTC+04:30, but in the rest of year, it is UTC+03:30. As a matter of fact daylight saving time is used in Iran.
IMPORTANT: Sometimes we make a decision and change database server location from Iran to Europe or elsewhere.
Now I have some questions:
What data type is more convenient to save date with time? TIMESTAMP (TIMESTAMP WITHOUT TIMEZONE) or TIMESTAMPZ (TIMESTAMP WITH TIME ZONE)?
What data type is more convenient to save time only? TIME or TIME WITH TIME ZONE?
What data type is more convenient to save date only? DATE (Gregorian) String (Jalali) or a custom data type?
How can I set TIMEZONE for our database once? I wanna set TIMEZONE for database one time and after that all queries over TIMESTAMPZ columns will be saved and fetched within that TIMEZONE and also it considered daylight saving time ?
What SQL query is the best when I wan to save current date and time ?
a- INSERT INTO test(d) VALUES(now());
b- INSERT INTO test(d) VALUES(now() at time zone 'utc');
c- INSERT INTO test(d) VALUES(now() at time zone 'Asia/Tehran');
d- INSERT INTO test(d) VALUES(current_timestamp);
e- INSERT INTO test(d) VALUES(now() at time zone 'utc');
f- INSERT INTO test(d) VALUES(now() at time zone 'Asia/Tehran');
Is number 5-c and 5-f considered daylight saving times when save it or not?
Is number 5-a and 5-d saved in 'Asia/Tehran' when database has 'Asia/Tehran' time zone ?
When I wan to query from database, what options is the best in my situation ?
a- SELECT d FROM test;
b- SELECT d at time zone 'utc' FROM test;
c- SELECT d at time zone 'Asia/Tehran' FROM test;
Is number 7-c considered daylight saving times?
Is number 7-a considered 'Asia/Tehran' time zone and daylight saving time when database has 'Asia/Tehran' time zone?
If I'd better use the timestamp to save date/time, then I have to add daylight saving time to it and convert it to Jalali date time and show it to the user, and vice versa.
https://www.postgresql.org/docs/current/static/datatype-datetime.html
For timestamp with time zone, the internally stored value is always in
UTC (Universal Coordinated Time, traditionally known as Greenwich Mean
Time, GMT). An input value that has an explicit time zone specified is
converted to UTC using the appropriate offset for that time zone. If
no time zone is stated in the input string, then it is assumed to be
in the time zone indicated by the system's TimeZone parameter, and is
converted to UTC using the offset for the timezone zone.
When a timestamp with time zone value is output, it is always
converted from UTC to the current timezone zone, and displayed as
local time in that zone. To see the time in another time zone, either
change timezone or use the AT TIME ZONE construct
timestamptz
timestamptz and select date only
timestamptz and select time only
database saves tz aware time stamps in UTC, no matter your locale or settings. stamps are always converted to UTC adjusting it by TimeZone parameter. Same deconvert happens on displaying data every time. TimeZone for server is only default value, used if client does not specify any. database TimeZone overrides the one in postgresql.conf for specified database, but still, client settings will override the database ones.
I need to convert local time to UTC using a function. The inputs I have is the local time, timezone of the local time (e.g. 'Pacific/Auckland'). What I need to get from the procedure is the UTC time for the local time based on given timezone.
Can someone please help?
I am using version 8.3
This is covered in the manual, but it's not always obvious how to actually work with dates/times. The SQL spec is a bit bizarre.
In the case of your question it isn't clear whether you want to store the time in UTC but display it in the server's local time (TimeZone), or whether you wan to ignore TimeZone and always display it as UTC. I'll assume the latter.
For timestamps you'd just use AT TIME ZONE twice, like:
SELECT TIMESTAMP '2013-08-13 00:00:00' AT TIME ZONE 'Australia/Sydney' AT TIME ZONE 'UTC';
You need to use AT TIME ZONE twice. Once to convert the input timestamp to timestamptz according to the argument timezone, then another to convert that to a timestamp at UTC.
Unfortunately because of the (IMO insane) way the SQL spec defines AT TIME ZONE for TIME, you can't do the same thing for TIME. You'll have to manipulate the TimeZone variable instead:
SET TimeZone = 'UTC';
SELECT TIME '07:00' AT TIME ZONE 'Australia/Sydney';
This still leaves you with a timetz not a time. So its display value changes with the timezone setting.
You can use select now() at time zone 'UTC' for postgresql.
I think one of the two scourges of IT are timestamps and time zones (the other being character encoding) where one keeps stumbling upon again and again...
In that regard I currently have a problem related to different timestamps within a Java application storing into a PostgreSQL database.
For keeping things simple, assume having the following table:
CREATE TABLE ts_test
(
id integer NOT NULL,
utc timestamp without time zone,
local timestamp with time zone,
CONSTRAINT pk PRIMARY KEY (id)
)
So, I have to store a UTC time stamp and a local one, which in my case is central Europe summer time, so currently UTC+2.
Further assume having 2 entries in the table, which output on the psql console as follows (the database runs in UTC):
# select id, utc, local, local-utc as diff from ts_test;
id | utc | local | diff
----+---------------------+------------------------+----------
1 | 2012-06-27 12:00:00 | 2012-06-27 12:00:00+00 | 00:00:00
2 | 2012-06-27 12:00:00 | 2012-06-27 14:00:00+00 | 02:00:00
(2 rows)
Now, several questions arise:
What exactly does the output in the local column mean?
How does the system know the timezone, I inserted the value in?
How can I see the real raw value (e.g. milliseconds) stored?
I would have assumed, that the first line's local "12:00:00+00" means, that it's 12:00 in UTC, which again is 14:00 in CEST. But it seems (and so I was told by our database admin), that the 2nd line's local "14:00:00+00" is the correct value for 14:00 CEST - which is supported by the diff of 2 hours.
But to produce the 2nd line via sql insert, I have to write
insert into ts_test (id, utc, local) values (2, '2012-06-27 12:00:00', '2012-06-27 16:00:00+02');
which again does not support the predication.
So, to sum up this long question - can anyone enlighten me on how this whole thing works in detail, what the output is supposed to mean and how one should write local time stamps correctly into the database?
According to the output of the local column, the time zone of your SQL session is set to UTC or GMT and not the time zone where you live in. Presumably this is what you mean by: the database runs in UTC. This is the root of the problem, but let's try to elaborate.
The db itself, as a data repository, doesn't have a timezone, but each SQL session has its own timezone.
When they're requested by a SQL session, the values of timestamp without time zone are not rotated to the session's time zone and not presented with a time offset, whereas the values for timestamp with time zone are rotated to the session's timezone and presented with the time offset for this time zone. That's the difference between both.
The time zone is never stored in any of the datatypes, because when reading the value, all that matters is the time zone of the SQL session that is requesting this value.
Setting your SQL time zone to UTC is not a good idea because it contradicts this other part of your question:
So, I have to store a UTC time stamp and a local one, which in my case
is central Europe summer time, so currently UTC+2
Let the SQL session know your real time zone, and it will start to work as intended. If you don't, timestamp with time zone is essentially useless.
Also note that storing the same time in utc timestamp without time zone and local timestamp with time zone doesn't make sense, because you could always get the utc with:
SELECT local AT TIME ZONE 'UTC' FROM ts_test WHERE...
EDIT: answers to questions in the comments:
Q: you are saying that if my timezone is set to my local time in the
session, then I should see for example ...14:00:00+02 in local for a
utc value of ...12:00:00
Yes.
Q: And when writing something into the local field from my application,
it matters which timezone is set there?
Exactly.
Q: How does one set this in JDBS-session?
I don't know JDBC but at the SQL level, that would be for example:
SET timezone='Europe/Berlin';
Normally it's automatically set from the environment but it can be forced at various levels including postgresql.conf. Setting it explicitly in the session will override anything else.
Q: can I see the raw value of the timestamp somehow, to make sure it's
not just a representation problem when displaying
I'm not aware of how to do that except with pageinspect which operates at the lower level.
We're deploying our own stream gauges (a lot like this USGS gauge: http://waterdata.usgs.gov/usa/nwis/uv?site_no=03539600) so us kayakers know whether or not there's enough water to paddle the stream and don't waste time and gas to drive out there. We hope install a few of these across the southeast whitewater region which spans the eastern and central time zones.
I'm storing the time a record is inserted using the default value of current_time for the record. I'd like to later display the data using the MM/DD/YYYY HH12:MI AM TZ format, which outputs reading like 03/12/2012 01:00 AM CDT. I'd also like for the output to be aware of changes in day light savings time, so the last part of the previous sentence would change between CST and CDT when we 'spring forward' and 'fall back'. This change occurred on 3/11/2012 this year and I've included dates on both sides of this DST line below. I'm using my Windows 7 laptop for development and we will later be deploying on a Unix box. Postgres has apparently detected that my Windows computer is set to eastern US time zone. I'm trying this with a 'timestamp without time zone' field and a 'timestamp with time zone' field but can't get it to work.
I've tried using 'at time zone' in my selects and every thing is working until it's time to display the time zone. The actual hour is part of the time stamp is correctly subtracted by an hour when I ask for the time in CDT. But EDT is displayed in the output.
SELECT reading_time as raw,
reading_time at time zone 'CDT',
to_char(reading_time at time zone 'CDT',
'MM/DD/YYYY HH12:MI AM TZ') as formatted_time
FROM readings2;
"2012-04-29 17:59:35.65";"2012-04-29 18:59:35.65-04";"04/29/2012 06:59 PM EDT"
"2012-04-29 17:59:40.19";"2012-04-29 18:59:40.19-04";"04/29/2012 06:59 PM EDT"
"2012-03-10 00:00:00";"2012-03-10 00:00:00-05";"03/10/2012 12:00 AM EST"
"2012-03-11 00:00:00";"2012-03-11 00:00:00-05";"03/11/2012 12:00 AM EST"
"2012-03-12 00:00:00";"2012-03-12 01:00:00-04";"03/12/2012 01:00 AM EDT"
I'm storing the time zone that each of our gauges is located in a character varying field a separate table. I considered just appending this value to the end of the time output, but I want it to change from from CST to CDT without my intervention.
Thanks for your help.
Instead of using time zone abbreviations like CDT or CST, you could consider using full Olson-style time zone names. In the case of central time, you could choose a time zone. Either one that matches your location, such as America/Chicago, or just US/Central. This ensures PostgreSQL uses the Olson tz database to automatically figure out whether daylight saving time applies at any given date.
You definitely want a TIMESTAMP WITH TIME ZONE column (which is also known as timestamptz in PostgreSQL). That will store the timestamp in UTC, so that it represents a particular moment in time. Contrary to what the name suggests, it does not save a time zone in the column -- you can view the retrieved timestamp in the time zone of your choosing with the AT TIME ZONE phrase.
The semantics of TIMESTAMP WITHOUT TIME ZONE are confusing and nearly useless. I strongly recommend you don't use that type at all for what you are describing.
I'm really confused by the part of the question which talks about storing the timestamp in a CHARACTER VARYING column. That seems as though it might be part of the problem. If you can store it in timestamptz right from the start I suspect that you will have fewer problems. Barring that, it would be safest to use the -04 notation for offset from UTC; but that seems like more work to me for no benefit.
You can create a table of known timezones in the format suggested in Guan Yang's answer, and then use a foreign key column to this table. Valid timezones can be obtained from pg_timezone_names I've gone into more detail in this related answer.