Based on the first two answers, the question was unclear as originally posted, thus I am completely rewriting it:
The following question is concerned only with how and what data is stored, and is no way shape or form concerned with converting data upon retrieval. As such, converting at SELECT to the desired time zone is not an appropriate answer.
When inserting a value into a timestamp with time zone field, it is retrieved (and thus presumably stored) with the timestamp converted to the local time zone of the database at the time it was inserted.
That is so say, a timestamp inserted as 2012-01-01 00:00:00+00:00 is retrieved as 2011-12-31 19:00:00-05, where the local time zone of the database at the time of the insert was -05. Timestamps that were inserted during daylight savings time, when the database was at -04, are returned using the -04 time zone.
What I want is for all timestamps to use an arbitrary time zone when stored (and thus all be retrieved without any additional work as having that time zone). That is to say, were the server orbiting the planet, all times would be at +00:00 (arbitrary time zone), instead of -12:00 to +12:00.
Can I insert into a timestamp with time zone column such that all timestamps are stored relative to an arbitrary time zone? If so, how?
Original follows.
When inserting a value into a timestamp with time zone field, it is being converted to the server's current time zone.
Example: If I insert a value specifying a time zone of -1, retrieving it will give back the time at -5 (the time zone of the server at the time it was inserted).
Is it possible to specify that it should be stored using an arbitrary time zone?
Note: This question is not how to convert the returned time to another time zone, this is specific to how the time is stored.
You have to save the time zone offset in addition to the timestamp.
As #Milen already explained (and linked to the manual): a timestamp only saves a point in time (as abstract value). The time zone modifier is not saved, it only serves to adjust the timestamp relative to UTC.
Consider the following demo:
-- DROP TABLE tbl;
CREATE TEMP TABLE tbl (id int, myts timestamptz, mytz interval);
INSERT INTO tbl VALUES
(1, now() , EXTRACT (timezone from now()) * interval '1s')
,(2, '2012-01-01 00:00-05', interval '-5h')
,(3, '2012-01-01 00:00+04', interval '4h')
,(4, '2012-11-11 20:30+03', interval '3h');
SELECT *
,(myts AT TIME ZONE mytz)::text
|| CASE WHEN mytz > '0:0' THEN '+' ELSE '' END
|| to_char(mytz, 'FMHH24:mi') AS timestamp_at_origin
FROM tbl;
Run it locally to see. Pay special attention to the details of the AT TIME ZONE construct, and how I extract the time zone from the (local!) timestamp with time zone.
now() returns timestamp with time zone or timestamptz for short.
EXTRACT (timezone from now()) * interval '1s'
timestamp_at_origin displays the timestamp with time zone as seen at its origin. If I understood your question, then that is what you are looking for.
You could further improve formatting.
You may be interested in this related question which sheds some light on the ambiguities and pitfalls of time zones.
When inserting a value into a timestamp with time zone field what actually happens is the timestamp is converted to UTC. Another matter altogether is to what time zone that value is converted on output. There are a few ways to control that:
When a timestamp with time zone value is output, it is always
converted from UTC to the current timezone zone, and displayed as
local time in that zone. To see the time in another time zone, either
change timezone or use the AT TIME ZONE construct (see Section
9.9.3).
You can do your select with an at time zone operator:
select insertTs at time zone 'CST' from table
See more here.
I always store times in GMT so that the client can convert based on it's current GMT offset (the GMT offest is available in most language).
I write C# - so I can easily convert all DateTime objects to GMT using DateTime.ToUniversalTime() as I store data in the database.
I am not sure what language you are using or how to convert all times to GMT in postgressql, but from a logic standpoint - storing all times in GMT will allow a uniform time zone that all other time zones can easily relate to.
Hope this helps!
Greenwich Mean Time
Related
I am creating a table with timestamp column,I am stuggling with time zone settings, I want to specify the time zone on the column,as follows:
create table t1(a date, b timestamp with time zone 'America/Los_Angeles', c timestamp without time zone)
But the grammar is wrong, I would ask how to specify the time zone on the column, than
👉 You do not specify a time zone when defining the column.
CREATE TABLE t1
(
a DATE ,
b TIMESTAMP WITH TIME ZONE ,
c TIMESTAMP WITHOUT TIME ZONE
)
;
You need to read the documentation carefully. Programming by intuition tends to end badly.
The TIMESTAMP WITH TIME ZONE type in Postgres does not save a time zone. The type uses any offset or time zone info supplied with an input to adjust to an offset of zero hours-minutes-seconds from UTC. 👉 Every value in that column is set to an offset of zero. After adjusting to zero offset, the supplied time zone or offset info is discarded by Postgres.
If you care about the original time zone, you need to write that value into a separate column yourself.
In contrast, the TIMESTAMP WITHOUT TIME ZONE type lacks any concept of a time zone or offset from UTC. A column of this type stores simply a date and a time-of-day. So values in this column cannot represent a moment, cannot refer to a specific point on the timeline. If you write noon on the 23rd of last January, we have no way of knowing if you meant noon in Tokyo Japan, noon in Toulouse France, or noon in Toledo Ohio US. Those would be three different moments, several hours apart.
Some other databases share the same behavior as Postgres. But not all. The SQL standard barely touches on the subject of date-time, just mentioning the types but without much detail regarding prescribed behavior. As a consequence, date-time behavior varies widely across database engines.
I should mention that some tools have an anti-feature where they inject a default time zone, used to adjust a value stored in UTC to that zone. pgAdmin is, unfortunately, one such tool. While well-intentioned as a convenience to the user, this behavior creates the illusion of a time zone having been stored and retrieved. I would rather all tools “tell the truth”, and report retrieved values with an offset of zero. A workaround is to set the current default time zone of your database session to UTC.
All this has been covered many times already here on Stack Overflow, and also on the sister site https://dba.stackexchange.com/. Search to learn more.
I am using django models. TIMEZONE in django settings is UTC.
and constructing a timestamp by doing some arithmetic.
return queryset.annotate(
**{f"server_time":RawSQL(
f'''
SELECT TO_TIMESTAMP((FLOOR((EXTRACT(EPOCH FROM "{gmdts_table}"."date_key" AT TIME ZONE %s) + %s) / %s)::INTEGER * %s) - %s)::TIMESTAMP
''',
params=[str(requested_time_zone), bucket_offset, time_interval, time_interval, bucket_offset]
)}
)
I have timestamp being returned as 2021-07-26 00:00:00 when I am using ::timestamp
If I use ::TIMESTAMPTZ, it becomes 2021-07-26 00:00:00+00:00 even though requested_time_zone is 'America/New_York'
I want the output to be 2021-07-26 00:00:00-04:00 ie. show the same time with offset of 'America/New_York appended'. Essentially I just want to append the offset.
I have derived this timestamp by making some calculations and it actually belongs to the timezone 'America/New_York' (it can be any time zone say calculated_time_zone).
How can I update the timezone of this timestamp without actually changing the time for the output, which is:
2021-07-26 00:00:00-04:00
When I use AS TIME ZONE 'America/New_York', I get the output as 2021-07-26 04:00:00+00 which is not what I want.
In python, it can be easily done with a <datetime_object>.replace(tzinfo=new_time_zone). I am looking for the same thing in postgresql.
Any manner in which it can be done with django database functions would be helpful too (I wasn't able to find any)
It's important to understand that the timestamptz type does not store a timezone! It simply stores the UTC timestamp. Which means that your timestamp isn't "in the wrong timezone" because it isn't in any timezone. It is, however, displayed to you in a certain timezone, defined by your session settings.
If you need to shift your timestamp by four hours, as the case may be, then you can add an interval of 4 hours to it. But if you need to "reinterpret" the UTC timestamp to be a timestamp of a different timezone (which means you probably did something wrong when building the timestamp initially), then you can do so by switching to a timezone-naive timestamp and back again:
SELECT timestamp AT TIME ZONE 'UTC' AT TIME ZONE 'America/New_York'
But don't be surprised if this new timestamp is displayed in the same timezone as before, because that's defined by your session settings, not by the timestamp itself.
Strip off the timezone, then add back the new timezone you want. This can be done with:
(myfield::timestamp || '+00:00')::timestamptz
I'm gonna to save date and time in my PostgreSQL database and fetch and show it to the user in an appropriate format.
Suppose that application users are located in Iran and use Jalali date/time system. In the first half of year, Iran time is UTC+04:30, but in the rest of year, it is UTC+03:30. As a matter of fact daylight saving time is used in Iran.
IMPORTANT: Sometimes we make a decision and change database server location from Iran to Europe or elsewhere.
Now I have some questions:
What data type is more convenient to save date with time? TIMESTAMP (TIMESTAMP WITHOUT TIMEZONE) or TIMESTAMPZ (TIMESTAMP WITH TIME ZONE)?
What data type is more convenient to save time only? TIME or TIME WITH TIME ZONE?
What data type is more convenient to save date only? DATE (Gregorian) String (Jalali) or a custom data type?
How can I set TIMEZONE for our database once? I wanna set TIMEZONE for database one time and after that all queries over TIMESTAMPZ columns will be saved and fetched within that TIMEZONE and also it considered daylight saving time ?
What SQL query is the best when I wan to save current date and time ?
a- INSERT INTO test(d) VALUES(now());
b- INSERT INTO test(d) VALUES(now() at time zone 'utc');
c- INSERT INTO test(d) VALUES(now() at time zone 'Asia/Tehran');
d- INSERT INTO test(d) VALUES(current_timestamp);
e- INSERT INTO test(d) VALUES(now() at time zone 'utc');
f- INSERT INTO test(d) VALUES(now() at time zone 'Asia/Tehran');
Is number 5-c and 5-f considered daylight saving times when save it or not?
Is number 5-a and 5-d saved in 'Asia/Tehran' when database has 'Asia/Tehran' time zone ?
When I wan to query from database, what options is the best in my situation ?
a- SELECT d FROM test;
b- SELECT d at time zone 'utc' FROM test;
c- SELECT d at time zone 'Asia/Tehran' FROM test;
Is number 7-c considered daylight saving times?
Is number 7-a considered 'Asia/Tehran' time zone and daylight saving time when database has 'Asia/Tehran' time zone?
If I'd better use the timestamp to save date/time, then I have to add daylight saving time to it and convert it to Jalali date time and show it to the user, and vice versa.
https://www.postgresql.org/docs/current/static/datatype-datetime.html
For timestamp with time zone, the internally stored value is always in
UTC (Universal Coordinated Time, traditionally known as Greenwich Mean
Time, GMT). An input value that has an explicit time zone specified is
converted to UTC using the appropriate offset for that time zone. If
no time zone is stated in the input string, then it is assumed to be
in the time zone indicated by the system's TimeZone parameter, and is
converted to UTC using the offset for the timezone zone.
When a timestamp with time zone value is output, it is always
converted from UTC to the current timezone zone, and displayed as
local time in that zone. To see the time in another time zone, either
change timezone or use the AT TIME ZONE construct
timestamptz
timestamptz and select date only
timestamptz and select time only
database saves tz aware time stamps in UTC, no matter your locale or settings. stamps are always converted to UTC adjusting it by TimeZone parameter. Same deconvert happens on displaying data every time. TimeZone for server is only default value, used if client does not specify any. database TimeZone overrides the one in postgresql.conf for specified database, but still, client settings will override the database ones.
Let's frame this problem as a scheduling application. Users specify a minimum time in their timezone ("8am" in "America/New_York") after which they're willing to take appointments. (we also have a similar constraint on the maximum time)
Given an incoming appointment request (say, for 2016-09-07 03:00:00 in UTC time), we need to determine which users can take the appointment.
How can we determine whether a UTC timestamp is between two timezone-adjusted times?
My approach has been to convert the incoming timestamp to the user's timezone, extract the time portion and compare it to the time constraints:
SELECT * FROM users u WHERE
('2016-09-07 03:00:00' AT TIME ZONE u.timezone)::time >= u.earliest_start_time AND
('2016-09-07 03:00:00' AT TIME ZONE u.timezone)::time <= u.latest_stop_time
(earliest_start_time and latest_stop_time are of type time without time zone; timezone is an Olson tz string)
There's all sorts of odd behavior as timezone offsets causes days to roll over and I'm just generally very confused and hazy about this.
To make the code below more readable, suppose that the placeholder $1 holds a value of type timestamp with time zone. Then the following query will find all users that are available at that time:
SELECT * FROM users u WHERE
$1 BETWEEN
($1::date || ' ' || u.earliest_start_time)::timestamp without time zone AT TIME ZONE u.timezone
AND
($1::date || ' ' || u.latest_stop_time)::timestamp without time zone AT TIME ZONE u.timezone;
Explanation: we take $1 which is a timestamp with time zone and we keep only the date part by type casting it into a date. Then we take earliest_start_time which is a time and combine it with the previous date and we get full time stamp which doesn't have time zone information (yet). Then we specify the time zone to convert that into a timestamp with time zone. We do the same for latest_stop_time. So now that we have found those 2 timestamp with time zone values, we can just compare them with $1 which has the same type.
This will have problem with ambiguous timestamps though (i.e. in the transition from DST to standard time), but you cannot avoid that. If you only use it for typical businesses hours, it might be ok.
We have an application that fetches data from a source and that source present the data with a timestamp in UTC. When our application saves that data to Postgres, it stores that timestamp in a timestamp column without time zone. The default on postgres in our shop is set to our local time, Mountain Time. So that means, I think, that postgres assumes that timestamp is mountain time. How can I query that column so that my result set thinks it's UTC and not the local time zone?
More cleary stated, I need to perform some offsets on that timestamp (moving it to, say EST) and so the math of doing that is different if the resultset thinks it's UTC than my local time
The Answer by Kouber Saparev is mostly correct, though incorrect about storing a time zone.
Wrong data type in Postgres
a timestamp in UTC. When our application saves that data to Postgres, it stores that timestamp in a timestamp column without time zone.
As noted in his Answer, you are using the wrong data type in your Postgres database. When tracking moments, you must use a column of type TIMESTAMP WITH TIME ZONE. When supplying an input during an insert or update, any accompanying info about time zone or offset-from-UTC is used to adjust into UTC. The accompanying zone/offset is then discarded. If you need to remember the original zone/offset, you will need to define a second column and store that info there yourself.
The other type in Postgres, and the SQL standard, is TIMESTAMP WITHOUT TIME ZONE. This type purposely lacks any concept of time zone or offset-from-UTC. So this type cannot represent moments, cannot store points on the timeline. It stores values that represent potential moments along a range of about 26-27 hours, the range of various time zones around the globe. Use this type only when you mean a date with time-of-day everywhere or anywhere, but not specifically somewhere. Also used when you mean appointments far enough out in the future that we run the risk of politicians changing the offset used in any of the time zones we care about.
Always specify time zone
default on postgres in our shop is set to our local time, Mountain Time
Never depend on the current default time zone of your host OS, the database server, or your tools such as the Java Virtual Machine. Always specify the desired/expected time zone in your code.
Tip: Generally best to work in UTC for data storage, data exchange, and most of your business logic. Adjust from UTC to a time zone only for presentation to the user or where business rules require.
As explained above, Postgres always stores date-time values either in UTC or with no zone/offset at all. Beware: Tools used between you and Postgres may apply a time zone to the UTC value retrieved from the database. While well-intentioned, this anti-feature creates the illusion that the time zone was stored when in fact only UTC was stored in TIMESTAMP WITH TIME ZONE or no zone/offset at all in TIMESTAMP WITHOUT TIME ZONE.
Be aware that any zone information accompanying input to a column of TIMESTAMP WITHOUT TIME ZONE is simply ignored, the date and time-of-day taken as-is and stored.
I need to perform some offsets on that timestamp (moving it to, say EST)
Generally best to use your database just for storage, query, and retrieval of data. For massaging the data like adjusting time zone, do such work in your application. For example, in Java use the industry-leading java.time classes, in .NET the Noda Time project (a port of the predecessor of java.time, the Joda-Time project).
Example code in Java using JDBC 4.2 or later.
LocalDateTime
For a value in a column of TIMESTAMP WITHOUT TIME ZONE we use the corresponding type in Java, LocalDateTime, lacking any concept of time zone or offset-from-UTC.
LocalDateTime ldt = myResultSet.getObject( … , LocalDateTime.class ) ; // Retrieve value from database.
String output = ldt.toString() ; // Generate text representing this date-with-time value in standard ISO 8601 format.
2018-01-23T01:23:45.123
If you know for certain that this date and time was meant for UTC but was incorrectly stored without any zone/offset info, you can apply a zone or offset to repair the damage.
OffsetDateTime odt = ldt.atOffset( ZoneOffset.UTC ); // Apply an offset-from-UTC to a `LocalDateTime` lacking such information. Determines a moment.
OffsetDateTime
For a value in a column of TIMESTAMP WITH TIME ZONE we use the corresponding type in Java, OffsetDateTime (or Instant), representing a moment in UTC.
OffsetDateTime odt = myResultSet.getObject( … , OffsetDateTime.class ) ; // Retrieve value from database.
String output = odt.toString() ; // Generate text representing this date-with-time value in standard ISO 8601 format. A `Z` on the end indicates UTC, pronounced “Zulu”.
2018-01-23T01:23:45.123Z
ZonedDateTime
To see that OffsetDateTime value set in UTC through the lens of the wall-clock time used by the people of regions within the mid-west of North America, specify a time zone such as America/Edmonton or America/Denver.
Specify a proper time zone name in the format of continent/region, such as America/Montreal, Africa/Casablanca, or Pacific/Auckland. Never use the 2-4 letter abbreviation such as EST or IST as they are not true time zones, not standardized, and not even unique(!).
ZoneId z = ZoneId.of( "America/Denver" ) ;
ZonedDateTime zdt = odt.atZoneSameInstant( z ) ;
See this code run live at IdeOne.com. We see the same moment but with a different wall-clock time.
2018-01-22T18:23:45.123-07:00[America/Denver]
Beware of tools & middleware injecting a time zone
Unfortunately, many tools and middleware will volunteer to apply some default time zone to a moment retrieved from the database. While well-intentioned, this creates the illusion of the zone having been a part of the stored data when in fact the time zone was added after storage, upon retrieval. This anti-feature creates much confusion. I wish all the tools were clear and truthful by reporting the moment in UTC, as it was stored.
If you use Java, with JDBC 4.2 and later, you can exchange java.time (JSR 310) (tutorial) objects with the database and avoid this time zone injection.
There are two data types handling timestamps in PostgreSQL - timestamp, and timestamptz (timestamp with time zone). The latter stores the time zone along with the timestamp itself.
If you are using just a timestamp without time zone, then there is no way for the result set to think whether the timestamp is UTC or not. It is just a timestamp. It is up to the client application to interpret it and give it some time zone meaning.
On the contrary, if you use timestamptz, then PostgreSQL knows the time zone of that timestamp, and then it can calculate time zone offsets properly for you.
db=# select now();
now
-------------------------------
2014-12-04 19:27:06.044703+02
(1 row)
db=# select timezone('est', now());
timezone
----------------------------
2014-12-04 12:27:06.044703
(1 row)
So, back on the problem posed. You need to make sure that first the data is imported properly and then - when needed, it is returned and displayed properly to the end user. You have two options:
Continue using timestamp
In that case both the writing app and the reading app need to know that all the timestamps in the database are UTC and calculate offsets accordingly.
Switch to timestamptz
Then the only thing that the apps need to know is their own time zone, they just have to declare it after connecting to PostgreSQL and leave the rest to the database.
For example, let's connect as a writing app and declare our time zone as UTC.
db=# create table x (data timestamptz);
CREATE TABLE
db=# set timezone='utc';
SET
db=# insert into x values (now());
INSERT 0 1
db=# select * from x;
data
-------------------------------
2014-12-04 20:02:08.692329+00
(1 row)
Now, let's say a reading app connects and is in the EST time zone.
db=# set timezone='est';
SET
db=# select * from x;
data
-------------------------------
2014-12-04 15:02:08.692329-05
(1 row)
Changing the client time zone setting changes the way all the timestamps are returned, but that's the case only if you use timestamptz - timestamp with time zone. If you cannot switch to this data type, then the application will have to take care of all this magic.