Why
Select timestamptz '2022-03-19T00:00Z' as test1
give result : 2022-03-19 03:00:00+03 ??
You're indicating Postgresql a timestamptz at UTC time.
There, it's returning a strictly equivalent time, except it is displayed in a different time zone (UTC-03).
Why does Postgresql not return the exact same thing you indicated ?
When parsing a timestamp with time zone, Postgresql internally stores it in UTC time, using the provided TZ info (here 'Z, understood as Z00, which means UTC time) to determine the offset to apply to convert it (here, you're giving a perfectly fine UTC timestamp).
When displaying data however, Postgresql relies on the internal TimeZone info defined in the postgresql.conf file to choose the time zone to use.
In your case, the local time zone info might be set to the place you live (or where your server lies), which is in UTC-03. Just type show time zone to be sure of that.
If you want to retrieve the data at timezone UTC, you have two or three options:
the simplest one is to precise the timezone info you want in the select statement:
select timestamptz '2022-03-19T00:00Z' at time zone 'UTC' as test1;
which gives you a timestamp without timezone though;
or you can set the time zone info for your local session:
set timezone='UTC';
or if you want to set the time zone info to UTC permanently, the timezone field of the postgresql.conf file has to be changed.
This file is for instance in /opt/homebrew/var/postgres if you're running a local instance of postgresql on Mac (like me), installed from brew. On Linux, I believe it might be in /etc/postgresql.
You need to restart your postgresql instance after the change:
brew services restart postgresql on Mac for example.
Related
I want to get the current timezone name.
What I already achieved is to get the utc_offset / the timezone abbreviation via:
SELECT * FROM pg_timezone_names WHERE abbrev = current_setting('TIMEZONE')
This gives me all Continent / Capital combinations for this timezone but not the exact timezone. For example I get:
Europe/Amsterdam
Europe/Berlin
The server is in Berlin and I want to get the timezone name of the server.
The problem I have with CET that it is always UTC+01:00 and does not account for DST iirc.
I don't think this is possible using PostgreSQL alone in the most general case. When you install PostgreSQL, you pick a time zone. I'm pretty sure the default is to use the operating system's timezone. That will usually be reflected in postgresql.conf as the value of the parameter "timezone". But the value ends up as "localtime". You can see this setting with the SQL statement.
show timezone;
But if you change the timezone in postgresql.conf to something like "Europe/Berlin", then show timezone; will return that value instead of "localtime".
So I think your solution will involve setting "timezone" in postgresql.conf to an explicit value rather than the default "localtime".
It seems to work fine in Postgresql 9.5:
SELECT current_setting('TIMEZONE');
This may or may not help you address your problem, OP, but to get the timezone of the current server relative to UTC (UT1, technically), do:
SELECT EXTRACT(TIMEZONE FROM now())/3600.0;
The above works by extracting the UT1-relative offset in minutes, and then converting it to hours using the factor of 3600 secs/hour.
Example:
SET SESSION timezone TO 'Asia/Kabul';
SELECT EXTRACT(TIMEZONE FROM now())/3600.0;
-- output: 4.5 (as of the writing of this post)
(docs).
You can access the timezone by the following script:
SELECT * FROM pg_timezone_names WHERE name = current_setting('TIMEZONE');
current_setting('TIMEZONE') will give you Continent / Capital information of settings
pg_timezone_names The view pg_timezone_names provides a list of time zone names that are
recognized by SET TIMEZONE, along with their associated abbreviations, UTC offsets, and
daylight-savings status.
name column in a view (pg_timezone_names) is time zone name.
output will be :
name- Europe/Berlin,
abbrev - CET,
utc_offset- 01:00:00,
is_dst- false
See this answer: Source
If timezone is not specified in postgresql.conf or as a server command-line option, the server attempts to use the value of the TZ environment variable as the default time zone. If TZ is not defined or is not any of the time zone names known to PostgreSQL, the server attempts to determine the operating system's default time zone by checking the behavior of the C library function localtime(). The default time zone is selected as the closest match among PostgreSQL's known time zones. (These rules are also used to choose the default value of log_timezone, if not specified.) source
This means that if you do not define a timezone, the server attempts to determine the operating system's default time zone by checking the behavior of the C library function localtime().
If timezone is not specified in postgresql.conf or as a server command-line option, the server attempts to use the value of the TZ environment variable as the default time zone.
It seems to have the System's timezone to be set is possible indeed.
Get the OS local time zone from the shell. In psql:
=> \! date +%Z
PostgreSQL 9.3 / postgresql-9.3-1100-jdbc41.jar
I have a table with a column of type timestamp without time zone, this generates my Object with the applicable java.util.Timestamp property.
What I'm seeing, during insert, is jOOQ's binding process converting a java.util.Timestamp into a date with local timezone offset.
eg for a unix timestamp 1421109419 (13 Jan 2015 00:36:59 GMT) the property is set with new Timestamp(1421109419 * 1000).
from the jOOQ logger I see:
2015-01-13 14:14:31,482 DEBUG [http-bio-8002-exec-4] org.jooq.tools.LoggerListener#debug:255 - -> with bind values : insert into "foo"."bar" ("start_date") values (timestamp '2015-01-13 13:36:59.0') returning "foo"."bar"."id"
2015-01-13 14:14:31,483 TRACE [http-bio-8002-exec-4] org.jooq.impl.DefaultBinding#trace:179 - Binding variable 3 : 2015-01-13 13:36:59.0 (class java.sql.Timestamp)
and sure enough in the record is the value "2015-01-13 13:36:59".
The software is running on a machine in NZDT which explains the +13 offset.
Given the time is being supplied in a TimeZone agnostic container (Timestamp) I would have expected that to be honoured when creating the insert statement.
How can I have jOOQ create timestamps NOT in local time?
Unfortunately you have a few things working against you:
The PostgreSQL JDBC driver sets the timezone to your JVM timezone in the Postgres session. So even if your Database Server is running in UTC a TIMESTAMP field will be inserted using the time zone of your JVM. When you insert or query data the database server will always use the JVM time zone.
You are using TIMESTAMP instead of TIMESTAMPTZ. The description of these types do not reflect their actually usage. TIMESTAMPTZ actually means time zone agnostic. Whatever value you insert it will be adjusted to UTC using the session timezone.
Because of these two issues, if you have two different JVMs -- one using Los Angeles time and the other using New York time -- whenever you write a TIMESTAMP with one JVM it will be a different "UTC time" in the other JVM. TIMESTAMP takes the adjusted value and just uses it as given. If you change your TIMESTAMP columns to be TIMESTAMPTZ then the same time in both JVMs will always be the same UTC time.
If you look at the Postgres JDBC Driver's ConnectionFactoryImpl#openConnectionImp you can see where it sets your local JVM's time zone as the time zone for the database server's session zone.
So the only sane way to deal with this is to only ever use TIMESTAMPTZ instead of TIMESTAMP. Here's some more information on this:
PostgreSQL/JDBC and TIMESTAMP vs. TIMESTAMPTZ
http://justatheory.com/computers/databases/postgresql/use-timestamptz.html
The following (very nasty) code works for me:
eventsRecord.setCreatedOn(new Timestamp(System.currentTimeMillis()
- TimeZone.getDefault().getOffset(new Date().getTime())));
Alas jOOQ simply uses the local timezone when saving into PostgreSQL "timestamp without timezone" or MySQL "datetime" fields. The source code evidence for this travesty is here, it does not specify the timezone nor have any facility for the user to override this functionality and specify a timezone. This renders usage of this very basic datatype from jOOQ completely useless, with a multitude of clients all with different timezones writing data to the same field without recording their timezone nor normalizing the data to UTC.
JDBC provides an extra three-argument setTimestamp where the user can specify what timezone is desired (UTC is basically the only value that makes sense). However jOOQ "abstracts" away from JDBC and does not offer this facility.
We have an application that fetches data from a source and that source present the data with a timestamp in UTC. When our application saves that data to Postgres, it stores that timestamp in a timestamp column without time zone. The default on postgres in our shop is set to our local time, Mountain Time. So that means, I think, that postgres assumes that timestamp is mountain time. How can I query that column so that my result set thinks it's UTC and not the local time zone?
More cleary stated, I need to perform some offsets on that timestamp (moving it to, say EST) and so the math of doing that is different if the resultset thinks it's UTC than my local time
The Answer by Kouber Saparev is mostly correct, though incorrect about storing a time zone.
Wrong data type in Postgres
a timestamp in UTC. When our application saves that data to Postgres, it stores that timestamp in a timestamp column without time zone.
As noted in his Answer, you are using the wrong data type in your Postgres database. When tracking moments, you must use a column of type TIMESTAMP WITH TIME ZONE. When supplying an input during an insert or update, any accompanying info about time zone or offset-from-UTC is used to adjust into UTC. The accompanying zone/offset is then discarded. If you need to remember the original zone/offset, you will need to define a second column and store that info there yourself.
The other type in Postgres, and the SQL standard, is TIMESTAMP WITHOUT TIME ZONE. This type purposely lacks any concept of time zone or offset-from-UTC. So this type cannot represent moments, cannot store points on the timeline. It stores values that represent potential moments along a range of about 26-27 hours, the range of various time zones around the globe. Use this type only when you mean a date with time-of-day everywhere or anywhere, but not specifically somewhere. Also used when you mean appointments far enough out in the future that we run the risk of politicians changing the offset used in any of the time zones we care about.
Always specify time zone
default on postgres in our shop is set to our local time, Mountain Time
Never depend on the current default time zone of your host OS, the database server, or your tools such as the Java Virtual Machine. Always specify the desired/expected time zone in your code.
Tip: Generally best to work in UTC for data storage, data exchange, and most of your business logic. Adjust from UTC to a time zone only for presentation to the user or where business rules require.
As explained above, Postgres always stores date-time values either in UTC or with no zone/offset at all. Beware: Tools used between you and Postgres may apply a time zone to the UTC value retrieved from the database. While well-intentioned, this anti-feature creates the illusion that the time zone was stored when in fact only UTC was stored in TIMESTAMP WITH TIME ZONE or no zone/offset at all in TIMESTAMP WITHOUT TIME ZONE.
Be aware that any zone information accompanying input to a column of TIMESTAMP WITHOUT TIME ZONE is simply ignored, the date and time-of-day taken as-is and stored.
I need to perform some offsets on that timestamp (moving it to, say EST)
Generally best to use your database just for storage, query, and retrieval of data. For massaging the data like adjusting time zone, do such work in your application. For example, in Java use the industry-leading java.time classes, in .NET the Noda Time project (a port of the predecessor of java.time, the Joda-Time project).
Example code in Java using JDBC 4.2 or later.
LocalDateTime
For a value in a column of TIMESTAMP WITHOUT TIME ZONE we use the corresponding type in Java, LocalDateTime, lacking any concept of time zone or offset-from-UTC.
LocalDateTime ldt = myResultSet.getObject( … , LocalDateTime.class ) ; // Retrieve value from database.
String output = ldt.toString() ; // Generate text representing this date-with-time value in standard ISO 8601 format.
2018-01-23T01:23:45.123
If you know for certain that this date and time was meant for UTC but was incorrectly stored without any zone/offset info, you can apply a zone or offset to repair the damage.
OffsetDateTime odt = ldt.atOffset( ZoneOffset.UTC ); // Apply an offset-from-UTC to a `LocalDateTime` lacking such information. Determines a moment.
OffsetDateTime
For a value in a column of TIMESTAMP WITH TIME ZONE we use the corresponding type in Java, OffsetDateTime (or Instant), representing a moment in UTC.
OffsetDateTime odt = myResultSet.getObject( … , OffsetDateTime.class ) ; // Retrieve value from database.
String output = odt.toString() ; // Generate text representing this date-with-time value in standard ISO 8601 format. A `Z` on the end indicates UTC, pronounced “Zulu”.
2018-01-23T01:23:45.123Z
ZonedDateTime
To see that OffsetDateTime value set in UTC through the lens of the wall-clock time used by the people of regions within the mid-west of North America, specify a time zone such as America/Edmonton or America/Denver.
Specify a proper time zone name in the format of continent/region, such as America/Montreal, Africa/Casablanca, or Pacific/Auckland. Never use the 2-4 letter abbreviation such as EST or IST as they are not true time zones, not standardized, and not even unique(!).
ZoneId z = ZoneId.of( "America/Denver" ) ;
ZonedDateTime zdt = odt.atZoneSameInstant( z ) ;
See this code run live at IdeOne.com. We see the same moment but with a different wall-clock time.
2018-01-22T18:23:45.123-07:00[America/Denver]
Beware of tools & middleware injecting a time zone
Unfortunately, many tools and middleware will volunteer to apply some default time zone to a moment retrieved from the database. While well-intentioned, this creates the illusion of the zone having been a part of the stored data when in fact the time zone was added after storage, upon retrieval. This anti-feature creates much confusion. I wish all the tools were clear and truthful by reporting the moment in UTC, as it was stored.
If you use Java, with JDBC 4.2 and later, you can exchange java.time (JSR 310) (tutorial) objects with the database and avoid this time zone injection.
There are two data types handling timestamps in PostgreSQL - timestamp, and timestamptz (timestamp with time zone). The latter stores the time zone along with the timestamp itself.
If you are using just a timestamp without time zone, then there is no way for the result set to think whether the timestamp is UTC or not. It is just a timestamp. It is up to the client application to interpret it and give it some time zone meaning.
On the contrary, if you use timestamptz, then PostgreSQL knows the time zone of that timestamp, and then it can calculate time zone offsets properly for you.
db=# select now();
now
-------------------------------
2014-12-04 19:27:06.044703+02
(1 row)
db=# select timezone('est', now());
timezone
----------------------------
2014-12-04 12:27:06.044703
(1 row)
So, back on the problem posed. You need to make sure that first the data is imported properly and then - when needed, it is returned and displayed properly to the end user. You have two options:
Continue using timestamp
In that case both the writing app and the reading app need to know that all the timestamps in the database are UTC and calculate offsets accordingly.
Switch to timestamptz
Then the only thing that the apps need to know is their own time zone, they just have to declare it after connecting to PostgreSQL and leave the rest to the database.
For example, let's connect as a writing app and declare our time zone as UTC.
db=# create table x (data timestamptz);
CREATE TABLE
db=# set timezone='utc';
SET
db=# insert into x values (now());
INSERT 0 1
db=# select * from x;
data
-------------------------------
2014-12-04 20:02:08.692329+00
(1 row)
Now, let's say a reading app connects and is in the EST time zone.
db=# set timezone='est';
SET
db=# select * from x;
data
-------------------------------
2014-12-04 15:02:08.692329-05
(1 row)
Changing the client time zone setting changes the way all the timestamps are returned, but that's the case only if you use timestamptz - timestamp with time zone. If you cannot switch to this data type, then the application will have to take care of all this magic.
I have column 'event_date_time' in my 'events' table with type 'timestamp with timezone'. My python flask application is saving date like '2014-08-30 02:17:02+00:00' but postgres automatically converts it to '2014-08-30 07:17:02+05'. It converts the timestamp to my local timezone i-e Pakistan. I want to save it without converting.
I have tried
set timezone='UTC'
and it does change timezone to 'UTC' but pgadmin3 is still saving the converted time.
I am using MAC OS and Postgresql 9.3.
The reason pgadmin is displaying hours +5 is because your system timezone is set to this.
When you save a "timestamp with time zone" value at GMT + or - any value, the system offsets whatever timezone your input was to GMT (or UTC), so that when you go to retrieve it, you can specify the timezone you want it displayed in.
For example let's establish a current time for say... New York.
select now()::timestamp with time zone at time zone 'America/New_York';
At the time of asking it returned '2014-08-23 08:50:57.136817'. 8:50 Saturday morning, or 8:51 if you're being pedantic.
Now if we take that same time and display it in GMT we will see a different result:
select '2014-08-23 08:50:57.136817 America/New_York'::timestamp with time zone at time zone 'GMT';
Now have a new time of '2014-08-23 12:50:57.136817'... 5 hours into the "future"!
Finally let's get the original timestamp and display it in what I believe is the Pakistan time zone (PKT) and see what it shows
select '2014-08-23 08:50:57.136817 America/New_York'::timestamp with time zone at time zone 'PKT';
The result? '2014-08-23 17:50:57.136817' further into the future still!
Again I must stress the reason it can do this is because it is always converting the input time offset to UTC or GMT. Postgres processes all of its "timestamp with time zone" data types in this way. It is designed to avoid time zone problems such as daylight savings and so on.
Your issue appears to be that python is inserting the time at an offset of +00, and if this was supposed to be a local time then you will be 5 hours off as far as postgres is concerned. Without knowing exactly what queries python is making, I would assume you may want to look at that to make sure it is giving you the correct time, presumably set timezone='PKT' should be a fix. Either way, when you are viewing timestamp with time zone using a browser such as pgadmin, the timestamp is being converted to your local timezone and this is why you see +5.
Alternatively if you do wish to see those times at +00 then you must specify that you want this in your SELECT queries.
I think one of the two scourges of IT are timestamps and time zones (the other being character encoding) where one keeps stumbling upon again and again...
In that regard I currently have a problem related to different timestamps within a Java application storing into a PostgreSQL database.
For keeping things simple, assume having the following table:
CREATE TABLE ts_test
(
id integer NOT NULL,
utc timestamp without time zone,
local timestamp with time zone,
CONSTRAINT pk PRIMARY KEY (id)
)
So, I have to store a UTC time stamp and a local one, which in my case is central Europe summer time, so currently UTC+2.
Further assume having 2 entries in the table, which output on the psql console as follows (the database runs in UTC):
# select id, utc, local, local-utc as diff from ts_test;
id | utc | local | diff
----+---------------------+------------------------+----------
1 | 2012-06-27 12:00:00 | 2012-06-27 12:00:00+00 | 00:00:00
2 | 2012-06-27 12:00:00 | 2012-06-27 14:00:00+00 | 02:00:00
(2 rows)
Now, several questions arise:
What exactly does the output in the local column mean?
How does the system know the timezone, I inserted the value in?
How can I see the real raw value (e.g. milliseconds) stored?
I would have assumed, that the first line's local "12:00:00+00" means, that it's 12:00 in UTC, which again is 14:00 in CEST. But it seems (and so I was told by our database admin), that the 2nd line's local "14:00:00+00" is the correct value for 14:00 CEST - which is supported by the diff of 2 hours.
But to produce the 2nd line via sql insert, I have to write
insert into ts_test (id, utc, local) values (2, '2012-06-27 12:00:00', '2012-06-27 16:00:00+02');
which again does not support the predication.
So, to sum up this long question - can anyone enlighten me on how this whole thing works in detail, what the output is supposed to mean and how one should write local time stamps correctly into the database?
According to the output of the local column, the time zone of your SQL session is set to UTC or GMT and not the time zone where you live in. Presumably this is what you mean by: the database runs in UTC. This is the root of the problem, but let's try to elaborate.
The db itself, as a data repository, doesn't have a timezone, but each SQL session has its own timezone.
When they're requested by a SQL session, the values of timestamp without time zone are not rotated to the session's time zone and not presented with a time offset, whereas the values for timestamp with time zone are rotated to the session's timezone and presented with the time offset for this time zone. That's the difference between both.
The time zone is never stored in any of the datatypes, because when reading the value, all that matters is the time zone of the SQL session that is requesting this value.
Setting your SQL time zone to UTC is not a good idea because it contradicts this other part of your question:
So, I have to store a UTC time stamp and a local one, which in my case
is central Europe summer time, so currently UTC+2
Let the SQL session know your real time zone, and it will start to work as intended. If you don't, timestamp with time zone is essentially useless.
Also note that storing the same time in utc timestamp without time zone and local timestamp with time zone doesn't make sense, because you could always get the utc with:
SELECT local AT TIME ZONE 'UTC' FROM ts_test WHERE...
EDIT: answers to questions in the comments:
Q: you are saying that if my timezone is set to my local time in the
session, then I should see for example ...14:00:00+02 in local for a
utc value of ...12:00:00
Yes.
Q: And when writing something into the local field from my application,
it matters which timezone is set there?
Exactly.
Q: How does one set this in JDBS-session?
I don't know JDBC but at the SQL level, that would be for example:
SET timezone='Europe/Berlin';
Normally it's automatically set from the environment but it can be forced at various levels including postgresql.conf. Setting it explicitly in the session will override anything else.
Q: can I see the raw value of the timestamp somehow, to make sure it's
not just a representation problem when displaying
I'm not aware of how to do that except with pageinspect which operates at the lower level.