I really need to store local date and time (aircraft wheels up date and time) and the utc offset so I can convert to utc to calculate intervals (flight duration). People travel on local time and you need utc for cross time zone computations. I had hoped to use timestamptz but it absolutely does not work for this purpose. It converts everything to a function of the postgres time zone. In my case this is yyyy-dd-mm hh:mi:ss-07.
However, I investigated timetz just to cover all the bases. It stores exactly what I need. It preserves the local time while providing the offset to utc. Except that now I need two columns instead of one to store the information.
My questions are:
Why do the timestamptz and the timetz functions give different results?
Is there a way to make the timestamptz include the local time zone offset rather than the system time zone offset?
Here are the queries that illustrate the difference:
select cast('2015-05-01 11:25:00 america/caracas' as timestamptz)
-- 2015-05-01 08:55:00-07
;
select cast('2015-05-01 11:25:00 america/caracas' as timetz)
-- 11:25:00-04:30
;
I personally find the wording timestamp with time zone confusing when trying to understand PostgreSQL's timestamptz, because it doesn't store any time zone. According to the docs:
All timezone-aware dates and times are stored internally in UTC. They are converted to local time in the zone specified by the TimeZone configuration parameter before being displayed to the client.
Notice, on that page, that the storage characteristics and limits of timestamp and timestamptz are identical.
In my head, to keep things straight, I translate timestamptz to "timestamp in the real world", and plain timestamp to "you probably didn't mean to use this type" (because, so far, I've only found myself needing to store timestamps that are related to the real world.)
So:
Why do the timestamptz and the timetz functions give different results?
It appears to be because PostgreSQL didn't feel they were allowed to make timetz work like timestamptz does:
The type time with time zone is defined by the SQL standard, but the definition exhibits properties which lead to questionable usefulness.
My guess is that some of these "properties" are the ones that they don't like, and you do.
And:
Is there a way to make the timestamptz include the local time zone offset rather than the system time zone offset?
There's no offset being stored for timestamptz values. They're just real-world timestamps. So the way to store the local time zone is the way you've already thought of: store it separately.
create table my_table (
happened timestamptz,
local_time_zone varchar
);
select happened at time zone 'UTC', happened at time zone local_time_zone
from my_table;
Related
If I inherit a Postgresql database filled with TIMESTAMP type, and feel the need to convert them all to TIMESTAMPTZ type, is this a simple process requiring only:
ALTER TABLE mytable ALTER COLUMN the_time_stamp TYPE TIMESTAMPTZ;
I have such a database, and all the timestamps within it, created in a timezone different to mine (and the server's) behave such that they seem to be stored internally as TIMESTAMPTZ anyway. That is, I have two clients (one write, one read, neither of which is in UTC zone), and when I read, the timestamps written in zone1 are correct when read in zone2, implying (to me) they have been converted and stored in UTC format between the write and read. It makes me wonder if:
Are all timestamps stored in identical format in the database, and the only differences occur in the way input/output is handled, according to the timezone(s) of the connected client(s)?
This seems like such a clear and simple thing to state if it were true, but I cannot find it stated clearly and/or simply...
Internally, both timestamp and timestamp with time zone are stored in the same fashion: an 8-byte integer that is the offset from 2000-01-01 00:00:00 in microseconds.
The difference is the semantics: while timestamp with time zone is stored as offset from midnight 2000-01-01 in UTC, no time zone conversion is made for timestamp.
That means that the table has to be rewritten if you change the data type, because the point of reference changes. If the timezone parameter is set to UTC, the values won't change, but there is no optimization in PostgreSQL that avoids the table rewrite in this case.
We have a table that is filled with data from a legacy report of another system. The columns of that table reflect the same structure of the report.
Here are a abbreviated structure of the table:
CREATE TABLE IF NOT EXISTS LEGACY_TABLE (
REPORT_DATE DATE NOT NULL,
EVENT_ID BIGINT PRIMARY KEY NOT NULL,
START_HOUR TIMESTAMP WITHOUT TIME ZONE,
END_HOUR TIME WITHOUT TIME ZONE,
EXPECTED_HOUR TIME WITHOUT TIME ZONE
);
We are refactoring this table to deal with different time zones of different clients. The new structure would be something like:
CREATE TABLE IF NOT EXISTS LEGACY_TABLE (
REPORT_DATE DATE NOT NULL,
EVENT_ID BIGINT PRIMARY KEY NOT NULL,
START_HOUR TIMESTAMP WITH TIME ZONE,
END_HOUR TIME WITH TIME ZONE,
EXPECTED_HOUR TIME WITH TIME ZONE
);
These hour fields represents a specific point in time during the day represented by the REPORT_DATE column. What I mean by that is that every TIME column represents a moment during the day specified in REPORT_DATE.
Some other points to consider:
We don't know why the START_HOUR is in TIMESTAMP format in the report we receive from the legacy system. But we import the data the way it comes to us.
The fields in the report are formatted according to the timezone of the client, so to refactor this table we need to combine the timezone of the client (we have this info) to properly insert the timestamps/times in UTC.
But now to the problem. The value of these columns are used to compute another values multiple times in our system, something like the following:
START_HOUR - END_HOUR (the result of this operation is currently being casted to TIME WITHOUT TIME ZONE)
START_HOUR < END_HOUR
START_HOUR + EXPECTED_HOUR
EXPECTED_HOUR - END_HOUR
EXPECTED_HOUR < '05:00'
After some research I found that is not recommended to use the type TIME WITH TIME ZONE (Postgres time with time zone equality) and now I'm a bit confused about what is the best way to refactor this table to deal with different time zones and handle the different column operations that we need to.
Besides that, I already know that is safe to subtract two columns of type TIMESTAMP WITH TIME ZONE. This subtraction operation is taking into account DST changes (Subtracting two columns of type timestamp with time zone) but how about the others? And the one subtracting a TIME from a TIMESTAMP?.
And about the table refactoring, should we use TIME WITH TIME ZONE anyways? Should we continue using TIME WITHOUT TIME ZONE? Or is better to forget the type TIME altogether and combine the DATE with the TIME and change the columns to TIMESTAMP WITH TIME ZONE?
I think these questions are related because the new column types we choose to use, will define how we operate with the columns.
You asserted that:
every TIME column represents a moment during the day specified in REPORT_DATE.
So you never cross the a dateline within the same row. I suggest to save 1x date 3x time and the time zone (as text or FK column):
CREATE TABLE legacy_table (
event_id bigint PRIMARY KEY NOT NULL
, report_date date NOT NULL
, start_hour time
, end_hour time
, expected_hour time
, tz text -- time zone
);
Like you already found, timetz (time with time zone) should generally be avoided. It cannot deal with DST rules properly (daylight saving time).
So basically what you already had. Just drop the date component from start_hour, that's dead freight. Cast timestamp to time to cut off the date. Like: (timestamp '2018-03-25 1:00:00')::time
tz can be any string accepted by the AT TIME ZONE construct, but to deal with different time zones reliably, it's best to use time zone names exclusively. Any name you find in the system catalog pg_timezone_names.
To optimize storage, you could collect allowed time zone names in a small lookup table and replace tz text with tz_id int REFERENCES my_tz_table.
Two example rows with and without DST:
INSERT INTO legacy_table VALUES
(1, '2018-03-25', '1:00', '3:00', '2:00', 'Europe/Vienna') -- sadly, with DST
, (2, '2018-03-25', '1:00', '3:00', '2:00', 'Europe/Moscow'); -- Russians got rid of DST
For representation purposes or calculations you can do things like:
SELECT (report_date + start_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS start_utc
, (report_date + end_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS end_utc
, (report_date + expected_hour) AT TIME ZONE tz AT TIME ZONE 'UTC' AS expected_utc
-- START_HOUR - END_HOUR
, (report_date + start_hour) AT TIME ZONE tz
- (report_date + end_hour) AT TIME ZONE tz AS start_minus_end
FROM legacy_table;
You might create one or more views to readily display strings as needed. The table is for storing the information you need.
Note the parentheses! Else the operator + would bind before AT TIME ZONE due to operator precedence.
And behold the results:
db<>fiddle here
Since the time is manipulated in Vienna (like any place where silly DST rules apply), you get "surprising" results.
Related:
Accounting for DST in Postgres, when selecting scheduled items
Ignoring time zones altogether in Rails and PostgreSQL
I have a log table in a PostgreSQL database with an event column of type timestamp without time zone.
Now I have a bash script, which creates a CSV file from the log database:
...
psql .. -c "COPY (SELECT event, ... FROM logtable order by event desc) TO STDOUT WITH CSV" logdb > log.csv
...
This is executed on the cloud server on which the DB is hosted and therefore, the timestamp strings in log.csv are in local time of the timezone of the server.
However, I like to have the timestamp strings to represent the time of my own time zone. So I shall be able to let psql transform the timestamp -> string to a given timezone. How can I achieve this?
First of all, you should use timestamptz instead of timestamp whenever working with multiple times zones. Would avoid the problem completely.
Details:
Ignoring timezones altogether in Rails and PostgreSQL
You can use the AT TIME ZONE construct like #NuLo suggests, it may even work, but not exactly as described.
AT TIME ZONE converts the type timestamp (timestamp without time zone) to timestamptz (timestamp with time zone) and vice versa. The text representation of a timestamptz value depends on the current setting of the time zone in the session in which you run the command. These two timestamptz values are 100 % identical (denote the same point in time):
'2015-09-02 15:55:00+02'::timestamptz
'2015-09-02 14:55:00+01'::timestamptz
But the text representation is not. The display is for different time zones. If you take this string literal and feed it to a timestamp type, the time zone part is just ignored and you end up with different values. Hence, if you run your COPY statement in a session with the same time zone setting as your original timestamp values are for, the suggested operation happens to work.
The clean way, however, is to produce correct timestamp values to begin with by applying AT TIME ZONE twice:
SELECT event AT TIME ZONE 'my_target_tz' AT TIME ZONE 'my_source_tz', ...
FROM logtable
ORDER BY event desc;
'my_target_tz' is "your own time zone" and 'my_source_tz' the time zone of the of the cloud server in the example. To make sure that DST is respected use time zone names, not time zone abbreviations. The documentation:
A time zone abbreviation, for example PST. Such a specification merely
defines a particular offset from UTC, in contrast to full time zone names
which can imply a set of daylight savings transition-date rules as well.
Related:
Accounting for DST in Postgres, when selecting scheduled items
Time zone names with identical properties yield different result when applied to timestamp
Or, much better yet, use timestamptz everywhere and it works correctly automatically.
I have a column added_at of type timestamp without time zone. I want it's default value to be the current date-time but without time zone. The function now() returns a timezone as well.
How do I solve that problem?
SELECT now()::timestamp;
The cast converts the timestamptz returned by now() to the corresponding timestamp in your time zone - defined by the timezone setting of the session. That's also how the standard SQL function LOCALTIMESTAMP is implemented in Postgres.
If you don't operate in multiple time zones, that works just fine. Else switch to timestamptz for added_at. The difference?
Ignoring time zones altogether in Rails and PostgreSQL
BTW, this does exactly the same, just more noisy and expensive:
SELECT now() AT TIME ZONE current_setting('timezone');
Well you can do something like:
SELECT now() AT TIME ZONE current_setting('TimeZone');
SELECT now() AT TIME ZONE 'Europe/Paris';
SELECT now() AT TIME ZONE 'UTC';
Not sure how that makes any sense for a column "added_at". You almost always want an absolute timestamp (timestamp with time zone) not a floating one.
Edit responding to points below:
Yes, should use timestamp with time zone (absolute time) unless you have a good reason not to.
The client timezone is given by SHOW TimeZone or current_setting(...) as shown above.
Do take some time to skim the manuals - they cover all this quite well.
"Current Date/Time":
CURRENT_TIME and CURRENT_TIMESTAMP deliver values with time zone; LOCALTIME and LOCALTIMESTAMP deliver values without time zone.
New, and Native Answer in 2020
In PostgreSQL, If you only want the current date-time by calling CURRENT_TIMESTAMP() without time zone, and fractional digits in the seconds field which come after the decimal point of the seconds field?
(Tested on PostgreSQL v12.4)
Then use this:
SELECT CURRENT_TIMESTAMP(0)::TIMESTAMP WITHOUT TIME ZONE;
If you define your column's data type as timestamp (not as timestamptz), then you can store the timestamp without time zone, in that case you don't neet to add TIMESTAMP WITHOUT TIME ZONE
Like this:
CREATE TABLE foo (created timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP(0))
In the above function, 0 is passed to get rid of the fractional digits in the seconds field.
If your application doesn't care about timezone, you can use SELECT LOCALTIMESTAMP for it.
Ex:
SELECT LOCALTIMESTAMP
-- Result: 2023-01-30 17:43:33.628952
Are timestamp values stored differently in PostgreSQL when the data type is WITH TIME ZONE versus WITHOUT TIME ZONE? Can the differences be illustrated with simple test cases?
The differences are covered at the PostgreSQL documentation for date/time types. Yes, the treatment of TIME or TIMESTAMP differs between one WITH TIME ZONE or WITHOUT TIME ZONE. It doesn't affect how the values are stored; it affects how they are interpreted.
The effects of time zones on these data types is covered specifically in the docs. The difference arises from what the system can reasonably know about the value:
With a time zone as part of the value, the value can be rendered as a local time in the client.
Without a time zone as part of the value, the obvious default time zone is UTC, so it is rendered for that time zone.
The behaviour differs depending on at least three factors:
The timezone setting in the client.
The data type (i.e. WITH TIME ZONE or WITHOUT TIME ZONE) of the value.
Whether the value is specified with a particular time zone.
Here are examples covering the combinations of those factors:
foo=> SET TIMEZONE TO 'Japan';
SET
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 00:00:00+09
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 06:00:00+09
(1 row)
foo=> SET TIMEZONE TO 'Australia/Melbourne';
SET
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 00:00:00+11
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP;
timestamp
---------------------
2011-01-01 00:00:00
(1 row)
foo=> SELECT '2011-01-01 00:00:00+03'::TIMESTAMP WITH TIME ZONE;
timestamptz
------------------------
2011-01-01 08:00:00+11
(1 row)
I try to explain it more understandably than the referred PostgreSQL documentation.
Neither TIMESTAMP variants store a time zone (or an offset), despite what the names suggest. The difference is in the interpretation of the stored data (and in the intended application), not in the storage format itself:
TIMESTAMP WITHOUT TIME ZONE stores local date-time (aka. wall calendar date and wall clock time). Its time zone is unspecified as far as PostgreSQL can tell (though your application may knows what it is). Hence, PostgreSQL does no time zone related conversion on input or output. If the value was entered into the database as '2011-07-01 06:30:30', then no mater in what time zone you display it later, it will still say year 2011, month 07, day 01, 06 hours, 30 minutes, and 30 seconds (in some format). Also, any offset or time zone you specify in the input is ignored by PostgreSQL, so '2011-07-01 06:30:30+00' and '2011-07-01 06:30:30+05' are the same as just '2011-07-01 06:30:30'.
For Java developers: it's analogous to java.time.LocalDateTime.
TIMESTAMP WITH TIME ZONE stores a point on the UTC time line. How it looks (how many hours, minutes, etc.) depends on your time zone, but it always refers to the same "physical" instant (like the moment of an actual physical event). The
input is internally converted to UTC, and that's how it's stored. For that, the offset of the input must be known, so when the input contains no explicit offset or time zone (like '2011-07-01 06:30:30') it's assumed to be in the current time zone of the PostgreSQL session, otherwise the explicitly specified offset or time zone is used (as in '2011-07-01 06:30:30+05'). The output is displayed converted to the current time zone of the PostgreSQL session.
For Java developers: It's analogous to java.time.Instant (with lower resolution though), but with JDBC and JPA 2.2 you are supposed to map it to java.time.OffsetDateTime (or to java.util.Date or java.sql.Timestamp of course).
Some say that both TIMESTAMP variations store UTC date-time. Kind of, but it's confusing to put it that way in my opinion. TIMESTAMP WITHOUT TIME ZONE is stored like a TIMESTAMP WITH TIME ZONE, which rendered with UTC time zone happens to give the same year, month, day, hours, minutes, seconds, and microseconds as they are in the local date-time. But it's not meant to represent the point on the time line that the UTC interpretation says, it's just the way the local date-time fields are encoded. (It's some cluster of dots on the time line, as the real time zone is not UTC; we don't know what it is.)
Here is an example that should help. If you have a timestamp with a timezone, you can convert that timestamp into any other timezone. If you haven't got a base timezone it won't be converted correctly.
SELECT now(),
now()::timestamp,
now() AT TIME ZONE 'CST',
now()::timestamp AT TIME ZONE 'CST'
Output:
-[ RECORD 1 ]---------------------------
now | 2018-09-15 17:01:36.399357+03
now | 2018-09-15 17:01:36.399357
timezone | 2018-09-15 08:01:36.399357
timezone | 2018-09-16 02:01:36.399357+03
Timestamptz vs Timestamp
The timestamptz field in Postgres is basically just the timestamp field where Postgres actually just stores the “normalised” UTC time, even if the timestamp given in the input string has a timezone.
If your input string is: 2018-08-28T12:30:00+05:30 , when this timestamp is stored in the database, it will be stored as 2018-08-28T07:00:00.
The advantage of this over the simple timestamp field is that your input to the database will be timezone independent, and will not be inaccurate when apps from different timezones insert timestamps, or when you move your database server location to a different timezone.
To quote from the docs:
For timestamp with time zone, the internally stored value is always in
UTC (Universal Coordinated Time, traditionally known as Greenwich Mean
Time, GMT). An input value that has an explicit time zone specified is
converted to UTC using the appropriate offset for that time zone. If
no time zone is stated in the input string, then it is assumed to be
in the time zone indicated by the system’s TimeZone parameter, and is
converted to UTC using the offset for the timezone zone. To give a
simple analogy, a timestamptz value represents an instant in time, the
same instant for anyone viewing it. But a timestamp value just
represents a particular orientation of a clock, which will represent
different instances of time based on your timezone.
For pretty much any use case, timestamptz is almost always a better choice. This choice is made easier with the fact that both timestamptz and timestamp take up the same 8 bytes of data.
source:
https://hasura.io/blog/postgres-date-time-data-types-on-graphql-fd926e86ee87/
The diffrences are shown in PostgreSQL official docs. Please refer the docs for deep digging.
In a nutshell TIMESTAMP WITHOUT TIME ZONE doesn't save any timezone related informations if you give date time with timezone info,it takes date & time only and ignores timezone
For example
When I save this 12:13, 11 June 2021 IST to PostgreSQL TIMESTAMP WITHOUT TIME ZONE will reject the timezone information and saves the date time 12:13,11 June 2021
But the the case of TIMESTAMP WITH TIME ZONE it saves the timezone info in UTC format.
For example
When I save this 12:13, 11 June 2021 IST to PostgreSQL TIMESTAMP WITH TIME ZONE type variable it will interpret this time to UTC value and
stored as shown in below 6:43,11 June 2021 UTC
NB : UTC + 5.30 is IST
During the time conversion time returned by TIMESTAMP WITH TIME ZONE will be stored in UTC format and we can convert it to the required timezone like IST or PST etc.
So the recommented timestamp type in PostgreSQL is TIMESTAMP WITH TIME ZONE or TIMESTAMPZ
Run the following to see diff in pgAdmin:
create table public.testts (tz timestamp with time zone, tnz timestamp without time zone);
insert into public.testts values(now(), now());
select * from public.testts;
If you have similar issues I had of timestamp precision in Angular / Typescript / Node API / PostgreSql environment, hope my complete answer and solution will help you out.