Today I encounter a strange postgres behavious. Let me explain:
Here is my table I will work on.
=># \d planning_time_slot
Table "public.planning_time_slot"
Column | Type | Collation | Nullable | Default
-------------+---------------------------+-----------+----------+------------------------------------------------
id | integer | | not null | nextval('planning_time_slot_id_seq'::regclass)
planning_id | integer | | not null |
day | character varying(255) | | not null |
start_time | time(0) without time zone | | not null |
end_time | time(0) without time zone | | not null |
day_id | integer | | not null | 0
Indexes:
"planning_time_slot_pkey" PRIMARY KEY, btree (id)
"idx_a9e3f3493d865311" btree (planning_id)
Foreign-key constraints:
"fk_a9e3f3493d865311" FOREIGN KEY (planning_id) REFERENCES planning(id)
what i want to do is something like:
select * from planning_time_slot where start_time > (CURRENT_TIME AT TIME ZONE 'Europe/Paris');
But it seems like postgres is comparing time before the time zone conversion.
Here is my tests:
=># select * from planning_time_slot where start_time > (CURRENT_TIME AT TIME ZONE 'Europe/Paris');
id | planning_id | day | start_time | end_time | day_id
-----+-------------+-----+------------+----------+--------
157 | 6 | su | 16:00:00 | 16:30:00 | 0
(1 row)
=># select (CURRENT_TIME AT TIME ZONE 'Europe/Paris');
timezone
--------------------
16:35:48.591002+02
(1 row)
When I try with a lot of entries it appears that the comparaison is done between start_time and CURRENT_TIME without the time zone cast.
For your information I also tried :
select * from planning_time_slot where start_time > timezone('Europe/Paris', CURRENT_TIME);
It has the exact same result.
I also tried to change the column type to time(0) with time zone. It makes the exact same result.
One last important point. I really need to set timezone I want, because later on I will change it dynamically depending on other stuffs. So it will not be 'Europe/Paris' everytime.
Does anyone have a clue or a hint please ?
psql (PostgreSQL) 11.2 (Debian 11.2-1.pgdg90+1)
(CURRENT_TIME AT TIME ZONE 'Europe/Paris') is, for example, 17:52:17.872082+02. But internally it is 15:52:17.872082+00. Both time and timetz (time with time zone) are all stored as UTC, the only difference is timetz is stored with a time zone. Changing the time zone does not change what point in time it represents.
So when you compare it with a time...
# select '17:00:00'::time < '17:52:17+02'::timetz;
?column?
----------
f
That is really...
# select '17:00:00'::time < '15:52:17'::time;
?column?
----------
f
Casting a timetz to a time will lop off the time zone.
test=# select (CURRENT_TIME AT TIME ZONE 'Europe/Paris')::time;
timezone
-----------------
17:55:57.099863
(1 row)
test=# select '17:00:00' < (CURRENT_TIME AT TIME ZONE 'Europe/Paris')::time;
?column?
----------
t
Note that this sort of comparison only makes sense if you want to store the notion that a thing happens at 17:00 according to the clock on the wall. For example, if you had a mobile phone game where an event starts "at 17:00" meaning 17:00 where the user is. This is referred to as a "floating time zone".
Assuming day is "day of week", I suggest storing it as an integer. It's easier to compare and localize.
Instead of separate start and end times, consider a single timerange. Then you can use range operators.
I think you have deeper problems.
You have a day, a start time and an end time, but no notion of time zone. So this will mean something different depending on the time zone of the observer.
I think you should add a tz column that stores which time zone that information is in. Then you can get the start time like this:
WHERE (day + start_time) AT TIME ZONE tz > current_timestamp
Related
How can I convert this to work in PostgreSQL?
TO_CHAR(CAST(FROM_TZ(CAST(columnname AS TIMESTAMP), 'GMT') AT TIME ZONE 'US/Eastern' AS DATE),'MM/DD/YY HH:MI AM') AS dt
testdb=# select TO_CHAR(CAST('2020-02-28T18:43' AS TIMESTAMP) AT TIME ZONE 'UTC' AT TIME ZONE 'US/Eastern','MM/DD/YY HH:MI AM') as dt;
dt
-------------------
02/28/20 01:43 PM
(1 row)
To make it clear what's going on, we'll start with the cast to TIMESTAMP, show that adding the first AT TIME ZONE makes it a tz-aware timestamp, and then how the 2nd does the timezone conversion.
testdb=# select CAST('2020-02-28T18:43' AS TIMESTAMP),
testdb-# CAST('2020-02-28T18:43' AS TIMESTAMP) AT TIME ZONE 'GMT',
testdb-# CAST('2020-02-28T18:43' AS TIMESTAMP) AT TIME ZONE 'GMT' AT TIME ZONE 'US/Eastern';
timestamp | timezone | timezone
---------------------+------------------------+---------------------
2020-02-28 18:43:00 | 2020-02-28 18:43:00+00 | 2020-02-28 13:43:00
(1 row)
See the timezone conversion docs for more details.
I have two columns one is a time the other a timestamp
ALTER TABLE public.tour
ADD COLUMN reprocess_toupdate timestamp without time zone DEFAULT NOW();
ALTER TABLE public.tour
ADD COLUMN reprocess_updated time without time zone DEFAULT NOW();
when I execute:
select reprocess_toupdate, reprocess_updated
from tour
where reprocess_toupdate::date > reprocess_updated::date;
I get an error:
ERROR: cannot cast type time without time zone to date
without ::date, I get this error:
ERROR: operator does not exist: timestamp without time zone > time without time zone
That is because a TIME column does not have a date component. It's range of values is 00:00:00 - 24:00:00. see Documentation Section 8.5 Date/Time Types. Since it does not have a date component you cannot cast it as date. The proper solution would to change the type to "timestamp without time zone". If that is not possible then compare just the times or to "reattach" the date then compare:
with dateset as
(select '2019-06-02 13:00:00'::time without time zone tm, (now() - interval '1 day')::timestamp without time zone dt)
select tm, dt, date_trunc('day', dt)+tm redt from dateset
Works here:
create temporary table so (id serial primary key, ts timestamp default now());
insert into so (ts) values (now());
select * from so where ts::date < now();
Output:
+------+----------------------------+
| id | ts |
|------+----------------------------|
| 1 | 2019-07-01 10:16:43.093662 |
+------+----------------------------+
Question: How is query 1 "semantically" different than the query 2?
Background:
To extract data from the table in a db which is at my localtime zone (AT TIME ZONE 'America/New_York').
The table has data for various time zones such as the 'America/Los_Angeles', America/North_Dakota/New_Salem and such time zones.
(Postgres stores the table data for various timezones in my local timezone)
So, everytime I retrieve data for a different location other than my localtime, I convert it to its relevant timezone for evaluation purposes..
Query 1:
test_db=# select count(id) from click_tb where date::date AT TIME ZONE 'America/Los_Angeles' = '2017-05-22'::date AT TIME ZONE 'America/Los_Angeles';
count
-------
1001
(1 row)
Query 2:
test_db=# select count(id) from click_tb where (date AT TIME ZONE 'America/Los_Angeles')::date = '2017-05-22'::date;
count
-------
5
(1 row)
Table structure:
test_db=# /d+ click_tb
Table "public.click_tb"
Column | Type | Modifiers | Storage | Stats target | Description
-----------------------------------+--------------------------+-------------------------------------------------------------+----------+--------------+-------------
id | integer | not null default nextval('click_tb_id_seq'::regclass) | plain | |
date | timestamp with time zone | | plain | |
Indexes:
"click_tb_id" UNIQUE CONSTRAINT, btree (id)
"click_tb_date_index" btree (date)
The query 1 and query 2 do not produce consistent results.
As per my tests, the below query 3, semantically addresses my requirement.
Your critical feedback is welcome.
Query 3:
test_db=# select count(id) from click_tb where ((date AT TIME ZONE 'America/Los_Angeles')::timestamp with time zone)::date = '2017-05-22'::date;
Do not convert the timestamp field. Instead, do a range query. Since your data is already using a timestamp with time zone type, just set the time zone of your query accordingly.
set TimeZone = 'America/Los_Angeles';
select count(id) from click_tb
where date >= '2017-01-02'
and date < '2017-01-03';
Note how this uses a half open interval of the dates (at start of day in the set time zone). If you want to compute the second date from your first date, then:
set TimeZone = 'America/Los_Angeles';
select count(id) from click_tb
where date >= '2017-01-02'
and date < (timestamp with time zone '2017-01-02' + interval '1 day');
This properly handles daylight saving time, and sargability.
I am trying to implement a function that calculates the spare time out of stored activities start and end times. I implemented my database on PostgreSQL 9.5.3. This is how the activity table looks like
activity_id | user_id | activity_title | starts_at | ends_at
(serial) | (integer) | (text) | (timestamp without time zone) |(timestamp without time zone)
---------------------------------------------------------------------------------------------------------------------------
1 | 1 | Go to school | 2016-06-12 08:00:00 | 2016-06-12 14:00:00
2 | 1 | Visit my uncle | 2016-06-12 16:00:00 | 2016-06-12 17:30:00
3 | 1 | Go shopping | 2016-06-12 18:00:00 | 2016-06-12 21:15:00
4 | 1 | Go to Library | 2016-06-13 10:00:00 | 2016-06-13 12:00:00
5 | 1 | Install some programs on my laptop | 2016-06-13 18:00:00 | 2016-06-13 19:00:00
Actual table definition of my real table:
CREATE TABLE public.activity (
activity_id serial,
user_id integer NOT NULL,
activity_title text,
starts_at timestamp without time zone NOT NULL,
start_tz text NOT NULL,
ends_at timestamp without time zone NOT NULL,
end_tz text NOT NULL,
recurrence text NOT NULL DEFAULT 'none'::text,
lat numeric NOT NULL,
lon numeric NOT NULL,
CONSTRAINT pk_activity PRIMARY KEY (activity_id),
CONSTRAINT fk_user_id FOREIGN KEY (user_id)
REFERENCES public.users (user_id) MATCH SIMPLE
ON UPDATE NO ACTION ON DELETE NO ACTION
)
I want to calculate every day spare time for this user using PL/pgSQL function that takes (user_id INTEGER, range_start TIMESTAMP, range_end TIMESTAMP) as parameters. I want the output of this SQL statement:
SELECT * from calculate_spare_time(1, '2016-06-12', '2016-06-13');
to be like this:
spare_time_id | user_id | starts_at | ends_at
(serial) | (integer) | (timestamp without time zone) |(timestamp without time zone)
----------------------------------------------------------------------------------------
1 | 1 | 2016-06-12 00:00:00 | 2016-06-12 08:00:00
2 | 1 | 2016-06-12 12:00:00 | 2016-06-12 16:00:00
3 | 1 | 2016-06-12 17:30:00 | 2016-06-12 18:00:00
4 | 1 | 2016-06-12 21:15:00 | 2016-06-13 00:00:00
5 | 1 | 2016-06-13 00:00:00 | 2016-06-13 10:00:00
6 | 1 | 2016-06-13 12:00:00 | 2016-06-13 18:00:00
7 | 1 | 2016-06-13 19:00:00 | 2016-06-14 00:00:00
I have the idea of subtracting the end time of one activity from the start time of the next activity happening on the same date, but I am stuck with implementing that with PL/pgSQL especially on how to deal with 2 rows in the same time.
To simplify things, I suggest to create a view - or better yet: a MATERIALZED VIEW showing gaps in the activities per user:
CREATE MATERIALIZED VIEW mv_gap AS
SELECT user_id, tsrange(a, z) AS gap
FROM (
SELECT user_id, ends_at AS a
, lead(starts_at) OVER (PARTITION BY user_id ORDER BY starts_at) AS z
FROM activity
) sub
WHERE z > a; -- weed out simple overlaps and the dangling "gap" till infinity
Note the range type tsrange.
ATTENTION: You mentioned possible overlaps, which complicate things. If one time range of a single user can be included in another, you need to do more! Merge time ranges to identify earliest start and latest end per block.
Remember to refresh the MV when needed.
Then your function can simply be:
CREATE OR REPLACE FUNCTION f_freetime(_user_id int, _from timestamp, _to timestamp)
RETURNS TABLE (rn int, gap tsrange) AS
$func$
SELECT row_number() OVER (ORDER BY g.gap)::int AS rn
, g.gap * tsrange(_from, _to) AS gap
FROM mv_gap g
WHERE g.user_id = _user_id
AND g.gap && tsrange(_from, _to)
ORDER BY g.gap;
$func$ LANGUAGE sql STABLE;
Call:
SELECT * FROM f_freetime(1, '2016-06-12 0:0', '2016-06-13 0:0');
Note the range operators * and &&.
Also note that I use a simple SQL function, after the problem has been simplified enough. If you need to add more, you might want to switch back to plpgsql and use RETURN QUERY ...
Or just use the query without function wrapper.
Performance
If you have many rows per user, to optimize query times, add an SP-GiST index (one reason to use a MV):
CREATE INDEX activity_gap_spgist_idx on mv_gap USING spgist (gap);
In addition to an index on (user_id).
Details in this related answer:
Perform this hours of operation query in PostgreSQL
I have a table with epoch values (one per minute, the epoch itself is in milliseconds) and temperatures.
select * from outdoor_temperature order by time desc;
time | value
---------------+-------
1423385340000 | 31.6
1423385280000 | 31.6
1423385220000 | 31.7
1423385160000 | 31.7
1423385100000 | 31.7
1423385040000 | 31.8
1423384980000 | 31.8
1423384920000 | 31.8
1423384860000 | 31.8
[...]
I want to get the highest single value in a given day, which I'm doing like this:
SELECT *
FROM
outdoor_temperature
WHERE
value = (
SELECT max(value)
FROM outdoor_temperature
WHERE
((timestamp with time zone 'epoch' + (time::float/1000) * interval '1 second') at time zone 'Australia/Sydney')::date
= '2015-02-05' at time zone 'Australia/Sydney'
)
AND
((timestamp with time zone 'epoch' + (time::float/1000) * interval '1 second') at time zone 'Australia/Sydney')::date
= '2015-02-05' at time zone 'Australia/Sydney'
ORDER BY time DESC LIMIT 1;
On my Linode, running CentOS 5 and Postgres 8.4, it returns perfectly (I get a single value, within that date, with the maximum temperature). On my MacBook Pro with Postgres 9.3.5, however, the exact same query against the exact same data doesn't return anything. I started simplifying everything to work out what was going wrong, and got to here:
SELECT max(value)
FROM outdoor_temperature
WHERE
((timestamp with time zone 'epoch' + (time::float/1000) * interval '1 second') at time zone 'Australia/Sydney')::date
= '2015-02-05' at time zone 'Australia/Sydney';
max
-----
(1 row)
It's empty, and yet returning one row?!
My questions are:
Firstly, why is that query working against Postgres 8.4 and doing something different on 9.3.5?
Secondly, is there a much simpler way to achieve what I'm trying to do? I feel like there should be but if so I've not managed to work it out. This ultimately needs to work on Postgres 8.4.
I'm not really sure why you're getting no results - you seem to simply miss data for this day.
But you really should use another query for selecting a date, as your query would not be able to use an index.
You should select like this:
select max(value) from outdoor_temperature where
time>=extract(
epoch from
'2015-02-05'::timestamp at time zone 'Australia/Sydney'
)
and
time<extract(
epoch from
('2015-02-05'::timestamp+'1 day'::interval) at time zone 'Australia/Sydney'
)
;
This is much simpler and this way your database would be able to use an index on time, which should be a primary key (with automatic index).