How does postgres query planner work on partitioned tables? - postgresql

Question

We have a postgres 11 database with tables having a quite big number of rows, so we use postgres declerative partitioning to ensure query performance.
Today while wirting a database function I noticed some strange behaviour of the postgres query planner:
In this particular case we have two tables track.track and sensor.location.
The function shall return all locations for a given track.
The relation between track.track and sensor.location is given by a user_vehicle_id and a time range.
sensor.location is partitioned monthly by range using the column time
A query for this problem could look like this:
WITH single_track AS (
SELECT
start_time, stop_time, user_vehicle_id
FROM
track.track
WHERE
id = 1350000744800)
SELECT *
FROM
sensor.location as l, single_track as t
WHERE
l.time >= t.start_time AND
l.time <= t.stop_time AND
l.user_vehicle_id = t.user_vehicle_id
I would expect that the query planer only looks at those partitions of location which match the given time frame from start_time to stop_time.
Instead it performs a Bitmap Heap/Index scan on all partitions:
Nested Loop (cost=8.59..9308018.00 rows=722021 width=106) (actual time=1.796..2.296 rows=1025 loops=1)
CTE single_track
-> Index Scan using track_pkey on track (cost=0.42..8.44 rows=1 width=24) (actual time=0.023..0.024 rows=1 loops=1)
Index Cond: (id = '1350000744800'::bigint)
-> CTE Scan on single_track t (cost=0.00..0.02 rows=1 width=24) (actual time=0.027..0.029 rows=1 loops=1)
-> Append (cost=0.15..9286171.84 rows=2183770 width=82) (actual time=1.750..1.998 rows=1025 loops=1)
-> Index Scan using location_p2011_01_pkey on location_p2011_01 l (cost=0.15..8.83 rows=1 width=136) (never executed)
Index Cond: (("time" >= t.start_time) AND ("time" <= t.stop_time) AND (user_vehicle_id = t.user_vehicle_id))
-> Seq Scan on location_p2011_02 l_1 (cost=0.00..7.71 rows=1 width=82) (never executed)
Filter: (("time" >= t.start_time) AND ("time" <= t.stop_time) AND (t.user_vehicle_id = user_vehicle_id))
-> Bitmap Heap Scan on location_p2011_03 l_2 (cost=643.94..3370.03 rows=2087 width=114) (never executed)
Recheck Cond: (("time" >= t.start_time) AND ("time" <= t.stop_time) AND (user_vehicle_id = t.user_vehicle_id))
...
-> Index Scan using location_p2020_10_pkey on location_p2020_10 l_117 (cost=0.15..8.83 rows=1 width=136) (never executed)
Index Cond: (("time" >= t.start_time) AND ("time" <= t.stop_time) AND (user_vehicle_id = t.user_vehicle_id))
-> Index Scan using location_p2020_11_pkey on location_p2020_11 l_118 (cost=0.15..8.83 rows=1 width=136) (never executed)
Index Cond: (("time" >= t.start_time) AND ("time" <= t.stop_time) AND (user_vehicle_id = t.user_vehicle_id))
-> Index Scan using location_p2020_12_pkey on location_p2020_12 l_119 (cost=0.15..8.83 rows=1 width=136) (never executed)
Index Cond: (("time" >= t.start_time) AND ("time" <= t.stop_time) AND (user_vehicle_id = t.user_vehicle_id))
Planning Time: 11.046 ms
Execution Time: 4.144 ms
While playing around I discoverd, that using the same query but passing the times explicitly:
EXPLAIN ANALYSE
WITH single_track AS (
SELECT
start_time,
stop_time,
user_vehicle_id
FROM
track.track
WHERE
id = 1350000744800)
SELECT *
FROM
sensor.location as l, single_track as t
WHERE
l.time >= '2016-04-12 18:04:59' AND
l.time <= '2016-04-12 18:22:49' AND
l.user_vehicle_id = t.user_vehicle_id
produces the expected behaviour:
Nested Loop (cost=9.00..2111.73 rows=141 width=102) (actual time=0.085..2.408 rows=1025 loops=1)
CTE single_track
-> Index Scan using track_pkey on track (cost=0.42..8.44 rows=1 width=24) (actual time=0.017..0.018 rows=1 loops=1)
Index Cond: (id = '1350000744800'::bigint)
-> CTE Scan on single_track t (cost=0.00..0.02 rows=1 width=24) (actual time=0.021..0.022 rows=1 loops=1)
-> Append (cost=0.56..2099.99 rows=328 width=78) (actual time=0.060..2.081 rows=1025 loops=1)
-> Index Scan using location_p2016_04_pkey on location_p2016_04 l (cost=0.56..2098.35 rows=328 width=78) (actual time=0.058..1.994 rows=1025 loops=1)
Index Cond: (("time" >= '2016-04-12 18:04:59'::timestamp without time zone) AND ("time" <= '2016-04-12 18:22:49'::timestamp without time zone) AND (user_vehicle_id = t.user_vehicle_id))
Planning Time: 4.709 ms
Execution Time: 2.494 ms
Can anyone explain this behaviour and help me how to overcome this issue?

Answer 1

This seems to be too complicated for the PostgreSQL executor.
I'd suggest trying
SELECT *
FROM
sensor.location as l, single_track as t
WHERE
l.time >= (SELECT start_time FROM track.track WHERE id = 1350000744800) AND
l.time <= (SELECT stop_time FROM track.track WHERE id = 1350000744800) AND
l.user_vehicle_id = (SELECT user_vehicle_id FROM track.track WHERE id = 1350000744800)
Then PostgreSQL knows at least that there will only be a single value.
If that still doesn't work, split the query in two parts:
First, get the values from track.track.
Then construct a query using the results and execute that.

Answer 2

I also tried this, which may be close to what Laurenz Albe suggested. I wasn't able to confirm that this causes the right behavior, as EXPLAIN ANALYSE does not show the query plan of a psql funtction.
CREATE OR REPLACE FUNCTION location_from_track_id(
_track_id bigint)
RETURNS SETOF sensor.location
LANGUAGE 'plpgsql'
AS
$BODY$
DECLARE
_user_vehicle_id bigint;
_start_time timestamp without time zone;
_stop_time timestamp without time zone;
BEGIN
SELECT
user_vehicle_id,
start_time,
stop_time
INTO
_user_vehicle_id,
_start_time,
_stop_time
FROM
track.track
WHERE id=_track_id;
RETURN QUERY
SELECT *
FROM sensor.location
WHERE
time BETWEEN _start_time AND _stop_time AND
user_vehicle_id = _user_vehicle_id;
END;
$BODY$;