PostgreSQL: Fast check whether all elements of an LTREE [] <# LTREE[] - postgresql

Question

I have the following tables (simplified):
CREATE TABLE groups
( id PRIMARY KEY,
path ltree,
...
);
CREATE TABLE items
( id bigserial,
path ltree,
...
PRIMARY KEY (id, path)
);
For each item, there is a list of groups that the item belongs too. A group is represented by its full path. There may be up to 10M items, each item belongs to about 20 groups.
I need to design the following query. Given (a) a "parent" group and (b) a list of up to 10 additional groups, find those immediate descendants of the "parent" group that have at least one item in their subtree that is contained in each of the groups in the search criteria.
For example, given the parent group "NorthAmerica.USA" and additional groups ["CandyLovers.ChocolateLovers", "Athletes.Footballers"], then "NorthAmerica.USA.CA" is a result if it there exists an item like "George" that is in the groups like ["NorthAmerica.USA.CA.LosAngeles", "Athletes.Footballers", "CandyLovers.ChocolateLovers.ChocolateDonutLovers"]
I tried a few different ways to write queries, and they scale very poorly: take minutes to return result on a set of 1M items and 3-4 paths in the search criteria. For example:
EXPLAIN ANALYZE
SELECT *
FROM groups
WHERE path ~ CAST ('1.2.22' || '.*{1}' AS lquery)
AND EXISTS
(SELECT 1
FROM
(SELECT array_agg(DISTINCT leaf_paths_sans_result_path.path) AS paths_of_a_match,
max(path_count) AS path_count
FROM items,
(SELECT path,
count(*) OVER() AS path_count
FROM (
VALUES (groups.path) , ('1.3'),('1.4')) t (path)) leaf_paths_sans_result_path
WHERE 1 = 1
AND items.path <# leaf_paths_sans_result_path.path
GROUP BY id) items_by_id
WHERE cardinality(paths_of_a_match) = path_count );
Results in the following:
Index Scan using idx_groups__path__gist on groups (cost=0.28..37013.74 rows=38 width=469) (actual time=11.735..322285.421 rows=950 loops=1)
Index Cond: (path ~ '1.2.22.*{1}'::lquery)
Filter: (SubPlan 1)
Rows Removed by Filter: 3
SubPlan 1
-> Subquery Scan on items_by_id (cost=0.55..1809359.86 rows=3752 width=0) (actual time=338.162..338.162 rows=1 loops=953)
-> GroupAggregate (cost=0.55..1809322.34 rows=3752 width=65) (actual time=338.159..338.159 rows=1 loops=953)
Group Key: ibt.id
Filter: (cardinality(array_agg(DISTINCT "*VALUES*".column1)) >= max(3))
Rows Removed by Filter: 7845
-> Nested Loop (cost=0.55..1809228.54 rows=3752 width=65) (actual time=0.044..307.087 rows=20423 loops=953)
Join Filter: (ibt.path <# "*VALUES*".column1)
Rows Removed by Join Filter: 651228
-> Index Scan using idx_items__id on items (cost=0.55..1752954.06 rows=1250543 width=193) (actual time=0.007..110.517 rows=223884 loops=953)
-> Materialize (cost=0.00..0.05 rows=3 width=32) (actual time=0.000..0.000 rows=3 loops=213361141)
-> Values Scan on "*VALUES*" (cost=0.00..0.04 rows=3 width=32) (actual time=0.002..0.003 rows=3 loops=953)
Planning time: 3.151 ms
Execution time: 322286.404 ms
(18 rows)
I can change the data model as needed in order to optimize for this query. I am running PostgreSQL v9.5
Many thanks! Sorry for a messy question.

Answer 1

Looks like you're using the ltree module? The following query avoids the intermediate array_agg arrays:
select *
from items i
join groups g
on i.groups = g.id
where g.path ~ '1.2.22.*' and
(
i.path ~ '*.1.3.*' or
i.path ~ '*.1.4.*'
)
group by
g.id
having count(distinct
case
when i.path ~ '*.1.3.*' then 1
when i.path ~ '*.1.4.*' then 2
end) = 2
The count constructs asserts that both conditions are met, not just two rows that match the same pattern.