Group rows by the largest grouping columns - postgresql

Question

I have a time like this
Item
title
image
hash
A
cat
cat_x.jpg
x-123
B
cat
cat_y.jpg
y-123
C
dog1
dog.jpg
xyx
D
dog2
dog.jpg
xyx
I can group items by the same title, image or hash column and have an array with groupped items.
For example, A and B could be grouped by title
SELECT
title,
array_agg(item) AS items
FROM products
GROUP BY 1
title
items
cat
{A,B}
dog1
{C}
dog2
{D}
but this is not good for C and D which should be grouped by image (dog.jpg)
SELECT
image,
array_agg(item) AS items
FROM products
GROUP BY 1
image
items
cat_x.jpg
{A}
cat_y.jpg
{B}
dog.jpg
{C,D}
and the same for "hash" column.
So if two rows have the same title, or image or hash (or all) should be merged into the same group. If two groups have at least one item in common, they need to merged into the same group.
If X and Y are grouped by title, and Y and Z are grouped by image. Then X,Y,Z should be into the same array.
In the end, I don't want two groups containing the same item.
Each Item should belong to only one group.
An idea could be:
Group items by title
Then group the result by image
Then group the result by hash
Products table contains about 100,000 records.
My progress so far
At the moment I tried with this but for 100,000 records it timeouts. Even if I am using temp tables + indexes.
https://www.db-fiddle.com/f/v4RsA4jRdYz3JHmph3Mykj/1
create temp table titles AS (
SELECT
title,
array_agg(item) AS items
FROM products
GROUP BY 1
);
create temp table images AS (
SELECT
image,
array_agg(item) AS items
FROM products
GROUP BY 1
);
create temp table hashes AS (
SELECT
hash,
array_agg(item) AS items
FROM products
GROUP BY 1
);
CREATE INDEX idx_items_titles on titles USING GIN ("items");
CREATE INDEX idx_items_images on images USING GIN ("items");
CREATE INDEX idx_items_hashes on hashes USING GIN ("items");
SELECT
ARRAY( SELECT DISTINCT e FROM unnest(items || title_items || hash_items) AS a(e) )
FROM images,
LATERAL (
SELECT
titles.items AS title_items
FROM titles
WHERE titles.items && images.items
) x,
LATERAL (
SELECT
hashes.items AS hash_items
FROM hashes
WHERE hashes.items && images.items
) y
GROUP BY 1
Desired output
Groups
{B,A}
{D,C}

Answer 1

Those lateral joins and the array columns look very expensive.
I would instead use a different approach where you add an extra column for the representative element of the respective group, like in a disjoint-set data structure.
A simple definition of a canonical representation for each row might be the smallest id of any other row that shares either the same image, title or hash. Using a window function, we can easily (and efficiently) compute that:
SELECT
array_agg(item) AS items,
array_agg(DISTINCT title) AS titles,
array_agg(DISTINCT image) AS images,
array_agg(DISTINCT hash) AS hashes
FROM (
SELECT *, LEAST (
MIN(item) OVER (PARTITION BY title),
MIN(item) OVER (PARTITION BY image),
MIN(item) OVER (PARTITION BY hash)
) AS group_id
FROM products
) AS tmp
GROUP BY tmp.group_id;
(online demo on your sample data)
Unfortunately, it is also wrong, since it doesn't handle transitive equivalence. You can check this example, where A and D contain no shared values, but they do both share some values (their titles) with other rows that have the same value for another column (their images).
To fix this issue, one will need to actually run the union-find algorithm, repeatedly merging the groups until you end up with the desired equivalence classes. This is not exactly trivial, but can be done with a recursive common table expression:
WITH RECURSIVE eqiv AS (
SELECT id, MIN(id) OVER (PARTITION BY title) AS group_id
FROM products
UNION
SELECT id, MIN(id) OVER (PARTITION BY image) AS group_id
FROM products
UNION
SELECT id, MIN(id) OVER (PARTITION BY hash) AS group_id
FROM products
),
rel AS (
SELECT id, id AS group_id
FROM products
UNION
SELECT eqiv.id, rel.group_id
FROM rel
JOIN eqiv ON (rel.id = eqiv.group_id)
)
SELECT
array_agg(id) AS items,
array_agg(DISTINCT title) AS titles,
array_agg(DISTINCT image) AS images,
array_agg(DISTINCT hash) AS hashes
FROM products
GROUP BY (SELECT MIN(group_id) FROM rel WHERE rel.id = products.id);
(online demo)