Create dependency statistics on not reference columns - postgresql

Question

Assume following tables:
CREATE TABLE main
(
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY
);
CREATE TABLE apple
(
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
main_id INTEGER NOT NULL REFERENCES main(id)
);
CREATE TABLE orange
(
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
main_id INTEGER NOT NULL REFERENCES main(id)
);
CREATE TABLE main_history
(
id INTEGER NOT NULL,
history_valid_from TIMESTAMPTZ NOT NULL,
history_valid_to TIMESTAMPTZ
);
CREATE TABLE apple_history
(
id INTEGER NOT NULL,
main_id INTEGER NOT NULL REFERENCES main(id), -- main_history does not have a PK
history_valid_from TIMESTAMPTZ NOT NULL,
history_valid_to TIMESTAMPTZ
);
CREATE TABLE orange_history
(
id INTEGER NOT NULL,
main_id INTEGER NOT NULL REFERENCES main(id), -- main_history does not have a PK
history_valid_from TIMESTAMPTZ NOT NULL,
history_valid_to TIMESTAMPTZ
);
So there is a root (main) and two tables which reference on it. The referenced table hold max. 20 records (in 99% of the cases) for each main record.
The problem starts when I try to retreive the history:
SELECT * FROM main_history
LEFT JOIN apple_history ON apple_history.main_id = main_history.id AND
apple_history.history_valid_from <= '2021-12-20T10:46:52.482620Z' AND
(apple_history.history_valid_to IS NULL OR '2021-12-20T10:46:52.482620Z' < apple_history.history_valid_to)
LEFT JOIN orange_history ON orange_history.main_id = main_history.id AND
orange_history.history_valid_from <= '2021-12-20T10:46:52.482620Z' AND
(orange_history.history_valid_to IS NULL OR '2021-12-20T10:46:52.482620Z' < orange_history.history_valid_to)
WHERE
main_history.id IN (1,2,3,4) AND
main_history.history_valid_from <= '2021-12-20T10:46:52.482620Z' AND
(main_history.history_valid_to IS NULL OR '2021-12-20T10:46:52.482620Z' < main_history.history_valid_to)
The point is: I'm joining now over a non-referencing column. PostgreSQL overestimates the rows up to 20.000 times (see https://explain.dalibo.com/plan/oIF).
Then I tried to
SET enable_hashjoin = off; SET enable_mergejoin = off
because I knew that there are not that much rows --> then it went down to 100msec.
When I change the query to:
SELECT * FROM main
LEFT JOIN apple_history ON apple_history.main_id = main_history.id AND
apple_history.history_valid_from <= '2021-12-20T10:46:52.482620Z' AND
(apple_history.history_valid_to IS NULL OR '2021-12-20T10:46:52.482620Z' < apple_history.history_valid_to)
LEFT JOIN orange_history ON orange_history.main_id = main_history.id AND
orange_history.history_valid_from <= '2021-12-20T10:46:52.482620Z' AND
(orange_history.history_valid_to IS NULL OR '2021-12-20T10:46:52.482620Z' < orange_history.history_valid_to)
WHERE
main.id IN (1,2,3,4)
It can use the original statitics - everything works fine.
My question: Can I create manual statistics for JOINing not REFERENCESed columns? Goal: use main_history as main table in the query without forcing JOIN strategies

Answer 1

You need some indexes.
To replicate 100ms performance all you need to do is create an index on main_history (main_id, valid_from, valid_to) and you should be set.
But why, though? Ok, so when you query main_history WHERE main_history.Id IN (1,2,3,4) postgres
doesn't know how many rows will match
doesn't know where the rows are on disk/in memory
It has no choice but to scan the entire main_history table to find the rows you're looking for. And the same goes for apples_history/oranges_history. It cannot possibly know where the corresponding main_ids are, nor how many there are. Full scans are needed, and Hash-joins are the best choices (since data isn't ordered)
So to help it along, you create indexes on the referenced/referencing columns. They give count and order, and will help postgres choose the correct plan.