select max(id) from joined table inefficient query plan - postgresql

Question

I have a view req_res which joins 2 tables - Request and Response inner joined on requestId. Request has primary key - ID column.
When I query (Query 1):
select max(ID) from req_res where ID > 1000000 and ID < 2000000;
Explain plan: hash join: Index scan of Request.ID and seqential scan of Response.request_id
query duration: 30s
When I lower the boundaries to 900k (Query 2):
select max(ID) from req_res where ID > 1000000 and ID < 1900000;
Plan: nested loop: Index scan of Request.ID and Index only scan of Response.request_id
query duration: 3s
When I play with first query and disable hash join - set enable_hashjoin=off; I get Merge join plan. When I disable also the merge join plan with set enable_mergejoin=off; I get nested loop, which completes in 3 seconds (instead of 30 using hash join).
Size of the Request table is ~70 Mil records. Most of the requests have response counterpart, but some of them don't.
Version: PostgreSQL 10.10
req_res DDL:
CREATE OR REPLACE VIEW public.req_res
AS SELECT req.id,
res.req_id,
res.body::character varying(500),
res.time,
res.duration,
res.balance,
res.header::character varying(100),
res.created_at
FROM response res
JOIN request req ON req.req_id = res.req_id;
Query 1 Plan:
Aggregate (cost=2834115.70..2834115.71 rows=1 width=8) (actual time=154709.729..154709.730 rows=1 loops=1)
Buffers: shared hit=467727 read=685320 dirtied=214, temp read=240773 written=239751
-> Hash Join (cost=2493060.64..2831172.33 rows=1177346 width=8) (actual time=143800.101..154147.080 rows=1198706 loops=1)
Hash Cond: (req.req_id = res.req_id)
Buffers: shared hit=467727 read=685320 dirtied=214, temp read=240773 written=239751
-> Append (cost=0.44..55619.59 rows=1177346 width=16) (actual time=0.957..2354.648 rows=1200001 loops=1)
Buffers: shared hit=438960 read=32014
-> Index Scan using "5_5_req_pkey" on _hyper_2_5_chunk rs (cost=0.44..19000.10 rows=399803 width=16) (actual time=0.956..546.231 rows=399999 loops=1)
Index Cond: ((id >= 49600001) AND (id <= 50800001))
Buffers: shared hit=178872 read=10742
-> Index Scan using "7_7_req_pkey" on _hyper_2_7_chunk rs_1 (cost=0.44..36619.50 rows=777543 width=16) (actual time=0.037..767.744 rows=800002 loops=1)
Index Cond: ((id >= 49600001) AND (id <= 50800001))
Buffers: shared hit=260088 read=21272
-> Hash (cost=1367864.98..1367864.98 rows=68583298 width=8) (actual time=143681.850..143681.850 rows=68568554 loops=1)
Buckets: 262144 Batches: 512 Memory Usage: 7278kB
Buffers: shared hit=28764 read=653306 dirtied=214, temp written=233652
-> Append (cost=0.00..1367864.98 rows=68583298 width=8) (actual time=0.311..99590.021 rows=68568554 loops=1)
Buffers: shared hit=28764 read=653306 dirtied=214
-> Seq Scan on _hyper_3_2_chunk wt (cost=0.00..493704.44 rows=24941244 width=8) (actual time=0.309..14484.420 rows=24950147 loops=1)
Buffers: shared hit=661 read=243631
-> Seq Scan on _hyper_3_6_chunk wt_1 (cost=0.00..503935.04 rows=24978804 width=8) (actual time=0.334..14487.931 rows=24963020 loops=1)
Buffers: shared hit=168 read=253979
-> Seq Scan on _hyper_3_8_chunk wt_2 (cost=0.00..370225.50 rows=18663250 width=8) (actual time=0.327..10837.291 rows=18655387 loops=1)
Buffers: shared hit=27935 read=155696 dirtied=214
Planning time: 3.986 ms
Execution time: 154709.859 ms
Query 2 Plan:
Finalize Aggregate (cost=2634042.50..2634042.51 rows=1 width=8) (actual time=5525.626..5525.627 rows=1 loops=1)
Buffers: shared hit=8764620 read=12779
-> Gather (cost=2634042.29..2634042.50 rows=2 width=8) (actual time=5525.609..5525.705 rows=3 loops=1)
Workers Planned: 2
Workers Launched: 2
Buffers: shared hit=8764620 read=12779
-> Partial Aggregate (cost=2633042.29..2633042.30 rows=1 width=8) (actual time=5515.507..5515.508 rows=1 loops=3)
Buffers: shared hit=8764620 read=12779
-> Nested Loop (cost=0.88..2632023.83 rows=407382 width=8) (actual time=5.383..5261.979 rows=332978 loops=3)
Buffers: shared hit=8764620 read=12779
-> Append (cost=0.44..40514.98 rows=407383 width=16) (actual time=0.035..924.498 rows=333334 loops=3)
Buffers: shared hit=446706
-> Parallel Index Scan using "5_5_req_pkey" on _hyper_2_5_chunk rs (cost=0.44..16667.91 rows=166585 width=16) (actual time=0.033..169.854 rows=133333 loops=3)
Index Cond: ((id >= 49600001) AND (id <= 50600001))
Buffers: shared hit=190175
-> Parallel Index Scan using "7_7_req_pkey" on _hyper_2_7_chunk rs_1 (cost=0.44..23847.07 rows=240798 width=16) (actual time=0.039..336.091 rows=200001 loops=3)
Index Cond: ((id >= 49600001) AND (id <= 50600001))
Buffers: shared hit=256531
-> Append (cost=0.44..6.33 rows=3 width=8) (actual time=0.011..0.011 rows=1 loops=1000001)
Buffers: shared hit=8317914 read=12779
-> Index Only Scan using "2_2_response_pkey" on _hyper_3_2_chunk wt (cost=0.44..2.11 rows=1 width=8) (actual time=0.003..0.003 rows=0 loops=1000001)
Index Cond: (req_id = req.req_id)
Heap Fetches: 0
Buffers: shared hit=3000005
-> Index Only Scan using "6_6_response_pkey" on _hyper_3_6_chunk wt_1 (cost=0.44..2.11 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=1000001)
Index Cond: (req_id = req.req_id)
Heap Fetches: 192906
Buffers: shared hit=3551440 read=7082
-> Index Only Scan using "8_8_response_pkey" on _hyper_3_8_chunk wt_2 (cost=0.44..2.10 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=443006)
Index Cond: (req_id = req.req_id)
Heap Fetches: 162913
Buffers: shared hit=1766469 read=5697
Planning time: 0.839 ms
Execution time: 5525.814 ms