Very slow SELECT from partitioned table in PostgreSQL

Very slow SELECT from partitioned table in PostgreSQL - postgresql

I use Postgresql 9.5. The settings are default.
I have split a table into ~7000 partitions. Then I've inserted one row.
When I query SELECT * FROM "Offer";, it is running 1,5 seconds.
When I query SELECT * FROM "Offer" WHERE bid=4793;, where bid -- partition's constraint (one table per bid), it is running 1 second.
Here is an EXPLAIN ANALYZE for second query:
Append (cost=0.00..12.14 rows=2 width=596) (actual time=0.014..0.014 rows=1 loops=1)
-> Seq Scan on "Offer" (cost=0.00..1.01 rows=1 width=344) (actual time=0.011..0.011 rows=0 loops=1)
Filter: (bid = 4793)
Rows Removed by Filter: 1
-> Seq Scan on "Offer-4793" (cost=0.00..11.12 rows=1 width=848) (actual time=0.002..0.002 rows=1 loops=1)
Filter: (bid = 4793)
Planning time: 996.243 ms
Execution time: 0.261 ms
Why so slow? What can I use to profile it?
I have only one guess -- postgresql does not keep partitioning constrains in RAM and reads them from HDD every time.
Expecting some help!
UPDATE:
I've tried to create cascading partitioning (as #jmelesky has written). Results are worse:
Append (cost=0.00..12.24 rows=5 width=848) (actual time=0.013..0.013 rows=1 loops=1)
-> Seq Scan on "Offer" (cost=0.00..1.11 rows=1 width=848) (actual time=0.006..0.006 rows=0 loops=1)
Filter: (bid = 4793)
-> Seq Scan on "Offer-ddd-3" (cost=0.00..0.00 rows=1 width=848) (actual time=0.001..0.001 rows=0 loops=1)
Filter: (bid = 4793)
-> Seq Scan on "Offer-dd-33" (cost=0.00..0.00 rows=1 width=848) (actual time=0.000..0.000 rows=0 loops=1)
Filter: (bid = 4793)
-> Seq Scan on "Offer-d-336" (cost=0.00..0.00 rows=1 width=848) (actual time=0.000..0.000 rows=0 loops=1)
Filter: (bid = 4793)
-> Seq Scan on "Offer-4793" (cost=0.00..11.12 rows=1 width=848) (actual time=0.006..0.006 rows=1 loops=1)
Filter: (bid = 4793)
Planning time: 1449.872 ms
Execution time: 0.354 ms

Related

Postgres cursor based pagination performance

I have created a dummy panda table with 1.000.000 rows of random data. Table has id primary key column of uuid type.
Using prisma I do a cursor based pagination forward and backward for the same id cursor. Forward generated query takes 900ms (~1s), backward takes 0.06ms (~0.006s)
Why first is slower and is there a way to improve it? Strange thing is that if I replace "order_cmp"."Panda_id_0" with actual value (second example) performance will be good again 0.06ms (~0.006s).
Forward:
explain analyze
select * from
"public"."panda",
(
select
"public"."panda"."id" as "Panda_id_0"
from
"public"."panda"
where
("public"."panda"."id") = ('ffff581b-ef0c-4cb8-9ece-25504e420348')) as "order_cmp"
where
"public"."panda"."id" >= "order_cmp"."Panda_id_0"
order by
"public"."panda"."id" asc
limit 4 offset 1;
Limit (cost=0.98..1.52 rows=4 width=86) (actual time=919.298..919.303 rows=4 loops=1)
-> Nested Loop (cost=0.85..44539.57 rows=333333 width=86) (actual time=919.295..919.301 rows=5 loops=1)
Join Filter: (panda.id >= panda_1.id)
Rows Removed by Join Filter: 999990
-> Index Scan using panda_pkey on panda (cost=0.42..29536.92 rows=1000000 width=70) (actual time=0.017..607.298 rows=999995 loops=1)
-> Materialize (cost=0.42..2.65 rows=1 width=16) (actual time=0.000..0.000 rows=1 loops=999995)
-> Index Only Scan using panda_pkey on panda panda_1 (cost=0.42..2.64 rows=1 width=16) (actual time=0.011..0.012 rows=1 loops=1)
Index Cond: (id = 'ffff581b-ef0c-4cb8-9ece-25504e420348'::uuid)
Heap Fetches: 0
Planning Time: 0.166 ms
Execution Time: 919.332 ms
Previous forward query with replaced "order_cmp"."Panda_id_0" to actual value
explain analyze
select * from
"public"."panda",
(
select
"public"."panda"."id" as "Panda_id_0"
from
"public"."panda"
where
("public"."panda"."id") = ('ffff581b-ef0c-4cb8-9ece-25504e420348')) as "order_cmp"
where
"public"."panda"."id" >= 'ffff581b-ef0c-4cb8-9ece-25504e420348'
order by
"public"."panda"."id" asc
limit 4 offset 1;
Limit (cost=1.73..5.26 rows=4 width=86) (actual time=0.032..0.040 rows=4 loops=1)
-> Nested Loop (cost=0.85..4412.29 rows=5001 width=86) (actual time=0.013..0.038 rows=5 loops=1)
-> Index Scan using panda_pkey on panda (cost=0.42..4347.13 rows=5001 width=70) (actual time=0.007..0.028 rows=5 loops=1)
Index Cond: (id >= 'ffff581b-ef0c-4cb8-9ece-25504e420348'::uuid)
-> Materialize (cost=0.42..2.65 rows=1 width=16) (actual time=0.001..0.001 rows=1 loops=5)
-> Index Only Scan using panda_pkey on panda panda_1 (cost=0.42..2.64 rows=1 width=16) (actual time=0.003..0.004 rows=1 loops=1)
Index Cond: (id = 'ffff581b-ef0c-4cb8-9ece-25504e420348'::uuid)
Heap Fetches: 0
Planning Time: 0.224 ms
Execution Time: 0.073 ms
Unrelated, just to compare:
Backward:
explain analyze
select * from
"public"."panda",
(
select
"public"."panda"."id" as "Panda_id_0"
from
"public"."panda"
where
("public"."panda"."id") = ('ffff581b-ef0c-4cb8-9ece-25504e420348')) as "order_cmp"
where
"public"."panda"."id" <= "order_cmp"."Panda_id_0"
order by
"public"."panda"."id" desc
limit 4 offset 1;
Limit (cost=0.97..1.51 rows=4 width=85) (actual time=0.025..0.030 rows=4 loops=1)
-> Nested Loop (cost=0.83..4503.05 rows=33280 width=85) (actual time=0.023..0.029 rows=5 loops=1)
Join Filter: (panda.id <= panda_1.id)
Rows Removed by Join Filter: 2
-> Index Scan Backward using panda_pkey on panda (cost=0.42..3002.81 rows=99840 width=69) (actual time=0.011..0.017 rows=7 loops=1)
-> Materialize (cost=0.42..2.64 rows=1 width=16) (actual time=0.001..0.001 rows=1 loops=7)
-> Index Only Scan using panda_pkey on panda panda_1 (cost=0.42..2.64 rows=1 width=16) (actual time=0.005..0.005 rows=1 loops=1)
Index Cond: (id = 'ffff581b-ef0c-4cb8-9ece-25504e420348'::uuid)
Heap Fetches: 0
Planning Time: 0.099 ms
Execution Time: 0.046 ms
Offset based:
explain analyze
select * from
"public"."panda"
order by
"public"."panda"."id" asc
limit 4 offset 500000;
Limit (cost=14768.67..14768.79 rows=4 width=70) (actual time=316.159..316.161 rows=4 loops=1)
-> Index Scan using panda_pkey on panda (cost=0.42..29536.92 rows=1000000 width=70) (actual time=0.012..298.960 rows=500004 loops=1)
Planning Time: 0.051 ms
Execution Time: 316.179 ms

Postgres hash join batches explosion

We are having some struggle identifying why Postgres is using too much batches to resolve a join.
Here it is the output of explain analyze of a problematic execution:
https://explain.dalibo.com/plan/xNJ#plan
Limit (cost=20880.87..20882.91 rows=48 width=205) (actual time=10722.953..10723.358 rows=48 loops=1)
-> Unique (cost=20880.87..21718.12 rows=19700 width=205) (actual time=10722.951..10723.356 rows=48 loops=1)
-> Sort (cost=20880.87..20930.12 rows=19700 width=205) (actual time=10722.950..10722.990 rows=312 loops=1)
Sort Key: titlemetadata_titlemetadata.creation_date DESC, titlemetadata_titlemetadata.id, titlemetadata_titlemetadata.title_type, titlemetadata_titlemetadata.original_title, titlemetadata_titlemetadata.alternative_ids, titlemetadata_titlemetadata.metadata,
titlemetadata_titlemetadata.is_adult, titlemetadata_titlemetadata.is_kids, titlemetadata_titlemetadata.last_modified, titlemetadata_titlemetadata.year, titlemetadata_titlemetadata.runtime, titlemetadata_titlemetadata.rating, titlemetadata_titlemetadata.video_provider, tit
lemetadata_titlemetadata.series_id_id, titlemetadata_titlemetadata.season_number, titlemetadata_titlemetadata.episode_number
Sort Method: quicksort Memory: 872kB
-> Hash Right Join (cost=13378.20..19475.68 rows=19700 width=205) (actual time=1926.352..10709.970 rows=2909 loops=1)
Hash Cond: (t4.titlemetadata_id = t3.id)
Filter: ((hashed SubPlan 1) OR (hashed SubPlan 2))
Rows Removed by Filter: 63248
-> Seq Scan on video_provider_offer t4 (cost=0.00..5454.90 rows=66290 width=16) (actual time=0.024..57.893 rows=66390 loops=1)
-> Hash (cost=11314.39..11314.39 rows=22996 width=221) (actual time=489.530..489.530 rows=60096 loops=1)
Buckets: 65536 (originally 32768) Batches: 32768 (originally 1) Memory Usage: 11656kB
-> Hash Right Join (cost=5380.95..11314.39 rows=22996 width=221) (actual time=130.024..225.271 rows=60096 loops=1)
Hash Cond: (video_provider_offer.titlemetadata_id = titlemetadata_titlemetadata.id)
-> Seq Scan on video_provider_offer (cost=0.00..5454.90 rows=66290 width=16) (actual time=0.011..32.950 rows=66390 loops=1)
-> Hash (cost=5129.28..5129.28 rows=20133 width=213) (actual time=129.897..129.897 rows=55793 loops=1)
Buckets: 65536 (originally 32768) Batches: 2 (originally 1) Memory Usage: 7877kB
-> Merge Left Join (cost=1.72..5129.28 rows=20133 width=213) (actual time=0.041..93.057 rows=55793 loops=1)
Merge Cond: (titlemetadata_titlemetadata.id = t3.series_id_id)
-> Index Scan using titlemetadata_titlemetadata_pkey on titlemetadata_titlemetadata (cost=1.30..4130.22 rows=20133 width=205) (actual time=0.028..62.949 rows=43921 loops=1)
Filter: ((NOT is_adult) AND (NOT (hashed SubPlan 3)) AND (((title_type)::text = 'MOV'::text) OR ((title_type)::text = 'TVS'::text) OR ((title_type)::text = 'TVP'::text) OR ((title_type)::text = 'EVT'::text)))
Rows Removed by Filter: 14121
SubPlan 3
-> Seq Scan on cable_operator_cableoperatorexcludedtitle u0_2 (cost=0.00..1.01 rows=1 width=8) (actual time=0.006..0.006 rows=0 loops=1)
Filter: (cable_operator_id = 54)
-> Index Scan using titlemetadata_titlemetadata_series_id_id_73453db4_uniq on titlemetadata_titlemetadata t3 (cost=0.41..3901.36 rows=58037 width=16) (actual time=0.011..9.375 rows=12887 loops=1)
SubPlan 1
-> Hash Join (cost=44.62..885.73 rows=981 width=8) (actual time=0.486..36.806 rows=5757 loops=1)
Hash Cond: (w2.device_id = w3.id)
-> Nested Loop (cost=43.49..866.20 rows=2289 width=16) (actual time=0.441..33.096 rows=20180 loops=1)
-> Nested Loop (cost=43.06..414.98 rows=521 width=8) (actual time=0.426..9.952 rows=2909 loops=1)
Join Filter: (w1.id = w0.video_provider_id)
-> Nested Loop (cost=42.65..54.77 rows=13 width=24) (actual time=0.399..0.532 rows=15 loops=1)
-> HashAggregate (cost=42.50..42.95 rows=45 width=16) (actual time=0.390..0.403 rows=45 loops=1)
Group Key: v0.id
-> Nested Loop (cost=13.34..42.39 rows=45 width=16) (actual time=0.095..0.364 rows=45 loops=1)
-> Hash Semi Join (cost=13.19..32.72 rows=45 width=8) (actual time=0.084..0.229 rows=45 loops=1)
Hash Cond: (v1.id = u0.id)
-> Seq Scan on cable_operator_cableoperatorprovider v1 (cost=0.00..17.36 rows=636 width=16) (actual time=0.010..0.077 rows=636 loops=1)
-> Hash (cost=12.63..12.63 rows=45 width=8) (actual time=0.046..0.046 rows=45 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 10kB
-> Index Scan using cable_operator_cableoperatorprovider_4d6e54b3 on cable_operator_cableoperatorprovider u0 (cost=0.28..12.63 rows=45 width=8) (actual time=0.016..0.035 rows=45 loops=1)
Index Cond: (cable_operator_id = 54)
-> Index Only Scan using video_provider_videoprovider_pkey on video_provider_videoprovider v0 (cost=0.15..0.20 rows=1 width=8) (actual time=0.002..0.002 rows=1 loops=45)
Index Cond: (id = v1.provider_id)
Heap Fetches: 45
-> Index Scan using video_provider_videoprovider_pkey on video_provider_videoprovider w1 (cost=0.15..0.25 rows=1 width=8) (actual time=0.002..0.002 rows=0 loops=45)
Index Cond: (id = v0.id)
Filter: ((video_provider_type)::text = 'VOD'::text)
Rows Removed by Filter: 1
-> Index Scan using video_provider_offer_da942d2e on video_provider_offer w0 (cost=0.42..27.22 rows=39 width=16) (actual time=0.026..0.585 rows=194 loops=15)
Index Cond: (video_provider_id = v0.id)
Filter: (((end_date > '2021-09-02 19:23:00-03'::timestamp with time zone) OR (end_date IS NULL)) AND (access_criteria && '{vtv_mas,TBX_LOGIN,urn:spkg:tve:fox-premium,urn:tve:mcp,AMCHD,AMC_CONSORCIO,ANIMAL_PLANET,ASUNTOS_PUBLI
COS,ASUNTOS_PUBLICOS_CONSORCIO,CINECANALLIVE,CINECANAL_CONSORCIO,DISCOVERY,DISCOVERY_KIDS_CONSORCIO,DISCOVERY_KIDS_OD,DISNEY,DISNEY_CH_CONSORCIO,DISNEY_XD,DISNEY_XD_CONSORCIO,EL_CANAL_HD,EL_CANAL_HD_CONSORCIO,EL_GOURMET_CONSORCIO,ESPN,ESPN2_HD_CONSORCIO,ESPN3_HD_CONSORCIO
,ESPNMAS_HD_CONSORCIO,ESPN_BASIC,ESPN_HD_CONSORCIO,ESPN_PLAY,EUROPALIVE,EUROPA_EUROPA,EUROPA_EUROPA_CONSORCIO,FILMANDARTS_DISPOSITIVOS,FILMS_ARTS,FILM_AND_ARTS_CONSORCIO,FOXLIFE,FOX_LIFE_CONSORCIO,FOX_SPORTS_1_DISPOSITIVOS,FOX_SPORTS_2_DISPOSITIVOS,FOX_SPORTS_2_HD_CONSORC
IO,FOX_SPORTS_3_DISPOSITIVOS,FOX_SPORTS_3_HD_CONSORCIO,FOX_SPORTS_HD_CONSORCIO,FRANCE24_DISPOSITIVOS,FRANCE_24_CONSORCIO,GOURMET,GOURMET_DISPOSITIVOS,HOME_HEALTH,INVESTIGATION_DISCOVERY,MAS_CHIC,NATGEOKIDS_DISPOSITIVOS,NATGEO_CONSORCIO,NATGEO_DISPOSITIVOS,NATGEO_KIDS_CONS
ORCIO,PASIONES,PASIONES_CONSORCIO,SVOD_TYC_BASIC,TBX_LOGIN,TCC_2_CONSORCIO,TCC_2_HD,TLC,TVE,TVE_CONSORCIO,TYC_SPORTS_CONSORCIO,VTV_LIVE,clarosports,discoverykids,espnplay_south_alt,urn:spkg:tve:fox-basic,urn:tve:babytv,urn:tve:cinecanal,urn:tve:discoverykids,urn:tve:foxli
fe,urn:tve:fp,urn:tve:fx,urn:tve:natgeo,urn:tve:natgeokids,urn:tve:natgeowild,urn:tve:thefilmzone}'::character varying(50)[]) AND ((((content_type)::text = 'VOD'::text) AND ((start_date < '2021-09-02 19:23:00-03'::timestamp with time zone) OR (start_date IS NULL))) OR ((c
ontent_type)::text = 'LIV'::text)))
Rows Removed by Filter: 5
-> Index Only Scan using video_provider_offer_devices_offer_id_device_id_key on video_provider_offer_devices w2 (cost=0.42..0.81 rows=6 width=16) (actual time=0.004..0.007 rows=7 loops=2909)
Index Cond: (offer_id = w0.id)
Heap Fetches: 17828
-> Hash (cost=1.10..1.10 rows=3 width=8) (actual time=0.029..0.029 rows=2 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
-> Seq Scan on platform_device_device w3 (cost=0.00..1.10 rows=3 width=8) (actual time=0.024..0.027 rows=2 loops=1)
Filter: ((device_code)::text = ANY ('{ANDROID,ott_dual_tcc,ott_k2_tcc}'::text[]))
Rows Removed by Filter: 5
SubPlan 2
-> Hash Join (cost=44.62..885.73 rows=981 width=8) (actual time=0.410..33.580 rows=5757 loops=1)
Hash Cond: (w2_1.device_id = w3_1.id)
-> Nested Loop (cost=43.49..866.20 rows=2289 width=16) (actual time=0.375..29.886 rows=20180 loops=1)
-> Nested Loop (cost=43.06..414.98 rows=521 width=8) (actual time=0.366..9.134 rows=2909 loops=1)
Join Filter: (w1_1.id = w0_1.video_provider_id)
-> Nested Loop (cost=42.65..54.77 rows=13 width=24) (actual time=0.343..0.476 rows=15 loops=1)
-> HashAggregate (cost=42.50..42.95 rows=45 width=16) (actual time=0.333..0.347 rows=45 loops=1)
Group Key: v0_1.id
-> Nested Loop (cost=13.34..42.39 rows=45 width=16) (actual time=0.083..0.311 rows=45 loops=1)
-> Hash Semi Join (cost=13.19..32.72 rows=45 width=8) (actual time=0.076..0.202 rows=45 loops=1)
Hash Cond: (v1_1.id = u0_1.id)
-> Seq Scan on cable_operator_cableoperatorprovider v1_1 (cost=0.00..17.36 rows=636 width=16) (actual time=0.005..0.057 rows=636 loops=1)
-> Hash (cost=12.63..12.63 rows=45 width=8) (actual time=0.038..0.038 rows=45 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 10kB
-> Index Scan using cable_operator_cableoperatorprovider_4d6e54b3 on cable_operator_cableoperatorprovider u0_1 (cost=0.28..12.63 rows=45 width=8) (actual time=0.007..0.020 rows=45 loops=1)
Index Cond: (cable_operator_id = 54)
-> Index Only Scan using video_provider_videoprovider_pkey on video_provider_videoprovider v0_1 (cost=0.15..0.20 rows=1 width=8) (actual time=0.002..0.002 rows=1 loops=45)
Index Cond: (id = v1_1.provider_id)
Heap Fetches: 45
-> Index Scan using video_provider_videoprovider_pkey on video_provider_videoprovider w1_1 (cost=0.15..0.25 rows=1 width=8) (actual time=0.002..0.002 rows=0 loops=45)
Index Cond: (id = v0_1.id)
Filter: ((video_provider_type)::text = 'VOD'::text)
Rows Removed by Filter: 1
-> Index Scan using video_provider_offer_da942d2e on video_provider_offer w0_1 (cost=0.42..27.22 rows=39 width=16) (actual time=0.022..0.536 rows=194 loops=15)
Index Cond: (video_provider_id = v0_1.id)
Filter: (((end_date > '2021-09-02 19:23:00-03'::timestamp with time zone) OR (end_date IS NULL)) AND (access_criteria && '{vtv_mas,TBX_LOGIN,urn:spkg:tve:fox-premium,urn:tve:mcp,AMCHD,AMC_CONSORCIO,ANIMAL_PLANET,ASUNTOS_PUBLI
COS,ASUNTOS_PUBLICOS_CONSORCIO,CINECANALLIVE,CINECANAL_CONSORCIO,DISCOVERY,DISCOVERY_KIDS_CONSORCIO,DISCOVERY_KIDS_OD,DISNEY,DISNEY_CH_CONSORCIO,DISNEY_XD,DISNEY_XD_CONSORCIO,EL_CANAL_HD,EL_CANAL_HD_CONSORCIO,EL_GOURMET_CONSORCIO,ESPN,ESPN2_HD_CONSORCIO,ESPN3_HD_CONSORCIO
,ESPNMAS_HD_CONSORCIO,ESPN_BASIC,ESPN_HD_CONSORCIO,ESPN_PLAY,EUROPALIVE,EUROPA_EUROPA,EUROPA_EUROPA_CONSORCIO,FILMANDARTS_DISPOSITIVOS,FILMS_ARTS,FILM_AND_ARTS_CONSORCIO,FOXLIFE,FOX_LIFE_CONSORCIO,FOX_SPORTS_1_DISPOSITIVOS,FOX_SPORTS_2_DISPOSITIVOS,FOX_SPORTS_2_HD_CONSORC
IO,FOX_SPORTS_3_DISPOSITIVOS,FOX_SPORTS_3_HD_CONSORCIO,FOX_SPORTS_HD_CONSORCIO,FRANCE24_DISPOSITIVOS,FRANCE_24_CONSORCIO,GOURMET,GOURMET_DISPOSITIVOS,HOME_HEALTH,INVESTIGATION_DISCOVERY,MAS_CHIC,NATGEOKIDS_DISPOSITIVOS,NATGEO_CONSORCIO,NATGEO_DISPOSITIVOS,NATGEO_KIDS_CONS
ORCIO,PASIONES,PASIONES_CONSORCIO,SVOD_TYC_BASIC,TBX_LOGIN,TCC_2_CONSORCIO,TCC_2_HD,TLC,TVE,TVE_CONSORCIO,TYC_SPORTS_CONSORCIO,VTV_LIVE,clarosports,discoverykids,espnplay_south_alt,urn:spkg:tve:fox-basic,urn:tve:babytv,urn:tve:cinecanal,urn:tve:discoverykids,urn:tve:foxli
fe,urn:tve:fp,urn:tve:fx,urn:tve:natgeo,urn:tve:natgeokids,urn:tve:natgeowild,urn:tve:thefilmzone}'::character varying(50)[]) AND ((((content_type)::text = 'VOD'::text) AND ((start_date < '2021-09-02 19:23:00-03'::timestamp with time zone) OR (start_date IS NULL))) OR ((c
ontent_type)::text = 'LIV'::text)))
Rows Removed by Filter: 5
-> Index Only Scan using video_provider_offer_devices_offer_id_device_id_key on video_provider_offer_devices w2_1 (cost=0.42..0.81 rows=6 width=16) (actual time=0.003..0.006 rows=7 loops=2909)
Index Cond: (offer_id = w0_1.id)
Heap Fetches: 17828
-> Hash (cost=1.10..1.10 rows=3 width=8) (actual time=0.015..0.015 rows=2 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
-> Seq Scan on platform_device_device w3_1 (cost=0.00..1.10 rows=3 width=8) (actual time=0.010..0.011 rows=2 loops=1)
Filter: ((device_code)::text = ANY ('{ANDROID,ott_dual_tcc,ott_k2_tcc}'::text[]))
Rows Removed by Filter: 5
Planning time: 8.255 ms
Execution time: 10723.830 ms
(100 rows)
The weird part is that the same query, sometimes just uses a single batch. Here is an example: https://explain.dalibo.com/plan/zTv#plan
Here is the work_mem being used:
show work_mem;
work_mem
----------
8388kB
(1 row)
I'm not interested in changing the query to be more performant, but in understanding why is the different behavior.
I've found this thread apparently related with this, but I don't quite understand what are they talking about: https://www.postgresql.org/message-id/flat/CA%2BhUKGKWWmf%3DWELLG%3DaUGbcugRaSQbtm0tKYiBut-B2rVKX63g%40mail.gmail.com
Can anyone tell me why is this different behavior? The underlying data is the same in both cases.

If the hash is done in memory, there will only be a single batch.
A difference with the original hash batch numbers is due to Postgres choosing to increase the number of batches in order to reduce memory consumption.
You might find this EXPLAIN glossary useful (disclaimer: I'm one of the authors), here is the page on Hash Batches which also links to the PostgreSQL source code (it's very nicely documented in plain English).
While not a perfect heuristic, you can see that the memory required for the operations with multiple batches are around or above your work_mem setting. They can be lower than it, due to operations on disk generally requiring less memory overall.
I'm not 100% sure why in your exact case one was chosen over the other, but it does look like there are some very slight row estimate differences, which might be a good place to start.
As of PostgreSQL 13 there is also now a hash_mem_multiplier setting that can be used to give more memory to hashes without doing so for other operations (like sorts).

We where able to solve the problem just by doing VACUUM FULL ANALYZE;.
After that, everything started to work as expected (https://explain.depesz.com/s/eoqH#html)
Side note: we where not aware that we should do this on daily basis.

Recursive query slow on strange conditions

The following query is part of a much bigger one that runs perfectly fast on a filled DB but on a nearly empty one it is very long.
In this simplified form, it takes ~400ms to execute but if you remove either line (1) or lines (2) and (3) then it takes ~35ms. Why ? And how do I make it work normally ?
Some background about the DB :
DB is VACUUMed and ANALYZEd
ctract is empty
contrats contains only 2 lines, none of which has a idtypecontrat IN (4,5)
so tmpctr1 is empty
copyrightad contains 280 rows, only one matches the filters idoeu=13 and role IN ('E','CE')
in all cases, query returns ONE row (the one returned by the first part of the recursive CTE)
line (1) is absolutely not used in this version but removing it hides the problem for some reason
WITH RECURSIVE tmpctr1 AS (
SELECT ced.idad AS cedant, ced.idclient
FROM contrats c
JOIN CtrAct ced ON c.idcontrat=ced.idcontrat AND ced.isassignor
JOIN CtrAct ces ON c.idcontrat=ces.idcontrat AND NOT COALESCE(ces.isassignor,FALSE) --(1)
WHERE idtypecontrat IN (4,5)
)
,rec1 AS (
SELECT ca.idoeu,ca.idad AS chn,1 AS idclient, 1 AS level
FROM copyrightad ca
WHERE ca.role IN ('E','CE')
AND ca.idoeu = 13
UNION
SELECT r.idoeu,0, 0, r.level+1
FROM rec1 r
LEFT JOIN tmpctr1 c ON r.chn=c.cedant
LEFT JOIN tmpctr1 c2 ON r.idclient=c2.idclient -- (2)
WHERE r.level<20
AND (c.cedant is not null
OR c2.cedant is not null --(3)
)
)
select * from rec1
Query plan #1 : slow
QUERY PLAN
CTE Scan on rec1 (cost=1662106.61..2431078.65 rows=38448602 width=16) (actual time=384.975..398.182 rows=1 loops=1)
CTE tmpctr1
-> Hash Join (cost=36.06..116.37 rows=148225 width=8) (actual time=0.009..0.010 rows=0 loops=1)
Hash Cond: (c.idcontrat = ces.idcontrat)
-> Hash Join (cost=1.04..28.50 rows=385 width=16) (actual time=0.009..0.009 rows=0 loops=1)
Hash Cond: (ced.idcontrat = c.idcontrat)
-> Seq Scan on ctract ced (cost=0.00..25.40 rows=770 width=12) (actual time=0.008..0.008 rows=0 loops=1)
Filter: isassignor
-> Hash (cost=1.02..1.02 rows=1 width=4) (never executed)
-> Seq Scan on contrats c (cost=0.00..1.02 rows=1 width=4) (never executed)
Filter: (idtypecontrat = ANY ('{4,5}'::integer[]))
-> Hash (cost=25.40..25.40 rows=770 width=4) (never executed)
-> Seq Scan on ctract ces (cost=0.00..25.40 rows=770 width=4) (never executed)
Filter: (NOT COALESCE(isassignor, false))
CTE rec1
-> Recursive Union (cost=0.00..1661990.25 rows=38448602 width=16) (actual time=384.973..398.179 rows=1 loops=1)
-> Seq Scan on copyrightad ca (cost=0.00..8.20 rows=2 width=16) (actual time=384.970..384.981 rows=1 loops=1)
Filter: (((role)::text = ANY ('{E,CE}'::text[])) AND (idoeu = 13))
Rows Removed by Filter: 279
-> Merge Left Join (cost=21618.01..89301.00 rows=3844860 width=16) (actual time=13.193..13.193 rows=0 loops=1)
Merge Cond: (r.idclient = c2.idclient)
Filter: ((c_1.cedant IS NOT NULL) OR (c2.cedant IS NOT NULL))
Rows Removed by Filter: 1
-> Sort (cost=3892.89..3905.86 rows=5188 width=16) (actual time=13.179..13.180 rows=1 loops=1)
Sort Key: r.idclient
Sort Method: quicksort Memory: 25kB
-> Hash Right Join (cost=0.54..3572.76 rows=5188 width=16) (actual time=13.170..13.171 rows=1 loops=1)
Hash Cond: (c_1.cedant = r.chn)
-> CTE Scan on tmpctr1 c_1 (cost=0.00..2964.50 rows=148225 width=4) (actual time=0.011..0.011 rows=0 loops=1)
-> Hash (cost=0.45..0.45 rows=7 width=16) (actual time=13.150..13.150 rows=1 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
-> WorkTable Scan on rec1 r (cost=0.00..0.45 rows=7 width=16) (actual time=13.138..13.140 rows=1 loops=1)
Filter: (level < 20)
-> Materialize (cost=17725.12..18466.25 rows=148225 width=8) (actual time=0.008..0.008 rows=0 loops=1)
-> Sort (cost=17725.12..18095.68 rows=148225 width=8) (actual time=0.007..0.007 rows=0 loops=1)
Sort Key: c2.idclient
Sort Method: quicksort Memory: 25kB
-> CTE Scan on tmpctr1 c2 (cost=0.00..2964.50 rows=148225 width=8) (actual time=0.000..0.000 rows=0 loops=1)
Planning Time: 0.270 ms
JIT:
Functions: 53
Options: Inlining true, Optimization true, Expressions true, Deforming true
Timing: Generation 5.064 ms, Inlining 4.491 ms, Optimization 236.336 ms, Emission 155.206 ms, Total 401.097 ms
Execution Time: 403.549 ms
Query plan #2 : fast : line (1) is hidden
QUERY PLAN
CTE Scan on rec1 (cost=240.86..245.90 rows=252 width=16) (actual time=0.030..0.058 rows=1 loops=1)
CTE tmpctr1
-> Hash Join (cost=1.04..28.50 rows=385 width=8) (actual time=0.001..0.001 rows=0 loops=1)
Hash Cond: (ced.idcontrat = c.idcontrat)
-> Seq Scan on ctract ced (cost=0.00..25.40 rows=770 width=12) (actual time=0.001..0.001 rows=0 loops=1)
Filter: isassignor
-> Hash (cost=1.02..1.02 rows=1 width=4) (never executed)
-> Seq Scan on contrats c (cost=0.00..1.02 rows=1 width=4) (never executed)
Filter: (idtypecontrat = ANY ('{4,5}'::integer[]))
CTE rec1
-> Recursive Union (cost=0.00..212.35 rows=252 width=16) (actual time=0.029..0.056 rows=1 loops=1)
-> Seq Scan on copyrightad ca (cost=0.00..8.20 rows=2 width=16) (actual time=0.027..0.041 rows=1 loops=1)
Filter: (((role)::text = ANY ('{E,CE}'::text[])) AND (idoeu = 13))
Rows Removed by Filter: 279
-> Hash Right Join (cost=9.97..19.91 rows=25 width=16) (actual time=0.013..0.013 rows=0 loops=1)
Hash Cond: (c2.idclient = r.idclient)
Filter: ((c_1.cedant IS NOT NULL) OR (c2.cedant IS NOT NULL))
Rows Removed by Filter: 1
-> CTE Scan on tmpctr1 c2 (cost=0.00..7.70 rows=385 width=8) (actual time=0.000..0.000 rows=0 loops=1)
-> Hash (cost=9.81..9.81 rows=13 width=16) (actual time=0.009..0.009 rows=1 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
-> Hash Right Join (cost=0.54..9.81 rows=13 width=16) (actual time=0.008..0.008 rows=1 loops=1)
Hash Cond: (c_1.cedant = r.chn)
-> CTE Scan on tmpctr1 c_1 (cost=0.00..7.70 rows=385 width=4) (actual time=0.001..0.001 rows=0 loops=1)
-> Hash (cost=0.45..0.45 rows=7 width=16) (actual time=0.003..0.003 rows=1 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
-> WorkTable Scan on rec1 r (cost=0.00..0.45 rows=7 width=16) (actual time=0.002..0.002 rows=1 loops=1)
Filter: (level < 20)
Planning Time: 0.330 ms
Execution Time: 0.094 ms
Query plan #3 : fast : lines (2) and (3) are hidden
QUERY PLAN
CTE Scan on rec1 (cost=1829.46..2907.50 rows=53902 width=16) (actual time=0.050..0.074 rows=1 loops=1)
CTE rec1
-> Recursive Union (cost=0.00..1829.46 rows=53902 width=16) (actual time=0.049..0.072 rows=1 loops=1)
-> Seq Scan on copyrightad ca (cost=0.00..8.20 rows=2 width=16) (actual time=0.046..0.067 rows=1 loops=1)
Filter: (((role)::text = ANY ('{E,CE}'::text[])) AND (idoeu = 13))
Rows Removed by Filter: 279
-> Hash Join (cost=30.45..74.32 rows=5390 width=16) (actual time=0.003..0.003 rows=0 loops=1)
Hash Cond: (c.idcontrat = ced.idcontrat)
-> Hash Join (cost=1.04..28.50 rows=385 width=8) (actual time=0.002..0.002 rows=0 loops=1)
Hash Cond: (ces.idcontrat = c.idcontrat)
-> Seq Scan on ctract ces (cost=0.00..25.40 rows=770 width=4) (actual time=0.002..0.002 rows=0 loops=1)
Filter: (NOT COALESCE(isassignor, false))
-> Hash (cost=1.02..1.02 rows=1 width=4) (never executed)
-> Seq Scan on contrats c (cost=0.00..1.02 rows=1 width=4) (never executed)
Filter: (idtypecontrat = ANY ('{4,5}'::integer[]))
-> Hash (cost=29.08..29.08 rows=27 width=12) (never executed)
-> Hash Join (cost=0.54..29.08 rows=27 width=12) (never executed)
Hash Cond: (ced.idad = r.chn)
-> Seq Scan on ctract ced (cost=0.00..25.40 rows=766 width=8) (never executed)
Filter: (isassignor AND (idad IS NOT NULL))
-> Hash (cost=0.45..0.45 rows=7 width=12) (never executed)
-> WorkTable Scan on rec1 r (cost=0.00..0.45 rows=7 width=12) (never executed)
Filter: (level < 20)
Planning Time: 0.310 ms
Execution Time: 0.179 ms
PostgreSQL 12.2
Edit: the same query on the same DB on PostgreSQL 11.6 runs fast (still highly over-estimating rows on some parts) so I guess this is a regression.

Why?
The immediate reason for the big difference in query execution time is "Just-in-Time compilation", which is active by default in Postgres 12. Quoting the release notes:
Enable Just-in-Time (JIT) compilation by default, if the server
has been built with support for it (Andres Freund)
Note that this support is not built by default, but has to be selected
explicitly while configuring the build.
Turn it off in your session and test again:
SET jit = off
But JIT only amplifies the underlying problem: Estimates are way off in the query plan, which leads Postgres to assume a huge number of rows resulting from the joins in CTE tmpctr1, and assume that JIT would pay off.
Keep PostgreSQL from sometimes choosing a bad query plan
You asserted that ...
DB is VACUUMed and ANALYZEd
ctract is empty
But Postgres expects to find 770 rows in a sequential scan:
-> Seq Scan on ctract ced (cost=0.00..25.40 rows=770 width=12) (actual time=0.008..0.008 rows=0 loops=1)
Filter: isassignor
Bold emphasis mine. The number 770 comes directly from pg_class.reltuples, meaning that statistic is completely out of date. Maybe you relied on autovacuum but something kept it from kicking in, or its settings are not aggressive enough? Run this manually and retry:
ANALYZE ctract;
There is probably more potential to optimize, but I stopped processing here.
In a populated database, indexes will help a lot. Are you aware that partial or expression indexes can help with customized statistics? See:
Index that is not used, yet influences query
Get count estimates from pg_class.reltuples for given conditions
Abount (1):
JOIN CtrAct ces ON c.idcontrat=ces.idcontrat AND NOT COALESCE(ces.isassignor,FALSE) --(1)
Try replacing it with the equivalent:
JOIN CtrAct ces ON c.idcontrat=ces.idcontrat AND ces.isassignor IS NOT TRUE
It's clearer in any case. The convoluted expression may prevent index usage or better estimates (not the problem here).

postgres window function trebles query time

I'm using postgres 10, and have the following query
select
count(task.id) over() as _total_ ,
json_agg(u.*) as users,
task.*
from task
left outer join taskuserlink_history tu on (task.id = tu.taskid)
left outer join "user" u on (tu.userId = u.id)
group by task.id offset 10 limit 10;
this query takes approx 800ms to execute
if I remove the count(task.id) over() as _total_ , line, then it executes in 250ms
I have to confess being a complete sql noob, so the query itself may be completely borked
I was wondering if anyone could point to the flaws in the query, and make suggestions on how to speed it up.
The number of tasks is approx 15k, with an average of 5 users per task, linked through taskuserlink
I have looked at the pgadmin "explain" diagram
but to be honest can't really figure it out yet ;)
the table definitions are
task , with id (int) as primary column
taskuserlink_history, with taskId (int) and userId (int) (both as foreign key constraints, indexed)
user, with id (int) as primary column
the query plan is as follows
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=4.74..12.49 rows=10 width=44) (actual time=1178.016..1178.043 rows=10 loops=1)
Buffers: shared hit=3731, temp read=6655 written=6914
-> WindowAgg (cost=4.74..10248.90 rows=13231 width=44) (actual time=1178.014..1178.040 rows=10 loops=1)
Buffers: shared hit=3731, temp read=6655 written=6914
-> GroupAggregate (cost=4.74..10083.51 rows=13231 width=36) (actual time=0.417..1049.294 rows=13255 loops=1)
Group Key: task.id
Buffers: shared hit=3731
-> Nested Loop Left Join (cost=4.74..9586.77 rows=66271 width=36) (actual time=0.103..309.372 rows=66162 loops=1)
Join Filter: (taskuserlink_history.userid = user_archive.id)
Rows Removed by Join Filter: 1182904
Buffers: shared hit=3731
-> Merge Left Join (cost=0.58..5563.22 rows=66271 width=8) (actual time=0.044..73.598 rows=66162 loops=1)
Merge Cond: (task.id = taskuserlink_history.taskid)
Buffers: shared hit=3629
-> Index Only Scan using task_pkey on task (cost=0.29..1938.30 rows=13231 width=4) (actual time=0.026..7.683 rows=13255 loops=1)
Heap Fetches: 13255
Buffers: shared hit=1810
-> Index Scan using taskuserlink_history_task_fk_idx on taskuserlink_history (cost=0.29..2764.46 rows=66271 width=8) (actual time=0.015..40.109 rows=66162 loops=1)
Filter: (timeend IS NULL)
Rows Removed by Filter: 13368
Buffers: shared hit=1819
-> Materialize (cost=4.17..50.46 rows=4 width=36) (actual time=0.000..0.001 rows=19 loops=66162)
Buffers: shared hit=102
-> Bitmap Heap Scan on user_archive (cost=4.17..50.44 rows=4 width=36) (actual time=0.050..0.305 rows=45 loops=1)
Recheck Cond: (archived_at IS NULL)
Heap Blocks: exact=11
Buffers: shared hit=102
-> Bitmap Index Scan on user_unique_username (cost=0.00..4.16 rows=4 width=0) (actual time=0.014..0.014 rows=46 loops=1)
Buffers: shared hit=1
SubPlan 1
-> Aggregate (cost=8.30..8.31 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=45)
Buffers: shared hit=90
-> Index Scan using task_assignedto_idx on task task_1 (cost=0.29..8.30 rows=1 width=4) (actual time=0.002..0.002 rows=0 loops=45)
Index Cond: (assignedtoid = user_archive.id)
Buffers: shared hit=90
Planning time: 0.989 ms
Execution time: 1191.451 ms
(37 rows)
without the window function it is
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=4.74..12.36 rows=10 width=36) (actual time=0.510..1.763 rows=10 loops=1)
Buffers: shared hit=91
-> GroupAggregate (cost=4.74..10083.51 rows=13231 width=36) (actual time=0.509..1.759 rows=10 loops=1)
Group Key: task.id
Buffers: shared hit=91
-> Nested Loop Left Join (cost=4.74..9586.77 rows=66271 width=36) (actual time=0.073..0.744 rows=50 loops=1)
Join Filter: (taskuserlink_history.userid = user_archive.id)
Rows Removed by Join Filter: 361
Buffers: shared hit=91
-> Merge Left Join (cost=0.58..5563.22 rows=66271 width=8) (actual time=0.029..0.161 rows=50 loops=1)
Merge Cond: (task.id = taskuserlink_history.taskid)
Buffers: shared hit=7
-> Index Only Scan using task_pkey on task (cost=0.29..1938.30 rows=13231 width=4) (actual time=0.016..0.031 rows=11 loops=1)
Heap Fetches: 11
Buffers: shared hit=4
-> Index Scan using taskuserlink_history_task_fk_idx on taskuserlink_history (cost=0.29..2764.46 rows=66271 width=8) (actual time=0.009..0.081 rows=50 loops=1)
Filter: (timeend IS NULL)
Rows Removed by Filter: 11
Buffers: shared hit=3
-> Materialize (cost=4.17..50.46 rows=4 width=36) (actual time=0.001..0.009 rows=8 loops=50)
Buffers: shared hit=84
-> Bitmap Heap Scan on user_archive (cost=4.17..50.44 rows=4 width=36) (actual time=0.040..0.382 rows=38 loops=1)
Recheck Cond: (archived_at IS NULL)
Heap Blocks: exact=7
Buffers: shared hit=84
-> Bitmap Index Scan on user_unique_username (cost=0.00..4.16 rows=4 width=0) (actual time=0.012..0.012 rows=46 loops=1)
Buffers: shared hit=1
SubPlan 1
-> Aggregate (cost=8.30..8.31 rows=1 width=8) (actual time=0.005..0.005 rows=1 loops=38)
Buffers: shared hit=76
-> Index Scan using task_assignedto_idx on task task_1 (cost=0.29..8.30 rows=1 width=4) (actual time=0.003..0.003 rows=0 loops=38)
Index Cond: (assignedtoid = user_archive.id)
Buffers: shared hit=76
Planning time: 0.895 ms
Execution time: 1.890 ms
(35 rows)|

I believe the LIMIT clause is making the difference. LIMIT is limiting the number of rows returned, not neccessarily the work involved:
Your second query can be aborted early after 20 rows have been constructed (10 for OFFSET and 10 for LIMIT).
However, your first query needs to go through the whole set to calculate the count(task.id).
Not what you were asking, but I say it anyway:
"user" is not a table, but a view. That is were both queries actually get slower than they should be (The "Materialize" in the plan).
Using OFFSET for paging calls for trouble because it will get slow when the OFFSET increases
Using OFFSET and LIMIT without an ORDER BY is most likely not what you want. The result sets might not be identical on consecutive calls.

Explain postgres query, why is the query that much longer with WHERE and LIMIT

I'm using postgres v9.6.5. I have a query which seems not that complicated and was wondering why is it so "slow" (it's not really that slow, but I don't have a lot of data actually - like a few thousand rows).
Here is the query:
SELECT o0.*
FROM "orders" AS o0
JOIN "balances" AS b1 ON b1."id" = o0."balance_id"
JOIN "users" AS u3 ON u3."id" = b1."user_id"
WHERE (u3."partner_id" = 3)
ORDER BY o0."id" DESC LIMIT 10;
And that's query plan:
Limit (cost=0.43..12.84 rows=10 width=148) (actual time=0.062..53.866 rows=4 loops=1)
-> Nested Loop (cost=0.43..4750.03 rows=3826 width=148) (actual time=0.061..53.864 rows=4 loops=1)
Join Filter: (b1.user_id = u3.id)
Rows Removed by Join Filter: 67404
-> Nested Loop (cost=0.43..3945.32 rows=17856 width=152) (actual time=0.025..38.457 rows=16852 loops=1)
-> Index Scan Backward using orders_pkey on orders o0 (cost=0.29..897.80 rows=17856 width=148) (actual time=0.016..11.558 rows=16852 loops=1)
-> Index Scan using balances_pkey on balances b1 (cost=0.14..0.16 rows=1 width=8) (actual time=0.001..0.001 rows=1 loops=16852)
Index Cond: (id = o0.balance_id)
-> Materialize (cost=0.00..1.19 rows=3 width=4) (actual time=0.000..0.000 rows=4 loops=16852)
-> Seq Scan on users u3 (cost=0.00..1.18 rows=3 width=4) (actual time=0.023..0.030 rows=4 loops=1)
Filter: (partner_id = 3)
Rows Removed by Filter: 12
Planning time: 0.780 ms
Execution time: 54.053 ms
I actually tried without LIMIT and I got quite different plan:
Sort (cost=874.23..883.80 rows=3826 width=148) (actual time=11.361..11.362 rows=4 loops=1)
Sort Key: o0.id DESC
Sort Method: quicksort Memory: 26kB
-> Hash Join (cost=3.77..646.55 rows=3826 width=148) (actual time=11.300..11.346 rows=4 loops=1)
Hash Cond: (o0.balance_id = b1.id)
-> Seq Scan on orders o0 (cost=0.00..537.56 rows=17856 width=148) (actual time=0.012..8.464 rows=16852 loops=1)
-> Hash (cost=3.55..3.55 rows=18 width=4) (actual time=0.125..0.125 rows=24 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
-> Hash Join (cost=1.21..3.55 rows=18 width=4) (actual time=0.046..0.089 rows=24 loops=1)
Hash Cond: (b1.user_id = u3.id)
-> Seq Scan on balances b1 (cost=0.00..1.84 rows=84 width=8) (actual time=0.011..0.029 rows=96 loops=1)
-> Hash (cost=1.18..1.18 rows=3 width=4) (actual time=0.028..0.028 rows=4 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
-> Seq Scan on users u3 (cost=0.00..1.18 rows=3 width=4) (actual time=0.014..0.021 rows=4 loops=1)
Filter: (partner_id = 3)
Rows Removed by Filter: 12
Planning time: 0.569 ms
Execution time: 11.420 ms
And also without WHERE (but with LIMIT):
Limit (cost=0.43..4.74 rows=10 width=148) (actual time=0.023..0.066 rows=10 loops=1)
-> Nested Loop (cost=0.43..7696.26 rows=17856 width=148) (actual time=0.022..0.065 rows=10 loops=1)
Join Filter: (b1.user_id = u3.id)
Rows Removed by Join Filter: 139
-> Nested Loop (cost=0.43..3945.32 rows=17856 width=152) (actual time=0.009..0.029 rows=10 loops=1)
-> Index Scan Backward using orders_pkey on orders o0 (cost=0.29..897.80 rows=17856 width=148) (actual time=0.007..0.015 rows=10 loops=1)
-> Index Scan using balances_pkey on balances b1 (cost=0.14..0.16 rows=1 width=8) (actual time=0.001..0.001 rows=1 loops=10)
Index Cond: (id = o0.balance_id)
-> Materialize (cost=0.00..1.21 rows=14 width=4) (actual time=0.001..0.001 rows=15 loops=10)
-> Seq Scan on users u3 (cost=0.00..1.14 rows=14 width=4) (actual time=0.005..0.007 rows=16 loops=1)
Planning time: 0.286 ms
Execution time: 0.097 ms
As you can see, without WHERE it's much faster. Can someone provide me with some information where can I look for explanations for those plans to better understand them? And also what can I do to make those queries faster (or I shouldn't worry cause with like 100 times more data they will still be fast enough? - 50ms is fine for me tbh)

PostgreSQL thinks that it will be fastest if it scans orders in the correct order until it finds a matching users entry that satisfies the WHERE condition.
However, it seems that the data distribution is such that it has to scan almost 17000 orders before it finds a match.
Since PostgreSQL doesn't know how values correlate across tables, there is nothing much you can do to change that.
You can force PostgreSQL to plan the query without the LIMIT clause like this:
SELECT *
FROM (<your query without ORDER BY and LIMIT> OFFSET 0) q
ORDER BY id DESC LIMIT 10;
With a top-N-sort this should perform better.

We Keep Coding

iphone swift flutter scala powershell matlab mongodb postgresql perl eclipse