Recently we experienced a performance problem in Production Aurora PG cluster. This is an EXPLAIN ANALYZE of the query.
The majority of the time is spent on Bitmap Index Scan on job_stage (cost=0.00..172.93 rows=9666 width=0) (actual time=238.410..238.410 rows=2019444 loops=1) where 2019444 are scanned. However, what troubles me is that there are only 70k rows in this table. Autovacuum is turned on, but the RDS was overloaded recently from another issue. We suspect that the autovacuum was running behind. If that is the case, would it explain our observation the scanned row exceeds actual row in table?
Nested Loop (cost=229.16..265.28 rows=1 width=464) (actual time=239.815..239.815 rows=0 loops=1)
-> Nested Loop (cost=228.62..252.71 rows=1 width=540) (actual time=239.814..239.814 rows=0 loops=1)
Join Filter: (job.scanner_uuid = scanner_resource_pool.resource_uuid)
Rows Removed by Join Filter: 1
-> Index Scan using scanner_resource_pool_scanner_index on scanner_resource_pool (cost=0.41..8.43 rows=1 width=115) (actual time=0.017..0.019 rows=1 loops=1)
Index Cond: ((box_uuid = '5d8a7e0c-23ff-4853-bb6d-ffff6a38afa7'::text) AND (scanner_uuid = '9be9ac50-de05-4ddd-9545-ddddc484dce'::text))
-> Bitmap Heap Scan on job (cost=228.22..244.23 rows=4 width=464) (actual time=239.790..239.791 rows=1 loops=1)
Recheck Cond: ((box_uuid = '5d8a7e0c-23ff-4853-bb6d-ffff6a38afa7'::text) AND (stage = 'active'::text))
Rows Removed by Index Recheck: 6
Heap Blocks: exact=791
-> BitmapAnd (cost=228.22..228.22 rows=4 width=0) (actual time=238.913..238.913 rows=0 loops=1)
-> Bitmap Index Scan on job_box_status (cost=0.00..55.04 rows=1398 width=0) (actual time=0.183..0.183 rows=899 loops=1)
Index Cond: (box_uuid = '5d8a7e0c-23ff-4853-bb6d-ffff6a38afa7'::text)
-> Bitmap Index Scan on job_stage (cost=0.00..172.93 rows=9666 width=0) (actual time=238.410..238.410 rows=2019444 loops=1)
Index Cond: (stage = 'active'::text)
-> Index Only Scan using uc_box_uuid on scanner (cost=0.54..12.56 rows=1 width=87) (never executed)
Index Cond: ((box_uuid = '5d8a7e0c-23ff-4853-bb6d-ffff6a38afa7'::text) AND (uuid = '9be9ac50-de05-4ddd-9545-ddddc484dce'::text))
Heap Fetches: 0
Planning time: 1.274 ms
Execution time: 239.876 ms
I found my answer by confirming with AWS. If autovacuum was running behind, the EXPLAIN ANALYZE result may show this discrepancy.
Related
We have a complex query which is dynamically built depending on different options on the customer and runs a query for data. We have functions in Azure which are running these queries to build reporting data every night, we run approx. 30k of these. The queries in isolation are about as fast as I can get them, approximately 100ms but when we are running functions in parallel on the consumption plan in Azure (restricted to a maximum of 5 functions running at the same time) the performance of the queries is dropping off and some are even timing out at 5 minutes, some which are timing out I have tested in isolation and are coming in at under 100ms. There are no writes as this is using a read replica in Azure to load this data.
We are running in Postgres 11.6 on hosted Azure with PgBouncer on a VM. All these queries are going to a read replica which is configured to a 4 vCore Memory Optimized.
What changes can we make to allow more parallel execution of these queries or is scaling up our only option?
I would like to share the EXPLAIN ANALYZE but this is restricted by the business. Please let me know what information would help and I will try to provide as much as possible.
CTE Scan on bravo_zulu romeo (cost=2151.89..2151.94 rows=1 width=204) (actual time=27.756..84.147 rows=36 loops=1)
CTE bravo_zulu
-> Nested Loop (cost=13.84..2151.89 rows=1 width=139) (actual time=27.744..84.009 rows=36 loops=1)
-> Nested Loop (cost=13.42..2151.43 rows=1 width=139) (actual time=26.811..76.983 rows=36 loops=1)
-> Nested Loop (cost=12.86..130.51 rows=1 width=44) (actual time=7.471..19.361 rows=29 loops=1)
-> Nested Loop (cost=4.88..97.73 rows=1 width=24) (actual time=7.410..10.480 rows=24 loops=1)
-> Index Scan using yankee on xray_zulu foxtrot_uniform (cost=0.28..8.29 rows=1 width=8) (actual time=1.339..1.340 rows=1 loops=1)
Index Cond: ("juliet" = 20)
-> Bitmap Heap Scan on golf_delta hotel_six (cost=4.60..89.43 rows=1 width=20) (actual time=6.064..9.123 rows=24 loops=1)
Recheck Cond: ("delta_oscar_hotel" = foxtrot_uniform."lima")
Filter: ("juliet" = ANY ('foxtrot_oscar'::integer[]))
Rows Removed by Filter: 442
Heap Blocks: exact=65
-> Bitmap Index Scan on papa (cost=0.00..4.60 rows=42 width=0) (actual time=0.024..0.024 rows=466 loops=1)
Index Cond: ("delta_oscar_hotel" = foxtrot_uniform."lima")
-> Bitmap Heap Scan on delta_sierra_two bravo_hotel (cost=7.98..32.76 rows=2 width=20) (actual time=0.321..0.363 rows=1 loops=24)
Recheck Cond: ((hotel_six."juliet" = "xray_india") OR (hotel_six."juliet" = "foxtrot_foxtrot"))
Filter: ("hotel_golf" = 23)
Rows Removed by Filter: 10
Heap Blocks: exact=240
-> BitmapOr (cost=7.98..7.98 rows=9 width=0) (actual time=0.066..0.066 rows=0 loops=24)
-> Bitmap Index Scan on delta_sierra_sierra (cost=0.00..3.99 rows=5 width=0) (actual time=0.063..0.063 rows=11 loops=24)
Index Cond: (hotel_six."juliet" = "xray_india")
-> Bitmap Index Scan on xray_sierra (cost=0.00..3.99 rows=4 width=0) (actual time=0.002..0.002 rows=0 loops=24)
Index Cond: (hotel_six."juliet" = "foxtrot_foxtrot")
-> Index Only Scan using echo on xray_papa victor (cost=0.56..2020.44 rows=48 width=102) (actual time=1.606..1.986 rows=1 loops=29)
Index Cond: (("five_lima" = 23) AND ("seven_yankee" = bravo_hotel."november") AND ("charlie_hotel" five_romeo NULL))
Filter: (("three" = 'charlie_romeo'::text) AND (("alpha" = 'golf_bravo'::text) OR ("alpha" = 'delta_echo'::text)) AND ((("alpha" = ANY ('mike_juliet'::text[])) AND ("mike_lima" >= 'xray_whiskey'::date) AND ("mike_lima" <= 'uniform'::date)) OR (("alpha" = ANY ('kilo'::text[])) AND ("quebec_uniform" >= 'xray_whiskey'::date) AND ("quebec_uniform" <= 'uniform'::date)) OR (("alpha" = 'quebec_alpha_quebec'::text) AND ("quebec_uniform" >= 'xray_whiskey'::date) AND ("quebec_uniform" <= 'uniform'::date) AND ("mike_lima" >= 'xray_whiskey'::date) AND ("mike_lima" <= 'uniform'::date))) AND ((("alpha" = ANY ('oscar'::text[])) AND ("seven_india" = ANY ('four'::text[]))) OR (("alpha" = ANY ('quebec_alpha_delta'::text[])) AND ("seven_charlie" = ANY ('four'::text[])))))
Rows Removed by Filter: 1059
Heap Fetches: 0
-> Index Scan using bravo_papa on tango sierra (cost=0.42..0.45 rows=1 width=16) (actual time=0.194..0.194 rows=1 loops=36)
Index Cond: (("bravo_two" = 23) AND ("delta_tango" = six1."delta_oscar_romeo"))
SubPlan
-> Result (cost=0.01..0.02 rows=1 width=32) (actual time=0.001..0.001 rows=1 loops=36)
One-Time Filter: ((romeo.zulu = 'golf_bravo'::text) AND (romeo.golf_uniform = 20) AND (romeo.charlie_two = 'charlie_romeo'::text))
SubPlan
-> Result (cost=0.01..0.02 rows=1 width=32) (actual time=0.000..0.000 rows=0 loops=36)
One-Time Filter: ((romeo.zulu = 'delta_echo'::text) AND (romeo.charlie_two = 'charlie_romeo'::text) AND (romeo.golf_uniform = 20))
Planning time: 19.385 ms
Execution time: 84.373 ms
Above is an anonymised execution plan, this same query when running the functions in Azure in parallel timed out.
Table sizes are not large, largest is 8m rows but all others are low 100k.
When analysing a problem like this I find it usefull to restate what we know:
Your queries go from taking 100ms to timing out after 5 minutes.
This is happening on a 4 core system that is restricted to a maximum of 5 functions running at the same time.
This sounds more like a deadlock problem than a load problem.
There are 2 things that you could try:
Change to run one report at a time, to see if the timeouts disappear
Check your SQL code for and "with update" that could be leading to database locks
Edit:
Baced on answer in comment we can assume that it is not a database lock problem
Next thing to check is the connection to the database. It could be that the system is running out of available connections to the database. Things to check:
Max number of connections available
Is connection pooling used
Are connections being closed /released back to the pool, when they are no longer needed
I am facing a problem with a specific query on postgressql.
Look the explain:
-> Nested Loop Left Join (cost=21547.86..87609.16 rows=123 width=69) (actual time=28.997..562.299 rows=32710 loops=1)
-> Hash Join (cost=21547.30..87210.72 rows=123 width=53) (actual time=28.913..74.682 rows=32710 loops=1)
Hash Cond: (registry.id = profile.registry_id)
-> Bitmap Heap Scan on registry (cost=726.99..66218.46 rows=65503 width=53) (actual time=5.123..32.794 rows=66496 loops=1)
Recheck Cond: ((tenant_id = 1009469) AND active AND (excluded_at IS NULL))
Heap Blocks: exact=12563
-> Bitmap Index Scan on registry_tenant_id_excluded_at (cost=0.00..710.61 rows=65503 width=0) (actual time=3.589..3.589 rows=66496 loops=1)
Index Cond: (tenant_id = 1009469)
-> Hash (cost=20202.82..20202.82 rows=49399 width=16) (actual time=23.738..23.738 rows=32710 loops=1)
Buckets: 65536 Batches: 1 Memory Usage: 2046kB
-> Index Only Scan using profile_tenant_id_registry_id on profile (cost=0.56..20202.82 rows=49399 width=16) (actual time=0.019..19.173 rows=32710 loops=1)
Index Cond: (tenant_id = 1009469)
Heap Fetches: 29493
It misestimate the hash join, even if both the scans are accurate.
I already tried to boost the statistics on the related columns but it just estimated from 117 to 123, so I guess this is not the issue.
Why it is misestimating so hard?
The nested loop takes a lot of work for the database.
It looks like rows with same tenant_id also mostly have the same value for registry_id/registry.id. But the planner doesn't understand that. It thinks that registry_id=registry.id will be true as often for the actually selected rows as it will be for randomly selected pairs of rows.
I don't think there is anything you can do about this.
Table: Customer
Type:
telephone1 | character varying(255)
telephone2 | character varying(255)
location_id | integer
Index:
"idx_customers_location_id" btree (location_id)
"idx_customers_telephone1_txt" btree (telephone1 text_pattern_ops)
"idx_customers_trim_telephone_1" btree (btrim(telephone1::text))
"idx_customers_trim_telephone2" btree (btrim(telephone2::text))
I have a table called customers, total rows are 141182. I was checking values in two columns (telephone1, telephone2), all the column telephone1 has data, but only 8 rows have value for the column telephone2
When I check for the value 1, getting this below execution time.
SELECT customers.id, location_id, telephone1, telephon2 FROM "customers" INNER JOIN "locations" ON
"locations"."id" = "customers"."location_id" WHERE (customers.location_id = 189 AND (telephone1 = '1'
OR telephone2 = '1')) GROUP BY customers.id LIMIT 20 OFFSET 0;
Limit (cost=519.62..519.64 rows=4 width=125) (actual time=25.895..25.898 rows=1 loops=1)
-> GroupAggregate (cost=519.62..519.64 rows=4 width=125) (actual time=25.893..25.896 rows=1 loops=1)
Group Key: customers.id
-> Sort (cost=519.62..519.62 rows=4 width=127) (actual time=25.876..25.879 rows=1 loops=1)
Sort Key: customers.id
Sort Method: quicksort Memory: 25kB
-> Nested Loop (cost=8.62..519.61 rows=4 width=127) (actual time=10.740..25.869 rows=1 loops=1)
-> Index Scan using locations_pkey on locations (cost=0.06..4.06 rows=1 width=70) (actual time=0.027..0.029 rows=1 loops=1)
Index Cond: (id = 189)
-> Bitmap Heap Scan on customers (cost=8.56..515.54 rows=4 width=61) (actual time=10.707..25.832 rows=1 loops=1)
Recheck Cond: (((telephone1)::text = '1'::text) OR ((telephone2)::text = '1'::text))
Filter: (location_id = 189)
Rows Removed by Filter: 1048
Heap Blocks: exact=1737
-> BitmapOr (cost=8.56..8.56 rows=259 width=0) (actual time=3.445..3.446 rows=0 loops=1)
-> Bitmap Index Scan on idx_customers_telephone1_txt (cost=0.00..2.10 rows=7 width=0) (actual time=0.065..0.066 rows=99 loops=1)
Index Cond: ((telephone1)::text = '1'::text)
-> Bitmap Index Scan on idx_customers_telephone2_txt (cost=0.00..6.47 rows=253 width=0) (actual time=3.378..3.378 rows=1664 loops=1)
Index Cond: ((telephone2)::text = '1'::text)
Planning Time: 0.419 ms
Execution Time: 25.995 ms
When I check for value 0 there is a huge change in the execution time (7753.216 ms)
Limit (cost=0.14..2440.90 rows=10 width=125) (actual time=5900.924..7753.133 rows=4 loops=1)
-> GroupAggregate (cost=0.14..292402.20 rows=1198 width=125) (actual time=5900.922..7753.129 rows=4 loops=1)
Group Key: customers.id
-> Nested Loop (cost=0.14..292395.61 rows=1198 width=127) (actual time=4350.358..7753.087 rows=4 loops=1)
-> Index Scan using customers_pkey on customers (cost=0.09..292387.36 rows=1198 width=61) (actual time=4350.338..7753.054 rows=4 loops=1)
Filter: ((location_id = 189) AND (((telephone1)::text = '0'::text) OR ((telephone2)::text = '0'::text)))
Rows Removed by Filter: 8484280
-> Materialize (cost=0.06..4.06 rows=1 width=70) (actual time=0.005..0.005 rows=1 loops=4)
-> Index Scan using locations_pkey on locations (cost=0.06..4.06 rows=1 width=70) (actual time=0.013..0.013 rows=1 loops=1)
Index Cond: (id = 189)
Planning Time: 0.322 ms
Execution Time: 7753.216 ms
Is there any particular reason, that takes more time to execute for the value 0 ? or anything wrong here?
One more thing I have noticed this issue happens only with column telephone2.
but only 8 rows have value for the column telephone2
Your explain plan indicates otherwise, finding 1664 rows with one specific value for telephone2. Now maybe most of those are not visible, but in that case you really need to VACUUM ANALYZE the table.
Nested Loop (cost=0.14..292395.61 rows=1198 width=127) (actual time=4350.358..7753.087 rows=4 loops=1)
With this second query, it thinks it will find 1198 rows (if run to completion). But it thinks it can stop after the first 20, so that would be 1.67% of index. But instead there are only 4 rows, so it unexpectedly has to scan the entire index without getting to stop early.
Why are the estimates off by so much? I don't know, it could just be stale statistics (again, VACUUM ANALYZE the table), or there could be some interrelation between the columns that make the estimation hard to do even with accurate statistics.
What is the point in joining to locations at all?
I have the following two tables.
person_addresses
address_normalization
The person_addresses table has a field named address_id as the primary key and address_normalization has the corresponding field address_id which has an index on it.
Now, when I explain the following query, I see a sequential scan.
SELECT
count(*)
FROM
mp_member2.person_addresses pa
JOIN mp_member2.address_normalization an ON
an.address_id = pa.address_id
WHERE
an.sr_modification_time >= 1550692189468;
-- Result: 2654
Please refer to the following screenshot.
You see that there is a sequential scan after the hash join. I'm not sure I understand this part; why would a sequential scan follow a hash join.
And as seen in the query above, the set of records returned is also low.
Is this expected behaviour or am I doing something wrong?
Update #1: I also have indices on the sr_modification_time fields of both the tables
Update #2: Full execution plan
Aggregate (cost=206944.74..206944.75 rows=1 width=0) (actual time=2807.844..2807.844 rows=1 loops=1)
Buffers: shared hit=4629 read=82217
-> Hash Join (cost=2881.95..206825.15 rows=47836 width=0) (actual time=0.775..2807.160 rows=2654 loops=1)
Hash Cond: (pa.address_id = an.address_id)
Buffers: shared hit=4629 read=82217
-> Seq Scan on person_addresses pa (cost=0.00..135924.93 rows=4911993 width=8) (actual time=0.005..1374.610 rows=4911993 loops=1)
Buffers: shared hit=4588 read=82217
-> Hash (cost=2432.05..2432.05 rows=35992 width=18) (actual time=0.756..0.756 rows=1005 loops=1)
Buckets: 4096 Batches: 1 Memory Usage: 41kB
Buffers: shared hit=41
-> Index Scan using mp_member2_address_normalization_mod_time on address_normalization an (cost=0.43..2432.05 rows=35992 width=18) (actual time=0.012..0.424 rows=1005 loops=1)
Index Cond: (sr_modification_time >= 1550692189468::bigint)
Buffers: shared hit=41
Planning time: 0.244 ms
Execution time: 2807.885 ms
Update #3: I tried with a newer timestamp and it used an index scan.
EXPLAIN (
ANALYZE
, buffers
, format TEXT
) SELECT
COUNT(*)
FROM
mp_member2.person_addresses pa
JOIN mp_member2.address_normalization an ON
an.address_id = pa.address_id
WHERE
an.sr_modification_time >= 1557507300342;
-- count: 1364
Query Plan:
Aggregate (cost=295.48..295.49 rows=1 width=0) (actual time=2.770..2.770 rows=1 loops=1)
Buffers: shared hit=1404
-> Nested Loop (cost=4.89..295.43 rows=19 width=0) (actual time=0.038..2.491 rows=1364 loops=1)
Buffers: shared hit=1404
-> Index Scan using mp_member2_address_normalization_mod_time on address_normalization an (cost=0.43..8.82 rows=14 width=18) (actual time=0.009..0.142 rows=341 loops=1)
Index Cond: (sr_modification_time >= 1557507300342::bigint)
Buffers: shared hit=14
-> Bitmap Heap Scan on person_addresses pa (cost=4.46..20.43 rows=4 width=8) (actual time=0.004..0.005 rows=4 loops=341)
Recheck Cond: (address_id = an.address_id)
Heap Blocks: exact=360
Buffers: shared hit=1390
-> Bitmap Index Scan on idx_mp_member2_person_addresses_address_id (cost=0.00..4.46 rows=4 width=0) (actual time=0.003..0.003 rows=4 loops=341)
Index Cond: (address_id = an.address_id)
Buffers: shared hit=1030
Planning time: 0.214 ms
Execution time: 2.816 ms
That is the expected behavior because you don't have index for sr_modification_time so after create the hash join db has to scan the whole set to check each row for the sr_modification_time value
You should create:
index for (sr_modification_time)
or composite index for (address_id , sr_modification_time )
I have one complexe query generated by Hibernate for JBPM. I can't really modify it and i'm searching to optimize it as much as possible.
I found out that ORDER BY DESC is way slower than ORDER BY ASC, do you have any idea ?
PostgreSQL Version : 9.4
Schema : https://pastebin.com/qNZhrbef
Query :
select
taskinstan0_.ID_ as ID1_27_,
taskinstan0_.VERSION_ as VERSION3_27_,
taskinstan0_.NAME_ as NAME4_27_,
taskinstan0_.DESCRIPTION_ as DESCRIPT5_27_,
taskinstan0_.ACTORID_ as ACTORID6_27_,
taskinstan0_.CREATE_ as CREATE7_27_,
taskinstan0_.START_ as START8_27_,
taskinstan0_.END_ as END9_27_,
taskinstan0_.DUEDATE_ as DUEDATE10_27_,
taskinstan0_.PRIORITY_ as PRIORITY11_27_,
taskinstan0_.ISCANCELLED_ as ISCANCE12_27_,
taskinstan0_.ISSUSPENDED_ as ISSUSPE13_27_,
taskinstan0_.ISOPEN_ as ISOPEN14_27_,
taskinstan0_.ISSIGNALLING_ as ISSIGNA15_27_,
taskinstan0_.ISBLOCKING_ as ISBLOCKING16_27_,
taskinstan0_.LOCKED as LOCKED27_,
taskinstan0_.QUEUE as QUEUE27_,
taskinstan0_.TASK_ as TASK19_27_,
taskinstan0_.TOKEN_ as TOKEN20_27_,
taskinstan0_.PROCINST_ as PROCINST21_27_,
taskinstan0_.SWIMLANINSTANCE_ as SWIMLAN22_27_,
taskinstan0_.TASKMGMTINSTANCE_ as TASKMGM23_27_
from JBPM_TASKINSTANCE taskinstan0_, JBPM_VARIABLEINSTANCE stringinst1_, JBPM_PROCESSINSTANCE processins2_, JBPM_VARIABLEINSTANCE variablein3_
where stringinst1_.CLASS_='S'
and taskinstan0_.PROCINST_=processins2_.ID_
and taskinstan0_.ID_=variablein3_.TASKINSTANCE_
and variablein3_.NAME_ = 'NIR'
and taskinstan0_.QUEUE = 'ERT_TPS'
and (processins2_.ORGAPATH_ like '/ERT%')
and taskinstan0_.ISOPEN_= 't'
and variablein3_.ID_=stringinst1_.ID_
order by stringinst1_.STRINGVALUE_ ASC limit '10';
Explain result for ASC :
Limit (cost=1.71..11652.93 rows=10 width=646) (actual time=6.588..82.407 rows=10 loops=1)
-> Nested Loop (cost=1.71..6215929.27 rows=5335 width=646) (actual time=6.587..82.402 rows=10 loops=1)
-> Nested Loop (cost=1.29..6213170.78 rows=5335 width=646) (actual time=6.578..82.363 rows=10 loops=1)
-> Nested Loop (cost=1.00..6159814.66 rows=153812 width=13) (actual time=0.537..82.130 rows=149 loops=1)
-> Index Scan Backward using totoidx10 on jbpm_variableinstance stringinst1_ (cost=0.56..558481.07 rows=11199905 width=13) (actual time=0.018..11.914 rows=40182 loops=1)
Filter: (class_ = 'S'::bpchar)
-> Index Scan using jbpm_variableinstance_pkey on jbpm_variableinstance variablein3_ (cost=0.43..0.49 rows=1 width=16) (actual time=0.002..0.002 rows=0 loops=40182)
Index Cond: (id_ = stringinst1_.id_)
Filter: ((name_)::text = 'NIR'::text)
Rows Removed by Filter: 1
-> Index Scan using jbpm_taskinstance_pkey on jbpm_taskinstance taskinstan0_ (cost=0.29..0.34 rows=1 width=641) (actual time=0.001..0.001 rows=0 loops=149)
Index Cond: (id_ = variablein3_.taskinstance_)
Filter: (isopen_ AND ((queue)::text = 'ERT_TPS'::text))
Rows Removed by Filter: 0
-> Index Only Scan using idx_procin_2 on jbpm_processinstance processins2_ (cost=0.42..0.51 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=10)
Index Cond: (id_ = taskinstan0_.procinst_)
Filter: ((orgapath_)::text ~~ '/ERT%'::text)
Heap Fetches: 0
Planning time: 2.598 ms
Execution time: 82.513 ms
Explain result for DESC :
Limit (cost=1.71..11652.93 rows=10 width=646) (actual time=8144.871..8144.986 rows=10 loops=1)
-> Nested Loop (cost=1.71..6215929.27 rows=5335 width=646) (actual time=8144.870..8144.984 rows=10 loops=1)
-> Nested Loop (cost=1.29..6213170.78 rows=5335 width=646) (actual time=8144.858..8144.951 rows=10 loops=1)
-> Nested Loop (cost=1.00..6159814.66 rows=153812 width=13) (actual time=8144.838..8144.910 rows=20 loops=1)
-> Index Scan using totoidx10 on jbpm_variableinstance stringinst1_ (cost=0.56..558481.07 rows=11199905 width=13) (actual time=0.066..2351.727 rows=2619671 loops=1)
Filter: (class_ = 'S'::bpchar)
Rows Removed by Filter: 906237
-> Index Scan using jbpm_variableinstance_pkey on jbpm_variableinstance variablein3_ (cost=0.43..0.49 rows=1 width=16) (actual time=0.002..0.002 rows=0 loops=2619671)
Index Cond: (id_ = stringinst1_.id_)
Filter: ((name_)::text = 'NIR'::text)
Rows Removed by Filter: 1
-> Index Scan using jbpm_taskinstance_pkey on jbpm_taskinstance taskinstan0_ (cost=0.29..0.34 rows=1 width=641) (actual time=0.002..0.002 rows=0 loops=20)
Index Cond: (id_ = variablein3_.taskinstance_)
Filter: (isopen_ AND ((queue)::text = 'ERT_TPS'::text))
-> Index Only Scan using idx_procin_2 on jbpm_processinstance processins2_ (cost=0.42..0.51 rows=1 width=8) (actual time=0.003..0.003 rows=1 loops=10)
Index Cond: (id_ = taskinstan0_.procinst_)
Filter: ((orgapath_)::text ~~ '/ERT%'::text)
Heap Fetches: 0
Planning time: 2.080 ms
Execution time: 8145.053 ms
Tables infos :
jbpm_variableinstance 12100592 rows
jbpm_taskinstance 69913 rows
jbpm_processinstance 97546 rows
If you have any idea, thanks
This typically only happens when OFFSET and / or LIMIT are involved (as is the case here).
The key difference is this line in the EXPLAIN output for the query with DESC:
Rows Removed by Filter: 906237
Meaning that while the first 10 rows in the index totoidx10 match when scanning backwards (which matches your ASC ordering, obviously), Postgres has to filter ~ 900k rows before it finally finds qualifying rows when scanning the same index forward.
A matching multicolumn index (with the right sort order) might help a lot.
Or, since Postgres chooses an unfavorable query plan, maybe just updated (or more detailed) table statistics or cost settings.
Related:
Keep PostgreSQL from sometimes choosing a bad query plan
Optimizing queries on a range of timestamps (two columns)