Very slow execution under two conditions like in the query - postgresql

Question

Plan query:
Finalize Aggregate (cost=20465.85..20465.86 rows=1 width=8) (actual time=21363.036..21363.037 rows=1 loops=1)
-> Gather (cost=20465.63..20465.84 rows=2 width=8) (actual time=21363.014..21363.108 rows=3 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (cost=19465.63..19465.64 rows=1 width=8) (actual time=19548.388..19548.388 rows=1 loops=3)
-> Nested Loop (cost=10766.23..19465.62 rows=6 width=0) (actual time=7204.795..19548.383 rows=1 loops=3)
-> Hash Join (cost=10765.81..15255.58 rows=1142 width=16) (actual time=115.799..19505.353 rows=3937 loops=3)
Hash Cond: (exc.packageid = ex.id)
Join Filter: (ex1.notificationcontent ~~* (('%'::text || (CASE WHEN (exc.packageuid IS NOT NULL) THEN exc.packageuid ELSE ex.packageuid END)::text) || '%'::text))
Rows Removed by Join Filter: 1044143
-> Merge Join (cost=9575.89..13397.74 rows=254444 width=496) (actual time=104.064..456.209 rows=1048080 loops=3)
Merge Cond: (ex1.correspondent_uid = exc.correspondentuid)
-> Sort (cost=8011.08..8014.32 rows=1295 width=492) (actual time=96.345..97.537 rows=4011 loops=3)
Sort Key: ex1.correspondent_uid
Sort Method: quicksort Memory: 5797kB
-> Parallel Seq Scan on ex_in ex1 (cost=0.00..7944.14 rows=1295 width=492) (actual time=0.082..90.635 rows=4011 loops=3)
Filter: ((doctypeid = ANY ('{201,140005}'::integer[])) AND (notificationcontent ~~* '%success%'::text))
Rows Removed by Filter: 12263
-> Sort (cost=1564.81..1596.49 rows=12675 width=36) (actual time=7.714..132.269 rows=1048164 loops=3)
Sort Key: exc.correspondentuid
Sort Method: quicksort Memory: 1376kB
-> Seq Scan on ex_out_correspondents exc (cost=0.00..701.03 rows=12675 width=36) (actual time=0.032..4.151 rows=12694 loops=3)
Filter: (exchangestate = 2)
Rows Removed by Filter: 20268
-> Hash (cost=884.00..884.00 rows=24473 width=36) (actual time=11.028..11.028 rows=24489 loops=3)
Buckets: 32768 Batches: 1 Memory Usage: 1874kB
-> Seq Scan on ex_out ex (cost=0.00..884.00 rows=24473 width=36) (actual time=0.025..6.597 rows=24489 loops=3)
Filter: ((doctypeid < 200) AND (exchangetype = 7))
Rows Removed by Filter: 2780
-> Index Scan using ipk__ds_documents__73ba3083 on ds_documents d (cost=0.42..3.69 rows=1 width=16) (actual time=0.010..0.010 rows=0 loops=11811)
Index Cond: (uid = ex.docuid)
Filter: ((NOT pruz) AND ((reg_date)::date >= '2019-02-01'::date) AND ((reg_date)::date <= '2019-02-28'::date))
Rows Removed by Filter: 1
Planning time: 4.169 ms
Execution time: 21363.604 ms
If I remove one of the like conditions, then the sample is constructed in 100 milliseconds, versus 30 seconds, if I leave both conditions.
EXPLAIN(ANALYZE)
SELECT count(*)
FROM DS_DOCUMENTS d
INNER JOIN EX_Out ex ON d.UID = ex.DocUID and ExchangeType IN (7) and DocTypeID < 200 and d.pruz = false
INNER JOIN EX_Out_Correspondents exc ON ex.ID = exc.PackageID and exc.ExchangeState = 2
INNER JOIN Ex_in ex1 ON ex1.Correspondent_UID = exc.CorrespondentUID
and ex1.DocTypeID in (201, 140005)
and ex1.NotificationContent ilike '%' || (case when exc.PackageUID is not null then exc.PackageUID else ex.PackageUID end)|| '%'
and ex1.NotificationContent ilike '%success%'
WHERE cast(reg_date as date) between '01.02.2019' and '28.02.2019'
What am I doing wrong?
The problem is in this condition, but I do not know why.
and ex1.NotificationContent ilike '%' || (case when exc.PackageUID is not null then exc.PackageUID else ex.PackageUID end)|| '%'
and ex1.NotificationContent ilike '%success%'

Answer 1

Here is an example of building the query piece-wise with the idea of trimming the working set prior to adding the expensive JOIN condition (which if possible should be optimized). I tried to handle the CASE condition prior to the JOIN also. Is the 'success' string located in a common place in ex1.NotificationContent? If so, apply that in the first condition.
WITH d AS
(
SELECT *
FROM DS_DOCUMENTS d
WHERE reg_date::date BETWEEN '01.02.2019' AND '28.02.2019'
), ex_exc AS
(
SELECT d.* -- you may choose to select only the required fields in place of *
, ex.*
, exc.*
, (CASE WHEN exc.PackageUID IS NOT NULL THEN exc.PackageUID ELSE ex.PackageUID END) AS join_package_uid
FROM d
JOIN EX_Out ex ON (d.UID = ex.DocUID
AND ExchangeType = 7
AND DocTypeID < 200
AND d.pruz = FALSE
)
JOIN EX_Out_Correspondents exc ON (ex.ID = exc.PackageID
AND exc.ExchangeState = 2
)
)
SELECT count(*)
FROM ex_exc
JOIN Ex_in ex1 ON (ex1.Correspondent_UID = exc.CorrespondentUID
AND ex1.DocTypeID IN (201, 140005)
AND ex1.NotificationContent ILIKE '%' || ex_exc.join_package_uid || '%'
AND ex1.NotificationContent ILIKE '%success%'
)
;

Answer 2

From your execution plan it seems like the first of these conditions is your main problem:
-> Hash Join (cost=10765.81..15255.58 rows=1142 width=16) (actual time=115.799..19505.353 rows=3937 loops=3)
Hash Cond: (exc.packageid = ex.id)
Join Filter: (ex1.notificationcontent ~~* (('%'::text || (CASE WHEN (exc.packageuid IS NOT NULL) THEN exc.packageuid ELSE ex.packageuid END)::text) || '%'::text))
Rows Removed by Join Filter: 1044143
All your time is spent there, and it must be the 1000000 executions of the expression match that take so long.
Maybe you can come up with a cheaper test. Even if you can find an additional cheaper condition that reduces the number of rows for the expensive test, you would win.