Poor performance of UNION query Redshift - amazon-redshift

Question

I have a Redshift UNION query that performs very poorly. Query goes like this:
WITH a1 AS (SELECT
revenue_month,
SUM(revenue) AS revenue,
SUM(cost1) AS cost1,
SUM(cost2) AS cost2,
SUM(cost3) AS cost3
FROM orders1
GROUP BY revenue_month),
a2 AS (SELECT
revenue_month,
SUM(revenue) AS revenue,
SUM(cost1) AS cost1,
SUM(cost2) AS cost2,
SUM(cost3) AS cost3
FROM orders2
GROUP BY revenue_month),
b1 AS (SELECT
revenue_month,
amount_type,
SUM(amount) AS amount
FROM monthly
GROUP BY revenue_month,amount_type)
SELECT 'a1' AS data_set, 'revenue' AS amount_type, a1.revenue AS amount FROM a1 UNION
SELECT 'a1' AS data_set, 'cost1' AS amount_type, a1.cost1 AS amount FROM a1 UNION
SELECT 'a1' AS data_set, 'cost2' AS amount_type, a1.cost2 AS amount FROM a1 UNION
SELECT 'a1' AS data_set, 'cost3' AS amount_type, a1.cost3 AS amount FROM a1 UNION
SELECT 'a2' AS data_set, 'revenue' AS amount_type, a2.revenue AS amount FROM a2 UNION
SELECT 'a2' AS data_set, 'cost1' AS amount_type, a2.cost1 AS amount FROM a2 UNION
SELECT 'a2' AS data_set, 'cost2' AS amount_type, a2.cost2 AS amount FROM a2 UNION
SELECT 'a2' AS data_set, 'cost3' AS amount_type, a2.cost3 AS amount FROM a2 UNION
SELECT 'b1' AS data_set, b1.amount_type, b2.amount FROM b2
The goal of the UNION part is to transform a1 and a2 to have the same result set schema as b1 and eventually have one combined data set.
The a1 and a2 subqueries, when run on its own each takes around 60 secs to complete with 6000 rows, while b1 runs for 5 secs with 500 rows. These run-times are acceptable for me, however, the "combined" query above runs for a whopping 20 mins.
I think the fetching part is what takes too much time for this query. I have tried using UNION ALL but performance did not improve that much. If I can somehow transform a1 and a2 to b1 schema without having to use UNION would be great but I haven't been able to do so.
Any help will be greatly appreciated. Thank you

Answer 1

You basically want to unpivot the a1 and a2 tables.
I would do it like this:
WITH
seq (idx) AS (
select 'revenue' UNION ALL
select 'cost1' UNION ALL
select 'cost2' UNION ALL
select 'cost3'
),
a1 AS (SELECT
revenue_month,
SUM(revenue) AS revenue,
SUM(cost1) AS cost1,
SUM(cost2) AS cost2,
SUM(cost3) AS cost3
FROM orders1
GROUP BY revenue_month),
a2 AS (SELECT
revenue_month,
SUM(revenue) AS revenue,
SUM(cost1) AS cost1,
SUM(cost2) AS cost2,
SUM(cost3) AS cost3
FROM orders2
GROUP BY revenue_month),
b1 AS (SELECT
revenue_month,
amount_type,
SUM(amount) AS amount
FROM monthly
GROUP BY revenue_month,amount_type)
SELECT
'a1' AS data_set,
seq.idx AS amount_type,
CASE seq.idx
WHEN 'revenue' THEN a1.revenue
WHEN 'cost1' THEN a1.cost1
WHEN 'cost2' THEN a1.cost2
WHEN 'cost3' THEN a1.cost3
END AS amount
FROM a1 CROSS JOIN seq
UNION ALL
SELECT
'a2' AS data_set,
seq.idx AS amount_type,
CASE seq.idx
WHEN 'revenue' THEN a1.revenue
WHEN 'cost1' THEN a1.cost1
WHEN 'cost2' THEN a1.cost2
WHEN 'cost3' THEN a1.cost3
END AS amount
FROM a2 CROSS JOIN seq
UNION ALL
SELECT
'b1' AS data_set,
b1.amount_type,
b1.amount
FROM b1

Answer 2

Thanks #botchniaque for all your help on this. Your CROSS JOIN suggestion solved this. There's something about that query pattern though that Redshift fails to read. The final query that worked for me is something like this:
WITH a1 AS (SELECT
revenue_month,
SUM(revenue) AS revenue,
SUM(cost1) AS cost1,
SUM(cost2) AS cost2,
SUM(cost3) AS cost3
FROM orders1
GROUP BY revenue_month),
a2 AS (SELECT
revenue_month,
SUM(revenue) AS revenue,
SUM(cost1) AS cost1,
SUM(cost2) AS cost2,
SUM(cost3) AS cost3
FROM orders2
GROUP BY revenue_month),
b1 AS (SELECT
revenue_month,
SUM(CASE WHEN amount_type = 'revenue' THEN amount ELSE 0 END) AS revenue,
SUM(CASE WHEN amount_type = 'cost1' THEN amount ELSE 0 END) AS cost1,
SUM(CASE WHEN amount_type = 'cost2' THEN amount ELSE 0 END) AS cost2,
SUM(CASE WHEN amount_type = 'cost3' THEN amount ELSE 0 END) AS cost3
FROM (SELECT
revenue_month,
amount_type,
SUM(amount) AS amount
FROM monthly
GROUP BY revenue_month,amount_type) AS b0
GROUP BY revenue_month)
SELECT
ab.data_set,
ab.revenue_month,
seq.amount_type,
CASE seq.amount_type
WHEN 'revenue' THEN ab.revenue
WHEN 'cost1' THEN ab.cost1
WHEN 'cost2' THEN ab.cost2
WHEN 'cost3' THEN ab.cost3
END AS amount
FROM
(SELECT a1.revenue_month, a1.revenue, a1.cost1, a1.cost2, a1.cost3 FROM a1 UNION ALL
SELECT a2.revenue_month, a2.revenue, a2.cost1, a2.cost2, a2.cost3 FROM a2 UNION ALL
SELECT b1.revenue_month, b1.revenue, b1.cost1, b1.cost2, b1.cost3 FROM b1) AS ab
CROSS JOIN (SELECT 'revenue' AS amount_type UNION ALL
SELECT 'cost1' AS amount_type UNION ALL
SELECT 'cost2' AS amount_type UNION ALL
SELECT 'cost3' AS amount_type) AS seq
Basically it pivots b1 first to have the same schema as a1 and a2. And then combines all three datasets with UNION resulting to ab. And then finally unpivots the combined dataset with CROSS JOIN