Postgres - ranking with additional conditions, then pulling the top 3 results - postgresql

Question

I want to combine 2 conditions and get the top 3 rank result.
My dataset has 2 components, i.e. 1 major group and with sub-groups mapping to each major group
For e.g. i take part of the dataset for Group A
Group | Sub-group | Revenue | Source
---------------------------------------
A | A-1 | 50 | Y
A | A-2 | 40 | Y
A | A-3 | 60 | Y
A | A-4 | 80 | Y
A | A-5 | 100 | Y
A | A-6 | 140 | X
A | A-7 | 20 | X
A | A-8 | 300 | X
And under Revenue, there are different sources, such as revenue from source X and source Y. I am only interested at source Y.
So I want to pull get a list of eligible results, with top 3 revenue > rank them. A separate step to pull data of Group A for sub-groups that have revenue from source Y. My steps:
WITH
sum_revenue AS (
SELECT DISTINCT
group,
subgroup,
SUM(revenue) AS total_rev
FROM table t
GROUP BY 1, 2
),
subgroup_rev AS (
SELECT DISTINCT
group,
total_rev,
ROW_NUMBER() OVER (
PARTITION BY group ORDER BY total_rev DESC
)
AS row
FROM sum_revenue
GROUP BY 1, 2
),
source_rev AS (
SELECT DISTINCT
group,
subgroup,
SUM(revenue) AS total_sourceY_rev
FROM table t
WHERE
subgroup = 'Y'
GROUP BY 1, 2
),
eligible AS (
SELECT DISTINCT
group,
subgroup,
total_revenue,
FROM source_rev
WHERE total_sourceY_rev < 1 --identify top 3 revenue with source Y rev < $1
),
agg_list AS (
SELECT DISTINCT
group,
STRING_AGG(DISTINCT eligible.subgroup) AS eligible_list
INNER JOIN eligible USING (group)
WHERE subgroup_rev.row <= 3
GROUP BY 1
)
SELECT DISTINCT
group,
eligible_list
FROM agg_list
WHERE group IN ('A')
I'm expecting to get aggregated list of top 3 revenue of subgroup which fulfils the condition with source Y revenue <$1.
)
But I am getting full list (or partial list aggregated (not top 3), my result gave me 7. I tried running without aggregate and it did return 7 individual rows as well.
what could have gone wrong ? I thought i have filtered with row <=3, and I also tried with INNER JOIN when i thought i might introduced redundant subgroup if using LEFT JOIN.
seems to be issue with joining with eligible alias, because when i break down to test with pulling directly from subgroup_rev without joining eligible, i can get the top 3 revenue subgroups.
However, I need the condition with the subgroups having source Y revenue, that are having highest 3, so ideally i shouldn't hardcode row as 3 (because some cases the some of the top 3s might not necessarily have source Y revenue.
[EDIT] based on feedback
Result:
Group | eligible_list
---------------------------
A | A-1,A-2,A-3,A-4,A-5
Expected result:
Group | eligible_list
---------------------------
A | A-3,A-4,A-5

Answer 1

It may be your subgroup_rev subquery.
Since you are using a OVER with a PARTITION as well as a GROUP BY clause, you may just end of creating groups for each combination of group and subgroup. I believe your subgroup_rev subquery can just be:
subgroup_rev AS (
SELECT DISTINCT
group,
total_rev,
ROW_NUMBER() OVER (
PARTITION BY group ORDER BY total_rev DESC
)
AS row
FROM sum_revenue)
Like #GMB said, it's hard to follow without seeing how sources play into the revenue column, but too much grouping may ruin your rankings.
You may end up with a all groups having only 1 member so all of their rankings would be 1 and thus would all be returned in by your final WHERE clause.

Answer 2

I restructured my funnel of data and finally got it work:
WITH
source_rev AS (
SELECT DISTINCT
group,
subgroup,
SUM(revenue) AS total_sourceY_rev
FROM table t
WHERE
subgroup = 'Y'
GROUP BY 1, 2
),
eligible AS (
SELECT DISTINCT
group,
subgroup,
total_revenue,
FROM source_rev
WHERE total_sourceY_rev < 1 --identify top 3 revenue with source Y rev < $1
),
sum_revenue AS (
SELECT DISTINCT
group,
subgroup,
SUM(revenue) AS total_rev
FROM table t
INNER JOIN eligible
USING (group)
GROUP BY 1, 2
),
subgroup_rev AS (
SELECT DISTINCT
group,
subgroup,
sum_revenue.total_rev,
ROW_NUMBER() OVER (
PARTITION BY group ORDER BY sum_revenue.total_rev DESC
)
AS row
FROM eligible
INNER JOIN sum_revenue
USING (group)
GROUP BY 1, 2,3
),
agg_list AS (
SELECT DISTINCT
group,
STRING_AGG(DISTINCT eligible.subgroup) AS eligible_list
WHERE subgroup_rev.row <= 3
GROUP BY 1
)
SELECT DISTINCT
group,
eligible_list
FROM agg_list
WHERE group IN ('A')
The difference is that
i first narrow the data with revenue from source Y and
find those eligible data with conditional select
then i am having an alias that pull total revenue of all source and finally INNER JOIN them together
do the ranking, so that the ranking is only being performed on the eligible data (i.e. source Y revenue < $1), and finally
aggregating them.
Result with eligible list with source Y revenue < $1:
Group | eligible_list
---------------------------
A | A-3,A-4,A-5
I note that some share feedback with me of having too many subgroups. Happy to get any other feedback for a better way to do this if there's more efficient suggestion. Thanks!