PostgreSQL array_agg order for window functions - postgresql

The answer to my question was almost here: PostgreSQL array_agg order
Except that I wanted to array_agg over a window function:
select distinct c.concept_name,
array_agg(c2.vocabulary_id||':'||c2.concept_name
order by c2.vocabulary_id, c2.concept_name)
over (partition by ca.min_levels_of_separation),
ca.min_levels_of_separation
from concept c
join concept_ancestor ca on c.concept_id = ca.descendant_concept_id
and max_levels_of_separation > 0
join concept c2 on ca.ancestor_concept_id = c2.concept_id
where
c.concept_code = '44054006'
order by min_levels_of_separation;
So, maybe this will work in some future version, but I get this error
ERROR: aggregate ORDER BY is not implemented for window functions
LINE 2: select distinct c.concept_name, array_agg(c2.vocabulary_id||...
^
I should probably be selecting from a subquery like the first answer to the quoted question above suggests. I was hoping for something as simple as the order by (in that question's second answer). Or maybe I'm just being lazy about the query and should be doing a group by instead of select distinct.
I did try putting the order by in the windowing function (over (partition by ca.min_levels_of_separation order by c2.vocabulary_id, c2.concept_name)), but I get these sort of repeated rows that way:
"Type 2 diabetes mellitus";"{"MedDRA:Diabetes mellitus"}";1
"Type 2 diabetes mellitus";"{"MedDRA:Diabetes mellitus","MedDRA:Diabetes mellitus (incl subtypes)"}";1
"Type 2 diabetes mellitus";"{"MedDRA:Diabetes mellitus","MedDRA:Diabetes mellitus (incl subtypes)","SNOMED:Diabetes mellitus"}";1
(btw: http://www.ohdsi.org/ if you happen to be curious about where I got the medical vocabulary tables)

Yes, it does look like I was being muddle-headed and didn't need the window function. This seems to work:
select c.concept_name,
array_agg(c2.vocabulary_id||':'||c2.concept_name
order by c2.vocabulary_id, c2.concept_name),
ca.min_levels_of_separation
from concept c
join concept_ancestor ca on c.concept_id = ca.descendant_concept_id
and max_levels_of_separation > 0
join concept c2 on ca.ancestor_concept_id = c2.concept_id
where c.concept_code = '44054006'
group by c.concept_name, ca.min_levels_of_separation
order by min_levels_of_separation
I won't accept my answer for a while since it just avoids the question instead of actually answering it, and someone might have something more useful to say on the matter.

like this :
select distinct c.concept_name,
array_agg(c2.vocabulary_id||':'||c2.concept_name ) over (partition by ca.min_levels_of_separation order by c2.vocabulary_id, c2.concept_name),
ca.min_levels_of_separation
from concept c
join concept_ancestor ca on c.concept_id = ca.descendant_concept_id
and max_levels_of_separation > 0
join concept c2 on ca.ancestor_concept_id = c2.concept_id
where
c.concept_code = '44054006'
order by min_levels_of_separation;

Related

SQL Server: How to customize one column in Order By clause out of many other columns in order by

I have store procedure which return data fine and it was developed by some one else who now not in touch.
Output now looks like
Here i am attaching a part of the query which return data.
SET #sql = '
Select XX.*,'''' scale,Isnull(AllowComma,''FALSE'') AllowComma,Isnull(AllowedDecimalPlace,''0'') AllowedDecimalPlace,
Isnull(AllowPercentageSign,''FALSE'') AllowPercentageSign,Isnull(CurrencySign,'''') CurrencySign,Isnull(BM_Denominator,'''') BM_Denominator
From
(
---- Broker Detail
Select AA.Section,AA.LineItem,Csm.DisplayInCSM ,AA.BrokerCode Broker,AA.BrokerName,'''' BM_Element,'''' BM_Code,AA.Ord,AA.[Revise Date],AA.LineItemId,
Csm.ID,[FontName],[FontStyle],[FontSize],[UnderLine],[BGColor],[FGColor],[Indent],[Box],[HeadingSubHeading],
'+#PeriodCols+','+#PeriodColsComment +',LineItem_Comment,BrokerName_Comment,Date_Comment
From tblCSM_ModelDetails Csm LEFT OUTER JOIN (
Select b.*,L.ID LineItemId
From #TmpAll_Broker_LI b
INNER JOIN TblLineItemTemplate L ON TickerID='''+#TickerID+''' AND b.LineItem= L.LineItem
) AA ON Csm.LineItemId=AA.LineItemId
WHERE Csm.CSM_ID='+TRIM(CONVERT(CHAR(10),#CSM_Id))+' AND Csm.BMID=0 AND Type !=''SHEET''
UNION
----- Consensus
Select Section, b.LineItem,DisplayInCSM, '''' Broker,'''' BrokerName,'''' BM_Element,'''' BM_Code, Ord,'''' [Revise Date],L.ID LineItemID,
Csm.ID,[FontName],[FontStyle],[FontSize],[UnderLine],[BGColor],[FGColor],[Indent],[Box],[HeadingSubHeading],
'+#PeriodCols+','+#PeriodColsComment +',LineItem_Comment,BrokerName_Comment,Date_Comment
From #TmpZacksCons b
INNER JOIN TblLineItemTemplate L ON TickerID='''+#TickerID+''' AND b.LineItem= L.LineItem
INNER JOIN tblCSM_ModelDetails Csm ON Csm.LineItemID=L.ID
WHERE Csm.CSM_ID='+TRIM(CONVERT(CHAR(10),#CSM_Id))+' AND Csm.BMID=0
---- Blue Metrics
UNION
Select Section, b.LineItem,DisplayInCSM,'''' Broker,'''' BrokerName,BM_Element,Code BM_Code, Ord,'''' [Revise Date],L.ID LineItemID,
Csm.ID,[FontName],[FontStyle],[FontSize],[UnderLine],[BGColor],[FGColor],[Indent],[Box],[HeadingSubHeading],
'+#PeriodCols+','+#PeriodColsComment +',LineItem_Comment,BrokerName_Comment,Date_Comment
From #TmpBM b
INNER JOIN TblLineItemTemplate L ON TickerID='''+#TickerID+''' AND b.LineItem= L.LineItem
INNER JOIN tblCSM_ModelDetails Csm ON Csm.BMID=b.code AND Csm.LineItemID=L.ID
WHERE Csm.CSM_ID='+TRIM(CONVERT(CHAR(10),#CSM_Id))+'
AND Ord IS NOT NULL
) XX
Left Outer Join tblLiConfig ZZ
On XX.Section=ZZ.Section And XX.LineItem=ZZ.LI And ZZ.Ticker='''+#Ticker+'''
Order by ID,Ord,BM_Code,LineItem,BrokerName'
Now broker Name is not coming as alphabetical order and it is the issue.
see this line at the bottom Order by ID,Ord,BM_Code,LineItem,BrokerName
When i try to change this order by like Order by ID,Ord,BM_Code,LineItem,BrokerName IN (SELECT BrokerName FROM #Brokers ORDER BY BrokerName ASC)' then getting error like clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP, OFFSET or FOR XML is also specified.
in my order by there are many columns and data is getting order by that way but i need to show broker name in alphabetical order but i am not being able. so please some one guide me how can i customize this sql.
Here i have not attached my full store procedure code because it is very large. looking for suggestion & help. Thanks
Short version
The ORDER BY is doing what is expected - ordering first by ID, then Ord, then BM_Code, then LineItem, then BrokerName.
Within ID 76187, the next field to order by is Ord - which it sorts from 30911, to 31097.
If it previously ordered by BrokerName, it was only by chance - or that Ord was ordered the same way as BrokerName.
My initial suggestion is to re-order your sort e.g., ORDER BY ID, BM_Code, LineItem, BrokerName, Ord
Longer explanation of issue
In SQL, underlying data is treated as a set and ordering doesn't matter.
For example, if you have a variable #x and you were testing IF #x IN (1,2,3,4,5) will produce the same result as IF #x in (5,4,3,2,1).
In your example, you're putting an ORDER BY into the sub-query you're checking with the IN e.g., ORDER BY ... BrokerName IN (SELECT BrokerName FROM #Brokers ORDER BY BrokerName ASC). The order of that sub-query isn't allowed, and wouldn't do anything anyway.
The only sort that matters (other than for a few things like TOP) is the final sort - when displaying the data.
That being said, even if you removed the ORDER BY in the sub-query, it wouldn't help you with your issue
The SQL is not likely to work anyway - ORDER BY needs a value - you may have needed to make it CASE WHEN BrokerName IN (...) THEN 0 ELSE 1 END
Which also won't help, as the issue is that Ord is sorted before BrokerName anyway.
UPDATE following comment
Fundamentally, the statement that provides the actual report is
SET #sql = '
Select XX.*,'''' scale,Isnull(AllowComma,''FALSE'') AllowComma,Isnull(AllowedDecimalPlace,''0'') AllowedDecimalPlace,
Isnull(AllowPercentageSign,''FALSE'') AllowPercentageSign,Isnull(CurrencySign,'''') CurrencySign,Isnull(BM_Denominator,'''') BM_Denominator
From (<a whole lot of calculations/cpode>) XX
Left Outer Join tblLiConfig ZZ
On XX.Section=ZZ.Section And XX.LineItem=ZZ.LI And ZZ.Ticker='''+#Ticker+'''
Order by ID,Ord,BM_Code,LineItem,BrokerName'
The last line on there provides the ordering of the data coming from this procedure.
To get a different order, you need to change the order of the fields shown - moving BrokerName more towards the start of the list, and Ord towards the end.
e.g.,
SET #sql = '
Select XX.*,'''' scale,Isnull(AllowComma,''FALSE'') AllowComma,Isnull(AllowedDecimalPlace,''0'') AllowedDecimalPlace,
Isnull(AllowPercentageSign,''FALSE'') AllowPercentageSign,Isnull(CurrencySign,'''') CurrencySign,Isnull(BM_Denominator,'''') BM_Denominator
From (<a whole lot of calculations/cpode>) XX
Left Outer Join tblLiConfig ZZ
On XX.Section=ZZ.Section And XX.LineItem=ZZ.LI And ZZ.Ticker='''+#Ticker+'''
Order by ID,BrokerName,BM_Code,LineItem,Ord'
The above probably sorts by BrokerName too early - but it's up to you to determine what you need.

Cannot get a result by Max date

I'm trying to get the highlighted result only as it's the latest date. First time I've asked a question here so I apologize in advance if this isn't clear. Thanks
By using the following query
SELECT
MAX(A.Insp_Date) AS Last_Insp_Date
,A.Doc_ID
,A.Service_Call_ID
,A.Customer_ID
,A.Address_Code
,A.State
,A.Branch
,B.HydLoc
,B.FlwOutSz
,B.StaticPSI
,B.ResidualPSI
,B.PititPSI
,B.FlwGPM
FROM [dbo].[fofHydrntInspHdr] AS A
LEFT OUTER JOIN [dbo].[fofHYD2800FlwTstRT] AS B
ON A.Doc_ID = B.Doc_ID
WHERE A.Doc_ID > 0
AND A.Address_Code = 'GEN0021'
GROUP BY
A.Doc_ID
,A.Service_Call_ID
,A.Customer_ID
,A.Address_Code
,A.State
,A.Branch
,B.HydLoc
,B.FlwOutSz
,B.StaticPSI
,B.ResidualPSI
,B.PititPSI
,B.FlwGPM
I've also tried using max doc_id and it still doesn't work. Appreciate any help.
Another option that shouldn't require two scans of your table is to filter for the latest using a window function:
with r as
(
SELECT
A.Insp_Date AS Last_Insp_Date
,A.Doc_ID
,A.Service_Call_ID
,A.Customer_ID
,A.Address_Code
,A.State
,A.Branch
,B.HydLoc
,B.FlwOutSz
,B.StaticPSI
,B.ResidualPSI
,B.PititPSI
,B.FlwGPM
,DENSE_RANK() OVER (ORDER BY A.Insp_Date DESC) AS r
FROM [dbo].[fofHydrntInspHdr] AS A
LEFT OUTER JOIN [dbo].[fofHYD2800FlwTstRT] AS B
ON A.Doc_ID = B.Doc_ID
WHERE A.Doc_ID > 0
AND A.Address_Code = 'GEN0021'
)
SELECT
Insp_Date AS Last_Insp_Date
,Doc_ID
,Service_Call_ID
,Customer_ID
,Address_Code
,State
,Branch
,HydLoc
,FlwOutSz
,StaticPSI
,ResidualPSI
,PititPSI
,FlwGPM
FROM r
WHERE r = 1;
As an aside, I would advise against aliasing your tables with A, B, C etc as they don't relate to the table and make understanding the query later on more awkward. In this case, aliases like h and ft would convey that one table is the Headers and the other the Flow Tests, whilst also reducing character count.
It also looks like you have some bad duplication going on in your results there, which suggests that either your query is not joining and filtering appropriately or your data is messy.

How to use GROUP BY with Firebird?

I'm trying create a SELECT with GROUP BY in Firebird but I can't have any success. How could I do this ?
Exception
Can't format message 13:896 -- message file C:\firebird.msg not found.
Dynamic SQL Error.
SQL error code = -104.
Invalid expression in the select list (not contained in either an aggregate function or the GROUP BY clause).
(49,765 sec)
trying
SELECT FA_DATA, FA_CODALUNO, FA_MATERIA, FA_TURMA, FA_QTDFALTA,
ALU_CODIGO, ALU_NOME,
M_CODIGO, M_DESCRICAO,
FT_CODIGO, FT_ANOLETIVO, FT_TURMA
FROM FALTAS Falta
INNER JOIN ALUNOS Aluno ON (Falta.FA_CODALUNO = Aluno.ALU_CODIGO)
INNER JOIN MATERIAS Materia ON (Falta.FA_MATERIA = Materia.M_CODIGO)
INNER JOIN FORMACAOTURMAS Turma ON (Falta.FA_TURMA = Turma.FT_CODIGO)
WHERE (Falta.FA_CODALUNO = 238) AND (Turma.FT_ANOLETIVO = 2015)
GROUP BY Materia.M_CODIGO
Simple use of group by in firebird,group by all columns
select * from T1 t
where t.id in
(SELECT t.id FROM T1 t
INNER JOIN T2 j ON j.id = t.jid
WHERE t.id = 1
GROUP BY t.id)
Using GROUP BY doesn't make sense in your example code. It is only useful when using aggregate functions (+ some other minor uses). In any case, Firebird requires you to specify all columns from the SELECT column list except those with aggregate functions in the GROUP BY clause.
Note that this is more restrictive than the SQL standard, which allows you to leave out functionally dependent columns (ie if you specify a primary key or unique key, you don't need to specify the other columns of that table).
You don't specify why you want to group (because it doesn't make much sense to do it with this query). Maybe instead you want to ORDER BY, or you want the first row for each M_CODIGO.

Greatest N per group in Open SQL

Selecting the rows from a table by (partial) key with the maximum value in a particular column is a common task in SQL. This question has some excellent answers that cover a variety of approaches to it. Unfortunately I'm struggling to replicate this in my ABAP program.
None of the commonly used approaches seem to be supported:
Joining on a subquery is not supported in syntax: SELECT * FROM X as x INNER JOIN ( SELECT ... ) AS y
Using IN for a composite key is not supported in syntax as far as I know: SELECT * FROM X WHERE (key1, key2) IN ( SELECT key1 key2 FROM ... )
Left join to itself with smaller-than comparison is not supported, outer joins only support EQ comparisons: SELECT * FROM X AS x LEFT JOIN X as xmax ON x-key1 = xmax-key1 AND x-key2 < xmax-key2 WHERE xmax-key IS INITIAL
After trying each of these solutions in turn only to discover that ABAP doesn't seem to support them and being unable to find any equivalents I'm starting to think that I'll have no choice but to dump the data of the subquery to an itab.
What is the best practice for this common programming requirement in ABAP development?
First of all, specific requirement, would give you a better answer. As it happens I bumped into this question when working on a program, that uses 3 distinct methods of pseudo-grouping, (while looking for alternatives) and ALL 3 can be used to answer your question, depending on what exactly you need to do. I'm sure there are more ways to do it.
For instance, you can pull maximum values within a group by simply selecting max( your_field ) and grouping by some fields, if that's all you need.
select bname, nation, max( date_from ) from adrp group by bname, nation. "selects highest "from" date for each bname
If you need to use that max value as a filter condition within a query, you can do it by performing pseudo-grouping using sub-query and max within sub-query like this (notice how I move out the BNAME check into sub query, which means I don't have to check both fields using in (subquery) addition):
select ... from adrp as b_adrp "Pulls the latest person info for a user (some conditions are missing, but this is a part of an actual query)
where b_adrp~date_from in (
select max( date_from ) "Highest date_from where both dates are valid
from adrp where persnumber = b_adrp~persnumber and nation = b_adrp~nation and date_from <= #sy-datum )
The query above allows you to select selects all user info from base query and (where the first one only allows to take aggregated and grouped data).
Finally, If you need to check based on composite key and compare it to multiple agregate function results, the implementation will heavily depend on specifics of your requirement (and since your question has none, I'll provide a generic one). Easiest option is to use exists / not exists instead of in (subquery), in exact same way and form the subquery to check for existance of specific key or condition rather than pull a list ( you can nest subqueries if you have to ):
select * from bkpf where exists ( select 1 from bkpf as b where belnr = bkpf~belnr and gjahr = bkpf~gjahr group by belnr, gjahr having max( budat ) = bkpf~budat ) "Took an available example, that I had in testing program.
All 3 queries will get you max value of a column within a group and in fact, all 3 can use joins to achieve identical results.
please find my answers below your questions.
Joining on a subquery is not supported in syntax: SELECT * FROM X as x INNER JOIN ( SELECT ... ) AS y
Putting the subquery in your where condition should do the work SELECT * FROM X AS x INNER JOIN Y AS y ON x~a = y~b WHERE ( SELECT * FROM y WHERE ... )
Using IN for a composite key is not supported in syntax as far as I know: SELECT * FROM X WHERE (key1, key2) IN ( SELECT key1 key2 FROM ... )
You have to split your WHERE clause: SELECT * FROM X WHERE key1 IN ( SELECT key1 FROM y ) AND key2 IN ( SELECT key2 FROM y )
Left join to itself with smaller-than comparison is not supported, outer joins only support EQ comparisons.
Yes, thats right at the moment.
Left join to itself with smaller-than comparison is not supported, outer joins only support EQ comparisons:
SELECT * FROM X AS x LEFT JOIN X as xmax ON x-key1 = xmax-key1 AND x-key2 < xmax-key2 WHERE xmax-key IS INITIAL
This is not true. This SELECT is perfectly valid:
SELECT b1~budat
INTO TABLE lt_bkpf
FROM bkpf AS b1
LEFT JOIN bkpf AS b2
ON b2~belnr < b1~belnr
WHERE b1~bukrs <> ''.
And was valid at least since 7.40 SP08, since July 2013, so at the time you asked this question it was valid as well.

Combine SUM and CAST - not working?

PostgreSQL Unicode 9.01 doesn't like:
SELECT table1.fielda,
SUM (CAST (table2.fielda AS INT)) AS header.specific
FROM *etc*
What is wrong with SUM-CAST?
Error Message:
Incorrect column expression: 'SUM (CAST
(specifics_nfl_3pl_work_order_item.delivery_quantity AS INT))
Query:
SELECT specifics_nfl_3pl_work_order.work_order_number,
specifics_nfl_3pl_work_order.goods_issue_date,
specifics_nfl_3pl_work_order.order_status_id,
SUM (CAST (specifics_nfl_3pl_work_order_item.delivery_quantity AS INT)) AS units
FROM public.specifics_nfl_3pl_work_order specifics_nfl_3pl_work_order,
public.specifics_nfl_3pl_work_order_item specifics_nfl_3pl_work_order_item,
public.specifics_nfl_order_status specifics_nfl_order_status
WHERE specifics_nfl_3pl_work_order.order_status_id In (3,17,14)
AND specifics_nfl_3pl_work_order_item.specifics_nfl_work_order_id=
specifics_nfl_3pl_work_order.id
AND ((specifics_nfl_3pl_work_order.sold_to_id<>'0000000000')
AND (specifics_nfl_3pl_work_order.goods_issue_date>={d '2013-08-01'}))
It would be really great if you can help.
If I were you, then I would do these steps:
give your table short aliases
format the query
use proper ANSI joins:
remove spaces between function name and (
select
o.work_order_number,
o.goods_issue_date,
o.order_status_id,
sum(cast(oi.delivery_quantity as int)) as units
from public.specifics_nfl_3pl_work_order as o
inner join public.specifics_nfl_3pl_work_order_item as oi on
oi.specifics_nfl_work_order_id = o.id
-- inner join public.specifics_nfl_order_status os -- seems redundant
where
o.order_status_id In (3,17,14) and
o.sold_to_id <> '0000000000' and
o.goods_issue_date >= {d '2013-08-01'}
Actually I really think you need group by clause here:
select
o.work_order_number,
o.goods_issue_date,
o.order_status_id,
sum(cast(oi.delivery_quantity as int)) as units
from public.specifics_nfl_3pl_work_order as o
inner join public.specifics_nfl_3pl_work_order_item as oi on
oi.specifics_nfl_work_order_id = o.id
where
o.order_status_id In (3,17,14) and
o.sold_to_id <> '0000000000' and
o.goods_issue_date >= {d '2013-08-01'}
group by
o.work_order_number,
o.goods_issue_date,
o.order_status_id
if it still doesn't work - try to comment sum and see is it working?
But you have a table2 or only table1?
Try:
SELECT table1.fielda,
SUM (CAST (table1.fielda AS INT)) AS "header.specific"
FROM etc
In addition to what #Roman already cleared up, there are more problems here:
SELECT o.work_order_number
,o.goods_issue_date
,o.order_status_id
,SUM(CAST(oi.delivery_quantity AS INT)) AS units -- suspicious
FROM public.specifics_nfl_3pl_work_order o,
JOIN public.specifics_nfl_3pl_work_order_item oi
ON oi.specifics_nfl_work_order_id = o.id
CROSS JOIN public.specifics_nfl_order_status os -- probably wrong
WHERE o.order_status_id IN (3,17,14)
AND o.sold_to_id <> '0000000000' -- suspicious
AND o.goods_issue_date> = {d '2013-08-01'} -- nonsense
GROUP BY 1, 2, 3
o.goods_issue_date> = {d '2013-08-01'} is syntactical nonsense. Maybe you mean:
o.goods_issue_date> = '2013-08-01'
You have the table specifics_nfl_order_status in your FROM list, but without any expression connecting it to the rest. This effectively results in a CROSS JOIN, which results in a Cartesian product and is almost certainly wrong in a very expensive way: every row is combined with every row of the rest:
CROSS JOIN public.specifics_nfl_order_status os
Either remove the table (since you don't use it) or add a WHERE or ON clause to connect it to the rest. Note, that it is not just redundant, it has a dramatic effect on the result as it is.
This WHERE clause is suspicious:
AND o.sold_to_id <> '0000000000'
Seems like you are storing numbers as strings or otherwise confusing the two.
Also, CAST (oi.delivery_quantity AS INT) should not be needed to begin with. The column should be of data type integer or some other appropriate numeric type to begin with. Be sure to use proper data types.
The default setting of search_path includes public, and you may not need to schema-qualify tables. Instead of public.specifics_nfl_3pl_work_order, it may suffice to use:
specifics_nfl_3pl_work_order
GROUP BY 1, 2, 3 is using positional parameters, just a notational shortcut for:
GROUP BY o.work_order_number, o.goods_issue_date, o.order_status_id
Details in the manual.
According to comments you are using MS Query to create the query. This is not the best of ideas. Produces the kind of inferior code you presented us with. You may want to get rid of that while you are working with PostgreSQL.