I have store procedure which return data fine and it was developed by some one else who now not in touch.
Output now looks like
Here i am attaching a part of the query which return data.
SET #sql = '
Select XX.*,'''' scale,Isnull(AllowComma,''FALSE'') AllowComma,Isnull(AllowedDecimalPlace,''0'') AllowedDecimalPlace,
Isnull(AllowPercentageSign,''FALSE'') AllowPercentageSign,Isnull(CurrencySign,'''') CurrencySign,Isnull(BM_Denominator,'''') BM_Denominator
From
(
---- Broker Detail
Select AA.Section,AA.LineItem,Csm.DisplayInCSM ,AA.BrokerCode Broker,AA.BrokerName,'''' BM_Element,'''' BM_Code,AA.Ord,AA.[Revise Date],AA.LineItemId,
Csm.ID,[FontName],[FontStyle],[FontSize],[UnderLine],[BGColor],[FGColor],[Indent],[Box],[HeadingSubHeading],
'+#PeriodCols+','+#PeriodColsComment +',LineItem_Comment,BrokerName_Comment,Date_Comment
From tblCSM_ModelDetails Csm LEFT OUTER JOIN (
Select b.*,L.ID LineItemId
From #TmpAll_Broker_LI b
INNER JOIN TblLineItemTemplate L ON TickerID='''+#TickerID+''' AND b.LineItem= L.LineItem
) AA ON Csm.LineItemId=AA.LineItemId
WHERE Csm.CSM_ID='+TRIM(CONVERT(CHAR(10),#CSM_Id))+' AND Csm.BMID=0 AND Type !=''SHEET''
UNION
----- Consensus
Select Section, b.LineItem,DisplayInCSM, '''' Broker,'''' BrokerName,'''' BM_Element,'''' BM_Code, Ord,'''' [Revise Date],L.ID LineItemID,
Csm.ID,[FontName],[FontStyle],[FontSize],[UnderLine],[BGColor],[FGColor],[Indent],[Box],[HeadingSubHeading],
'+#PeriodCols+','+#PeriodColsComment +',LineItem_Comment,BrokerName_Comment,Date_Comment
From #TmpZacksCons b
INNER JOIN TblLineItemTemplate L ON TickerID='''+#TickerID+''' AND b.LineItem= L.LineItem
INNER JOIN tblCSM_ModelDetails Csm ON Csm.LineItemID=L.ID
WHERE Csm.CSM_ID='+TRIM(CONVERT(CHAR(10),#CSM_Id))+' AND Csm.BMID=0
---- Blue Metrics
UNION
Select Section, b.LineItem,DisplayInCSM,'''' Broker,'''' BrokerName,BM_Element,Code BM_Code, Ord,'''' [Revise Date],L.ID LineItemID,
Csm.ID,[FontName],[FontStyle],[FontSize],[UnderLine],[BGColor],[FGColor],[Indent],[Box],[HeadingSubHeading],
'+#PeriodCols+','+#PeriodColsComment +',LineItem_Comment,BrokerName_Comment,Date_Comment
From #TmpBM b
INNER JOIN TblLineItemTemplate L ON TickerID='''+#TickerID+''' AND b.LineItem= L.LineItem
INNER JOIN tblCSM_ModelDetails Csm ON Csm.BMID=b.code AND Csm.LineItemID=L.ID
WHERE Csm.CSM_ID='+TRIM(CONVERT(CHAR(10),#CSM_Id))+'
AND Ord IS NOT NULL
) XX
Left Outer Join tblLiConfig ZZ
On XX.Section=ZZ.Section And XX.LineItem=ZZ.LI And ZZ.Ticker='''+#Ticker+'''
Order by ID,Ord,BM_Code,LineItem,BrokerName'
Now broker Name is not coming as alphabetical order and it is the issue.
see this line at the bottom Order by ID,Ord,BM_Code,LineItem,BrokerName
When i try to change this order by like Order by ID,Ord,BM_Code,LineItem,BrokerName IN (SELECT BrokerName FROM #Brokers ORDER BY BrokerName ASC)' then getting error like clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions, unless TOP, OFFSET or FOR XML is also specified.
in my order by there are many columns and data is getting order by that way but i need to show broker name in alphabetical order but i am not being able. so please some one guide me how can i customize this sql.
Here i have not attached my full store procedure code because it is very large. looking for suggestion & help. Thanks
Short version
The ORDER BY is doing what is expected - ordering first by ID, then Ord, then BM_Code, then LineItem, then BrokerName.
Within ID 76187, the next field to order by is Ord - which it sorts from 30911, to 31097.
If it previously ordered by BrokerName, it was only by chance - or that Ord was ordered the same way as BrokerName.
My initial suggestion is to re-order your sort e.g., ORDER BY ID, BM_Code, LineItem, BrokerName, Ord
Longer explanation of issue
In SQL, underlying data is treated as a set and ordering doesn't matter.
For example, if you have a variable #x and you were testing IF #x IN (1,2,3,4,5) will produce the same result as IF #x in (5,4,3,2,1).
In your example, you're putting an ORDER BY into the sub-query you're checking with the IN e.g., ORDER BY ... BrokerName IN (SELECT BrokerName FROM #Brokers ORDER BY BrokerName ASC). The order of that sub-query isn't allowed, and wouldn't do anything anyway.
The only sort that matters (other than for a few things like TOP) is the final sort - when displaying the data.
That being said, even if you removed the ORDER BY in the sub-query, it wouldn't help you with your issue
The SQL is not likely to work anyway - ORDER BY needs a value - you may have needed to make it CASE WHEN BrokerName IN (...) THEN 0 ELSE 1 END
Which also won't help, as the issue is that Ord is sorted before BrokerName anyway.
UPDATE following comment
Fundamentally, the statement that provides the actual report is
SET #sql = '
Select XX.*,'''' scale,Isnull(AllowComma,''FALSE'') AllowComma,Isnull(AllowedDecimalPlace,''0'') AllowedDecimalPlace,
Isnull(AllowPercentageSign,''FALSE'') AllowPercentageSign,Isnull(CurrencySign,'''') CurrencySign,Isnull(BM_Denominator,'''') BM_Denominator
From (<a whole lot of calculations/cpode>) XX
Left Outer Join tblLiConfig ZZ
On XX.Section=ZZ.Section And XX.LineItem=ZZ.LI And ZZ.Ticker='''+#Ticker+'''
Order by ID,Ord,BM_Code,LineItem,BrokerName'
The last line on there provides the ordering of the data coming from this procedure.
To get a different order, you need to change the order of the fields shown - moving BrokerName more towards the start of the list, and Ord towards the end.
e.g.,
SET #sql = '
Select XX.*,'''' scale,Isnull(AllowComma,''FALSE'') AllowComma,Isnull(AllowedDecimalPlace,''0'') AllowedDecimalPlace,
Isnull(AllowPercentageSign,''FALSE'') AllowPercentageSign,Isnull(CurrencySign,'''') CurrencySign,Isnull(BM_Denominator,'''') BM_Denominator
From (<a whole lot of calculations/cpode>) XX
Left Outer Join tblLiConfig ZZ
On XX.Section=ZZ.Section And XX.LineItem=ZZ.LI And ZZ.Ticker='''+#Ticker+'''
Order by ID,BrokerName,BM_Code,LineItem,Ord'
The above probably sorts by BrokerName too early - but it's up to you to determine what you need.
I'm working to convert a known working SQL query to work in pyspark, given two dataframes, using methods such as: .join, .where, filter, etc.
Here are examples of SQL queries that work (only selecting r.id where I will normally select more columns):
# "invalid" records, where there is a matching `record_id` for rv_df
SELECT DISTINCT(r.id) FROM core_record AS r LEFT OUTER JOIN core_recordvalidation rv ON r.id = rv.record_id WHERE r.job_id = 41 AND rv.record_id is not null;
# "valid" records, where there is no matching `record_id` for rv_df
SELECT DISTINCT(r.id) FROM core_record AS r LEFT OUTER JOIN core_recordvalidation rv ON r.id = rv.record_id WHERE r.job_id = 41 AND rv.record_id is not null;
I'm 80/20 close, but having trouble wrapping my head around the the last few steps, and/or how to do this most efficiently.
I've got a Dataframe r_df with column id that I'd like to join with Dataframe rv_df on column record_id. As output, I'd like only distinct r.id, and only columns from r_df, none from rv_df. Finally, I'd like two different calls where there is a match (what will be "invalid" records for me), and where there is not a match (what I consider "valid" records).
I have pyspark queries that get close, but not terribly clear on how to ensure that r_df.id is distinct, and select only columns from r_df, none from rv_df.
Any help would be much appreciated!
Just had to walk away for a couple hours. Found a solution that works for my use case.
First, selecting only distinct record_id from rv_df:
rv_df = rv_df.select('record_id').distinct()
Then use that for intersection and disjoints:
# Intersection:
j_df = r_df.join(rv_df, r_df.id == rv_df.record_id, 'leftsemi').select(r_df['*'])
# Disjoint:
j_df = r_df.join(rv_df, r_df.id == rv_df.record_id, 'leftanti').select(r_df['*'])
The answer to my question was almost here: PostgreSQL array_agg order
Except that I wanted to array_agg over a window function:
select distinct c.concept_name,
array_agg(c2.vocabulary_id||':'||c2.concept_name
order by c2.vocabulary_id, c2.concept_name)
over (partition by ca.min_levels_of_separation),
ca.min_levels_of_separation
from concept c
join concept_ancestor ca on c.concept_id = ca.descendant_concept_id
and max_levels_of_separation > 0
join concept c2 on ca.ancestor_concept_id = c2.concept_id
where
c.concept_code = '44054006'
order by min_levels_of_separation;
So, maybe this will work in some future version, but I get this error
ERROR: aggregate ORDER BY is not implemented for window functions
LINE 2: select distinct c.concept_name, array_agg(c2.vocabulary_id||...
^
I should probably be selecting from a subquery like the first answer to the quoted question above suggests. I was hoping for something as simple as the order by (in that question's second answer). Or maybe I'm just being lazy about the query and should be doing a group by instead of select distinct.
I did try putting the order by in the windowing function (over (partition by ca.min_levels_of_separation order by c2.vocabulary_id, c2.concept_name)), but I get these sort of repeated rows that way:
"Type 2 diabetes mellitus";"{"MedDRA:Diabetes mellitus"}";1
"Type 2 diabetes mellitus";"{"MedDRA:Diabetes mellitus","MedDRA:Diabetes mellitus (incl subtypes)"}";1
"Type 2 diabetes mellitus";"{"MedDRA:Diabetes mellitus","MedDRA:Diabetes mellitus (incl subtypes)","SNOMED:Diabetes mellitus"}";1
(btw: http://www.ohdsi.org/ if you happen to be curious about where I got the medical vocabulary tables)
Yes, it does look like I was being muddle-headed and didn't need the window function. This seems to work:
select c.concept_name,
array_agg(c2.vocabulary_id||':'||c2.concept_name
order by c2.vocabulary_id, c2.concept_name),
ca.min_levels_of_separation
from concept c
join concept_ancestor ca on c.concept_id = ca.descendant_concept_id
and max_levels_of_separation > 0
join concept c2 on ca.ancestor_concept_id = c2.concept_id
where c.concept_code = '44054006'
group by c.concept_name, ca.min_levels_of_separation
order by min_levels_of_separation
I won't accept my answer for a while since it just avoids the question instead of actually answering it, and someone might have something more useful to say on the matter.
like this :
select distinct c.concept_name,
array_agg(c2.vocabulary_id||':'||c2.concept_name ) over (partition by ca.min_levels_of_separation order by c2.vocabulary_id, c2.concept_name),
ca.min_levels_of_separation
from concept c
join concept_ancestor ca on c.concept_id = ca.descendant_concept_id
and max_levels_of_separation > 0
join concept c2 on ca.ancestor_concept_id = c2.concept_id
where
c.concept_code = '44054006'
order by min_levels_of_separation;
I am using iReport 4.2 and I am trying to do the following
from database I get data in BOM like format (or tree format) and now I need to shift fields right according to the level. I am doing a hierarchical tree
for example
A.
A.a
A.b
B.
B.a
B.a.a
Any suggestions. I did some research and I don't even know what to look for :)
I wrote the following select where I changed the ids with blanks multiplied with the level
select ocs_season_questionnaire_id ,id, code, title, description, 0, '1'
from ocs_season_report_unit
where supervisor_id is null and ocs_season_questionnaire_id = 5
union all
select C.ocs_season_questionnaire_id, C.id, C.code, C.title, C.description, depth+1, xid||'.'||C.id
from sumthis P
inner join ocs_season_report_unit C on P.id = C.supervisor_id
)
select id, lpad('', depth*2)||title as title, description from sumthis where ocs_season_questionnaire_id = $P{qu_id}
It is not the best solution so if anybody knows a better one please respond.
thank you
I have 2 complex queries that are both subqueries in postgres, the results of which are:
q1_results = id , delta , metric_1
q2_results = id , delta , metric_2
i'd like to combine the results of the queries, so the outer query can access either:
results_a = id , delta , metric_1 , metric_2
results_b = id , delta , combined_metric
i can't figure out how to do this. online searches keep leading me to UNION , but that keeps the metrics in the same column. i need to keep them split.
It's not entirely clear what you're asking in the question and the comments, but it sounds like you might be looking for a full join with a bunch of coalesce statements, e.g.:
-- create view at your option, e.g.:
-- create view combined_query as
select coalesce(a.id, b.id) as id,
coalesce(a.delta, b.delta) as delta,
a.metric1 as metric1,
b.metric2 as metric2,
coalesce(a.metric1,0) + coalesce(b.metric2,0) as combined
from (...) as results_a a
full join (...) as results_b b on a.id = b.id -- and a.delta = b.delta maybe?