Postgres XML parsing - Calculate the sum of multiple nodes of the same name using XPath - postgresql

I have an xml snippet as below:
<OpCodeLaborInfo JobStatus="F" UpSellFlag="N" JobNo="1" OpCode="02TTZ10K" OpCodeDesc="10K SERVICE">
<TechInfo ActualHrsWorked="2.50" CustTechRate="27.00" TechHrs="0.00" TechName="JEFF SELLERS" TechNo="4816" />
<TechInfo ActualHrsWorked="0.00" CustTechRate="27.00" TechHrs="0.70" TechName="JEFF SELLERS" TechNo="4816" />
<BillTimeRateHrs BillRate="129.97" />
<CCCStmts Correction="PERFORMED 10K SERVICE" Complaint="LUBE OIL FILTER CHANGE, TIRE ROTATION, PERFORM MULTI POINT" />
<CCCStmts Correction="X" Complaint="INSPECTION, INSPECT FILTERS AND RECOMMEND, INSPECT BRAKES," />
<CCCStmts Complaint="BELTS AND HOSES" />
<RoAmts DlrCost="18.90" PayType="Cust" AmtType="Job" TotalAmt="59.12" />
</OpCodeLaborInfo>
<OpCodeLaborInfo JobStatus="F" UpSellFlag="N" JobNo="2" OpCode="02TTZ10K" OpCodeDesc="10K SERVICE">
<TechInfo ActualHrsWorked="2.50" CustTechRate="27.00" TechHrs="1.00" TechName="JEFF SELLERS" TechNo="4816" />
<TechInfo ActualHrsWorked="0.00" CustTechRate="27.00" TechHrs="0.00" TechName="JEFF SELLERS" TechNo="4816" />
<BillTimeRateHrs BillRate="129.97" />
<CCCStmts Correction="PERFORMED 10K SERVICE" Complaint="LUBE OIL FILTER CHANGE, TIRE ROTATION, PERFORM MULTI POINT" />
<CCCStmts Correction="X" Complaint="INSPECTION, INSPECT FILTERS AND RECOMMEND, INSPECT BRAKES," />
<CCCStmts Complaint="BELTS AND HOSES" />
<RoAmts DlrCost="18.90" PayType="Cust" AmtType="Job" TotalAmt="59.12" />
</OpCodeLaborInfo>
I need to calculate the sum of the TechInfo/#TechHrs for each OpCodeLaborInfo. I tried the following:
unnest(xpath('sum(//dns:RepairOrder/dns:RoRecord/dns:Rolabor/dns:OpCodeLaborInfo/dns:TechInfo/#TechHrs[1])'::text,
data_detail.ro_data_xml,
ARRAY[ARRAY['dns'::text, 'http://www.starstandards.org/STAR'::text]]))::text::numeric AS lbrsoldhours
but this seems to return the sum of the Tech Hours inside both the OpCodeLaborInfo nodes. Could someone be able to tell me how I can tweak the xpath so as to get the desired result.
So basically I need :
Job
Tech Hrs
1
sum(0.00+0.70)
2
sum(1.00+0.00)

I would solve this using xmltable()
select d.job, sum(t.hours)
from data_detail d
cross join xmltable (
'/OpCodeLaborInfo/TechInfo'
passing d.ro_data_xml
columns hours numeric path '#TechHrs') as t
group by d.job;
Online example
The XPath is probably not correct as the XPath you have shown doesn't match your sample XML data. Your sample XML also doesn't contain a namespace, so I am not sure why you are passing one to xpath()
But if you need one, you can use something like this:
cross join xmltable (
xmlnamespaces ('http://www.starstandards.org/STAR' as dns),
'/dns:OpCodeLaborInfo/dns:TechInfo'
passing d.ro_data_xml
columns hours numeric path '#TechHrs') as t

xpath can also work fine:
SELECT job
, SUM((xpath('//#TechHrs', element))[1]::text::decimal)
FROM data_detail
, LATERAL (SELECT unnest(xpath('/OpCodeLaborInfo/TechInfo', ro_data_xml))) u(element)
GROUP BY job;
fiddle
(based on the fiddle of #a_horse_with_no_name)

Related

Unable to set the ID of an instance

What is unique about my situation is that the ID's can not be randomly assigned so I set it's value within the instance. I created several instances of an entity using the modeler. Below is the XML created:
<cf:entity name="Test4" namespace="Amikids.TimeTracking" categoryPath="/Amikids.TimeTracking">
<cf:property name="Id" key="true" typeName="int" />
<cf:property name="Name" />
<cf:instance>
<cf:instanceValue name="Id">10</cf:instanceValue>
<cf:instanceValue name="Name">Test 1</cf:instanceValue>
</cf:instance>
<cf:instance>
<cf:instanceValue name="Id">20</cf:instanceValue>
<cf:instanceValue name="Name">Test 2</cf:instanceValue>
</cf:instance>
<cf:instance>
<cf:instanceValue name="Id">30</cf:instanceValue>
<cf:instanceValue name="Name">Test 3</cf:instanceValue>
</cf:instance>
</cf:entity>
There are 2 things that are not working as expected:
The records inserted do not use the ID specificed in the model/xml. Instead they were created incrementally starting at 1:
(The below is displayed in a code snippet only to prevent StackOverflow from reformatting my list so all records appear on one line)
ID Name
1 Test 1
2 Test 2
3 Test 3
When I build the model a second time duplicate records are inserted.
(The below is displayed in a code snippet only to prevent StackOverflow from reformatting my list so all records appear on one line)
ID Name
1 Test 1
2 Test 2
3 Test 3
4 Test 1
5 Test 2
6 Test 3
Although specifying the ID in the instance does not appear to be work, as a simple work around I created the records using code, which allowed me to specify the ID. This has been verified with the following code snippet.
Amikids.TimeTracking.Test4 test4 = new Amikids.TimeTracking.Test4();
test4.Id = 100;
test4.Name = "Test 100";
test4.Save();
test4 = new Amikids.TimeTracking.Test4();
test4.Id = 200;
test4.Name = "Test 200";
test4.Save();

How can I convert this T-SQL to Fetch-XML?

I want to use this SQL in a Dynamics CRM report. I can't work out how to convert it to Fetch-XML.
select p.ProductNumber "Plan Number",p.Name,p.price "Monthly Rate",count(*) "Group", '0' "Direct Debit"
from contact c,product p
where c.integ_schemeid = p.ProductId
and c.ParentCustomerId is not null
group by p.ProductNumber,p.Name,p.price
union
select p.ProductNumber "Plan Number",p.Name,p.price "Monthly Rate", '0' "Group", count(*) "Direct Debit"
from contact c,product p
where c.integ_schemeid = p.ProductId
and c.ParentCustomerId is null
group by p.ProductNumber,p.Name,p.price
http://www.sql2fetchxml.com fails on this occasion.
You can't convert that query to FetchXML, because unions aren't supported. What you'll need to do is look into merging the queries into 1 if possible. Even if it means duplicating columns and using conditional statements within your report to display the relevant data instead. For example, if you simplified the query to this:
select p.ProductNumber "Plan Number", p.Name, p.price, c.ParentCustomerId
from product p
inner join contact c on c.integ_schemeid = p.ProductId
This can be converted to the following fetchxml:
<fetch mapping="logical">
<entity name="product">
<attribute name="ProductNumber" alias="Plan Number" />
<attribute name="Name" />
<attribute name="price" />
<link-entity name="contact" to="ProductId" from="integ_schemeid" alias="c" link-type="inner">
<attribute name="ParentCustomerId" />
</link-entity>
</entity>
</fetch>
Then, because you have all of the data in 1 dataset including contacts with null ParentCustomerIds, you can group within your report instead of within the query.

How to use SQL XML and XMLAGG to return multiple xmlelements

In SQL, I need to create xml code that looks like this:
<Phone>
<PhoneTypeCode tc="12">Mobile</PhoneTypeCode>
<Area>801</Area>
<DialNumber>9996666</DialNumber>
</Phone>
<Phone>
<PhoneTypeCode tc="2">Business</PhoneTypeCode>
<Area>801</Area>
<DialNumber>1113333</DialNumber>
</Phone>
using xmlagg, but it is throwing an error on the ',' after p.desc
How does this IBM DB2 SQL function need to be fixed to achieve the above xml?
select
xmlelement(Name "Phone",
xmlagg(xmlelement(name "PhoneTypeCode",
xmlattributes(trim(p.phtype) as "tc"), trim(p.desc)),
xmlelement(name "AreaCode", p.area),
xmlelement(name "DialNumber", p.phone)
)
) as xml
from phone p
where p.entityid = #entity_id
I also wanted to add that it does compile and run with this:
select
xmlelement(Name "Phone",
xmlagg(xmlelement(name "PhoneTypeCode",
xmlattributes(trim(p.phtype) as "tc"), trim(p.desc))
)
) as xml
from phone p
where p.entityid = #entity_id
Here is what it returns:
<Phone>
<PhoneTypeCode tc="12">Mobile</PhoneTypeCode>
<PhoneTypeCode tc="2">Business</PhoneTypeCode>
</Phone>
But of course, I need the Area and DialNumber. It is as if you can't have more than one xmlelement within an xmlagg.
How does this IBM DB2 SQL function need to be fixed to achieve the above xml?
Firstly, you may want to count your parentheses. Normally one would want as many closing parentheses as there are opening parentheses.
Secondly, you don't need XMLAGG() at all. You'd use it when inserting multiple elements of the same type, based on multiple relational records, into a single outer element, like
<phones>
<phone no="1" .../>
<phone no="2" .../>
...
</phones>
For you something like this should work:
with phone (phtype, desc, area, phone) as
(values ('home','blah','555','555-5555'),('office','blah','555','555-1111'))
select
xmlelement(
Name "Phone",
xmlelement(
name "PhoneTypeCode",
xmlattributes(
trim(p.phtype) as "tc"
),
trim(p.desc)
),
xmlelement(name "AreaCode", p.area),
xmlelement(name "DialNumber", p.phone)
) as xml
from phone p

How to avoid namespace in child nodes using FOR XML PATH?

I want to create a sitemap xml file (including images) directly from the database without another process (like transformation or another trick).
My query is:
;WITH XMLNAMESPACES(
DEFAULT 'http://www.sitemaps.org/schemas/sitemap/0.9',
'http://www.google.com/schemas/sitemap-image/1.1' as [image] )
SELECT
(SELECT
'mysite' as [loc],
(select
'anotherloc'
as [image:loc]
for XML path('image:image'), type
)
for xml path('url'), type
)
for xml path('urlset'), type
Returns:
<urlset xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<loc>mysite</loc>
<image:image xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<image:loc>anotherloc</image:loc>
</image:image>
</url>
</urlset>
But I need this output, without repeated namespace declaration:
<urlset xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>mysite</loc>
<image:image>
<image:loc>anotherloc</image:loc>
</image:image>
</url>
</urlset>
I'm sure you realise that the additional otiose namespace declarations don't change the meaning of the XML document, so if the result is going to be consumed by an XML-conformant tool, they shouldn't matter. Nevertheless I know there are some tools out there which don't do XML Namespaces correctly, and in a large XML instance superfluous repeated namespace declarations can bloat the size of the result significantly, which may cause its own problems.
In general there is no getting around the fact that each SELECT...FOR XML statement within the scope of a WITH XMLNAMESPACES prefix will generate namespace declarations on the outermost XML element(s) in its result set, in all XML-supporting versions of SQL Server up to SQL Server 2012.
In your specific example, you can get fairly close to the desired XML by separating the SELECTs rather than nesting them, and using the ROOT syntax for the enveloping root element, thus:
DECLARE #inner XML;
WITH XMLNAMESPACES('http://www.google.com/schemas/sitemap-image/1.1' as [image])
SELECT #inner =
(
SELECT
'anotherloc' AS [image:loc]
FOR XML PATH('image:image'), TYPE
)
;WITH XMLNAMESPACES(
DEFAULT 'http://www.sitemaps.org/schemas/sitemap/0.9'
)
SELECT
'mysite' AS [loc],
#inner
FOR XML PATH('url'), ROOT('urlset'), TYPE
The result being:
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>mysite</loc>
<image:image xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns="">
<image:loc>anotherloc</image:loc>
</image:image>
</url>
</urlset>
But this approach doesn't provide a completely general solution to the problem.
You can use UDF. Example:
ALTER FUNCTION [dbo].[udf_get_child_section] (
#serviceHeaderId INT
)
RETURNS XML
BEGIN
DECLARE #result XML;
SELECT #result =
(
SELECT 1 AS 'ChildElement'
FOR XML PATH('Child')
)
RETURN #result
END
GO
DECLARE #Ids TABLE
(
ID int
)
INSERT INTO #Ids
SELECT 1 AS ID
UNION ALL
SELECT 2 AS ID
;WITH XMLNAMESPACES (DEFAULT 'http://www...com/content')
SELECT
[dbo].[udf_get_child_section](ID)
FROM
#Ids
FOR XML PATH('Parent')
Result:
<Parent xmlns="http://www...com/content">
<Child xmlns="">
<ChildElement>1</ChildElement>
</Child>
</Parent>
<Parent xmlns="http://www...com/content">
<Child xmlns="">
<ChildElement>1</ChildElement>
</Child>
</Parent>
Maybe too late for answer, but this is a quick solution.
`DECLARE #PageNumber Int = 1;
DECLARE #siteMapXml XML ;
;WITH XMLNAMESPACES (
'http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd' as "schemaLocation",
'http://www.w3.org/2001/XMLSchema-instance' as xsi,
'http://www.google.com/schemas/sitemap-image/1.1' as [image],
DEFAULT 'http://www.sitemaps.org/schemas/sitemap/0.9'
)
SELECT #siteMapXml = (
SELECT
Slug loc,
convert(varchar(300),[Image]) as [image:image/image:loc]
,
Convert(char(10), UpdatedOnUtc, 126) as lastmod,
'hourly' as changefreq,
'0.5' as priority
FROM Products(NOLOCK)
WHERE Pagenumber = #PageNumber
FOR XML PATH ('url'), ROOT ('urlset'))
SELECT #siteMapXml = REPLACE(CAST(#siteMapXml AS NVARCHAR(MAX)), ' xmlns:schemaLocation=', ' xsi:schemaLocation=')
SELECT #siteMapXml`

OLAP/MDX - define calculated member, sum all time to date data

I would like to define "all time to date" calculated member in OLAP cube. I'm able to calculate YTD by using the following:
SUM(YTD([Time].[Month].CurrentMember), [Measures].[Suits])
How can I include all dates since the beginning of my data? My time dimension looks like:
<Dimension type="TimeDimension" visible="true" foreignKey="granularity" highCardinality="false" name="Time">
<Hierarchy name="Time" visible="true" hasAll="true" primaryKey="eom_date">
<Table name="v_months" schema="bizdata">
</Table>
<Level name="Year" visible="true" column="year_number" type="String" uniqueMembers="false" levelType="TimeYears" hideMemberIf="Never">
</Level>
<Level name="Quarter" visible="true" column="quarter_number" type="String" uniqueMembers="false" levelType="TimeQuarters" hideMemberIf="Never">
</Level>
<Level name="Month" visible="true" column="month_number" type="String" uniqueMembers="false" levelType="TimeMonths" hideMemberIf="Never">
</Level>
</Hierarchy>
</Dimension>
Not sure if relevant: I'm using mondrian olap server (running on tomcat), Saiku as frontend, postgres as database
I've tried a lot of combinations, but I can't figure it out.
Update: I've tried to use syntax suggested by Gonsalu:
<CalculatedMember name="YTD Suits" formatString="" formula="SUM(YTD([Time].[Month].CurrentMember), [Measures].[Suits])" dimension="Measures" visible="true">
</CalculatedMember>
<CalculatedMember name="PTD Suits" formatString="" formula="Sum({NULL:[Time].[Month].CurrentMember },[Measures].[Suits])" dimension="Measures" visible="true">
</CalculatedMember>
Using this I get the following error message when starting mondrian (note that YTD function works well without the second calculated member):
Caused by: mondrian.olap.MondrianException: Mondrian Error:Failed to parse query
'WITH
MEMBER [Measures].[Measures].[YTD Suits]
AS 'SUM(YTD([Time].[Month].CurrentMember), [Measures].[Suits])',
[$member_scope] = 'CUBE',
MEMBER_ORDINAL = 6
MEMBER [Measures].[Measures].[PTD Suits]
AS 'Sum({NULL:[Time].[Month].CurrentMember },[Measures].[Suits])',
[$member_scope] = 'CUBE',
MEMBER_ORDINAL = 7
SELECT FROM [Project Performance]'
Thank you for any ideas.
I haven't used Mondrian, but in SQL Server Analysis Services (SSAS), using the NULL member causes the range to go from one end of the level to the specified member.
In your case, the calculated member you're looking for might be something like this:
Sum( { NULL : [Time].[Month].CurrentMember }
, [Measures].[Suits]
)
You could also do a to the end of times calculated member using the NULL member on the other end, like so:
{ [Time].[Month].CurrentMember : NULL }
You can use PeriodsToDate function along with the allMember.
In your case it would be:
PeriodsToDate([Time.Time].[all_Time_member_name],[Time.Time].CurrentMember)