Our application receives an XML message from another system. The XML is structured like this:
<params>
<param name="FOO" value="BAR"/>
...
</params>
What is the best way, using Scala's native XML processing, to return the value BAR for the parameter which is FOO, so that:
val foo = "BAR"
Thanks
I assume your xml is invalid with missing param closing tag, it should be, for example
var x = <params>
<param name="FOO" value="BAR" />
<param name="FOO2" value="BAR2" />
</params>
If you want to extract the only param FOO, I don't think you will find anything much better than
(x \ "param" find (n => (n \ "#name").toString == "FOO")).get \ "#value"
If you want to get all params, you can iterate over them:
x \ "param" foreach {n => println(n \ "#name" + " -> " + n \ "#value")}
Related
I need to parse and print ns4:feature part. Karate prints it in json format. I tried referring to this answer. But, i get 'ERROR: 'Namespace for prefix 'xsi' has not been declared.' error, if used suggested xPath. i.e.,
* def list = $Test1/Envelope/Body/getPlan/planSummary/feature[1]
This is my XML: It contains lot many parts with different 'ns' values, but i have given here an extraxt.
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Header/>
<S:Body>
<ns9:getPlan xmlns:ns10="http://xmlschema.test.com/xsd_v8" xmlns:ns9="http://xmlschema.test.com/srv/SMO_v4" xmlns:ns8="http://xmlschema.test.com/xsd/Customer_v2" xmlns:ns7="http://xmlschema.test.com/xsd/Customer/Customer_v4" xmlns:ns6="http://schemas.test.com/eca/common_types_2_1" xmlns:ns5="http://xmlschema.test.com/xsd/Customer/BaseTypes_1_0" xmlns:ns4="http://xmlschema.test.com/xsd_v4" xmlns:ns3="http://xmlschema.test.com/xsd/Enterprise/BaseTypes/types/ping_v1" xmlns:ns2="http://xmlschema.test.com/xsd/common/exceptions/Exceptions_v1_0">
<ns9:planSummary xsi:type="ns4:Plan" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ns5:code>XPBSMWAT</ns5:code>
<ns5:description>Test Plan</ns5:description>
<ns4:category xsi:nil="true"/>
<ns4:effectiveDate>2009-11-05</ns4:effectiveDate>
<ns4:sharingGroupList>
<ns4:sharingCode>CAD_DATA</ns4:sharingCode>
<ns4:contributingInd>true</ns4:contributingInd>
</ns4:sharingGroupList>
<ns4:feature>
<ns5:code>ABC</ns5:code>
<ns5:description>Service</ns5:description>
<ns5:descriptionFrench>Service</ns5:descriptionFrench>
<ns4:poolGroupId xsi:nil="true"/>
<ns4:switchCode/>
<ns4:type/>
<ns4:dtInd>false</ns4:dtInd>
<ns4:usageCharge>0.0</ns4:usageCharge>
<ns4:connectInd>false</ns4:connectInd>
</ns4:feature>
</ns9:planSummary>
</ns9:getPlan>
</S:Body>
</S:Envelope>
This is the xPath i used;
Note: I saved above xml in a separate file test1.xml. I am just reading it and parsing the value.
* def Test1 = read('classpath:PP1/data/test1.xml')
* def list = $Test1/Envelope/Body/*[local-name()='getPlan']/*[local-name()='planSummary']/*[local-name()='feature']/*
* print list
This is the response i am getting;
16:20:10.729 [ForkJoinPool-1-worker-1] INFO com.intuit.karate - [print] [
"ABC",
"Service",
"Service",
"",
"",
"",
"false",
"0.0",
"false"
]
How can i get the same in XML?
This is interesting, I haven't seen this before. The problem was you have an attribute with a namespace xsi:nil="true" which is causing problems when you take a sub-set of the XML but the namespace is not defined anymore. If you remove it first, things will work.
Try this:
* remove Test1 //poolGroupId/#nil
* def temp = $Test1/Envelope/Body/getPlan/planSummary/feature
Another approach you could have tried is to do a string replace to remove troublesome stuff in the XML before doing XPath.
EDIT: added info on how to do a string replace using Java. The below will strip out the entire xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns4:Plan" part.
* string temp = Test1
* string temp = temp.replaceAll("xmlns:xsi[^>]*", "")
* print temp
So you get the idea. Just use regex.
Also see: https://stackoverflow.com/a/50372295/143475
In my performance tests in gatling I'm using ElFileBody to read xml, in which seqNumber will be used in one of the tags, so that each document is unique
<document>
<id>${seqNumber}</id>
</document>
Each unique document I want to then sign, so that I get:
<document>
<id>1</id>
<ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:SignedInfo>...</ds:Signature>
</document>
I have tried to use processRequestBody however printing body.toString inside processRequestBody gives me "" so I'm not sure how to get the string value of xml and transform it to add a signature tag.
during(100 seconds) {
forever("seqNumber") {
exec(http("Post New Document")
.post("/document")
.body(ElFileBody("bodies/document.xml"))
.processRequestBody
({ body => {
print(body.toString)
val xml = signatureHelper.sign(body.toString);
StringBody(xml)
}})
.processRequestBody(gzipBody)
.header("Content-Type", "application/xml")
.header("Content-Encoding",
"gzip").check(status.not(400),status.not(500))).exitHereIfFailed
}
}
I'm trying to request a line from Azure Table Storage using the REST API and C++, but always got the following error:
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
<cod_e>JsonFormatNotSupported</cod_e>
<message xml:lang="en-US">JSON format is not supported.
RequestId:0ccb3b9b-0002-0029-3389-0d2fa1000000
Time:2016-09-13T06:39:13.3155742Z</message>
</error>
Here is my request:
GET https://<myaccount>.table.core.windows.net/<mytable>(PartitionKey='<mypartition>',RowKey='<myrow>')?<sharedsignature>
Here how I fill request headers, as from https://msdn.microsoft.com/en-us/library/dd179428.aspx:
std::string sharedAccessSignature("<sharedsignature>");
std::string dateTime(GetDateTime());
std::string stringToSign(dateTime + "\n/" + account + "/" + "<mytable>");
std::string request("(PartitionKey='<mypartition>',RowKey='<myrow>')");
stringToSign += request;
std::string signatureString(HMACSHA256(stringToSign, sharedAccessSignature));
headers["Authorization"] = "SharedKeyLite " + account + ":" + signatureString;
headers["DataServiceVersion"] = "3.0;NetFx";
headers["MaxDataServiceVersion"] = "3.0;NetFx";
headers["x-ms-version"] = "2015-12-11";
headers["x-ms-date"] = dateTime;
headers["Accept"] = "application/json;odata=verbose";
headers["Accept-Charset"] = "UTF-8";
The table exists and is not empty.
Please advise what's wrong?
Update 1
Removing sharedsignature from request, i.e. GET https://<myaccount>.table.core.windows.net/<mytable>(PartitionKey='<mypartition>',RowKey='<myrow>') leads to the same result.
Removing Authorization header from the request leads to the same result too.
Update 2
Putting https://<myaccount>.table.core.windows.net/<mytable>(PartitionKey='<mypartition>',RowKey='<myrow>')?<sharedsignature> in a browser produces a valid response, but in Atom format.
<?xml version="1.0" encoding="utf-8"?>
<entry
xml:base="https://<myaccount>.table.core.windows.net/"
xmlns="http://www.w3.org/2005/Atom"
xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"
xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
m:etag="W/"datetime'2016-09-13T05%3A29%3A51.211538Z'"">
<id>https://<myaccount>.table.core.windows.net/<mytable> (PartitionKey='<mypartition>',RowKey='<myrow>')</id>
<category term="<myaccount>.<mytable>" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
<link rel="edit" title="<mytable>" href="<mytable> (PartitionKey='<mypartition>',RowKey='<myrow>')" />
<title />
<updated>2016-09-13T11:25:19Z</updated>
<author><name /></author>
<content type="application/xml">
<m:properties>
<d:PartitionKey><mypartition></d:PartitionKey>
<d:RowKey><myrow></d:RowKey>
<d:Timestamp m:type="Edm.DateTime">2016-09-13T05:29:51.211538Z</d:Timestamp>
<d:Score m:type="Edm.Int32">1050</d:Score>
</m:properties>
</content>
</entry>
Update 3
Investigation situation using curl I found that adding Accept: application/json;odata=fullmetadata to the request headers leads to the error above. Default Accept */* in headers produces valid Atom response.
Finally, got it!
The issue was in my shared signature.
While looking at it I found sv=2012-02-12 part, and assumed, that it means API version. The version, before JSON support was introduced! I created a new shared signature and finally got JSON with the following curl command.
curl -v -H "Accept: application/json;odata=nometadata" -H "x-ms-version: 2015-12-11" "https://<myaccount>.table.core.windows.net/<mytable>(PartitionKey='<mypartition>',RowKey='<myrow>')?<mysharedsignature>"
So, for everyone, who face the same issue in the future: please check your signature first!
I'm currently doing some parsing of very large xml files > 40 MB. I have just started developing in scala so I browsed the net for some good libs and stumbled upon Scala Scales which seems to be very good at handling large files.
I have read:
http://scala-scales.googlecode.com/svn/sites/scales/scales-xml_2.9.1/0.2/ScalesXmlIntro.html
,
http://scala-scales.googlecode.com/svn/sites/scales/scales-xml_2.9.2/0.4.4/PullParsing.html
and then tested the pullXml function, to make sure all libs are imported correctly.
val pull = pullXml(new FileReader("/Users/mycrazyxml/tmp/large.xml"))
while( pull.hasNext ){
pull.next match {
case Left( i : XmlItem ) =>
// Handle XmlItem
Logger.info("XmlItem: "+i)
case Left( e : Elem ) => {
// Handle Element
Logger.info("Element: "+e)
}
case Right(endElem) =>
// Handle endElement
Logger.info("Endelement: "+endElem)
}
}
This results in that the entire file is printed to the console! Nice!
Now it's time create the objects and save to the db, but I'm having
trouble in grasping how to do this in a good way. I would really need some good examples
of how to do this.
Eg. following XML has several Enterprise elements which can consist of one or several LocalUnits.
The idea here is to create an Enterprise object with an array of LocalUnits. When
the endElement is the closing tag for an Enterprise call the save method with the Enterprise object with it's LocalUnits.
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE Info SYSTEM "info.dtd">
<Info>
<Enterprise>
<RegNo>12345678</RegNo>
<Address>
<StreetInfo>
<StreetName>Infinite Loop</StreetName>
<StreetNumber>1</StreetNumber>
</StreetInfo>
</Address>
<EName>
<Legal>Crazy Company</Legal>
</EName>
<SNI>
<Code>00000</Code>
<Rank>1</Rank>
</SNI>
<LocalUnit>
<CFARNo>987654321</CFARNo>
<LUType>1</LUType>
<LUName>Crazy Company Gym</LUName>
<LUStatus>1</LUStatus>
<SNI>
<Code>46772</Code>
<Rank>1</Rank>
</SNI>
<SNI>
<Code>68203</Code>
<Rank>2</Rank>
</SNI>
<Address>
<StreetInfo>
<StreetName>Infinite Loop</StreetName>
<StreetNumber>1</StreetNumber>
</StreetInfo>
</Address>
</LocalUnit>
<LocalUnit>
<CFARNo>987654322</CFARNo>
<LUType>1</LUType>
<LUName>Crazy Company Restaurant</LUName>
<LUStatus>1</LUStatus>
<SNI>
<Code>46772</Code>
<Rank>1</Rank>
</SNI>
<SNI>
<Code>68203</Code>
<Rank>2</Rank>
</SNI>
<Address>
<StreetInfo>
<StreetName>Infinite Loop</StreetName>
<StreetNumber>1</StreetNumber>
</StreetInfo>
</Address>
</LocalUnit>
</Enterprise>
<Enterprise>
<RegNo>12345671220</RegNo>
<Address>
<StreetInfo>
<StreetName>Cupertino Road</StreetName>
<StreetNumber>2</StreetNumber>
</StreetInfo>
</Address>
<EName>
<Legal>Fun Company HQ</Legal>
</EName>
<SNI>
<Code>00000</Code>
<Rank>1</Rank>
</SNI>
<LocalUnit>
<CFARNo>987654321</CFARNo>
<LUType>1</LUType>
<LUName>Fun Company</LUName>
<LUStatus>1</LUStatus>
<SNI>
<Code>46772</Code>
<Rank>1</Rank>
</SNI>
<SNI>
<Code>68203</Code>
<Rank>2</Rank>
</SNI>
<Address>
<StreetInfo>
<StreetName>Cupertino road</StreetName>
<StreetNumber>2</StreetNumber>
</StreetInfo>
</Address>
</LocalUnit>
</Enterprise>
</Info>
To sum it up. For the given xml how should I use pullXml to create my objects and call the save method with them?
val xmlFile = resource(this, "/data/enterprise_info.xml")
val xml = pullXml(xmlFile)
val Info = NoNamespaceQName("Info")
val Enterprise = NoNamespaceQName("Enterprise")
val LocalUnit = NoNamespaceQName("LocalUnit")
val LocalUnitName = NoNamespaceQName("LUName")
val EName = NoNamespaceQName("EName")
val Legal = NoNamespaceQName("Legal")
val EnterprisePath = List(Info, Enterprise)
// iterate over each Enterprise
// only an Enterprise at a time is in memory
val itr = iterate(EnterprisePath, xml)
for {
enterprise <- itr
enterpriseName <- enterprise \* EName \* Legal
} {
println("enterprise "+text(enterpriseName) +" has units:")
for {
localUnits <- enterprise \* LocalUnit
localName <- localUnits \* LocalUnitName
}{
println(" " + text(localName))
}
//do a save
}
Pulling in each LocalUnit lazily is more difficult at the moment, you must separate Paths for each subsection which isn't a LocalUnit.
Hth
Maybe this could help you? I think using small snippets of xml works very well in Scala.
http://joncook.github.io/blog/2013/11/03/xml-processing-with-scala/
i try to copying the examples in wiki
http://wiki.liftweb.net/index.php/Hello_Darwin
in the example of HelloForm2.scala
"submit" -> submit(?("Send"), () => {println("value:" + who + " :: " + param("whoField"))}),
It always prints
value:Full(hogehoge) :: Empty" even if i set the who as "object who extends RequestVar(Full("world"))
am i do something wrong?
sorry for forgetting to post full code, i already try the second one in the wiki like below.
index.html
<lift:surround with="default" at="content">
<h2>Welcome to your project!</h2>
<lift:HelloWorld.show form="POST">
Hello <hello:who />
<br />
<label for="whoField">Who :</label>
<hello:whoField />
<hello:submit />
</lift:HelloWorld.show>
</lift:surround>
and HelloWorld.scala
class HelloWorld {
object who extends RequestVar(Full("world"));
def show(xhtml: NodeSeq): NodeSeq ={
bind("hello", xhtml,
"whoField" -> text(who.openOr(""), v => who(Full(v))) % ("size" -> "10") % ("id" -> "whoField"),
"submit" -> submit(?("Send"), () => {println("value:" + who.openOr("") + " :: " + param("whoField"))}),
"who" -> who.openOr("")
)
}
}
now, the who shows correct in the rendered page, but console still prints
value:hogehoge :: Empty
im using lift 1.0
thanks.
You have to change that code too, as shown in the example in the wiki page, which I'll copy here:
bind("hello", xhtml,
"whoField" -> text(who.openOr(""), v => who(Full(v))) % ("size" -> "10") % ("id" -> "whoField"),
"submit" -> submit(?("Send"), () => {println("value:" + who.openOr("") + " :: " + param("whoField"))}),
"who" -> who.openOr("")
)
Note that whoField is defined very differently.