How to remove SOAP Optional elements in the request - soap

Using Karate, I would like to build a soap request.
There are Optional elements in the SOAP Request, how to remove them based on the Scenario Outline: Example?
Shared an example, for the purpose of discussion. If there is already a sample code, please share, thank you.
Feature: SOAP request to get Customer Details
Background:
Given url 'http://localhost:8080/CustomerService_V2_Ws'
Scenario Outline:
* def removeElement =
"""
function(parameters, inputXml) {
if (parameters.city = null)
karate.remove(inputXml, '/Envelope/Header/AutHeader/ClientContext/city');
if (parameters.zipcode = null)
karate.remove(inputXml, '/Envelope/Header/AutHeader/ClientContext/zipcode');
return inputXml;
}
"""
* def inputXml = read('soap-request.xml')
* def updatedXml = removeElement(parameters,inputXml)
Given request updatedXml
When soap action ''
Then status <http_code>
Examples:
| CustomerId | ZipCode | City |
| 001 | null | null |
| 002 | 41235 | null |
| 003 | null | New York |
**Contents of "soap-request.xml"**
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header>
<wsc1:AutHeader xmlns:wsc1="http://example.com/ws/WSCommon_v22">
<wsc1:SourceApplication>ABC</wsc1:SourceApplication>
<wsc1:DestinationApplication>SoapUI</wsc1:DestinationApplication>
<wsc1:Function>CustomerService.readDetails</wsc1:Function>
<wsc1:Version>2</wsc1:Version>
<wsc1:ClientContext>
<wsc1:customerid>10000</wsc1:customerid>
<!--Optional:-->
<wsc1:zipcode>11111</wsc1:zipcode>
<!--Optional:-->
<wsc1:city>xyz</wsc1:city>
</wsc1:ClientContext>
</wsc1:AutHeader>
</soapenv:Header>
<soapenv:Body />
</soapenv:Envelope>

Yes my suggestion is use embedded expressions. When the expression is prefixed with 2 hash signs, this will "delete if null" which was designed for your exact use case. Here is an example:
Scenario: set / remove xml chunks using embedded expressions
* def phone = '123456'
# this will remove the <acc:phoneNumberSearchOption> element
* def searchOption = null
* def search =
"""
<acc:getAccountByPhoneNumber>
<acc:phoneNumber>#(phone)</acc:phoneNumber>
<acc:phoneNumberSearchOption>##(searchOption)</acc:phoneNumberSearchOption>
</acc:getAccountByPhoneNumber>
"""
* match search ==
"""
<acc:getAccountByPhoneNumber>
<acc:phoneNumber>123456</acc:phoneNumber>
</acc:getAccountByPhoneNumber>
"""
Do note that there are many more examples here: xml.feature

Related

Karate: Replace text at Envelope level [duplicate]

I need to parse and print ns4:feature part. Karate prints it in json format. I tried referring to this answer. But, i get 'ERROR: 'Namespace for prefix 'xsi' has not been declared.' error, if used suggested xPath. i.e.,
* def list = $Test1/Envelope/Body/getPlan/planSummary/feature[1]
This is my XML: It contains lot many parts with different 'ns' values, but i have given here an extraxt.
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Header/>
<S:Body>
<ns9:getPlan xmlns:ns10="http://xmlschema.test.com/xsd_v8" xmlns:ns9="http://xmlschema.test.com/srv/SMO_v4" xmlns:ns8="http://xmlschema.test.com/xsd/Customer_v2" xmlns:ns7="http://xmlschema.test.com/xsd/Customer/Customer_v4" xmlns:ns6="http://schemas.test.com/eca/common_types_2_1" xmlns:ns5="http://xmlschema.test.com/xsd/Customer/BaseTypes_1_0" xmlns:ns4="http://xmlschema.test.com/xsd_v4" xmlns:ns3="http://xmlschema.test.com/xsd/Enterprise/BaseTypes/types/ping_v1" xmlns:ns2="http://xmlschema.test.com/xsd/common/exceptions/Exceptions_v1_0">
<ns9:planSummary xsi:type="ns4:Plan" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ns5:code>XPBSMWAT</ns5:code>
<ns5:description>Test Plan</ns5:description>
<ns4:category xsi:nil="true"/>
<ns4:effectiveDate>2009-11-05</ns4:effectiveDate>
<ns4:sharingGroupList>
<ns4:sharingCode>CAD_DATA</ns4:sharingCode>
<ns4:contributingInd>true</ns4:contributingInd>
</ns4:sharingGroupList>
<ns4:feature>
<ns5:code>ABC</ns5:code>
<ns5:description>Service</ns5:description>
<ns5:descriptionFrench>Service</ns5:descriptionFrench>
<ns4:poolGroupId xsi:nil="true"/>
<ns4:switchCode/>
<ns4:type/>
<ns4:dtInd>false</ns4:dtInd>
<ns4:usageCharge>0.0</ns4:usageCharge>
<ns4:connectInd>false</ns4:connectInd>
</ns4:feature>
</ns9:planSummary>
</ns9:getPlan>
</S:Body>
</S:Envelope>
This is the xPath i used;
Note: I saved above xml in a separate file test1.xml. I am just reading it and parsing the value.
* def Test1 = read('classpath:PP1/data/test1.xml')
* def list = $Test1/Envelope/Body/*[local-name()='getPlan']/*[local-name()='planSummary']/*[local-name()='feature']/*
* print list
This is the response i am getting;
16:20:10.729 [ForkJoinPool-1-worker-1] INFO com.intuit.karate - [print] [
"ABC",
"Service",
"Service",
"",
"",
"",
"false",
"0.0",
"false"
]
How can i get the same in XML?
This is interesting, I haven't seen this before. The problem was you have an attribute with a namespace xsi:nil="true" which is causing problems when you take a sub-set of the XML but the namespace is not defined anymore. If you remove it first, things will work.
Try this:
* remove Test1 //poolGroupId/#nil
* def temp = $Test1/Envelope/Body/getPlan/planSummary/feature
Another approach you could have tried is to do a string replace to remove troublesome stuff in the XML before doing XPath.
EDIT: added info on how to do a string replace using Java. The below will strip out the entire xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns4:Plan" part.
* string temp = Test1
* string temp = temp.replaceAll("xmlns:xsi[^>]*", "")
* print temp
So you get the idea. Just use regex.
Also see: https://stackoverflow.com/a/50372295/143475

Printing SOAP response in Karate DSL

I have this feature file and i get a response correctly. I want to print obtained value from response but somehow I am not able to do that. Tried to research some stuff but I couldnt help myself.
Can anyone help please? Thanks in advance
Feature:
test of soap
Background:
* url 'http://www.dataaccess.com/webservicesserver/numberconversion.wso'
Scenario: soap 1.1
Given request
"""
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:web="http://www.dataaccess.com/webservicesserver/">
<soapenv:Header/>
<soapenv:Body>
<web:NumberToDollars>
<web:dNum>10</web:dNum>
</web:NumberToDollars>
</soapenv:Body>
</soapenv:Envelope>
"""
When soap action 'Conversion'
Then status 200
* print '\n', response
#working
* match response /Envelope/Body/NumberToDollarsResponse/NumberToDollarsResult == 'ten dollars'
#not working
* print response.Envelope.Body.NumberToDollarsResponse.NumberToDollarsResult
#not working
* print response /Envelope/Body/NumberToDollarsResponse/NumberToDollarsResult
#not working
* def x = response /Envelope/Body/NumberToDollarsResponse/NumberToDollarsResult
* print x
If you read the docs, print only handles JS on the right-hand-side, not XPath.
For what you want, please do in 2 steps:
* def temp = /Envelope/Body/NumberToDollarsResponse/NumberToDollarsResult
* print temp

Cannot find x path from request in wiremock

I'm trying to create stub in wirkmock. But it's show 'request not match' when I hit the endpoint
it's work when I use a simple check
In code:
stubFor(get(urlEqualTo("/v1/test.svc"))
.withHeader("Content-Type", equalTo("application/xml"))
.withRequestBody(containing("<b:Name>Test</b:Name>"))
.willReturn(aResponse()
.withStatus(200)
.withHeader("Content-Type", "application/xml")
.withBody("Passed")));
Request:
endpoint
http://{{url}}/v1/test.svc
header
Content-Type:application/xml
body
<test xmlns="http://abc.example.com">
<request xmlns:b="http://abc.example.com/b" xmlns:i="http://abc.example.com/i">
<b:Name>Test</b:Name>
<b:Age>18</b:Age>
<b:Count>2020</b:Count>
</request>
</test>
Result:
Passed
But it's not working when I try to use xpath for check some values
In code:
stubFor(get(urlEqualTo("/paymentapi/paymentservice.svc"))
.withHeader("Content-Type", equalTo("application/xml"))
.withRequestBody(containing("<b:Name>Test</b:Name>"))
.withRequestBody(matchingXPath("//Age/text()",equalTo("18")))
.willReturn(aResponse()
.withStatus(200)
.withHeader("Content-Type", "application/xml")
.withBody("Passed")));
Result:
Request was not matched
=======================
-----------------------------------------------------------------------------------------------------------------------
| Closest stub | Request |
-----------------------------------------------------------------------------------------------------------------------
|
GET | GET
/vi/test.svc | /v1/test.svc
|
Content-Type: application/xml | Content-Type: application/xml
|
//Age/text() | <test xmlns="http://abc.example.com">
<request <<<<< Body does not match
| xmlns:b="http://abc.example.com/b"
| xmlns:i="http://abc.example.com/i">
<b:Name>Test</b:Name>
<b:Age>18</b:Age>
<b:Count>2020</b:Count>
</request>
</test>
|
-----------------------------------------------------------------------------------------------------------------------
Can anyone tell me how to check value for this?
Age element is under the http://abc.example.com/b namespace URI. From Wiremock documentation you can declare the in scope namespace for your XPath expression like this:
.withRequestBody(matchingXPath("//b:Age[.=18]")
.withXPathNamespace("b", "http://abc.example.com/b"))

Parsing XML and ignoring sections with powershell

I want to parse out the key fields and Data table information from here with PowerShell.
I only want the datatable name if there is a keyfield so in the example below I do not want CC:Attribute.
I also want to output things to a text file.
I want to have a text file that is created that holds the Data table name & Access as well as all the key fields and what they are.
This is the code I have so far:
[xml]$global:xmldata = get-content "C:\hackathon\Mfg.xml"
$xmldata2 = $xmldata.SchemaPackage.Tables
$SField = $xmldata2.DataTable.KeyFields | %{$_.StringField}
$Reffield = $xmldata2.DataTable.KeyFields | %{$_.ReferenceField}
$table = $xmldata2 | %{$_.DataTable}
Xml File:
<?xml version="1.0" encoding="utf-8"?>
<SchemaPackage Namespace="Mfg" xmlns="" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<DataTable Name="CC::Attribute">
<DataFields>
</DataFields>
</DataTable>
<DataTable Name="PlannerCode" Access="WW">
<Licenses>Manufacturing, DemandManagement</Licenses>
<Flags>
</Flags>
<KeyFields>
<StringField Name="Value"/>
<ReferenceField Name="Site" Target="Core::Site" SetField="PlannerCodes"/>
</KeyFields>
<DataFields>
<StringField Name="Description"/>
</DataFields>
</DataTable>
</SchemaPackage>
Despite the edit you made, I can only get your XML to validate if I modify it slightly (cleaning the opening XML tag and removing the SchemaPackage namespace). Regardless,
if you're experiencing no issues with your XML import then it's fine.
Here I'm just constructing the XML object from a herestring because I haven't got it in a file on disk.
[xml]$xmldata = #"
<xml>
<DataTable Name="CC::Attribute">
<DataFields>
</DataFields>
</DataTable>
<DataTable Name="PlannerCode">
<Licenses>Manufacturing, DemandManagement</Licenses>
<Flags>
</Flags>
<KeyFields>
<StringField Name="Value"/>
<ReferenceField Name="Site" Target="Core::Site" SetField="PlannerCodes"/>
</KeyFields>
<DataFields>
<StringField Name="Description"/>
</DataFields>
</DataTable>
</xml>
"#
# Filter DataTable nodes for those with a KeyFields child node.
$DataTablesWithKeyFields = $xmldata.xml.DataTable | Where-Object { $_.KeyFields }
$DataTableName = $DataTablesWithKeyFields.Name
$StringFieldData = $DataTablesWithKeyFields.KeyFields.ReferenceField
$ReferenceFieldData = $DataTablesWithKeyFields.KeyFields.ReferenceField
I'm not sure if that's what you're after. $DataTablesWithKeyFields could be an array depending on your XML file so you may need to loop it to extract the information you require.
Since we're working with XML, one of the querying options is XPath!
You can select only DataTable nodes that have a KeyFields child with the following XPath expression:
/SchemaPackage/DataTable[KeyFields]
You can use Select-Xml:
Select-Xml -Path C:\hackathon\Mfg.xml -XPath /SchemaPackage/DataTable[KeyFields] |Select-Object -Expand Node
or pass the expression as an argument to the SelectSingleNodes() method:
[xml]$xmldata = Get-Content C:\hackathon\Mfg.xml
$xmldata.SelectNodes('/SchemaPackage/DataTable[KeyFields]')

spark dataframe handle option some none

Similar to Apache Spark: dealing with Option/Some/None in RDDs I have a function which is applied via df.mapPartitions
def mapToTopics(iterator: Iterator[RawRecords]): Iterator[TopicContent] = {
iterator.map(k => {
browser.parseString(k.content) >> elementList("doc").map(d => {
TopicContent((d >> text("docno")).head, (d >> text("text")).head, k.path)
})
})
}
The following is also defined:
#transient lazy val browser = JsoupBrowser()
case class TopicContent(topic: String, content: String, filepath: String)
case class RawRecords(path: String, content: String)
Above will throw an error (NoSuchElementException) if no xml tags with text exist (which happens for some malformed documents)
How can I correct and simplify this code to properly handle the options?
When trying to use a util.Try as outlined by the link above and applying a flatMap my code would fail, as instead of Element it was using Char
edit
try {
Some(TopicContent((d >> text("docno")).head, (d >> text("text")).head, k.path))
} catch {
case noelem: NoSuchElementException => {
println(d.head)
None
}
}
})
val flattended = results.flatten
Will unfortunately only return a Option[Nothing]
edit4
https://gist.github.com/geoHeil/bfb01427b88cf58ea755f912ce539712 a minimal sample without spark (and full code below as well)
import net.ruippeixotog.scalascraper.browser.JsoupBrowser
import net.ruippeixotog.scalascraper.dsl.DSL.Extract._
import net.ruippeixotog.scalascraper.dsl.DSL._
import net.ruippeixotog.scalascraper.scraper.ContentExtractors.elementList
#transient lazy val browser = JsoupBrowser()
val broken =
"""
|<docno>
| LA051089-0001
| </docno>
| <docid>
| 54901
| </docid>
| <date>
| <p> May 10, 1989, Wednesday, Home Edition </p>
| </date>
| <section>
| <p> Metro; Part 2; Page 2; Column 2 </p>
| </section>
| <graphic>
| <p> Photo, Cloudy and Clear A stormy afternoon provides a clear view of Los Angeles' skyline, with the still-emerging Library Tower rising above its companion buildings. KEN LUBAS / Los Angeles Times </p>
| </graphic>
| <type>
| <p> Wild Art </p>
| </type>
""".stripMargin
val correct =
"""
|<DOC>
|<DOCNO> FR940104-0-00001 </DOCNO>
|<PARENT> FR940104-0-00001 </PARENT>
|<TEXT>
|
|<!-- PJG FTAG 4700 -->
|
|<!-- PJG STAG 4700 -->
|
|<!-- PJG ITAG l=90 g=1 f=1 -->
|
|<!-- PJG /ITAG -->
|
|<!-- PJG ITAG l=90 g=1 f=4 -->
|Federal Register
|<!-- PJG /ITAG -->
|
|<!-- PJG ITAG l=90 g=1 f=1 -->
|&blank;/&blank;Vol. 59, No. 2&blank;/&blank;Tuesday, January 4, 1994&blank;/&blank;Rules and Regulations
|
|<!-- PJG 0012 frnewline -->
|
|<!-- PJG /ITAG -->
|
|<!-- PJG ITAG l=01 g=1 f=1 -->
|Vol. 59, No. 2
|<!-- PJG 0012 frnewline -->
|
|<!-- PJG /ITAG -->
|
|<!-- PJG ITAG l=02 g=1 f=1 -->
|Tuesday, January 4, 1994
|<!-- PJG 0012 frnewline -->
|
|<!-- PJG 0012 frnewline -->
|
|<!-- PJG /ITAG -->
|
|<!-- PJG /STAG -->
|
|<!-- PJG /FTAG -->
|</TEXT>
|</DOC>
""".stripMargin
case class RawRecords(path: String, content: String)
case class TopicContent(topic: String, content: String, filepath: String)
val raw = Seq(RawRecords("first", correct), RawRecords("second", broken))
val result = mapToTopics(raw.iterator)
// Variant 1
def mapToTopics(iterator: Iterator[RawRecords]): Iterator[TopicContent] = {
iterator.flatMap(k => {
val documents = browser.parseString(k.content) >> elementList("doc")
documents.map(d => {
val docno = d >> text("docno")
// try {
val textContent = d >> text("text")
TopicContent(docno, textContent, k.path)
// } catch {
// case _:NoSuchElementException => TopicContent(docno, None, k.path)
// }
}) //.filter(_.content !=None)
})
}
// When broken down even further you see the following will produce Options of strings
browser.parseString(raw(0).content) >> elementList("doc").map(d => {
val docno = d >> text("docno")
val textContent = d >> text("text")
(docno.headOption, textContent.headOption)
})
// while below will now map to characters. What is wrong here?
val documents = browser.parseString(raw(0).content) >> elementList("doc")
documents.map(d => {
val docno = d >> text("docno")
val textContent = d >> text("text")
(docno.headOption, textContent.headOption)
})
The difference between the two examples lies in the precedence of the operators. When you're doing browser.parseString(raw(0).content) >> elementList("doc").map(...), you're calling map on elementList("doc"), and not on the whole expression. In order for the first example to behave the same as the second one, you need to write either (browser.parseString(raw(0).content) >> elementList("doc")).map(...) (recommended) or browser.parseString(raw(0).content) >> elementList("doc") map(...).
In the context of scala-scraper, the library you're using, the two expressions mean very different things. With browser.parseString(raw(0).content) >> elementList("doc") you're extracting a List[Element] from a document, and calling map on that does just what you'd expect from a collection. On the other hand, elementList("doc") is an HtmlExtractor[List[Element]] and calling map on an extractor creates a new HtmlExtractor with the results of the original extractor transformed. That's the reason why you end up with two very different results.
I am unfamiliar with the API you are using, but using headOpton in a for comprehension might help you:
import net.ruippeixotog.scalascraper.dsl.DSL._
import net.ruippeixotog.scalascraper.dsl.DSL.Extract._
import net.ruippeixotog.scalascraper.dsl.DSL.Parse._
iterator.map(k => {
browser.parseString(k.content) >> elementList("doc").flatMap(d => {
for {
docno <- text("docno")).headOption
text <- (d >> text("text")).headOption
} yield TopicContent(docno, text, k.path)
})
})
This way you only construct TopicContent, really a Some(TopicContent), when both docno and text are present--and None otherwise. Then the flatMap removes all the Nones and extracts the content in Somes leaving you with a collection of TopicContent instances created for all valid XML.