I'm getting very strange error when I trying to convert a string to XML in MS SQL Server:
Msg 9420, Level 16, State 1, Line 5
XML parsing: line 1, character 8071, illegal xml character
If I check the string in some text editor, I can see that its length is 8070. Why is it complaining about character 8071 if it does not exist?
This is how I'm converting string to XML:
CAST(REPLACE(SUBSTRING(
REPLACE(REPLACE(REPLACE(ResponseData,'ä','a'),'ö','o'),'å','a'),
PATINDEX('%<?xml%',ResponseData), PATINDEX('%sonType>', ResponseData)+6),
'<?xml version="1.0" encoding="utf-16"?>',
'<?xml version="1.0" encoding="utf-8"?>')as XML) as ResponseData
Are any of replaces causing the problem?
UPD: The problem also is that in ResponseData column the XML string is stored together with some other data. Example:
Error from service: <Some error description>. Sent request: <?xml version="1.0" encoding="utf-16"?><Contents of the XML>
So I need to get that XML string from the column and then convert it to XML.
You could try to change original encoding from UTF-16 to ISO-8859-1, or a more precise encoding for your characters:
DECLARE #data varchar(max) = '<?xml version="1.0" encoding="utf-16"?><...>'
SELECT CAST(REPLACE(#data,
'<?xml version="1.0" encoding="utf-16"?>',
'<?xml version="1.0" encoding="iso-8859-1"?>') AS XML) ResponseData
Related
Facing issues while parsing this response using cypress,
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Header/>
<soapenv:Body>
<p385:summaryOutputArray xmlns:p385="http://dataobject.simulation.com">
<p385:userName>MS</p385:userName>
</p385:summaryOutputArray>
</soapenv:Body>
</soapenv:Envelope>
Using Cypress How to read userName TAG? I can see in the log that whole xml is printed but can't get to the particular tag. Also, while using the function to get to the particular tag to get the value, I am getting null property
Firstly I used this. This is giving error. property reading null
const parser = new DOMParser();
const xmlDOM = parser.parseFromString(quoteResp, 'text/xml');
cy.log('xmlDOM ' + quoteResp.);
cy.wrap(Cypress.$(quoteResp)).then(quoteResp => {
const txt = quoteResp.filter('Body').find('p385:userName').text();
cy.log('value:' + txt);
});
Using this I can see the whole response in logs
then(quoteResp => {cy.log('xmlDOM ' + quoteResp.body);
It's not necessary to wrap the response, just query the xmlDOM like this
const xmlDOM = parser.parseFromString(quoteResp, 'application/xml');
const userName = xmlDOM.querySelector('userName').textContent
expect(userName).to.eq('MS')
I need to parse and print ns4:feature part. Karate prints it in json format. I tried referring to this answer. But, i get 'ERROR: 'Namespace for prefix 'xsi' has not been declared.' error, if used suggested xPath. i.e.,
* def list = $Test1/Envelope/Body/getPlan/planSummary/feature[1]
This is my XML: It contains lot many parts with different 'ns' values, but i have given here an extraxt.
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Header/>
<S:Body>
<ns9:getPlan xmlns:ns10="http://xmlschema.test.com/xsd_v8" xmlns:ns9="http://xmlschema.test.com/srv/SMO_v4" xmlns:ns8="http://xmlschema.test.com/xsd/Customer_v2" xmlns:ns7="http://xmlschema.test.com/xsd/Customer/Customer_v4" xmlns:ns6="http://schemas.test.com/eca/common_types_2_1" xmlns:ns5="http://xmlschema.test.com/xsd/Customer/BaseTypes_1_0" xmlns:ns4="http://xmlschema.test.com/xsd_v4" xmlns:ns3="http://xmlschema.test.com/xsd/Enterprise/BaseTypes/types/ping_v1" xmlns:ns2="http://xmlschema.test.com/xsd/common/exceptions/Exceptions_v1_0">
<ns9:planSummary xsi:type="ns4:Plan" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ns5:code>XPBSMWAT</ns5:code>
<ns5:description>Test Plan</ns5:description>
<ns4:category xsi:nil="true"/>
<ns4:effectiveDate>2009-11-05</ns4:effectiveDate>
<ns4:sharingGroupList>
<ns4:sharingCode>CAD_DATA</ns4:sharingCode>
<ns4:contributingInd>true</ns4:contributingInd>
</ns4:sharingGroupList>
<ns4:feature>
<ns5:code>ABC</ns5:code>
<ns5:description>Service</ns5:description>
<ns5:descriptionFrench>Service</ns5:descriptionFrench>
<ns4:poolGroupId xsi:nil="true"/>
<ns4:switchCode/>
<ns4:type/>
<ns4:dtInd>false</ns4:dtInd>
<ns4:usageCharge>0.0</ns4:usageCharge>
<ns4:connectInd>false</ns4:connectInd>
</ns4:feature>
</ns9:planSummary>
</ns9:getPlan>
</S:Body>
</S:Envelope>
This is the xPath i used;
Note: I saved above xml in a separate file test1.xml. I am just reading it and parsing the value.
* def Test1 = read('classpath:PP1/data/test1.xml')
* def list = $Test1/Envelope/Body/*[local-name()='getPlan']/*[local-name()='planSummary']/*[local-name()='feature']/*
* print list
This is the response i am getting;
16:20:10.729 [ForkJoinPool-1-worker-1] INFO com.intuit.karate - [print] [
"ABC",
"Service",
"Service",
"",
"",
"",
"false",
"0.0",
"false"
]
How can i get the same in XML?
This is interesting, I haven't seen this before. The problem was you have an attribute with a namespace xsi:nil="true" which is causing problems when you take a sub-set of the XML but the namespace is not defined anymore. If you remove it first, things will work.
Try this:
* remove Test1 //poolGroupId/#nil
* def temp = $Test1/Envelope/Body/getPlan/planSummary/feature
Another approach you could have tried is to do a string replace to remove troublesome stuff in the XML before doing XPath.
EDIT: added info on how to do a string replace using Java. The below will strip out the entire xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns4:Plan" part.
* string temp = Test1
* string temp = temp.replaceAll("xmlns:xsi[^>]*", "")
* print temp
So you get the idea. Just use regex.
Also see: https://stackoverflow.com/a/50372295/143475
Sample text file contains:
`<?xml version="1.0" encoding="UTF-8" ?>
<Document xmlns ="urn:iso:std:iso:20022:tech:xsd:camt.056.001.01"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<FIToFIPmtCxlReq>
<Assgnmt>
<Id>ID123456</Id>
<Assgnr>
<Agt>
<FinInstnId>
<BIC>BICSEND</BIC>
</FinInstnId>
</Agt>
</Assgnr>
<Assgne>
<Agt>
<FinInstnId>
<BIC>BICRCV</BIC>
</FinInstnId>
</Agt>
</Assgne>
<CreDtTm>2020-12-16T09:05:15.0Z</CreDtTm>
</Assgnmt>
<CtrlData>
<NbOfTxs>1</NbOfTxs>
<CtrlSum>0</CtrlSum>
</CtrlData>
<Undrlyg>
<TxInf>
<CxlId>20201216.105.19344855940590400</CxlId>
<OrgnlGrpInf>
<OrgnlMsgId>REF123456789</OrgnlMsgId>
<OrgnlMsgNmId>pacs.008</OrgnlMsgNmId>
</OrgnlGrpInf>
<OrgnlInstrId>FT123456</OrgnlInstrId>
<OrgnlEndToEndId>NOTPROVIDED</OrgnlEndToEndId>
<OrgnlTxId>20201216.100.02202020</OrgnlTxId>
<OrgnlIntrBkSttlmAmt Ccy="EUR">25.23</OrgnlIntrBkSttlmAmt>
<OrgnlIntrBkSttlmDt>2020-12-16</OrgnlIntrBkSttlmDt>`
Please be informed that I would like to code PowerShell to extract the data in tag <OrgnlIntrBkSttlmAmt> (please note that the data length can change since this is an amount field) and then replace the "0" in tag <CtrlSum> with "25.23".
Can someone help me with this.
Thank you for your time.
The xml you show us is invalid as it is missing the following closing tags:
</TxInf>
</Undrlyg>
</FIToFIPmtCxlReq>
</Document>
If I add these, you could do this to update the value in the <CtrlSum> tag:
# load the xml from file
[xml]$xml = Get-Content -Path 'D:\Test\test.xml' -Raw
# get the amount from the 'OrgnlIntrBkSttlmAmt' tag
$amount = $xml.Document.FIToFIPmtCxlReq.Undrlyg.TxInf.OrgnlIntrBkSttlmAmt.'#text'
# use that amount to put in the 'CtrlSum' tag
$xml.Document.FIToFIPmtCxlReq.CtrlData.CtrlSum = $amount
# save the updated xml to file
$xml.Save('D:\Test\test.xml')
I'm using:
PHPUnit 3.6.12 / PHP 5.3.1 / MySQL 5.1.30
I'm trying to compare the value inserted by a function in a database with the value I expect.
The value is a string CONTAINING ACCENTS.
So I created a xml file: expectedValue.xml (file encoded in UTF-8)
<?xml version="1.0" encoding="UTF-8"?>
<dataset>
<table name="MyTable">
<column>MyColumn</column>
<row>
<value>résumé</value>
</row>
</table>
</dataset>
Here is the code in the test method (file encoded in UTF-8 too)
public function testSave()
{
// this function saves the data in an UTF-8 database
save('résumé');
$queryTable = $this->getConnection()->createQueryTable('MyTable', 'SELECT MyColumn FROM MyTable') ;
$expectedTable = $this->createXMLDataSet('expectedValue.xml)->getTable('MyTable') ;
$this->assertTablesEqual($expectedTable, $queryTable) ;
}
And here is the result I get:
Failed asserting that
MYTable
MyColumn
résumé
is equal to expected
MyTable
MyColumn
résumé
Does anyone know where this encoding problem may come from ??
Thanks !!
Could possibly be the database connection
When you're connecting to MySQL (in your getConnection() method), you need to make sure you explicitly set UTF-8.
$pdo = new PDO(
'mysql:host=hostname;dbname=defaultDbName',
'username',
'password',
array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8")
);
If you're not using MySQL, you can search for ways to set the charset.
I am feeling like i am doing something really not correct.
When doing a soap they return me with an xml which may or may not contain an error.
I would like to check if the error exists if not read the values.
somehow, I can't grab it directly :(
Below is a sample return of something with results and one which gives an error (name not found)
<?xml version="1.0" encoding="utf-8"?>
<soapEnvelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
<envHeader xmlns:env="http://www.w3.org/2003/05/soap-envelope">
<wsaAction>http://www.rechtspraak.nl/namespaces/ccr01/searchPersonResponse</wsaAction>
<wsaMessageID>urn:uuid:b75d2932-5687-4871-9d07-3b74b084978a</wsaMessageID>
<wsaRelatesTo>urn:uuid:9112d870-248d-4d07-acd0-d88e4a48d547</wsaRelatesTo>
<wsaTo>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsaTo>
<wsseSecurity>
<wsuTimestamp wsu:Id="Timestamp-061df7b5-32a2-4021-852d-2df98953e076">
<wsuCreated>2011-05-27T12:11:45Z</wsuCreated>
<wsuExpires>2011-05-27T12:16:45Z</wsuExpires>
</wsuTimestamp>
</envHeader>
<soapBody>
<searchPersonResponse xmlns="http://www.rechtspraak.nl/namespaces/ccr01">
<searchPersonResult>
<CCR_WS xmlns="http://www.rechtspraak.nl/namespaces/ccr">
<curandus>
<ccn>1</ccn>
<cur_voornamen>Jan</cur_voornamen>
<cur_voorvoegsels>van</cur_voorvoegsels>
<cur_achternaam>Beek</cur_achternaam>
<geboorte_datum>1980-01-02</geboorte_datum>
<geboorte_plaats>Werkendam</geboorte_plaats>
</curandus>
</CCR_WS>
</searchPersonResult>
</searchPersonResponse>
</soapBody>
</soapEnvelope>
and the one without results
<?xml version="1.0" encoding="utf-8"?>
<soapEnvelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
<envHeader xmlns:env="http://www.w3.org/2003/05/soap-envelope">
<wsaAction>http://www.rechtspraak.nl/namespaces/ccr01/searchPersonResponse</wsaAction>
<wsaMessageID>urn:uuid:b75d2932-5687-4871-9d07-3b74b084978a</wsaMessageID>
<wsaRelatesTo>urn:uuid:9112d870-248d-4d07-acd0-d88e4a48d547</wsaRelatesTo>
<wsaTo>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsaTo>
<wsseSecurity>
<wsuTimestamp wsu:Id="Timestamp-061df7b5-32a2-4021-852d-2df98953e076">
<wsuCreated>2011-05-27T12:11:45Z</wsuCreated>
<wsuExpires>2011-05-27T12:16:45Z</wsuExpires>
</wsuTimestamp>
</envHeader>
<soapBody>
<searchPersonResponse xmlns="http://www.rechtspraak.nl/namespaces/ccr01">
<searchPersonResult>
<CCR_WS xmlns="http://www.rechtspraak.nl/namespaces/ccr">
<exceptie errorcode="1">No Results found.</exceptie>
</CCR_WS>
</searchPersonResult>
</searchPersonResponse>
</soapBody>
</soapEnvelope>
Here is my code to select the namespace, then check
$results = simplexml_load_string($response);
$results->registerXPathNamespace('ccr','http://www.rechtspraak.nl/namespaces/ccr');
$lijst = $results->xpath('//ccr:CCR_WS');
$errorcode = $lijst[0]->exceptie->attributes()->errorcode;
$error = $lijst[0]->exceptie;
if (isset($errorcode) AND $errorcode != "") {
// do things with the error code
} else {
$lijst = $results->xpath('//ccr01:searchPersonResult');
$cur = $lijst[0]->CCR_WS->curandus;
echo $cur->ccn."<BR>";
echo $cur->cur_voornamen."<BR>";
echo $cur->cur_voorvoegsels."<BR>";
echo $cur->cur_achternaam."<BR>";
echo $cur->geboorte_datum."<BR>";
echo $cur->geboorte_plaats."<BR>";
}
surely there is a better way of grabbing
$lijst[0]->exceptie->attributes()->errorcode
for example...
...Don't know if this is a "better way" to everyone, but here is a direct XPath expression to select the errorcode. You can make it shorter and less efficient by dropping steps and using // (in the beginning or in the middle). Attributes are selected with # (or with attribute:: axis if you prefer the longer syntax). If attribute (or the exceptie element) doesn't exist, nothing is returned.
/*/*/ccr01:searchPersonResponse/ccr01:searchPersonResult/ccr:CCR_WS/ccr:exceptie/#errorcode
Remember to register all the namespace prefixes that yo use in your XPath expression.