PHPUnit - assertTablesEqual encoding accents - encoding

I'm using:
PHPUnit 3.6.12 / PHP 5.3.1 / MySQL 5.1.30
I'm trying to compare the value inserted by a function in a database with the value I expect.
The value is a string CONTAINING ACCENTS.
So I created a xml file: expectedValue.xml (file encoded in UTF-8)
<?xml version="1.0" encoding="UTF-8"?>
<dataset>
<table name="MyTable">
<column>MyColumn</column>
<row>
<value>résumé</value>
</row>
</table>
</dataset>
Here is the code in the test method (file encoded in UTF-8 too)
public function testSave()
{
// this function saves the data in an UTF-8 database
save('résumé');
$queryTable = $this->getConnection()->createQueryTable('MyTable', 'SELECT MyColumn FROM MyTable') ;
$expectedTable = $this->createXMLDataSet('expectedValue.xml)->getTable('MyTable') ;
$this->assertTablesEqual($expectedTable, $queryTable) ;
}
And here is the result I get:
Failed asserting that
MYTable
MyColumn
résumé
is equal to expected
MyTable
MyColumn
résumé
Does anyone know where this encoding problem may come from ??
Thanks !!

Could possibly be the database connection
When you're connecting to MySQL (in your getConnection() method), you need to make sure you explicitly set UTF-8.
$pdo = new PDO(
'mysql:host=hostname;dbname=defaultDbName',
'username',
'password',
array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8")
);
If you're not using MySQL, you can search for ways to set the charset.

Related

html2pdf not showing character correctly, encoding for ē

I'm struggling with some characters in a PDF I'm trying to create with html2pdf. The following code creates the PDF, but ē is shown an e.
$html2pdf=new Html2Pdf();
$html2pdf->writeHTML('<h1>Fēnix</h1>');
$html2pdf->output();
When getting the name from my database, ē is shown a ?.
$query=$mysqli->query('SELECT name FROM table WHERE id=1;');
$result=$query->fetch_assoc();
$html2pdf=new Html2Pdf();
$html2pdf->writeHTML('<h1>'.$result['name'].'</h1>');
$html2pdf->output();
This is the way I connect to my database:
$mysqli=new mysqli('host', 'user', 'pass', 'db');
I have also tried adding a charset:
$mysqli->set_charset('utf8');
Or initiating the class with parameters:
$html2pdf=new Html2Pdf('P', 'A4', 'nl');
$html2pdf=new Html2Pdf('P', 'A4', 'nl', true, 'UTF8');
Other characters that are giving issues are: Ś ą ł ś
Both server and database are UTF-8.
The solution is to apply a UTF-8 font to all elements.
* { font-family:freeserif; }

Karate: Replace text at Envelope level [duplicate]

I need to parse and print ns4:feature part. Karate prints it in json format. I tried referring to this answer. But, i get 'ERROR: 'Namespace for prefix 'xsi' has not been declared.' error, if used suggested xPath. i.e.,
* def list = $Test1/Envelope/Body/getPlan/planSummary/feature[1]
This is my XML: It contains lot many parts with different 'ns' values, but i have given here an extraxt.
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Header/>
<S:Body>
<ns9:getPlan xmlns:ns10="http://xmlschema.test.com/xsd_v8" xmlns:ns9="http://xmlschema.test.com/srv/SMO_v4" xmlns:ns8="http://xmlschema.test.com/xsd/Customer_v2" xmlns:ns7="http://xmlschema.test.com/xsd/Customer/Customer_v4" xmlns:ns6="http://schemas.test.com/eca/common_types_2_1" xmlns:ns5="http://xmlschema.test.com/xsd/Customer/BaseTypes_1_0" xmlns:ns4="http://xmlschema.test.com/xsd_v4" xmlns:ns3="http://xmlschema.test.com/xsd/Enterprise/BaseTypes/types/ping_v1" xmlns:ns2="http://xmlschema.test.com/xsd/common/exceptions/Exceptions_v1_0">
<ns9:planSummary xsi:type="ns4:Plan" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<ns5:code>XPBSMWAT</ns5:code>
<ns5:description>Test Plan</ns5:description>
<ns4:category xsi:nil="true"/>
<ns4:effectiveDate>2009-11-05</ns4:effectiveDate>
<ns4:sharingGroupList>
<ns4:sharingCode>CAD_DATA</ns4:sharingCode>
<ns4:contributingInd>true</ns4:contributingInd>
</ns4:sharingGroupList>
<ns4:feature>
<ns5:code>ABC</ns5:code>
<ns5:description>Service</ns5:description>
<ns5:descriptionFrench>Service</ns5:descriptionFrench>
<ns4:poolGroupId xsi:nil="true"/>
<ns4:switchCode/>
<ns4:type/>
<ns4:dtInd>false</ns4:dtInd>
<ns4:usageCharge>0.0</ns4:usageCharge>
<ns4:connectInd>false</ns4:connectInd>
</ns4:feature>
</ns9:planSummary>
</ns9:getPlan>
</S:Body>
</S:Envelope>
This is the xPath i used;
Note: I saved above xml in a separate file test1.xml. I am just reading it and parsing the value.
* def Test1 = read('classpath:PP1/data/test1.xml')
* def list = $Test1/Envelope/Body/*[local-name()='getPlan']/*[local-name()='planSummary']/*[local-name()='feature']/*
* print list
This is the response i am getting;
16:20:10.729 [ForkJoinPool-1-worker-1] INFO com.intuit.karate - [print] [
"ABC",
"Service",
"Service",
"",
"",
"",
"false",
"0.0",
"false"
]
How can i get the same in XML?
This is interesting, I haven't seen this before. The problem was you have an attribute with a namespace xsi:nil="true" which is causing problems when you take a sub-set of the XML but the namespace is not defined anymore. If you remove it first, things will work.
Try this:
* remove Test1 //poolGroupId/#nil
* def temp = $Test1/Envelope/Body/getPlan/planSummary/feature
Another approach you could have tried is to do a string replace to remove troublesome stuff in the XML before doing XPath.
EDIT: added info on how to do a string replace using Java. The below will strip out the entire xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="ns4:Plan" part.
* string temp = Test1
* string temp = temp.replaceAll("xmlns:xsi[^>]*", "")
* print temp
So you get the idea. Just use regex.
Also see: https://stackoverflow.com/a/50372295/143475

MYSQLi PHP connect

Is the following a valid mysqli php connect insert statement?
if($mysql_query = $conn->query("INSERT INTO users (username,password,question,securityanswer) VALUES ('$uname','$upass','$selectquestion','$answer')"))
{
?>
<script>alert('successfully registered ');</script>
<?php
}
else
{
?>
<script>alert('error while registering you...');</script>
<?php
}
}
However I am getting error as "error while registering you..."
Thanks
Issue is resolved. There was an issue with the "user_id" column primary key which was being updated duplicate(0 value) with this error "Duplicate entry '0' for key 'PRIMARY'. Upon updating user_id column with auto increment in phpmyadmin, the INSERT query got updated successfully.
Using the below code to check for errors helped.
die(mysqli_error($conn));
A simple example:
<?php
//Create the connection
$con = mysqli_connect("HostName","UserName","password","DBName") or die("Some error occurred during connection " . mysqli_error($con));
// Write query
$strSQL = "SELECT username FROM MyTable";
// Execute the query.
$query = mysqli_query($con, $strSQL);
while($result = mysqli_fetch_array($query))
{
echo $result["username"]."
";
}
// Close the connection
mysqli_close($con);
?>
Can you paste the whole block with connection statement?
Otherwise SQL statement looks fine. Excepct you should escape inputs to MySQL
if($mysql_query = $conn->query("INSERT INTO users (username,password,question,securityanswer) VALUES ('$uname','$upass','$selectquestion','$answer')"))
Why using if($mysql_query = $conn->query ?
better
if($conn->query("INSERT INTO users (username,password,question,securityanswer) VALUES ('$uname','$upass','$selectquestion','$answer')") === TRUE)

XML parsing, illegal character in the end of the string

I'm getting very strange error when I trying to convert a string to XML in MS SQL Server:
Msg 9420, Level 16, State 1, Line 5
XML parsing: line 1, character 8071, illegal xml character
If I check the string in some text editor, I can see that its length is 8070. Why is it complaining about character 8071 if it does not exist?
This is how I'm converting string to XML:
CAST(REPLACE(SUBSTRING(
REPLACE(REPLACE(REPLACE(ResponseData,'ä','a'),'ö','o'),'å','a'),
PATINDEX('%<?xml%',ResponseData), PATINDEX('%sonType>', ResponseData)+6),
'<?xml version="1.0" encoding="utf-16"?>',
'<?xml version="1.0" encoding="utf-8"?>')as XML) as ResponseData
Are any of replaces causing the problem?
UPD: The problem also is that in ResponseData column the XML string is stored together with some other data. Example:
Error from service: <Some error description>. Sent request: <?xml version="1.0" encoding="utf-16"?><Contents of the XML>
So I need to get that XML string from the column and then convert it to XML.
You could try to change original encoding from UTF-16 to ISO-8859-1, or a more precise encoding for your characters:
DECLARE #data varchar(max) = '<?xml version="1.0" encoding="utf-16"?><...>'
SELECT CAST(REPLACE(#data,
'<?xml version="1.0" encoding="utf-16"?>',
'<?xml version="1.0" encoding="iso-8859-1"?>') AS XML) ResponseData

How to send email to recipient with umlauts in domain name?

In my app I have to send email to recipient who has umlauts in domain name.
Example:
"test#äöü.test.com"
I'm using cfmail tag and I'm getting such error:
"invalid definition for attribute to at tag mail"
"Invalid E-Mail Address definition (test#äöü.test.com)"
Is there any way to send email to such recipients in coldfusion?
There is even a easier solution! Why not use Oracles built in class:
http://download.oracle.com/javase/6/docs/api/java/net/IDN.html#toUnicode(java.lang.String)
Then you only have to do this (example shows from punycode to Unicode):
<cfset strUrl = "xn--land-poa.se" />
<!--- Create a Java URL. --->
<cfset jUrl = CreateObject( "java", "java.net.IDN" ).toUnicode(strUrl) />
<cfoutput>
#jUrl#
You don´t have to download anything!
I'm no I18N expert but I was intrigued enough to investigate and come up with the following solution.
The problem is essentially how to send mail to Internationalised Domain Names (IDN), i.e. those which contain non-ASCII characters. IDNs are valid nowadays but not recognized by many systems including Java (and therefore ColdFusion, which uses the Java validation for CFMAIL address fields - hence the error you're seeing).
For a system to recognise an IDN it needs to be converted to an ASCII form called Punycode. For example müller.org needs to be converted to xn--mller-kva.org
LibIdn is an OS java library that will do this and the following code shows how you can hook it up to CF using Mark Mandel's JavaLoader.
<cffunction name="convertIdnToAscii" returntype="string" output="false">
<cfargument name="domain" type="string" required="true">
<cfscript>
var local = {};
// these paths assume the JavaLoader folder and the libidn-1.22.jar are in the same folder as the cfm template.
local.javaLoaderPath = "javaLoader.JavaLoader";
local.idnLibPath = ExpandPath( "libidn-1.22.jar" );
// convert the IDN lib path to an array which is what JavaLoader expects
local.libPathArray = [ local.idnLibPath ];
//load the IDN Lib
loader = CreateObject( "component",local.javaLoaderPath ).init( local.libPathArray );
// create an instance of the IDN lib
local.idn = loader.create( "gnu.inet.encoding.IDNA" ).init();
// convert the domain name
return local.idn.toASCII( arguments.domain );
</cfscript>
</cffunction>
<cffunction name="convertIdnAddress" returntype="string" output="false">
<cfargument name="address" type="string" required="true">
<cfscript>
var local = {};
local.domain = GetToken( arguments.address,2,"#" );
local.converted = convertIdnToAscii( local.domain );
return Replace( arguments.address,local.domain,local.converted );
</cfscript>
</cffunction>
<!--- Loop over a list of addresses and convert them if necessary --->
<cfset processedAddresses = []>
<cfloop list="test#äöü.test.com,test#example.com" index="address">
<cfif( NOT IsValid( "email",address ) )>
<cfset address = convertIdnAddress( address )>
</cfif>
<cfmail server="0.0.0.0" from="sender#mydomain.com" to="#address#" subject="test">Message</cfmail>
<cfset ArrayAppend( processedAddresses,address )>
</cfloop>
<cfdump var="#processedAddresses#">
This will send 2 emails (to a non-existent mailserver) and dump the converted addresses:
test#xn--4ca0bs.test.com
test#example.com
Notes:
To get the libidn jar file, download and extract the tar and look for it in the Java directory
The above assumes the libidn jar and JavaLoader package are located in the same folder as the template contain the CF code
The above should work on CF8 and above, although I've only tested it on CF9.
Be aware there's no error handling for addresses that might be invalid for reasons other than it containing an IDN.