I have read some stuff about form data encoding, but one thing remains unclear. In case of enctype="application/x-www-form-urlencoded" we need to urlencode data by hand, don't we?
... Forms submitted with this content type must be encoded as follows
Must be encoded by whom? By browsers? Or by application developers?
The other thing is -- what encoding (if any) is used, or should be used, in case of multipart/form-data?
I'm kindda mislead so big thx in advance.
Actually, browsers url-encode data automatically. And this w3 docs is first of all for those who make browsers. So that phrase, Forms submitted with this content type must be encoded as follows means that data should be encoded by browsers. Anyway, one can check it by viewing raw post in the form data handling script (in case of php in looks like file_get_contents("php://input");)
Related
Looking at the MS-OXPROPS, MS-OXCMSG and MS-OXCMAIL documentation, it is said that the user should include PidTagInternetCodePage to indicate the appropriate code page for the HTML content in order to parse it properly.
However, opening up the ole streams of the msg files, I could not find the 0x3FDE stream that indicates the code page id, but only found some semblance of a code page id in the compressed RTF stream (first line).
Am I looking at the streams wrongly or are the other properties hidden in other streams? If so, how do I look for them?
Thanks in advance.
The PidTagInternetCodePage property is not guaranteed to be present and is in no way required, especially if it is a Unicode MSG file. The HTML body can include the meta tag with encoding in the header, and even then, it won't be necessary if all Unicode characters in the HTML body are properly HTML-encoded (which is always a good idea).
I created my own persistence for SQL Server, and the CRUD works fine,
BUT I'm having some trouble with the enconding i think,
i receive the xml text from the XForms like that when i'm going to save something
?xml version="1.0" encoding="UTF-8"?xhtml:html xmlns:xhtml="http://www.w3 ...............
metadata
application-name w4/application-name
form-name usuario/form-name
title xml:lang="en"Cadastro/title
description xml:lang="en"Usuário/description ---------PROBLEM!!!
metadata
xforms:instance....................
Any ideas how to solve this??
In general, you need to make sure, when you are decoding the XML, to properly deal with the character encoding. How exactly to do that depends on the programming language or framework you are using, but you should:
if possible use an XML parser and just feed it the bytes (the parser will take care of handling the encoding by itself)
never assume a default or platform encoding when converting bytes to characters (Java in particular has a number of APIs which, for very wrong reasons, use a default encoding which is platform-dependent)
I am trying to support arbitrary unicode from a variety of international users. They have already put a bunch of data into sqlite databases on their iPhones, and now I want to capture the data into a database, then send it back to their device. Right now I am using a php page that is sending data back to from an internet mysql database. The data is saved in the mysql database properly, but when it's sent back it comes out as unicode text, such as
Frank\u00e2\u0080\u0099s iPad
instead of just
Frank's iPad
where the apostrophe should really be a curly apostrophe.
The answer posted to another question indicates that there is no built-in Cocoa methods to convert the "\u00e2\u0080\u0099" portion of the unicode string from the webserver to an NSString object. Is this correct?
That seems really surprising (and scarily disappointing), since Cocoa definitely allows input from many different Unicode characters, and I need to support any arbitrary language that I have never heard of, and all of the possible characters. I save them to and from the local sqlite database just fine now, but once I send it to a web server, then perhaps pull down different data, I want to ensure the data pulled from the web server is correctly formatted.
[...] there is no built-in Cocoa methods to convert [...]. Is this
correct?
It's not correct.
You might be interested in CFStringTransform and it's capabilities. It is a full blown ICU transformation engine, which can (also) perform your requested transformation.
See Using Objective C/Cocoa to unescape unicode characters, ie \u1234
All NSStrings are Unicode.
The problem with the “Frank\u00e2\u0080\u0099s iPad” data isn't that it's Unicode; it's that it's escaped to ASCII. “Frank’s iPad” is valid Unicode in any UTF, and is what you need.
So, you need to see whether the database is returning the data escaped or the PHP layer is escaping it at some point. If either of those is the case, fix it if you can; the PHP resource should return UTF-8/16/32. Only if that approach fails should you seek to unescape the string on the Cocoa side.
You're correct that there is no built-in way to unescape the string in Cocoa. If you get to that point, see if you can find some open-source code to do it; if not, you'll need to do it yourself, probably using NSScanner.
Check that your web service response has Content type and charset. Also that xml has encoding specified. In PHP you need to add the following before printing XML:
header('Content-type: text/xml; charset=UTF-8');
print '<?xml version="1.0" encoding="UTF-8"?>';
I guess there is just no encoding specified.
We have a system that sends out regular emails with links in, many of which contain URL encoded parameters such as this:
href="http://www.mydomain.com/login.aspx?returnurl=http%3A%2F%2Fwww.mydomain.com%2Fview.aspx%3Fid%3D1234%26alert%3Dtrue"
You can see that the "returnurl" parameter is encoded. However, it seems that a large number of our users (seemingly hotmail) are receiving the emails with this paramater partly decoded such as:
href="http://www.mydomain.com/login.aspx?returnurl=http://www.mydomain.com/view.aspx?view.aspx%3Fid%3D1234%26alert%3Dtrue"
Why would it decode like this? Why only partly decode?? I therefore have no idea how to deal with it. I thought of base-64 encoding but that base64 strings contain characters that would need decoding too... I thought of double encoding but then I will not know whether to double-decode the parameter or not... Can anyone help? Thanks.
One reason this could be happening is because url rules for encoding are different before and after ? so if mechanism that is doing decoding does it from the 'back' of url and apples query decoding rules until it finds first ? then this could cause problem you are describing...
Not sure how to deal with it though as I understand system that does this inappropriate decoding is outside of your control. I would try to hide the ? in return url query somehow...
I'm using a Google Gears Worker to submt a POST httprequest (using var request = google.gears.factory.create('beta.httprequest'); )
with a parameter containing the string
"bford%20%24%23%26!%3F%40%20%E5%BE%B3%E5%8A%9B%E5%9F%BA%E5%BD%A6"
but the Django HttpRequest is receiving it as "bford $#&!?# å¾³å\u008a\u009bå\u009fºå½¦"
How do I specify to one or the other of the parties in the transaction to leave it untranslated?
Check the HttpRequest.encoding and the DEFAULT_CHARSET settings. Judging by the encoded value, this should be UTF-8 (which is indeed usually the right thing).
You can get the ‘untranslated’ (with %s still in) value by looking at the input stream (for POST) or environ QUERY_STRING (for GET) and decoding it manually, but it would be better to fix Django's incorrect string-to-unicode decoding really.
As I understand it, Django 1.0 should default to using UTF-8, so I'm not sure why it's not in your case.