I'm using AFNetworking in my iOS App. I've discovered a problem that occurs when request parameters contain percent signs. Then the encoding fails. The method AFURLEncodedStringFromStringWithEncoding returns nil.
I've tested this code and it prints (null):
NSLog(#"%#", AFURLEncodedStringFromStringWithEncoding(#"%", NSUTF8StringEncoding));
The expected output should be: %25. Other characters can be encoded with no problem.
The method is defined as follows:
NSString * AFURLEncodedStringFromStringWithEncoding(NSString *string, NSStringEncoding encoding) {
static NSString * const kAFLegalCharactersToBeEscaped = #"?!##$^&%*+,:;='\"`<>()[]{}/\\|~ ";
// Following the suggestion in documentation for `CFURLCreateStringByAddingPercentEscapes` to "pre-process" URL strings (using stringByReplacingPercentEscapesUsingEncoding) with unpredictable sequences that may already contain percent escapes.
return [(NSString *)CFURLCreateStringByAddingPercentEscapes(kCFAllocatorDefault, (CFStringRef)[string stringByReplacingPercentEscapesUsingEncoding:encoding], NULL, (CFStringRef)kAFLegalCharactersToBeEscaped, CFStringConvertNSStringEncodingToEncoding(encoding)) autorelease];
}
Any ideas what's going wrong here?
EDIT: The issue has been fixed in AFNetworking.
I don't know about the API but URL encoding is normally done by
UTF-8 encoding the string portions--path, arguments, etc--into bytes
applying percent-encoding of the byte values to obtain ASCII-only string pieces
assembling the URL pieces together with the separators :, /, ?, & etc.
encoding that string into bytes again
sending it over the wire
See https://www.rfc-editor.org/rfc/rfc3986#section-2.5 for nuances. Note that the RFC states that if the URL refers to a document on an EBCDIC system, the URL should specify the byte values for the EBCDIC encoding of its name, not the string name that human users of the EBCDIC system know it by. So a URL is ultimately a byte-string not a character-string.
As to how to get your API to represent that byte-string correctly, I am unsure.
Just removing the stringByReplacingPercentEscapesUsingEncoding:encoding is fine, no need to encode twice, so the return value should be:
return [(NSString *)CFURLCreateStringByAddingPercentEscapes(kCFAllocatorDefault,
(CFStringRef)string,
NULL,
(CFStringRef)kAFLegalCharactersToBeEscaped,
CFStringConvertNSStringEncodingToEncoding(encoding)) autorelease];
Related
I know how to encode a string in URL format (the smiley face is intentional):
let str = "www.mywebsite.com/๐.html"
let escapedStr = str.stringByAddingPercentEncodingWithAllowedCharacters(NSCharacterSet.URLPathAllowedCharacterSet())!
print(escapedStr)
// Output:
// www.mywebsite.com/%F0%9F%98%80.html
But if I attach http:// to the unescaped string Swift escapes the colon too:
let str = "http://www.mywebsite.com/๐.html"
let escapedStr = str.stringByAddingPercentEncodingWithAllowedCharacters(NSCharacterSet.URLPathAllowedCharacterSet())!
print(escapedStr)
// Output
// http%3A//www.mywebsite.com/%F0%9F%98%80.html
So short of removing and adding http:// manually, how can I properly escape those strings? There are other prefixes I must handle handle like https://, ftp:// or ssh://
: is not a legal character in the path part of an URL. You percent-encoded everything not in URLPathAllowedCharacterSet, so it shouldn't be surprising that the : was encoded.
Each part of an URL has different encoding rules. iOS can't correctly encode an URL until it knows what goes in what part, and it can't do that from an unencoded string (since it'd have to parse it first, and it can't parse it because it's not correctly encoded yet). In some systems (including older versions of iOS), it would use various heuristics that assumed "well, I guess you probably meant..." rather than actually following the URL-encoding rules. This was convenient common cases, while mis-encoding less common, but legal, cases (especially involving non-http URLs and non-Latin URLs). iOS now follows the rules, so things encode consistently, but it means you need to actually think about URLs and not just throw random stuff at the system and hope it figures it out.
The best way to do this (if you have to compute this stuff dynamically) is with NSURLComponents:
let url = NSURLComponents()
url.scheme = "http"
url.host = "www.mywebsite.com"
url.path = "/๐.html"
url.string // "http://www.mywebsite.com/%F0%9F%98%80.html"
url.percentEncodedPath // "/%F0%9F%98%80.html"
url.URL // http://www.mywebsite.com/%F0%9F%98%80.html
// etc.
See also NSURLComponents.URLReativeToURL if you have some base, static URL that you don't have to worry about dynamically encoding.
let baseURL = NSURL(string: "http://www.mywebsite.com")
let relative = NSURLComponents()
relative.path = "/๐.html"
let url = relative.URLRelativeToURL(baseURL)
url?.absoluteString
You're confusing things โ the special characters after the domain name need to be escaped using the "percent encoding" (I don't think that's 100% the correct term), according to the HTTP standard.
The Domain name itself can contain any unicode codepoint (and the client should then apply Punycode to map it to a DNS name), and the URL classifier (http:) must not be escaped.
So, yes, you'll need to handle these parts of your URL differently โ no way around that. Other protocols might require other encoding of special characters than HTTP does. For example, the ssh: URL class (which is pretty application specific. SSH as it is just a family of secure transports, not a means to describe a uniform ressource location) will probably have wildly different approaches to non-ASCII characters than HTTP, depending on what you actually mean with ssh: "URLs".
The fastest way to do:
In past you use to escape and encode your string to UTF8 coding the following statement:
let str = "http://www.mywebsite.com/๐.html"
let escapedStr = str.stringByAddingPercentEscapesUsingEncoding(NSUTF8StringEncoding)
Now this code is deprecated so the equivalent in swift 2.2 is:
let str = "http://www.mywebsite.com/๐.html"
let escapedStr = str.stringByAddingPercentEncodingWithAllowedCharacters(.URLQueryAllowedCharacterSet())
It encodes everything after the question mark in the URL string.
I'm in quite a pickle, I'm acquiring user input in Arabic which is stored in a NSString.
The string is displayed properly both in the app and the debug window.
Here's my problem: the string is added to a NSMutableDictionary then serialized to JSON
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:dictionary
options:NSJSONWritingPrettyPrinted
error:&error];
The data is then POSTed to the server, but the once Arabic content is represented as ????
This is obviously an encoding issue. I can't find the proper method to encode my NSString with UTF-8 and still add it to the Dictionary.
AddingPercentEncoding will encode it as URL UTF-8 encoding which is not what I'm after.
The only reasonable encoding I find is [NSString UTF8String] which turns the string to const char * that is not an NSObject.
What I'm really after is: ู
ุฑุญุจุง UTF-8 ---> รรยฑรยญรยจรยง
The decoding works perfectly, if the iOS app receives JSON with such representation, the resulting NSDictionary will hold a key that returns the proper value in Arabic.
Thank you for your help! :D
I need to send a SMS via HTTP Post with some special characters like "caiรฒ"
So, with my code , I try to convert my string in ISO-8859-1.
basically, if my string is converted in ISO-8859-1, and the method Uri.EscapeDataString() is invoked on it, in place of "รฒ" I should have "F2" Hexadecimal code. But I get %C3%A0 , that is the Hexadecimal code for UTF8 encoding.
Encoding iso = Encoding.etEncoding("ISO-8859-1");
string StringBody = iso.GetString(iso.GetBytes(Body));
UrlParameter += "&data=" + Uri.EscapeDataString(StringBody);
First of all, encoding a string and decoding it right back is just a pointless round-trip at best and causes data loss at worst.
Uri.EscapeDataString always correctly encodes as UTF-8. Any server should support this.
You can try HttpUtility.UrlEncode which accepts an encoding parameter:
Encoding iso = Encoding.getEncoding("ISO-8859-1");
UrlParameter += "&data=" + HttpUtility.UrlEncode(Body, iso);
I believe the text API shall have Unicode support... proper encoding of text message at your end is important... However text message API shall support that aswell... API Supporting the UNICODE messaging will do the Job....
Here is what i did in PHP... I am not much in to coding for other language sorry about that
if ($cmd['method'] == 'POST') {
$opts['http']['method'] = 'POST';
$opts['http']['header'] .= "\r\nContent-type: application/x-www-form-urlencoded";
$opts['http']['content'] = $post_data;
}
$context = stream_context_create($opts);
I used Innovative Text API Gateway which provided Unicode support and allowed me to send Chinese, Japani, Arabic and other language messages.
I just switched from ASIHTTPRequest to AFNetworking.
My app allows a user to enter a search term, and then I make an api call to my server (rails) to retrieve a list of relevant objects to display on the phone. Simple enough.
Too Long, Won't Read
With the ASI library, the search term sent to the server would be something like st%20helena. Now, with the AFNetworking, my search term sent to the server is st%2520helena. Oddly enough, this is actually making a difference. With %20, I receive no results. With %2520, I receive the results I was expecting.
My question is, why does this make a difference? I know that %2520 is an encoded '%' + 20, which equals a whitespace character, which in my mind should be identical to passing %20.
Details
Before, I was simply appending the search term to the URL after first encoding it:
NSString *encoded = [#"st helena" stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding];
NSString *url = [NSString stringWithFormat:#"http://www.example.com/?=%#, encoded];
Now, with AFNetworking, I'm passing an NSDictionary to the parameters argument of AFHTTPClient#getPath.
NSString *encoded = [#"st helena" stringByAddingPercentEscapesUsingEncoding:NSUTR8StringEncoding];
NSDictionary *dict = #{ #"q" : encoded };
[[myClient sharedClient] getPath:#"" parameters:dict ...]
Since AFNetworking encodes items in the dictionary, my search string is being double encoded, which is why the % of '%20' is being converted into '%25'.
My question is, why does this make a difference? I know that %2520 is an encoded '%' + 20, which equals a whitespace character, which in my mind should be identical to passing %20.
No, they aren't identical. The parsing code isn't supposed to just keep decoding over and over again until there aren't any more percent signs left. If that were the case, it would be impossible to transmit the actual percent character. It's only supposed to decode it once. This means that if %2520 goes over the wire, it is decoded into the data %20 and processing stops there. If %20 goes over the wire, it is decoded once into a space. The decoding is only supposed to happen once. If your web service is treating %2520 as a space, then it has a double-decoding bug.
i am attempting to use HttpUtility.UrlEncode to encode strings that ultimately are used in URLs.
example
/string/http://www.google.com
or
/string/my test string
where http://www.google.com is a parameter passed to a controller.
I have tried UrlEncode but it doesn't seem to work quite right
my route looks like:
routes.MapRoute(
"mStringView",
"mString/{sText}",
new { controller = "mString", action = "Index", sText = UrlParameter.Optional }
);
The problem is the encoded bits are decoded it seems somewhere in the routing.. except things like "+" which replace " " are not decoded..
Understanding my case, where a UrlParameter can be any string, including URL's.. what is the best way to encode them before pushing them into my db, and then handling the decode knowing they will be passed to a controller as a parameter?
thanks!
It seems this problem has come up in other forums and the general recommendation is to not rely on standard url encoding for asp.net mvc. The advantage is url encoding is not necessarily as user friendly as we want, which is one of the goals of custom routed urls. For example, this:
http://server.com/products/Goods+%26+Services
can be friendlier written as
http://server.com/products/Good-and-Services
So custom url encoding has advantages beyond working around this quirk/bug. More details and examples here:
http://www.dominicpettifer.co.uk/Blog/34/asp-net-mvc-and-clean-seo-friendly-urls
You could convert the parameter to byte array and use the HttpServerUtility.UrlTokenEncode
If the problem is that the "+" doesn't get decoded, use HttpUtility.UrlPathEncode to encode and the decoding will work as desired.
From the documentation of HttpUtility.UrlEncode:
You can encode a URL using with the UrlEncode method or the
UrlPathEncode method. However, the methods return different results.
The UrlEncode method converts each space character to a plus character
(+). The UrlPathEncode method converts each space character into the
string "%20", which represents a space in hexadecimal notation. Use
the UrlPathEncode method when you encode the path portion of a URL in
order to guarantee a consistent decoded URL, regardless of which
platform or browser performs the decoding.