Basically I'm trying to download the HTML source code of a URL object into a String...
I've followed step-by-step examples of how-to-do so on StackOverflow.
import Foundation
try! print(String(contentsOf: URL(string: "https://www.google.com")!, encoding: .utf16))
This is a small sample of the output I'm getting:
楤瑨㨱〰╽䁭敤楡汬笮杢ㅻ桥楧桴㨲㉰砻浡牧楮楧桴㨮㕥活癥牴楣慬ⵡ汩杮㩴潰紣杢慲筦汯慴
When I was really just expecting some plain HTML output.
The only thing I could think that's causing this issue has to be related to Networking(such as the DNS servers being queried). But, just in case I'm asking it on here to get a review of the code.
It's unclear why you want to encode the output to UTF-16, although removing it should work:
try! print(String(contentsOf: URL(string: "https://www.google.com")!))
That should return the html of the URL.
You can use an encoding, however, UTF-16 is likely not correct in this circumstance.
try! print(String(contentsOf: URL(string: "https://www.google.com")!, encoding: .ascii))
ASCII would likely work.
Related
I'm a little ashamed to post this as I use URLs all the time but this one has me stumped!
I keep getting an 'Unexpectedly found nil while unwrapping an Optional value'
Just to add some context, I use the same format for over 20 URLs which is making requests to an API. The only difference is the ID at the end of the URL.
Ignore the insecure URL as this is simply a debug endpoint and has no bearing on the issue as I use the same URL in other calls. Also, ignore the first part of the URL, I have purposefully changed it for security purposes.
The code I use is as follows:
func returnEventParticipantsEndPoint(eventID: String) -> URLRequest {
print("URL: \(debugEndPointURL)/api/friendgroups/\(eventID)")
var url: URL!
url = URL(string: "\(debugEndPointURL)/api/friendgroups/\(eventID)")!
let requestURL = URLRequest(url: url)
return requestURL
}
The URL printed in the console (top line) before I try to return the URLRequest prints as follows:
http://blahblahblah.blah.net/api/friendgroups/1c8264b95884409f90b4fd8ba9d830e1
Yet, it keeps returning nil and I'd really appreciate some help with this...thanks.
The reason that it is failing is that there are hidden Unicode characters in your string. Copying and pasting the url string you provided into a hidden Unicode character checker gives the following for your url
http://blahblahblah.blah.net/apiU+200B/friendgroupsU+200B/1c8264b95884409f90b4fd8ba9d830e1
Notice the additional hidden characters at the end of the words api and friendgroups
You can try it here https://www.soscisurvey.de/tools/view-chars.php
Have you copied and pasted the format for this? Examining the string you have some non printable Unicode characters after the words api and friendgroups. Retyping the url path and trying again worked for me
I am trying to make a web request to a URL that needs to keep accented characters instead of percent encoding them. E.g. é must NOT change to e%CC%81. I cannot change this.
These are the allowed characters that shouldn't be percent encoded: AaÁáBbCcDdEeÉéFfGgHhIiÍíJjKkLlMmNnOoÓóÖöŐőPpQqRrSsTtUuÚúÜüŰűVvWwXxYyZz0123456789-
Here is an example of a url I need
https://helyesiras.mta.hu/helyesiras/default/suggest?q=hány%20éves
You can try this url in your web borwser to confirm its working. (The site is in Hungarian.) If you try the proper percent encoded version of this url (https://helyesiras.mta.hu/helyesiras/default/suggest?q=ha%CC%81ny%20e%CC%81ves) then the website will give an error. (Also in Hungarian.)
I have my custom encoder to get this URL string. However to make a web request I need to convert the String to URL.
I tried 2 ways:
URL(string:)
let urlStr = "https://helyesiras.mta.hu/helyesiras/default/suggest?q=hány%20éves"
var url = URL(string: urlStr)
// ERROR: Returns nil
URLComponents with percentEncodedQueryItems
var urlComponents = URLComponents()
urlComponents.scheme = "https"
urlComponents.host = "helyesiras.mta.hu"
urlComponents.path = "/helyesiras/default/suggest"
urlComponents.percentEncodedQueryItems = [ // ERROR: invalid characters in percent encoded query items
URLQueryItem(name: "q", value: "hány%20éves")
]
let url = urlComponents.url
Is it possible to create URLs without Foundation APIs checking its validity? Or can I create my own validation rules?
Safari is percent-encoding the URL. You're just percent-encoding it differently (and in a way your server is rejecting). What Safari sends to the server is:
GET /helyesiras/default/suggest?q=h%C3%A1ny%20%C3%A9ves HTTP/1.1
You can check that using Charles. Your website is behaving correctly and does not appear to require unencoded URLs.
It is not valid to send unencoded URLs, and Safari doesn't. There's no way to do it with URLSession either. You'd have to connect to the socket directly and build your own HTTP stack, which is quite possible, but I don't think you want to do that.
As Leo notes, the correct way to do this is using:
URLQueryItem(name: "q", value: "hány éves")
Replacing the %20 with the unencoded " " so that you don't double-encode the percent.
If you encode the string by hand, you'll find the same encoding:
print("hány éves".addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed))
// Optional("h%C3%A1ny%20%C3%A9ves")
(But you should use URLComponents. addingPercentEncoding is extremely error-prone.)
The preferred UTF-8 encoding of á is as the unicode code point LATIN SMALL LETTER A WITH ACUTE (C3 A1). What you're encoding is LATIN SMALL LETTER A (61) followed by COMBINING ACUTE ACCENT (CC 81). I suspect your server is not applying Unicode normalization rules. While that's unfortunate the fix is simple: use URLComponents, and you'll get the same correct behavior as Safari.
I'm trying to craft a very specific HTTP request to a server (ie. defining the exact set of HTTP headers), but NSURLSession keeps "helpfully" inserting a bunch of HTTP headers like Accept, Accept-Language and Accept-Encoding.
Consider the following playground (Swift 2.x) which sends a request to a service that just echos the HTTP headers that were sent:
import Foundation
import XCPlayground
XCPlaygroundPage.currentPage.needsIndefiniteExecution = true
let url = NSURL(string: "http://httpbin.org/headers")!
let request = NSMutableURLRequest(URL: url, cachePolicy: .ReloadIgnoringLocalCacheData, timeoutInterval: 30000)
let configuration = NSURLSessionConfiguration.ephemeralSessionConfiguration()
let session = NSURLSession(configuration: configuration)
let task = session.dataTaskWithRequest(request) { (data: NSData?, response: NSURLResponse?, error: NSError?) in
print(NSString(data: data!, encoding: NSUTF8StringEncoding))
XCPlaygroundPage.currentPage.finishExecution()
}
task.resume()
You can see that there are three Accept headers being sent. How can I prevent that?
I've tried setting the header using request.setValue(nil, forHTTPHeaderField: "Accept-Language") but that gets ignored. Tried setting it to "", but no good. I've also tried manipulating the HTTPAdditionalHeaders property on NSURLSessionConfiguration, but no love.
How do I get NSURLSession to not be quite so helpful?
I doubt what you're asking for is possible. NSURLSession (and NSURLConnection) automatically provide a number of headers, and that's one of them.
There's also no valid reason to remove them. All three of those headers have been part of the spec since the original HTTP/0.9 spec (https://www.w3.org/Protocols/HTTP/HTRQ_Headers.html). There's absolutely no excuse for any server not either handling those correctly or ignoring them outright.
With that said, if you provide the wrong value for those fields (and the default may be wrong), the server may refuse to give you results. To solve that problem, first figure out what type of data the server is actually going to provide, and specify that value in the Accept header instead of the default.
For example, you might set Accept to "application/json" or one of the other variants (What is the correct JSON content type?) if you're expecting JSON data.
That said, if you really must avoid having those headers sent, you can always open a socket, construct the request manually, and send it. Assuming the server doesn't require chunked encoding, it is pretty easy to do. (Chunked encoding, however, is a horror of Herculean proportions, so if your server sends that back, you'll pretty much have to resort to adding libcurl into your project and using that to make requests instead.)
I know how to encode a string in URL format (the smiley face is intentional):
let str = "www.mywebsite.com/😀.html"
let escapedStr = str.stringByAddingPercentEncodingWithAllowedCharacters(NSCharacterSet.URLPathAllowedCharacterSet())!
print(escapedStr)
// Output:
// www.mywebsite.com/%F0%9F%98%80.html
But if I attach http:// to the unescaped string Swift escapes the colon too:
let str = "http://www.mywebsite.com/😀.html"
let escapedStr = str.stringByAddingPercentEncodingWithAllowedCharacters(NSCharacterSet.URLPathAllowedCharacterSet())!
print(escapedStr)
// Output
// http%3A//www.mywebsite.com/%F0%9F%98%80.html
So short of removing and adding http:// manually, how can I properly escape those strings? There are other prefixes I must handle handle like https://, ftp:// or ssh://
: is not a legal character in the path part of an URL. You percent-encoded everything not in URLPathAllowedCharacterSet, so it shouldn't be surprising that the : was encoded.
Each part of an URL has different encoding rules. iOS can't correctly encode an URL until it knows what goes in what part, and it can't do that from an unencoded string (since it'd have to parse it first, and it can't parse it because it's not correctly encoded yet). In some systems (including older versions of iOS), it would use various heuristics that assumed "well, I guess you probably meant..." rather than actually following the URL-encoding rules. This was convenient common cases, while mis-encoding less common, but legal, cases (especially involving non-http URLs and non-Latin URLs). iOS now follows the rules, so things encode consistently, but it means you need to actually think about URLs and not just throw random stuff at the system and hope it figures it out.
The best way to do this (if you have to compute this stuff dynamically) is with NSURLComponents:
let url = NSURLComponents()
url.scheme = "http"
url.host = "www.mywebsite.com"
url.path = "/😀.html"
url.string // "http://www.mywebsite.com/%F0%9F%98%80.html"
url.percentEncodedPath // "/%F0%9F%98%80.html"
url.URL // http://www.mywebsite.com/%F0%9F%98%80.html
// etc.
See also NSURLComponents.URLReativeToURL if you have some base, static URL that you don't have to worry about dynamically encoding.
let baseURL = NSURL(string: "http://www.mywebsite.com")
let relative = NSURLComponents()
relative.path = "/😀.html"
let url = relative.URLRelativeToURL(baseURL)
url?.absoluteString
You're confusing things – the special characters after the domain name need to be escaped using the "percent encoding" (I don't think that's 100% the correct term), according to the HTTP standard.
The Domain name itself can contain any unicode codepoint (and the client should then apply Punycode to map it to a DNS name), and the URL classifier (http:) must not be escaped.
So, yes, you'll need to handle these parts of your URL differently – no way around that. Other protocols might require other encoding of special characters than HTTP does. For example, the ssh: URL class (which is pretty application specific. SSH as it is just a family of secure transports, not a means to describe a uniform ressource location) will probably have wildly different approaches to non-ASCII characters than HTTP, depending on what you actually mean with ssh: "URLs".
The fastest way to do:
In past you use to escape and encode your string to UTF8 coding the following statement:
let str = "http://www.mywebsite.com/😀.html"
let escapedStr = str.stringByAddingPercentEscapesUsingEncoding(NSUTF8StringEncoding)
Now this code is deprecated so the equivalent in swift 2.2 is:
let str = "http://www.mywebsite.com/😀.html"
let escapedStr = str.stringByAddingPercentEncodingWithAllowedCharacters(.URLQueryAllowedCharacterSet())
It encodes everything after the question mark in the URL string.
I have looked at questions like this or that one but it is still not working.
Also I have a question about what should I enter into fileURL param from function multipartFormData.appendBodyPart?
Should it be a way to image from PC, or image must be added to Images.xcassets? What should I send here?
It looks like you have three issues that you need to fix.
Use .POST instead of POST.
The fileURL needs to be a valid NSURL that points to a file on the file system. You cannot just use the filename.
You are using the responseString serializer, but named the third parameter in the closure JSON. Then you are letting result into s and trying to print it out. The result parameter doesn't even exist anywhere. Instead, you should print(JSON).
Hopefully that helps clear things up a bit.
Try to use .POST not POST
As an alternative solution upload an encoded file and send it as a parameter of POST.
// `data` is NSData
let base64String = data!.base64EncodedStringWithOptions(NSDataBase64EncodingOptions.allZeros)
let parameters = ["image_data": base64String] as [String: AnyObject]
Alamofire.request(.POST, "http://your-url.com", parameters: parameters)
The cons of this method is that the data will get %33 larger due to encoding. If you have bandwidth problems it may not be a good solution.