How can I stay logged in when web scraping with URLSession? - swift

With my program I try to login and scrape the home page, however after I log in with a post request (in which the response says successful), when I try to get the home page it says my session has timed out.
I've tried using different types of session, modifying minor parts of the code, and Alamofire with no success
import Foundation
let session: URLSession = {
let configuration = URLSessionConfiguration.default
configuration.timeoutIntervalForRequest = 30 // seconds
configuration.timeoutIntervalForResource = 30 // seconds
return URLSession(configuration: .default)
}()
let url = URL(string: "examplesite.com")!
var request = URLRequest(url: url)
request.setValue("application/x-www-form-urlencoded", forHTTPHeaderField:
"Content-Type")
request.httpMethod = "POST"
let postString = "Email=exampleemail&Password=examplepass"
request.httpBody = postString.data(using: .utf8)
//log in
session.dataTask(with: request) { data, response, error in
guard let data = data, error == nil else {
// check for fundamental networking error
print("\(error)")
return
}
if let httpStatus = response as? HTTPURLResponse, httpStatus.statusCode
!= 200 { // check for http errors
print("\(httpStatus.statusCode)")
print("\(response)")
}
let responseString = String(data: data, encoding: .utf8)
print("\(responseString)")
let urlnext = URL(string: "example.com/HomePage")!
//get page after loggin in
session.dataTask(with: urlnext) { datanext, responsenext, errornext in
guard let datanext = datanext, errornext == nil else {
print("\(errornext)")
return
}
let stringnext = String(data: datanext, encoding: .utf8)
//get page html
print("\(stringnext)")
}.resume()
}.resume()
// Infinitely run the main loop to wait for our request.
// Only necessary if you are testing in the command line.
RunLoop.main.run()
the results are a successful login {\"msg\":\"\",\"valid\":\"1\"}, and the html code of a time out page instead of the home page

Related

How can I read the content of a form-data when the response is in html(Swift)

I'm building an iOS application and I need to access the information on a website. I've located the API endpoint, and was able to get a result in Postman
screenshot of API header and form data
So far I have this code which can allow me to make the request, but how do I parse the response(which is an HTML form, then display response in app
var urlRequest = URLRequest(url: url!)
urlRequest.setValue("application/form-data",forHTTPHeaderField: "Content-Type")
urlRequest.httpMethod = "POST"
let postString = "year=2021&season=Outdoor&province=ON&age_group=OPEN&age_sub_group_masters=ALL&age_sub_group_para=ALL&rankings_event=100m&best_by_athlete=1&rankings_event_spec_num=1&is_relay_EVENT=0&page=1"
urlRequest.httpBody = postString.data(using: .utf8)
urlRequest = .init(url: url!)```
I actually found a great resources that showed how to send POST Request with post body, and how to read the response
How To send POST Request
Now it's just a matter of parsing the HTML that is returned, and displaying it in app.
let url = URL(string: Constants.rankingAPI)
guard let requestUrl = url else {
fatalError()
}
var request = URLRequest(url: requestUrl)
request.httpMethod = "POST"
let postString = "year=2021&season=Outdoor&province=ON&age_group=OPEN&age_sub_group_masters=ALL&age_sub_group_para=ALL&rankings_event=100m&best_by_athlete=1&rankings_event_spec_num=1&is_relay_EVENT=0&page=1"
request.httpBody = postString.data(using: String.Encoding.utf8)
let task = URLSession.shared.dataTask(with: request) { (data, response, error) in
// Check for Error
if let error = error {
print("Error took place \(error)")
return
}
// Convert HTTP Response Data to a String
if let data = data, let dataString = String(data: data, encoding: .utf8) {
print("Response data string:\n \(dataString.htmlToString)")
}
}
task.resume()
}

Swift: Mailgun 401 response code--No valid API key provided

So I have my send email function as seen below:
func email() {
let session = URLSession.shared
let request = NSMutableURLRequest(url: NSURL(string: "https://api.mailgun.net/v3/sandbox################################/messages")! as URL)
request.httpMethod = "POST"
let credentials = "api:key-################################-########-########"
request.setValue("Basic \(credentials.toBase64())", forHTTPHeaderField: "Authorization")
let data = "from: Swift Email <(test#test.com)>&to: [myemail#gmail.com,(myemail#gmail.com)]&subject:Hello&text:Testing_some_Mailgun_awesomness"
request.httpBody = data.data(using: String.Encoding.ascii)
let task = session.dataTask(with: request as URLRequest, completionHandler: {(data, response, error) in
if let error = error {
print(error)
}
if let response = response {
print("url = \(response.url!)")
print("response = \(response)")
let httpResponse = response as! HTTPURLResponse
print("response code = \(httpResponse.statusCode)")
}
})
task.resume()
}
So I guess either my API key is wrong or my request URL is wrong. To find my request URL, I went to https://app.mailgun.com/app/dashboard and then scrolled down to the Sending Domains section and copied that. To get my API key I went to https://app.mailgun.com/app/account/security/api_keys and just copied the Private API Key. I'm really not sure why I'm getting this invalid code--thank you in advance if you figure it out!
Side note: Not really sure if the data constant is set up right (in terms of missing or having too many parens), so if you could check that too that would be phenomenal.
I have a software called "paw" it helps forming REST API call for Xcode, curl, php, etc.
I don't know if this could help you
class MyRequestController {
func sendRequest(somevar: String, completion: #escaping (Books) -> Void) {
/* Configure session, choose between:
* defaultSessionConfiguration
* ephemeralSessionConfiguration
* backgroundSessionConfigurationWithIdentifier:
And set session-wide properties, such as: HTTPAdditionalHeaders,
HTTPCookieAcceptPolicy, requestCachePolicy or timeoutIntervalForRequest.
*/
let sessionConfig = URLSessionConfiguration.default
/* Create session, and optionally set a URLSessionDelegate. */
let session = URLSession(configuration: sessionConfig, delegate: nil, delegateQueue: nil)
/* Create the Request:
(POST https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages)
*/
guard var URL = URL(string: "https://api.mailgun.net/v3/YOUR_DOMAIN_NAME/messages") else {return}
var request = URLRequest(url: URL)
request.httpMethod = "POST"
// Headers
request.addValue("Basic YXBpOllPVVJfQVBJX0tFWQ==", forHTTPHeaderField: "Authorization")
request.addValue("multipart/form-data; charset=utf-8; boundary=__X_PAW_BOUNDARY__", forHTTPHeaderField: "Content-Type")
// Body
let bodyString = "--__X_PAW_BOUNDARY__\r\nContent-Disposition: form-data; name=\"from\"\r\n\r\n'Excited User <mailgun#YOUR_DOMAIN_NAME>'\r\n--__X_PAW_BOUNDARY__\r\nContent-Disposition: form-data; name=\"to\"\r\n\r\nYOU#YOUR_DOMAIN_NAME\r\n--__X_PAW_BOUNDARY__\r\nContent-Disposition: form-data; name=\"to\"\r\n\r\nbar#example.com\r\n--__X_PAW_BOUNDARY__\r\nContent-Disposition: form-data; name=\"subject\"\r\n\r\n'Hello'\r\n--__X_PAW_BOUNDARY__\r\nContent-Disposition: form-data; name=\"text\"\r\n\r\n'Testing some Mailgun awesomeness!'\r\n--__X_PAW_BOUNDARY__--\r\n"
request.httpBody = bodyString.data(using: .utf8, allowLossyConversion: true)
/* Start a new Task */
let task = session.dataTask(with: request, completionHandler: { (data: Data?, response: URLResponse?, error: Error?) -> Void in
if (error == nil) {
// Success
let statusCode = (response as! HTTPURLResponse).statusCode
print("URL Session Task Succeeded: HTTP \(statusCode)")
}
else {
// Failure
print("URL Session Task Failed: %#", error!.localizedDescription);
}
})
task.resume()
session.finishTasksAndInvalidate()
}
}
**** and you CALL this function a bit like this
MyRequestController().sendRequest(somevar: "something")
take a look at https://www.youtube.com/watch?v=44APgBnapag for more details
this tutorial show how to do REST API calls with Xcode, this example it scan a barcode, send the scanned code via a function that calls the API and return infos...

Get the HTML content when hitting URL in swift 3

My question is i want to hit the url and when i hit the url on server side the php return the results just echo in php and i have to save that result in variable in swift 3, i tried the below code:
let URLstr = URL(string: strURL)
let request = URLRequest(url: URLstr!)
request.httpMethod = "POST"
print (request)
I didn't get the content of URL in swift which is much easier in objective C.
Use the string initializer with the url.
do {
let contents = try String(contentsOf: URLstr, encoding: .ascii)
} catch {
// handle error
}
Or you can use URLSession.
let task = URLSession.shared.dataTask(with: URLStr) { data, response, error in
guard data != nil else { // no data }
let contents = String(data: data!, encoding: .ascii)
}
task.resume()
I adjusted the code provided above. To fetch HTML code use URLSession.
Swift 5
let session = URLSession.shared
let url = URL(string: "https://yourwebsiteaddress.com/")!
let task = session.dataTask(with: url) { data, response, error in
// Check whether data is not nil
guard let loadedData = data else { return }
// Load HTML code as string
let contents = String(data: loadedData, encoding: .utf8)
print(contents)
}
task.resume()

Swift 3 IBM Lotus Domino authentication

I am very new to XCODE and iOS and I can't figure out how to do an authentication on IBM Domino server. I am trying to do a simple app to get some data from the server.
This is the latest version of my attempt. Bear in mind, that couldn't find information on what IBM Domino server needs. But I've done tons of authentications with javascript. I've deliberatly left the commented part, which is one of the attempts to do a login.
func doLogin(){
print("entered doLogin")
let myUrl: String = "https://url.com/names?login"
guard let url = URL(string: myUrl.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed)!)else {
print("Error: cannot create URL")
return
}
var request = URLRequest.init(url: url)
request.httpMethod = "POST"
let params = ["Username":String(describing: self.username), "Password":String(describing: self.password)]
request.httpBody = try? JSONSerialization.data(withJSONObject: params, options: [])
request.addValue("application/json", forHTTPHeaderField: "Content-Type")
//URLSession.shared.dataTask(with: request) { (data:Data?, response:URLResponse?, error:Error?) in
// if let safeData = data{
// print("response: \(String(data:safeData, encoding:.utf8))")
// }
//}
//let postString = "Username=\(String(describing: self.username))&Password=\(String(describing: self.password))"
//request.httpBody = postString.data(using:String.Encoding.utf8)
request.timeoutInterval = 30.0
//print(request)
let configuration = URLSessionConfiguration.default
let session = URLSession(configuration : configuration)
//let session = URLSession(configuration: configuration, delegate: self as! URLSessionDelegate, delegateQueue: nil)
let task = session.dataTask(with: request) {
(data : Data?, response:URLResponse?, error) in
// check for any errors
guard error == nil else {
print("error calling POST on url")
print(error!)
return
}
// make sure we got data
guard let responseData = data else {
print("Error: did not receive data")
return
}
do {
print ("responseData:")
print(responseData)
}
}
//print ("response:")
//print(task.response)
task.resume()

Connecting Swift with php server [duplicate]

This question already has answers here:
How can I get the Data from NSURLSession.sharedSession().dataTaskWithRequest
(2 answers)
Closed 6 years ago.
I'm trying to post from swift to php and then return json encoded data back to swift.
However, print(task.response) is nil. My link does what it is supposed to but my app still responds with nil. Any idea why?
let request = NSMutableURLRequest(URL: NSURL(string: "https://www.flashkase.com/API/createAccount.php")!)
request.HTTPMethod = "POST"
let postString = "username=frogg222"
request.HTTPBody = postString.dataUsingEncoding(NSUTF8StringEncoding)
let task = NSURLSession.sharedSession().dataTaskWithRequest(request) { data, response, error in
guard error == nil && data != nil else { // check for fundamental networking error
print("error=\(error)")
return
}
if let httpStatus = response as? NSHTTPURLResponse where httpStatus.statusCode != 200 { // check for http errors
print("statusCode should be 200, but is \(httpStatus.statusCode)")
print("response = \(response)")
}
let responseString = String(data: data!, encoding: NSUTF8StringEncoding)
print("responseString = \(responseString)")
}
task.resume()
print(task.response)
And my PHP file
<?php
include($_SERVER["DOCUMENT_ROOT"] . "/inc/database.php");
$arr = [];
$arr[] = 'Reporting for duty';
if($_SERVER['REQUEST_METHOD'] == 'POST'){
$arr[] = 'Post received';
$arr[] = $_POST['username'];
}
echo json_encode($arr);
?>
Update: How can I check for data after completion? If I uncomment sleep(3) it will display results...but that's obviously not ideal.
func postToServer(postURL: String, postString: String) {
let request = NSMutableURLRequest(URL: NSURL(string: postURL)!)
request.HTTPMethod = "POST"
request.HTTPBody = postString.dataUsingEncoding(NSUTF8StringEncoding)
let task = NSURLSession.sharedSession().dataTaskWithRequest(request){
data, response, error in
print("\(data), \(response), \(error)")
}
task.resume()
//sleep(3)
}
let sendURL: String = "https://www.flashkase.com/API/createAccount.php"
let sendData: String = "username=frogg222&email=fake123#gmail.com&password=fake1234"
postToServer(sendURL,postString: sendData)
dataTaskWithRequest is an asynchronous operation. Printing the response code immediately after the task starts will obviously print nil, as it hasn't had time to get a value. The completion handler already has a response object you can use, so inside your completion handler, call print(response) to see the properties and try to get a response code.