Flutter http_auth headers contain "carrier return" and "new line" characters (\r\n) - flutter

I am currently experimenting with flutter and tried to use http package. When I request a resource, which requires an authorization header (Authorization: Basic ) I create my request as follows:
import 'package:http/http.dart' as http;
void request() {
http.get('http://<my_url>', headers: {'Authorization': 'Basic <hash>'}).then( (resp) {
print('response status: ${resp.statusCode} ${resp.reasonPhrase}');
}
}
While executing this request, I was watching the request sent over Wireshark and noticed, that it contains carrier-return (\r) and new-line (\n) characters at the end:
Snapshot of Wireshark
After invoking my request function, I get a 401 Unauthorized message in my print statement. When I invoke the same request in Postman, everything is fine.
I assume, that my authentication failed, because of these characters (\r\n) at the end of each header entry. Can anyone help me with it or am I doing something wrong?
PS.: I have also tried the http_auth package but it also failed.
import 'package:http_auth/http_auth.dart' as http_auth;
void request(String user, String password) {
http_auth.BasicAuthClient(user, password).get('http://<my_url>', headers: {'Authorization': 'Basic <hash>'}).then( (resp) {
print('response status: ${resp.statusCode} ${resp.reasonPhrase}');
}
}

Those \r\n characters are meant to be there. They are the line separators between the headers. If you packet capture the wire bytes from curl --user foo:bar www.dart.dev you will see exactly the same thing.
You will notice that Dart has lower-cased the header name to authorization as is perfectly valid per the RFC. However, this sometimes catches out some servers that (incorrectly) treat the header names case sensitively.
Try your postman again (or a curl with a manually-created, lower-case header) with authorization. If the server doesn't accept that, contact the server maintainer.

Related

How to send JWT token in flutter in a request header?

I'm trying to send a request to a server that requires a parameter called 'fid' sent in the header for autherization.
Here's what you can help me with.
Payload to be sent in the header 👇
{ "fid" : "Some alphaNum value", "type": "some type string" }
I would like to know how to use jwt encoding in flutter and how to use that in the header of a request to encode and send this payload across to the server for autherization. I'm planning to use HS256 algo for encoding. Since I'm new to the concept of Jwt, there might be errors in the way I asked this question. Please share your thoughts. I would also like to know how to use the secret key too -- like how to generate the key, and where in the request to send this.
Request 👇
fetchData({required String type, required String fid, String url}) async{
<How to get the json web token>
http.get(url, header : <What do I send here>);
}
you can use http or Dio package for calling API requests in flutter,
you can set headers as follows,
https://pub.dev/packages/dio
https://pub.dev/packages/http
final response = http.get(url,
headers: {HttpHeaders.contentTypeHeader: "application/json", HttpHeaders.authorizationHeader: "Bearer $token"});

How to extract header values from http response?

I am using http post to send some information and get back the response.
final response = await http.post(
Uri.parse(
"some url"
),
headers: <String, String>{
'email': email,
'callbacktoken': callbacktoken
},
);
When I run this, my execution gets stuck at this codeblock ( I tried putting a print statement in the next line) when the response coming from the backend has header values, however if I send the response with no header from the backend (I am using django at the backend) then the my program runs with no issue.
So my question is how to handle responses with headers and how to extract them?
After a little researching I finally found the issue. My custom header name had space in it "New user" because of which http was not able to parse it. After changing the name to "newuser" everything is working fine.

Flutter - Making a web request

I would like to know if there is a way to format special characters when parsing JSON requests from Flutter. I have tested a http request as such:
void curlRequest() async
{
String urlRequest = "http://my.web.url/webservice/rest/server.php?wstoken=my_token&moodlewsrestformat=json&wsfunction=core_user_create_users&users[0][username]=test_user_4&users[0][firstname]=John&users[0][lastname]=Doe&users[0][email]=john.doe#gmail.com&users[0][password]=Johns_Password";
http.post(urlRequest, headers: {'Content-type': 'application/json; charset=utf-8'}).then((response){
print("Response status: ${response.statusCode}");
print("Response body: ${response.body}");
});
}
I get the following response:
http://my.web.url/webservice/rest/server.php?wstoken=my_token&moodlewsrestformat=json&wsfunction=core_user_create_users&users%5B0%5D%5Busername%5D=test_user_4&users%5B0%5D%5Bfirstname%5D=John&users%5B0%5D%5Blastname%5D=Doe&users%5B0%5D%5Bemail%5D=john.doe#gmail.com&users%5B0%5D%5Bpassword%5D=Johns_Password&i=1
The request was invalid due to special characters being used. How do I make a request so that the special characters are also handled properly?
In a http restful request, the http GET request must be url encoded, which means that most special characters must be encoded in a way understandable by a webserver.
As such, characters such as the plus sign (+) or the question mark (?) must be replaced by their URL encoded equivalents (%2b for the plus sign and %3F for a question mark for instance) and vice versa for decoding.
Several URL encoders can be found online such as http://www.url-encode-decode.com/
In Dart you can use Uri.encodeFull() to encode string and Uri.encodeComponent() to encode a string component
Similarly to decode you can use Uri.decodeFull() and others
read more about this here Uri class
Switching to a better web hosting service resolved my issue. I was using AeonFree, which was a free web hosting service and had limited access for obvious reasons.
I also switched from the http package to using the dio package.

Getting 400 bad request in Axios but in Postman status 200

I'm trying to request a Power BI API using axios.
var configReportInfos = {
method: 'get',
url: `https://api.powerbi.com/v1.0/myorg/groups/${process.env.GROUP_ID}/reports/${process.env.DASHBOARD_ID}`,
headers: {
'Content-Type': 'multipart/form-data',
'Authorization': 'Bearer ' + tokenResponse.accessToken
},
data: {}
};
let reportinfos
try {
reportinfos = await axios(configReportInfos)
tokenResponse['reportInfos'] = reportinfos
}
catch (error) {
console.log(error);
};
Actually I'm getting this error
Request failed with status code 400"
But strangely in Postman I'm getting 200 status.
I'm using same token.
I had the same problem. When l switched content-type in my app to application.json everything worked.
I faced the same issue (not in Axios) and struggled for a few hours.
I was able to make the API call from postman but from my java code I was getting a 400.
Turns out, this was a content issue. My request body had some Cyrillic content and it needed to be encoded before sending.
Postman was doing it by default and hence the requests were going through and responding with 200
I recommend to check your Request Header options in most cases some option is configured wrong.
In your case content-type seems wrong, did you try application/json in both calls.
To configure properly you can check the documentation
Content Type Configuration

Python requests module crashing when fed URL with emoji in it. Why?

I am trying to scrape the following site:
https://nypost.com/2020/06/27/milton-glaser-designer-of-i-%e2%99%a5%e2%80%8a-ny-logo-dead-at-91/
import requests
from bs4 import BeautifulSoup
site = 'https://nypost.com/2020/06/27/milton-glaser-designer-of-i-%e2%99%a5%e2%80%8a-ny-logo-dead-at-91/'
soup = BeautifulSoup(requests.get(site).content, 'html.parser')
I get:
raise TooManyRedirects('Exceeded {} redirects.'.format(self.max_redirects), response=resp)
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.
I would like to understand what is going on. I suspect some loop generated somehow by the special characters being interpreted but I am at a loss rn.
You asked why this happens. It is due to requests using urllib3. The urllib3 changes the percent-encoded bytes to upper case https://github.com/urllib3/urllib3/issues/1677 as per the recommendation of RFC 3986 to uppercase percent-encoded bytes during normalization. In normal circumstances that would be good. But this server has seems to want it's URLs lowercase. This can be see by :
import requests
url = 'https://nypost.com/2020/06/27/milton-glaser-designer-of-i-%e2%99%a5%e2%80%8a-ny-logo-dead-at-91/'
resp = requests.get(url, allow_redirects=False)
print(resp.status_code)
print(resp.headers['Location'])
print(resp.url)
Outputs:
301
https://nypost.com/2020/06/27/milton-glaser-designer-of-i-%e2%99%a5%e2%80%8a-ny-logo-dead-at-91/
https://nypost.com/2020/06/27/milton-glaser-designer-of-i-%E2%99%A5%E2%80%8A-ny-logo-dead-at-91/
This shows it is a HTTP 301 redirect. The URL it is redirecting to and the URL the request was made to.
You can test this by opening Firefox or Chrome, right clicking on a page, Select Inspect, then select Network, select disable cache, then paste the last URL and hit return. You will see the 301 redirect.
I expect there is a directive on the server to make all URLs lowercase by forcing a redirect. So it goes into a loop of requesting with uppercase percent-encoded bytes and being redirected to a URL with lowercase percent-encoded bytes to which it makes a request with uppercase percent-encoded bytes etc.
There is a way round it but it could lead to unexpected side-effects and I would only use it as a last resort and then only if you were certain all your URLs were formatted as the server expects them. But it explains the problem.
import requests.packages.urllib3.util.url as _url
import requests
def my_encode_invalid_chars(component, allowed_chars):
return component
_url._encode_invalid_chars = my_encode_invalid_chars
url = 'https://nypost.com/2020/06/27/milton-glaser-designer-of-i-%e2%99%a5%e2%80%8a-ny-logo-dead-at-91/'
resp = requests.get(url)
print(resp.status_code)
print(resp.headers)
print(resp.url)
print(resp.text)
Note the output is:
200
{'Server': 'nginx', ...
https://nypost.com/2020/06/27/milton-glaser-designer-of-i-%e2%99%a5%e2%80%8a-ny-logo-dead-at-91/
The response is HTTP 200 OK.
There is no Location header (I truncated the output).
The URL that was requested is lowercase.
Then it prints the page source.