Ionic Native Http Client - Some UTF8 chars breaking response.data - ionic-framework

I am using native HTTP client (cordova-plugin-advanced-http) in my Ionic 3 project and I think I have found a problem with UTF8 chars on Android.
I am getting the data back from a POST call and the response is usually a JSON that I can parse with the normal JSON.parse function. Whenever there are some non-ascii chars inside (like à è ò ì ù) the json content is returned incorrectly as follows:
Actual data in the HTTP response:
{"activityResponse":{"activityId":"2113000190","errorCode":"0002","errorDesc":"Guasto già chiuso"}}
Response.data from the native POST promise:
chiuso"}}
This problem seems to happen only on Android. The iOS side of the plugin is dealing with these chars just fine.
Unfortunately I have to use the native HTTP client for CORS issues and cannot switch back to the standard Angular HTTP client.
I am also setting the following options before the POST request:
this.http.setHeader('*', 'Content-Type', 'application/json; charset=UTF-8');
this.http.setHeader('*', 'Accept-Charset', 'UTF-8');
this.http.disableRedirect(true);
this.http.setDataSerializer('utf8');
Anyone having a similar issue or ideas on how to solve it?
Thanks
Mat

Related

Dart http.MultipartRequest not sending data to Lumen after relocating server to a different apache server, Postman works fine

I have a multipart request I'm trying to send from my Flutter app to my remote Lumen/Apache server. This code was working fine when I was hosting my Lumen server in Homestead locally, which runs nginx.
The Lumen server itself responds accurately when sending the request via Postman. I doubt the Flutter application is sending nothing at all, as it was working before the server move. Since it's not the Lumen app, as it works with Postman, then I presume it has something to do with apache.
The header I set, using http.MultipartRequest() is (in Postman I used form-data):
headers['Content-Type'] = 'multipart/form-data';
I also have an Authorization Bearer: token header which works fine, as the app would've presented an Unauthorized response before running the route.
I have the following code:
...
print("Multipart request fields: " + request.fields.toString());
print("Multipart request files: " + request.files.toString());
var streamedResponse = await request.send();
response = await http.Response.fromStream(streamedResponse);
if (response.statusCode == 200) print('Uploaded!');
I get the following output in debug:
I/flutter ( 7073): Multipart request fields: {productName: new, description: new, price: 30.0, currency: USD, quantity: -1.0, mass: 0.0, massUnit: lb, isData: true}
I/flutter ( 7073): Multipart request files: [Instance of 'MultipartFile']
In my Lumen application, I have a function that simply does:
var_dump($request->all());
die;
I get the following result:
I/flutter ( 7073): array(1) {
I/flutter ( 7073): ["_method"]=>
I/flutter ( 7073): string(4) "POST"
I/flutter ( 7073): }
This arrived from the request not passing validation checks. Before this, I had the longer function:
$validator = Validator::make($request->all(), [
'productName' => 'required',
'description' => 'required',
'image' => 'sometimes|image|mimetypes:image/jpeg,image/png|max:600',// TODO remove sometimes from this
'price' => 'required',
'currency' => 'required|string',
'quantity' => 'required',
'mass' => 'numeric',
'massUnit' => 'required|string',
'isData' => 'present|string|nullable',
]);
And it was failing the validation tests, even though the data was apparently being sent (and it used to pass before the server move).
What could the reason be? It seems the only difference is moving from an nginx local Homestead server to a remote apache server. I thought maybe I need a different header, possibly.
Update
With some more testing, I found that the request works without error if there is no image in the multipart/form-data request. The issue seems to rely on there being a file attached. Apparently, when and only when there is a file attached, the server reports that no data was sent.
I have two very similar requests, one that is POST and one that is PATCH. I found that with Postman, using the request that is a PATCH, sending it as a PATCH requests results in the same issue -- no data sent. Sending it as a POST request with the field "_method" set to "PATCH" does work, however. The request that is a POST request (to add a new item) works fine without the "_method" field, when set to a POST request.
The POST endpoint is responding as if http is not sending a "POST" request (empty data). With the PATCH endpoint, Lumen is responding as if it was a "POST" request.
Here's the difference between Postman and http.MultipartRequest:
With POST request:
Postman: POST request + "_method: POST": Works
http: POST request + "_method: POST": No data sent
With PATCH request:
Postman: POST request + "_method: PATCH": Works
Postman: PATCH request without "_method": No data sent
http: POST request + "_method: PATCH": 405 Method Not Allowed (as if it was a PATCH request)
Again, the properly formed requests work if there is no file attached (and everything else is the same). It's when the http request has a file that it suddenly works unexpectedly.
In the last server, I could not fully test the http functionality as I was using localtunnel.js to forward requests to my phone from Homestead, but it has a file size limit that I didn't want to work with. I did know that dart's http was sending the file, however, due to the 413 Request Entity Too Large responses that didn't occur without the image.
Here's the output of the request object in Flutter's DevTools (sorry it's a picture -- it doesn't permit selecting the text):
When not including a file, the request looks identical except for that files list is empty. Despite this, a request without a file is received by the server with data, while the other results in no data apparently sent.
The headers field of the request object looks fine too, with Content-Type being present. The authorization field clearly works, too, otherwise I would be getting unauthorized errors. Since the request object looks fine, something between request.send(); and the Lumen app is dropping all of the data if there is a file attached. Where it's coming from -- the Flutter app or the server -- is difficult to be sure about. (Considering the Flutter app was definitely sending something large when a file was attached when I was using localtunnel.js, it seems likely to be on the server side, as it seems the Flutter app is sending data, just the server is dropping it if there is a file. However, Postman works without hiccup, which suggests the server is functioning correctly.)
Update
My answer was a bit early. PHP was throwing away all of the data because the file was too large. After editing php.ini for post_max_size and upload_max_filesize, the data did start making it through, and, at first, I thought that indicated it was working.
However, it turns out that while now the non-file data is making it through, and using mod_dumpio, I see that the file is being sent, when the request comes from Flutter's http.multipartRequest, the server is dropping the file, specifically. When the request comes from Postman, everything works fine. I made sure to test with the exact same image, as well, and there seems to be no difference in the requests.
Update: This answer only partially solved the issue.
Using tcpdump on the server, I found that the server is indeed receiving data (even though Lumen was saying that it was not receiving data), and the amount of data seemed to correspond with whether a file is attached or not. From this, it seemed the problem was occurring with Apache, PHP, or Lumen.
I also enabled mod_dumpio, and tested the requests with and without a file and both appear to send data normally. The file request was clearly sending a large amount more, and the Content-Length reported seemed accurate, even though Lumen was reporting no data.
Sorting through error.log I did find the error, however. It was just that PHP was dropping the data because it exceeded the post_max_size size of 8MB. Once editing php.ini for that and upload_max_filesize, the application worked fine.
What's funny is that's usually the first error that occurs with file uploads and PHP servers, so it's weird I didn't suspect it. It would be nice if PHP issued a 413 Response by default.
Recently I was working with sending MultipartFile to our server on MongoDB/HapiJS server. While it worked in postman and with vue web app, I struggled on Flutter to find out why it didn't work.
In our case the solution was rather silly, but it worked:
When sending the file I used MultipartFile.fromBytes(byteData), and after changing it to MultipartFile.fromFile(filePath) it worked.
Could you check this (or other way around) and tell if it works?

flutter app connect to ip camera with digest auth

I'm trying to pass digest auth on my ip camera
I do it on Flutter but actually it doesn't matter.
I got http response from camera with status code 401 Not Authorized
and Headers:
{content-type: text/html, pragma: no-cache, cache-control: no-cache, www-authenticate: Digest
realm="goAhead", domain=":13237",qop="auth", nonce="a98326cc6022c2a2b7cc7e57a5956f77",
opaque="5ccc069c403ebaf9f0171e9517f40e41",algorithm="MD5", stale="FALSE", date: Thu Dec 26 16:31:43
2019, server: GoAhead-Webs}
from this source here I've found the needed Response constructor view like this:
var mResponse = "Digest username=\"$username\", realm=\"$realm\", nonce=\"$nonce\", uri=\"$uri\",
response=\"$_response\", cnonce=\"$cnonce\", nc=$nc, qop=\"$qop\"";
I create this:
Digest username="admin", realm="goAhead", nonce="a98326cc6022c2a2b7cc7e57a5956f77",
uri="/onvif/device_service", response="5313fe5265efcd3da37cec322d92ebd7", cnonce="1234567890",
nc=00000001, qop="auth"
and send http request to camera:
Future<http.Response> new_response = http.post(snapshotUrl, headers: {"Content-Type":"text/xml; charset=utf-8", "Authorization": mResponse }, body: mGetSnapshotUriAuth);
new_response.then((resp){
print(resp.statusCode);
});
and getting this error:
[ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: Invalid header field name
When I remove "Authorization" field of headers, camera response with 401 code.
I'm trying to get snapshot from ip camera works on Onvif protocol. I am not sure in Uri parameter.
The same error when I try send this http request via http Client on desktop machine screenshot
Question 2:
When I try to change uri param to something else e.g:
String uri = http://www.onvif.org/ver10/media/wsdl/GetSnapshotUri";
My camera stops reply to me. The same behaviour I can see when I start use Http sniffer or change Digest's response's field name. Sometimes reset camera helps(or even hard reset) and it start working(return 401 error) again, sometimes it's start working by itself. But by the way the camera continues working via others Camera Viewer Apps.
My screenshot from http sniffer
PS After many tests to solve this problem I think my param values are correct, because If change something e.g username, password and etc (except cnonce. When I change сnonce the result the same), I got other error: 401.
screenshot of Android studio with error
Writing SOAP requests by hand is BAD PRACTICE. I know nothing about dart, but I invite you to look for a tool like gsoap for generating the functions from thw WSDL files you can find here.
Don't write requests by hand, you'll end up in a mess when parsing the responses.

content-type in requests getting overwritten when Chopper library for flutter

I recently started working in flutter and now I want to request some data from a server. I used the Chopper library to make my work easy but I wasn't getting the expected response so after investigation I found out that my provided content-type in the header was getting over written somewhere in the chopper library or in lower level classes.
Any help would be appreciated. thank you!
I gave-> content-type: application/x-www-form-urlencoded
it got over written to-> content-type: application/json

How to show streams inline as HTTP chunked response with play framework

Am following the "Streaming HTTP Response" documentation of Play2! framework which describes how a file or stream could be sent as chunked responses. The http-action code I wrote after reading this is very simple -
Ok.chunked(
Enumerator.fromStream(istream).andThen(Enumerator.eof)
)
With this code everytime I refresh the URL the data in the stream gets downloaded as a file by the browser. Instead of downloading as a file I want the stream content to shown inline in the browser as text. The File example on the documentation page describes how one could do this with files... but looking at the APIs I dont see an inline option with streams. So is it possible to show stream data inline with chunked responses everytime I refresh the browser? If my expectation is invalid then a little explanation of why-so will be very welcome.
From my comment: You should set a content-type supported by your browser (like text/plain or text/xml) when sending the response, otherwise you're just sending bytes and the browser doesn't "know" it can display it.
Update: adding the exact code that solved the issue:
Ok.chunked( Enumerator.fromStream(istream).andThen(Enumerator.eof) ).as("text/html")

Fiddler2 - How do I URlDecode the Request body for Viewing?

I'm using Fiddler to debug some particularly painful AJAX code, and in the POST requests that are being sent across to the server the Request BODY is UrlEncoded. This leads to me having to cut and paste the text into an online app to UrlDecode the text into the JSON object for the request. There has to be a better way to do this.
Does anyone know how I can make fiddler automatically URLDecode the body of the POST Request?
Well, you can simply press CTRL+E to decode locally. But depending on the format, you may also be able to use the WebForms Inspector.
Fiddler can manipulate the HTTP request and response in any way you like:
https://stackoverflow.com/a/23615119/264181