Dart http.MultipartRequest not sending data to Lumen after relocating server to a different apache server, Postman works fine - flutter

I have a multipart request I'm trying to send from my Flutter app to my remote Lumen/Apache server. This code was working fine when I was hosting my Lumen server in Homestead locally, which runs nginx.
The Lumen server itself responds accurately when sending the request via Postman. I doubt the Flutter application is sending nothing at all, as it was working before the server move. Since it's not the Lumen app, as it works with Postman, then I presume it has something to do with apache.
The header I set, using http.MultipartRequest() is (in Postman I used form-data):
headers['Content-Type'] = 'multipart/form-data';
I also have an Authorization Bearer: token header which works fine, as the app would've presented an Unauthorized response before running the route.
I have the following code:
...
print("Multipart request fields: " + request.fields.toString());
print("Multipart request files: " + request.files.toString());
var streamedResponse = await request.send();
response = await http.Response.fromStream(streamedResponse);
if (response.statusCode == 200) print('Uploaded!');
I get the following output in debug:
I/flutter ( 7073): Multipart request fields: {productName: new, description: new, price: 30.0, currency: USD, quantity: -1.0, mass: 0.0, massUnit: lb, isData: true}
I/flutter ( 7073): Multipart request files: [Instance of 'MultipartFile']
In my Lumen application, I have a function that simply does:
var_dump($request->all());
die;
I get the following result:
I/flutter ( 7073): array(1) {
I/flutter ( 7073): ["_method"]=>
I/flutter ( 7073): string(4) "POST"
I/flutter ( 7073): }
This arrived from the request not passing validation checks. Before this, I had the longer function:
$validator = Validator::make($request->all(), [
'productName' => 'required',
'description' => 'required',
'image' => 'sometimes|image|mimetypes:image/jpeg,image/png|max:600',// TODO remove sometimes from this
'price' => 'required',
'currency' => 'required|string',
'quantity' => 'required',
'mass' => 'numeric',
'massUnit' => 'required|string',
'isData' => 'present|string|nullable',
]);
And it was failing the validation tests, even though the data was apparently being sent (and it used to pass before the server move).
What could the reason be? It seems the only difference is moving from an nginx local Homestead server to a remote apache server. I thought maybe I need a different header, possibly.
Update
With some more testing, I found that the request works without error if there is no image in the multipart/form-data request. The issue seems to rely on there being a file attached. Apparently, when and only when there is a file attached, the server reports that no data was sent.
I have two very similar requests, one that is POST and one that is PATCH. I found that with Postman, using the request that is a PATCH, sending it as a PATCH requests results in the same issue -- no data sent. Sending it as a POST request with the field "_method" set to "PATCH" does work, however. The request that is a POST request (to add a new item) works fine without the "_method" field, when set to a POST request.
The POST endpoint is responding as if http is not sending a "POST" request (empty data). With the PATCH endpoint, Lumen is responding as if it was a "POST" request.
Here's the difference between Postman and http.MultipartRequest:
With POST request:
Postman: POST request + "_method: POST": Works
http: POST request + "_method: POST": No data sent
With PATCH request:
Postman: POST request + "_method: PATCH": Works
Postman: PATCH request without "_method": No data sent
http: POST request + "_method: PATCH": 405 Method Not Allowed (as if it was a PATCH request)
Again, the properly formed requests work if there is no file attached (and everything else is the same). It's when the http request has a file that it suddenly works unexpectedly.
In the last server, I could not fully test the http functionality as I was using localtunnel.js to forward requests to my phone from Homestead, but it has a file size limit that I didn't want to work with. I did know that dart's http was sending the file, however, due to the 413 Request Entity Too Large responses that didn't occur without the image.
Here's the output of the request object in Flutter's DevTools (sorry it's a picture -- it doesn't permit selecting the text):
When not including a file, the request looks identical except for that files list is empty. Despite this, a request without a file is received by the server with data, while the other results in no data apparently sent.
The headers field of the request object looks fine too, with Content-Type being present. The authorization field clearly works, too, otherwise I would be getting unauthorized errors. Since the request object looks fine, something between request.send(); and the Lumen app is dropping all of the data if there is a file attached. Where it's coming from -- the Flutter app or the server -- is difficult to be sure about. (Considering the Flutter app was definitely sending something large when a file was attached when I was using localtunnel.js, it seems likely to be on the server side, as it seems the Flutter app is sending data, just the server is dropping it if there is a file. However, Postman works without hiccup, which suggests the server is functioning correctly.)
Update
My answer was a bit early. PHP was throwing away all of the data because the file was too large. After editing php.ini for post_max_size and upload_max_filesize, the data did start making it through, and, at first, I thought that indicated it was working.
However, it turns out that while now the non-file data is making it through, and using mod_dumpio, I see that the file is being sent, when the request comes from Flutter's http.multipartRequest, the server is dropping the file, specifically. When the request comes from Postman, everything works fine. I made sure to test with the exact same image, as well, and there seems to be no difference in the requests.

Update: This answer only partially solved the issue.
Using tcpdump on the server, I found that the server is indeed receiving data (even though Lumen was saying that it was not receiving data), and the amount of data seemed to correspond with whether a file is attached or not. From this, it seemed the problem was occurring with Apache, PHP, or Lumen.
I also enabled mod_dumpio, and tested the requests with and without a file and both appear to send data normally. The file request was clearly sending a large amount more, and the Content-Length reported seemed accurate, even though Lumen was reporting no data.
Sorting through error.log I did find the error, however. It was just that PHP was dropping the data because it exceeded the post_max_size size of 8MB. Once editing php.ini for that and upload_max_filesize, the application worked fine.
What's funny is that's usually the first error that occurs with file uploads and PHP servers, so it's weird I didn't suspect it. It would be nice if PHP issued a 413 Response by default.

Recently I was working with sending MultipartFile to our server on MongoDB/HapiJS server. While it worked in postman and with vue web app, I struggled on Flutter to find out why it didn't work.
In our case the solution was rather silly, but it worked:
When sending the file I used MultipartFile.fromBytes(byteData), and after changing it to MultipartFile.fromFile(filePath) it worked.
Could you check this (or other way around) and tell if it works?

Related

Replicatiing post request from website form using postman returns 500 internal server error

I'm trying to replicate a post request done normally by a website form via postman but the server returns 500 error.
the form website URL that I'm dealing with is here.
what I have done so far is investigate the network request using chrome or safari dev tools, copy the request as cURL, import the cURL in postman and do the request.
what can be the possible reasons for the failure and what are the alternative ways to achieve the same result?
Postman Headers:
Most probably you must have used invalid request body. The browser shows parsed json body and you might have copied incomple request body.
To get full body click view source and copy the full content.

What Http code should I return when replacing a resource (file)

I searched "HTTP code for replacing a resource" on Google but it really messes up with the keywords and don't give me the desired results. I have a file upload API and an interface that users interact to manage their images, two situations can occur: the user can replace an existing image or he can send more images. When the user replaces an existing image what HTTP code should I return?
Its's not a duplicate because I'm asking what HTTP code should I return when replacing a file not what people think about returning null in a PUT operation. If a person answered the possible duplicate question with information that answer my question that is another story, the question itself is not the same.
I have a file upload API and an interface that users interact to manage their images, two situations can occur: the user can replace an existing image or he can send more images. When the user replaces an existing image what HTTP code should I return?
Probably a 200 OK; the client made a request, you handled it successfully.
http://racksburg.com/choosing-an-http-status-code/
The actual implementation details of what the server actually did, or what the client intended, are not all that important; status codes are part of the meta data that allows generic http components to understand what's going on, so that they can behave appropriately (do we need to invalidate caches, should we pro-actively load some other resource, and so on).
HTTP code is mainly for HTTP server's status, eg: server error, file not found on server, etc.
In your application, your file upload API should return a JSON response to tell the user the upload result. and the HTTP code can always be 200 for succeeded operations. For failed operations, choose a code you like.
JSON response example:
HTTP 200
{
'result': 'ok',
}
HTTP 200
{
'result': 'replaced',
}
HTTP 403/405/etc
{
'result': 'failed',
'reason': 'file exists'
}
Ref:
HTTP status code for update and delete?
http://www.restapitutorial.com/lessons/httpmethods.html

OpenStreetMap Direct Routing Request via GET error

I have problem with OpenRouteService API (Direct Routing Request via GET described in here http://wiki.openstreetmap.org/wiki/OpenRouteService#Direct_Routing_Request_.28via_GET.29).
My request is:
http://openls.geog.uni-heidelberg.de/route?start=18.609%2C53.02&end=18.749%2C53.49&via=18.01%2C53.12&lang=pl&distunit=KM&routepref=Pedestrian&&weighting=Recommended&avoidAreas&useTMC=false&noMotorways=false&noTollways=false&noUnpavedroads=false&noSteps=false&noFerries=false&instructions=false
(from Toruń in Poland to Grudziądz via Bydgoszcz).
Unfortonatly, I get error:
"validation error: Expected element 'EndPoint#http://www.opengis.net/xls' instead of 'viaPoint#http://www.opengis.net/xls' here in element WayPointList#http://www.opengis.net/xls"
If I put antyhing in "via=" this error appear.
When I change "via=" to empty value:
http://openls.geog.uni-heidelberg.de/route?start=18.609%2C53.02&end=18.749%2C53.49&via=&lang=pl&distunit=KM&routepref=Pedestrian&&weighting=Recommended&avoidAreas&useTMC=false&noMotorways=false&noTollways=false&noUnpavedroads=false&noSteps=false&noFerries=false&instructions=false
all works fine.
Is it problem with my request or api isn't working correctly?
It seems that the frontend API of OpenRouteService receives the GET requests correctly, but after it builds the request in xml and relays it to the backend server the server fails to validate the request correctly. It should be a problem with the backend server.
The frontend php code is here for reference.
The web frontend, which POSTs the xml directly to another backend server, seems to work correctly with via points.

Yii2 Reading PUT request body after oauth2 server already did it

I am working on REST API with oauth2 authorization.
For Oauth2 server i use https://github.com/bshaffer/oauth2-server-php
Php doc says here http://php.net/manual/en/wrappers.php.php
Prior to PHP 5.6, a stream opened with php://input could only be read once; the stream did not support seek operations. However, depending on the SAPI implementation, it may be possible to open another php://input stream and restart reading. This is only possible if the request body data has been saved. Typically, this is the case for POST requests, but not other request methods, such as PUT or PROPFIND.
In short it means that it is possible to read POST body twice, but not PUT.
But Oauth2 server reads it first time here https://github.com/bshaffer/oauth2-server-php/blob/develop/src/OAuth2/Request.php#L114
So when i read raw body in Yii2 Request, it is empty. (only on PUT, on POST and PATCH it is ok and can be read twice).
https://github.com/yiisoft/yii2/blob/master/framework/web/Request.php#L345
I know that this is kind of expected, no bugs. But what would be the solution for this?
Before you create that auth server, run this (depending in where you do authentication, you can use beforeAction(), or even init():
$content = Yii::$app->request->rawBody;
$authentication = Request::createFromGlobals();
if ($content)
$authentication->content = $content;
Now, I don't know how/where you use the component, so it might not fully work, but in theory it should.

POST from WinForms app using HttpWebRequest to webservice doesn't work when sent through Fiddler

I'm using HttpWebRequest in a VB.Net WinForms app to get data from an inhouse webservice. The code I'm using works for both GET and POST when run while Fiddler is not running. If I have Fiddler running the GETs work and are captured but a POST doesn't complete. In this case Fiddler captures the initial request but never gets the response and the application doesn't get a response.
The code builds a HttpWebRequest for the POST setting the appropriate properties, encodes the data to be sent into JSON and then does this.
Using postStream As Stream = webrequestobj.GetRequestStream()
postStream.Write(WebServiceByteData, 0, WebServiceByteData.Length)
End Using
I used WireShark to capture the generated network packets and noticed that when a POST is sent without going through Fiddler the following happens.
When "postStream As Stream = webrequestobj.GetRequestStream()" is executed a packet with all of the header info is sent that includes a "Expect: 100-continue" header but doesn't have the request data.
When the postStrean.Write call is executed an additional packet is sent that has the request data.
With Fiddler running nothing is put on the wire until after the postStream.Write is executed. At that point both the header packet with the "Expect: 100-continue" header and the request data packet are sent back to back before the service has responded with the "100 Continue". I'm guessing that this confuses the webservice as it doesn't expect to get the request data packet yet. It doesn't respond with the requested data.
I used Composer to manually create the request without the "Expect: 100-continue" header. When this is executed the same two packets are generated and the service responds with the expected data.
So, in order to be able to use Fiddler to capture the POST traffic it looks like I need to either be able to tell HttpWebRequest to not issue the "Expect: 100-continue" header (I've looked but haven't found a way to do this) or for Fiddler to handle the packets differently, maybe not sending the second packet until it sees the "100 Continue" response or by stripping out the "Expect: 100-continue" header.
It's possible that I've missed a setup option in Fiddler but nothing I've tried so far makes any difference.
Thanks,
Dave
Old question, but the short answer is that the lack of a 100/Continue response shouldn't have mattered at all.
To learn more about Expect: Continue, including how to remove this header if you like, see http://blogs.msdn.com/b/fiddler/archive/2011/11/05/http-expect-continue-delays-transmitting-post-bodies-by-up-to-350-milliseconds.aspx