Proxied "multipart" request results in empty body - guzzle

Our team recently migrated to Laminas and in doing so, are now experiencing an issue with how our application handles file uploads. We have 2 applications (both MVC): one view application and one API application. Our view application handles all requests from the browser and proxies API request through our ApiProxyController. This works for every one of our requests except for our file uploads. For those requests (multipart/form-data;), we've seen that by the time the request gets to our API application, the body of the request is empty.
We've experienced this issue using both the latest version of the Guzzle HTTP client and the Laminas client. Here is a snippet of the code we're using to proxy the request from our view application:
/* #var $request \Laminas\Http\PhpEnvironment\Request */
$request = $this->getRequest();
// URI object used to build PSR-7 request; update URI for proper forwarding
$uri = $request->getUri();
$uri->setHost($this->apiHost);
$psr7Request = Psr7ServerRequest::fromLaminas($request)
->withoutHeader('Host')
->withHeader('Authorization', sprintf('Bearer %s', $identity->getAccessToken()->getToken()));
$guzzle = new \GuzzleHttp\Client([
// Turn off SSL certificate verification
'verify' => false,
// Prevent exceptions from being thrown due to HTTP status codes (e.g. 4xx, 5xx)
'exceptions' => false,
]);
$psr7Response = $guzzle->send($psr7Request);
$response = Psr7Response::toLaminas($psr7Response);

I had similar trouble in laravel. You need to remove content-type header from original request since it contains boundary.
Then you should modify your request to send body as multipart to generate new content-type and content-length headers automatically.

Related

Dart http.MultipartRequest not sending data to Lumen after relocating server to a different apache server, Postman works fine

I have a multipart request I'm trying to send from my Flutter app to my remote Lumen/Apache server. This code was working fine when I was hosting my Lumen server in Homestead locally, which runs nginx.
The Lumen server itself responds accurately when sending the request via Postman. I doubt the Flutter application is sending nothing at all, as it was working before the server move. Since it's not the Lumen app, as it works with Postman, then I presume it has something to do with apache.
The header I set, using http.MultipartRequest() is (in Postman I used form-data):
headers['Content-Type'] = 'multipart/form-data';
I also have an Authorization Bearer: token header which works fine, as the app would've presented an Unauthorized response before running the route.
I have the following code:
...
print("Multipart request fields: " + request.fields.toString());
print("Multipart request files: " + request.files.toString());
var streamedResponse = await request.send();
response = await http.Response.fromStream(streamedResponse);
if (response.statusCode == 200) print('Uploaded!');
I get the following output in debug:
I/flutter ( 7073): Multipart request fields: {productName: new, description: new, price: 30.0, currency: USD, quantity: -1.0, mass: 0.0, massUnit: lb, isData: true}
I/flutter ( 7073): Multipart request files: [Instance of 'MultipartFile']
In my Lumen application, I have a function that simply does:
var_dump($request->all());
die;
I get the following result:
I/flutter ( 7073): array(1) {
I/flutter ( 7073): ["_method"]=>
I/flutter ( 7073): string(4) "POST"
I/flutter ( 7073): }
This arrived from the request not passing validation checks. Before this, I had the longer function:
$validator = Validator::make($request->all(), [
'productName' => 'required',
'description' => 'required',
'image' => 'sometimes|image|mimetypes:image/jpeg,image/png|max:600',// TODO remove sometimes from this
'price' => 'required',
'currency' => 'required|string',
'quantity' => 'required',
'mass' => 'numeric',
'massUnit' => 'required|string',
'isData' => 'present|string|nullable',
]);
And it was failing the validation tests, even though the data was apparently being sent (and it used to pass before the server move).
What could the reason be? It seems the only difference is moving from an nginx local Homestead server to a remote apache server. I thought maybe I need a different header, possibly.
Update
With some more testing, I found that the request works without error if there is no image in the multipart/form-data request. The issue seems to rely on there being a file attached. Apparently, when and only when there is a file attached, the server reports that no data was sent.
I have two very similar requests, one that is POST and one that is PATCH. I found that with Postman, using the request that is a PATCH, sending it as a PATCH requests results in the same issue -- no data sent. Sending it as a POST request with the field "_method" set to "PATCH" does work, however. The request that is a POST request (to add a new item) works fine without the "_method" field, when set to a POST request.
The POST endpoint is responding as if http is not sending a "POST" request (empty data). With the PATCH endpoint, Lumen is responding as if it was a "POST" request.
Here's the difference between Postman and http.MultipartRequest:
With POST request:
Postman: POST request + "_method: POST": Works
http: POST request + "_method: POST": No data sent
With PATCH request:
Postman: POST request + "_method: PATCH": Works
Postman: PATCH request without "_method": No data sent
http: POST request + "_method: PATCH": 405 Method Not Allowed (as if it was a PATCH request)
Again, the properly formed requests work if there is no file attached (and everything else is the same). It's when the http request has a file that it suddenly works unexpectedly.
In the last server, I could not fully test the http functionality as I was using localtunnel.js to forward requests to my phone from Homestead, but it has a file size limit that I didn't want to work with. I did know that dart's http was sending the file, however, due to the 413 Request Entity Too Large responses that didn't occur without the image.
Here's the output of the request object in Flutter's DevTools (sorry it's a picture -- it doesn't permit selecting the text):
When not including a file, the request looks identical except for that files list is empty. Despite this, a request without a file is received by the server with data, while the other results in no data apparently sent.
The headers field of the request object looks fine too, with Content-Type being present. The authorization field clearly works, too, otherwise I would be getting unauthorized errors. Since the request object looks fine, something between request.send(); and the Lumen app is dropping all of the data if there is a file attached. Where it's coming from -- the Flutter app or the server -- is difficult to be sure about. (Considering the Flutter app was definitely sending something large when a file was attached when I was using localtunnel.js, it seems likely to be on the server side, as it seems the Flutter app is sending data, just the server is dropping it if there is a file. However, Postman works without hiccup, which suggests the server is functioning correctly.)
Update
My answer was a bit early. PHP was throwing away all of the data because the file was too large. After editing php.ini for post_max_size and upload_max_filesize, the data did start making it through, and, at first, I thought that indicated it was working.
However, it turns out that while now the non-file data is making it through, and using mod_dumpio, I see that the file is being sent, when the request comes from Flutter's http.multipartRequest, the server is dropping the file, specifically. When the request comes from Postman, everything works fine. I made sure to test with the exact same image, as well, and there seems to be no difference in the requests.
Update: This answer only partially solved the issue.
Using tcpdump on the server, I found that the server is indeed receiving data (even though Lumen was saying that it was not receiving data), and the amount of data seemed to correspond with whether a file is attached or not. From this, it seemed the problem was occurring with Apache, PHP, or Lumen.
I also enabled mod_dumpio, and tested the requests with and without a file and both appear to send data normally. The file request was clearly sending a large amount more, and the Content-Length reported seemed accurate, even though Lumen was reporting no data.
Sorting through error.log I did find the error, however. It was just that PHP was dropping the data because it exceeded the post_max_size size of 8MB. Once editing php.ini for that and upload_max_filesize, the application worked fine.
What's funny is that's usually the first error that occurs with file uploads and PHP servers, so it's weird I didn't suspect it. It would be nice if PHP issued a 413 Response by default.
Recently I was working with sending MultipartFile to our server on MongoDB/HapiJS server. While it worked in postman and with vue web app, I struggled on Flutter to find out why it didn't work.
In our case the solution was rather silly, but it worked:
When sending the file I used MultipartFile.fromBytes(byteData), and after changing it to MultipartFile.fromFile(filePath) it worked.
Could you check this (or other way around) and tell if it works?

Cannot preserve POST request in PingAccess

I am getting this error when trying to authenticate to my application.
Log extract: level":"ERROR","message":"Cannot preserve POST request with content type 'application/json; charset=UTF-8' (only 'application/x-www-form-urlencoded' is supported)
My question: Is there a way to configure PingFederate, so it can handle POST requests?
That's not a PingFederate error, that's a PingAccess error.
This issue is typically seen on AJAX requests from a Single Page Application, where that Application definition in PingAccess has not been appropriately defined as "Web+API" with the "SPA Support" box checked. It happens when the PingAccess web session has expired, and the application tries to POST an update to the backend.
It must be understood, however, that the application will need to be coded so that it either handles the 401 or follows the 302 redirect that it gets back, once you add SPA Support.

axios causing an unintended OPTIONS request on HERE Autocomplete api

I am getting a preflight error 405: Method not allowed from the HERE API when I request autocomplete as per the documentation.
UPDATE 2:
I have since determined that Axios was adding my default.common authentication headers from my app's API client onto the HERE API client. Axios is supposed to keep those defaults separate per-client, but it seems that it doesn't ... at least not the version I have. I replaced the defaults with a per-client request interceptor and it worked fine. The request no longer triggers an OPTION pre-flight. No issue with HERE's API other than that it doesn't support OPTION method.
UPDATE:
The reason it fails is because HERE does not support the OPTIONS method, only the GET. So now the question is: Why does axios trigger an OPTIONS request when I don't set any headers? An XMLHttpRequest() based GET request does not trigger OPTIONS for the same URL. Something is happening with axios but I don't know what and I can't seem to investigate the headers that axios is sending.
ORIGINAL:
I've tried to find information about this error, as well as HTTP vs HTTPS. I haven't seen others having this problem so I feel like I must be making a simple error. The URL is generated correctly because it works when pasted directly into the browser for example.
const hereClient = axios.create({
baseURL: 'https://autocomplete.geocoder.api.here.com/6.2/'
})
async function searchHere (query) {
let searchTerms = query.split(' ').join('+')
let result = await hereClient.get('suggest.json', {
params: {
app_id: '<APPID>',
app_code: '<APPCODE>',
query: searchTerms
}
})
return processHereSearchResults(result.data)
}
The GET request fails on the OPTION preflight with a 405: Method not allowed. But if I paste the generated URL into a browser then it returns the expected results. For example:
https://autocomplete.geocoder.api.here.com/6.2/suggest.json?app_id=APPID&app_code=APPCODE&query=8131
returns:
{"suggestions":[{"label":"Česko, Brandýs nad Orlicí, 3123","language":"cs","countryCode":"CZE","locationId":"N . . .
Same result whether http or https.
I have since determined that Axios was adding my default.common authentication headers from my app's API client onto the HERE API client. Axios is supposed to keep those defaults separate per-client, but it seems that it doesn't ... at least not the version I have. I replaced the default header setting with a per-client request interceptor to set my authentication and it worked fine. The request no longer triggers an OPTION pre-flight. No issue with HERE's API other than that it doesn't support OPTION method.

How to read server set cookie in angular 2 application, and how to send same cookie in request from angular 2 application?

Requirement : Our application needs to support same user opening our web application as separated session.
The problem is not how to use cookies in angular 2, but how can sever get cookie from HTTPServletRequest object when angular 2 application makes a rest call to server.
Implementation: Server side restful application has one filter to set user's browser session in cookie and then in HttpServletResponse. Angular client is making one call upon application bootstrap, which is going through server filter to set user's browser session in cookie.
Problem statement: angular client is making first rest call which goes through server filter to set the browser session cookie. When i open chrome developer tool, i do see that rest api response has "set-cookie" which has cookie set by server, but when i open the application tag in developer tool, i do not see any cookie set.
After that if I make any other rest call through angular application, it does not send the cookie in either request or request headers. Now, our application rest api depends on this cookie value to be present in HttpServletRequest and now it is failing.
Can someone please guide me here? I must have done something wrong on angular 2 application side, which i am not able to catch.
I have tried passing "withCredentials =true", but no change.
Another thing I noticed, if i make "GET" request, then i do see cookie in request header, but for "POST" request, I do not see anything for cookie.
Please advice.
server side code to set cookie
String uniqueId = RandomStringUtils.randomAlphanumeric(32);
Cookie userSessionCookie = new Cookie("userSessionId", uniqueId);
if (getDefaultDomain() != null) {
userSessionCookie.setDomain(getDefaultDomain());
}
httpServletResponse.addCookie(userSessionCookie); httpServletResponse.addHeader("Access-Control-Allow-Credenti‌​als", "true"); httpServletResponse.addHeader("access-control-allow-methods"‌​, "GET, POST, PUT, PATCH, DELETE, OPTIONS");
httpServletResponse.addHeader("Access-Control-Allow-Headers"‌​, "Content-Type, token,withCredentials");
angular 2 post request which expects server to get cookie from HttpServletRequest
renderFF() {
//prepare renderFInput object
var fcRenderInput = {};
let headers = new Headers({ 'Content-Type': 'application/json' });
let options = new RequestOptions({ headers: headers, withCredentials: true
});
this._http.post('/api/v1/render/feature/fc',fcRenderI‌​nput,options)
.subscribe((res) => {
console.log(res.json());
});
}
Just a suggestion if this is about only one browser and multiple tabs, in this case you can use the local storage while setting some flag in it. Also when you try to open the same application in the new tab. you check if the flag is there and user is trying to open the same web application in some other tab of the same browser. You also need to delete the local storage you had set after some point.
I hope if you can get some trick to solve this issue :)

Odd CORS error querying JIRA REST API [duplicate]

This question already has answers here:
Origin is not allowed by Access-Control-Allow-Origin
(18 answers)
Closed 6 years ago.
I am trying to develop a D3 visualisation project for our JIRA boards, but I've fallen at the first hurdle. I'm having trouble authenticating and getting a list of JIRA boards.
This code is entirely client-side and is in Angular 2 RC 3. My service looks like this:
public authenticate( username:string, password:string ):void {
let encodedAuth:string = window.btoa( `${username}:${password}` );
this.headers = new Headers();
this.headers.append( 'Content-Type', 'application/json' );
this.headers.append( 'Authorization', `Basic ${encodedAuth}` );
}
public getAllBoards():Observable<Boards> {
return this.http.get( `http://${this.host}/rest/agile/1.0/board`, this.headers )
.map( response => response.json() as Boards )
}
and the code in my component looks like this:
constructor( protected jiraService:JIRAService ) {
this.jiraService.authenticate('me#you.com', 'password');
this.jiraService.getAllBoards().subscribe(
boards => this.boards = boards
);
}
Unfortunately, this generates what looks like a CORS error in my browser:
XMLHttpRequest cannot load https://myjira.atlassian.net/rest/agile/1.0/board. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'null' is therefore not allowed access. The response had HTTP status code 401.
...which is a little unexpected. This same URL in the browser works fine. Examining the request in Charles I see the error "SSL Proxying not enabled for this host: enable in Proxy Settings, SSL locations", but cannot actually find this setting. I don't care if I can't see it in Charles actually, I just want to get it working!
I have tried several of the npm JIRA packages but none of them are remarkable and seem to be designed for server-side development.
Any help greatly appreciated.
You get this error when you access it from within an application that was initially loaded from a different URL. If you load a new page from this "other" URL then this is not a CORS situation because the initial page load is already from this URL.
The server need to provide the expected CORS headers for the browser to allow this request. This is not an Angular issue but only a server issue.
Workarounds are
JSONP (doesn't support custom headers)
provide support server-side where your Angular application calls to your server which then forwards to the other server and then forwards the response it gets to your Angular application.