Guzzle not sending PSR-7 POST body correctly - guzzle

It is either not being sent, or not being received correctly. Using curl direct from the command line (using the -d option) or from PHP (using CURLOPT_POSTFIELDS) does work.
I start with a PSR-7 request:
$request = GuzzleHttp\Psr7\Request('POST', $url);
I add authentication header, which authenticates against the API correctly:
$request = $request->withHeader('Authorization', 'Bearer ' . $accessToken);
Then I add the request body:
// The parameter for the API function
$body = \GuzzleHttp\Psr7\stream_for('args=dot');
$request = $request->withBody($body);
I can send the message to the API:
$client = new \GuzzleHttp\Client();
$response = $client->send($request, ['timeout' => 2]);
The response I get back indicates that the "args" parameter was simply not seen by the API. I have tried moving the authentication token to the args:
'args=dot&access_token=123456789'
This should work, and does work with curl from the command line (-d access_token=123456789) but the API fails to see that parameter also when sending cia curl (6.x) as above.
I can see the message does contain the body:
var_dump((string)$request->getBody());
// string(8) "args=dot"
// The "=" is NOT URL-encoded in any way.
So what could be going wrong here? Are the parameters not being sent, or are they being sent in the wrong format (maybe '=' is being encoded?), or is perhaps the wrong content-type being used? It is difficult to see what is being sent "on the wire" when using Guzzle, since the HTTP message is formatted and sent many layer deep.
Edit: Calling up a local test script instead of the remote API, I get this raw message detail:
POST
CONNECTION: close
CONTENT-LENGTH: 62
HOST: acadweb.co.uk
USER-AGENT: GuzzleHttp/6.1.1 curl/7.19.7 PHP/5.5.9
args=dot&access_token=5e09d638965288937dfa0ca36366c9f8a44d4f3e
So it looks like the body is being sent, so I guess something else is missing to tell the remote API how to interpret that body.
Edit: the command-line curl that does work, sent to the same test script, gives me two additional header fields in the request:
CONTENT-TYPE: application/x-www-form-urlencoded
ACCEPT: */*
I'm going to guess it is the content-type header which is missing from the Guzzle request which is the source of the problem. So is this a Guzzle bug? Should it not always sent a Content-Type, based on the assumptions it makes that are listed in the documentation?

The Content-Type header was the issue. Normally, Guzzle will hold your hand and insert headers it deems necessary, and makes a good guess at the Content-Type based on what you have given it, and how you have given it.
With Guzzle's PSR-7 messages, none of that hand-holding is done. It strictly leaves all the headers for you to handle. So when adding POST parameters to a PSR-7 Request, you must explicitly set the Content-Type:
$params = ['Foo' => 'Bar'];
$body = \GuzzleHttp\Psr7\stream_for(http_build_query($params));
$request = $request->withBody($body);
$request = $request->withHeader('Content-Type', 'application/x-www-form-urlencoded');
The ability to pass in the params as an array and to leave Guzzle to work out the rest, does not apply to Guzzle's PSR-7 implementation. It's a bit clumsy, as you need to serialise the POST parameters into a HTTP query string, and then stick that into a stream, but there you have it. There may be an easier way to handle this (e.g. a wrapper class I'm not aware of), and I'll wait and see if any come up before accepting this answer.
Be aware also that if constructing a multipart/form-data Request message, you need to add the boundary string to the Content-Type:
$request = $request->withHeader('Content-Type', 'multipart/form-data; boundary=' . $boundary);
Where $boundary can be something like uniq() and is used in construction the multipart body.

The GuzzleHttp\Client provides all necessary wrapping.
$response = $client->post(
$uri,
[
'auth' => [null, 'Bearer ' . $token],
'form_params' => $parameters,
]);
Documentation available Guzzle Request Options
Edit: However, if your requests are being used within GuzzleHttp\Pool then, you can simply everything into the following:
$request = new GuzzleHttp\Psr7\Request(
'POST',
$uri,
[
'Authorization' => 'Bearer ' . $token,
'Content-Type' => 'application/x-www-form-urlencoded'
],
http_build_query($form_params, null, '&')
);

Related

Define Guzzle Content-Length

I am trying to send a request with a content length set by myself and not by guzzle. ( guzzle 6)
$res = $client->request('POST', $url, ['headers' => ['Content-Length' => 32]];
However this stops the request from sending !
I am doing this because my URL has a query string and i need the content length to be set. Not doing so sets a Content-Length: 0 by guzzle - how can i define all headers and stop guzzle from doing this?
Regards
Your code works perfectly with Guzzle (just add a missed bracket in the end ;).
You probably get an exception, but it's because the server responds with 413 Request entity too large, and Guzzle converts it to exception (see http_errors to control this behaviour). Because you don't provide the request body at all.
Anyway, I don't get it, how the query string is connected to Content-Length header. The header defined length of the request body, but the query string is not a part of the body, so there is no connection.

Perl - LWP::UserAgent; slow response - more then 4096 characters

I'm trying to retrieve interface statistics data via HTTPS GET from a remote network device. The script is working, however when a response is larger than 4098 characters the connection hangs for 60 seconds. Please help ! I'm slowly going crazy.
my $interface_getAll = HTTP::Request->new( GET => $URL );
$interface_getAll->content( 'session_id=' . $sessionID . '&format=json&method=network.interface.fetchAllStatistics' );
my $res = $ua->request($interface_getAll);
$interface_getAll_rData = $res->content;
my $InterfaceValues = decode_json($interface_getAll_rData);
print Dumper($InterfaceValues);
I can't replicate this issue when using curl or wget, I get a response instantly. The API on the remote device responds to GET requests only. If you think that my code is invalid I'm open to any suggestions.
request URL example/format:
https://192.168.99.51:443/services/rest/V2.1/session_id=b121c16aa0c0361e9bebe5bd67e60a&format=json&method=network.interface.fetchStatistics
You say you need to fetch url
https://192.168.99.51:443/services/rest/V2.1/session_id=b121c16aa0c0361e9bebe5bd67e60a&format=json&method=network.interface.fetchStatistics
but your code is fetching url
https://192.168.99.51:443/services/rest/V2.1/
The following is how you fetch the correct url:
use URI;
my $url = URI->new("https://192.168.99.51:443/services/rest/V2.1/");
$url->query_form(
session_id => $sessionID,
format => 'json',
method => 'network.interface.fetchAllStatistics',
);
my $response = $ua->get($url);
Using $url->query_form ensures your values are properly encoded.
The API on the remote device responds to GET requests only.
Using a message body with a GET request has no defined behavior. In fact RFC 7231 clearly says
A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.
Thus if your API really requires a GET with a message body you should consider the API broken.
But given the URL you show I would rather suggest that the API does not require a message body but that you simply use LWP in the wrong way.
According to your example URL the session_id=... is not inside the message body but actually part of the URL and should be added to this, i.e.
$URL = "https://192.168.99.51:443/services/rest/V2.1/session_id=...."
And while you could just append it directly to the URL a more clean and readable way would be to construct your URL with URI as suggested by ikegami#:
use URI;
my $url = URI->new("https://192.168.99.51:443/services/rest/V2.1/");
$url->query_form(
session_id => $sessionID,
format => 'json',
method => 'network.interface.fetchAllStatistics',
);
my $interface_getAll = HTTP::Request->new( GET => $url );
# no setting content() here

Adding headers to a LWP post in Perl

I have taken over some perl code and been asked to add a keep-alive header to the LWP post that happens.
Google tells me how to do this for certain setups, but I can't see how to do it for the way this code has been written. All the info I can find works on the basis of creating the LWP object, then creating the POST and parameters, then adding the headers, then actually POSTING the request, however in the code I have to deal with, the creating of the POST, adding the headers and sending are all in one line:
my $ua = LWP::UserAgent->new;
my $response = $ua->post( $URL, ['parm1'=>'val1']);
How/where can I add the headers in this setup, or do I need to re-write as per the examples I have found?
The LWP::UserAgent page tells you how to do this. You would set the handler request_prepare on the user agent object. That will pass you in the request object before it posts.
Actually, anything you put as a list of key-value pairs before the key 'Content' followed by the structure that you want to post, will translate into headers, per HTTP::Request::Common::POST
$ua->post( $URL, keep_alive => 1, Content => ['parm1'=>'val1']);
Or without the content tag, if you put the structure first, you can put header key-value pairs after:
$ua->post( $URL, ['parm1'=>'val1'], keep_alive => 1 );
Did they really asked you to add a keep-alive header only, or did they ask you to support keep alive, e.g. multiple HTTP requests within the same TCP connection. In the latter case you should use (according to the documentation of LWP::UserAgent):
my $ua = LWP::UserAgent->new( keep_alive => 10 );
$ua->get('http://foo.bar/page1');
$ua->get('http://foo.bar/page2'); # reuses connection from previous request
In this case it will keep at most 10 connections open at the same time. If you only do requests do the same site you can also set it to 1 so that it reuses the same TCP connection for all requests.
A Keep-Alive header has no meaning, what keep_alive => 1 within the user agent does is set up a connection cache and add a "Connection: keep-alive" header (with HTTP/1.1 keep-alive is implicite, so it does not need to add the header for HTTP/1.1 requests).

How to get URLencoded data from the body of a POST in CGI Perl

POSTDATA is not the correct answer. I have read the docs and still don't see how I can get the data.
I want to receive this request:
POST /cgi-bin/myscript.cgi HTTP/1.1
Host: myhost.com
Content-Length: 3
Content-Type: application/x-www-form-urlencoded
255
and have the server respond
You sent the string "255"
Please assist, I am a Perl beginner and have gotten a bunch of seemingly wrong and useless answers to this seemingly simple request.
CGI will automatically parse form data, so you need to hide that what you got is form data (or at least claims to be).
use CGI qw( );
$ENV{CONTENT_TYPE} = 'application/octet-stream';
my $cgi = CGI->new();
my $post_data = $cgi->param('POSTDATA');
Better solution: Have the requester use a correct content type (e.g. application/octet-stream), or have the requester actually send form data (e.g. data=255).
Unique solution for me, was change of ContentType on client's petition to 'application/octet-stream'
Module CGI CPAN says:
If POSTed data is not of type application/x-www-form-urlencoded or
multipart/form-data, then the POSTed data will not be processed, but
instead be returned as-is in a parameter named POSTDATA.
So if you can't change on clients petition to other ContentType, it won't be processed.
CGI (in recent versions at least) will stuff incorrectly encoded x-www-form-urlencoded params into a parameter named keywords. Better to send a proper content type though, then the POSTDATA works exactly as the docs say:
If POSTed data is not of type application/x-www-form-urlencoded or
multipart/form-data, then the POSTed data will not be processed...
use strictures;
use CGI::Emulate::PSGI;
use Plack::Test;
use HTTP::Request::Common;
use Test::More;
my $post = POST "/non-e-importa",
"Content-Length" => 5,
"Content-Type" => "application/x-www-form-urlencoded",
Content => "ohai\n";
my $cgis = CGI::Emulate::PSGI->handler( sub {
use CGI "param", "header";
my $incorrectly_encoded_body = param("keywords");
print header("text/plain"), $incorrectly_encoded_body;
});
test_psgi $cgis, sub {
my $cb = shift;
my $res = $cb->($post);
is $res->content, "ohai", "Soopersek437 param: keywords";
};
done_testing();
__END__
prove so-16846138 -v
ok 1 - Soopersek437 param: keywords
1..1
ok
All tests successful.
Result: PASS

Constructing a Paypal OAuth

I am trying to get an access token for Paypal's RESTful web services but unfortunately not making any headway. This is my first time dealing with REST, so please be patient with me :)
Here is what I have:
Client_id and secret as provided by Paypal for a sandbox account through the paypal developer website.
The ENDpoint: https://api.sandbox.paypal.com/v1/oauth2/token
The documentation that i am referring to is : https://developer.paypal.com/webapps/developer/docs/integration/direct/make-your-first-call/
Now the juicy part of making that API call. I am developing in PHP so I am using CURL to make the calls. something like this;
const CLIENT_ID = ****..*** ;
const SECRET = ***..***;
$base64EncodedClientID = base64_encode(self::CLIENT_ID . ":" . self::SECRET);
$headers = array("Authorization" => "Basic " . $base64EncodedClientId, "Accept" =>"*/*", "Content-type" => "multipart/form-data");
$params = array("grant_type"=>"client_credentials");
$url = "https://api.sandbox.paypal.com/v1/oauth2/token";
$ch = curl_init();
curl_setopt($ch,CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch,CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_POST, true);
curl_setopt($ch,CURLOPT_HEADER, $headers);
curl_setopt($ch,CURLOPT_POSTFIELDS,$params);
$response = curl_exec($ch);
Pretty vanilla right? Except that I do not get the JSON response that I expect from Paypal but false. This implies that my CURL request was not prepared well, perhaps I am setting the header incorrectly or the params are incorrect. Regardless, the URL is definitely accessible since I was able to access it through command line with the same credentials and got the desired JSON response.
The one glaring problem I have with the above code is that I am providing the client_id and secret as a header option. basic sense tells me that they need to be part of the POST field data However, if you look at line 89 of this Github code https://github.com/paypal/rest-api-sdk-php/blob/master/lib/PayPal/Auth/OAuthTokenCredential.php (Paypals' official PHP REST SDK), it clearly states that the credentials are being set in the header field.
Where am I messing up ?
With curl you don't need to manually generate the base64 encoded value for the Authorization header just use the CURLOPT_USERPWD option and pass the clientID and secret as the user:pwd.
curl_setopt($curl, CURLOPT_USERPWD, $clientId . ":" . $clientSecret);
here is a sample - look for the get_access_token() method:
https://github.com/paypal/rest-api-curlsamples/blob/master/execute_all_calls.php
Had the exact same problem you ran into. The issue is that PayPal accepts the content-type application/x-www-form-urlencoded. Your code is attempting to send multipart/form-data. CURL by default sends application/x-www-form-urlencoded, but you are passing your data as an array. Instead, you should be passing the data like a url encoded string since this is what application/x-www-form-urlencoded data looks like:
$params = "grant_type=client_credentials";
Your headers have the same problem. Pass it as an array of strings instead of a dictionary. For instance:
$headers = ["Authorization Basic " . $base64EncodedClientId];
Also, you don't need those other two headers you passed in. The 'Accept' header does nothing since you are accepting everything, and the Content-type is wrong for one, and two is defaulted to 'application/x-www-form-urlencoded' by CURL so unless you need to override that, there is no need.