LWP::UserAgent returns incomplete 2GB response message - perl

I am using LWP::UserAgentto send request on a URL. But sometime in the response I am getting incomplete XML response.
Code
$args->{pua} = LWP::UserAgent->new();
$args->{header} = HTTP::Headers->new;
$args->{header}->header("Content-Type" => "text/xml", "SOAPAction" => $args->{soapaction});
$request = HTTP::Request->new( "POST", $args->{endpoint}, $args->{header}, $args->{xml});
$response = $args->{pua}->simple_request($request);
my $xmlResponse = $response->content;
In the $xmlResponse sometime I am getting incomplete response. Why is it happening?
ResponseHeader
Connection: close
Date: Tue, 19 May 2015 11:07:37 GMT
Server: nginx/1.6.2
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Type: text/xml;charset=ISO-8859-1
Client-Date: Tue, 19 May 2015 11:07:40 GMT
Client-Peer: 202.77.98.11:80
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
X-Frame-Options: SAMEORIGIN

LWP may return incomplete response when it failed to read whole body because of the timeout or other read error. In this case $response->is_success will be true and $response->code will be 200, but response headers will contain special header called X-Died.
So you can check this header:
unless ($response->is_success) {
die "Response failed: ", $response->status_line;
}
if ($response->header('X-Died')) {
die "Response failed (internal): ", $response->header('X-Died');
}

Related

Wrong Content-Type in response in IErrorHandler

I would like to send response to my service in JSON format. I catch my custom errors in my custom behavior:
void IErrorHandler.ProvideFault(Exception error, MessageVersion version, ref Message fault)
{
XDocument errorMsg = XDocument.Parse("<errorMessage>" + error.Message + "</errorMessage>");
var jsonWriter = new JsonErrorBodyWriter(errorMsg);
fault = Message.CreateMessage(version, null, jsonWriter);
fault.Properties.Add(WebBodyFormatMessageProperty.Name, new WebBodyFormatMessageProperty(WebContentFormat.Json));
HttpResponseMessageProperty prop = new HttpResponseMessageProperty();
prop.StatusCode = HttpStatusCode.Unauthorized;
prop.Headers.Add("Content-Type", "application/json; charset=utf-8");
prop.Headers[HttpRequestHeader.ContentType] = "application/json; charset=utf-8";
--Tried different ways to achieve this
fault.Properties.Add(HttpResponseMessageProperty.Name, prop);
}
But I get wrong content-type in response. And also I couldn't manage to write any custom header like :
prop.Headers.Add("Test", "Value");
Reponse:
HTTP/1.1 401 Unauthorized
Content-Type: application/xml; charset=utf-8
Server: Microsoft-IIS/10.0
X-Powered-By: ASP.NET
Date: Wed, 30 Sep 2020 08:41:15 GMT
Content-Length: 37
{"description":"Autorization Failed"}
What is wrong in my code?

Can't redirect with status

Controller is called with
$this->get( '/read/{slug}', \Rib\Src\Apps\Blog\BlogControllers\IndexController::class . ':index' );
Inside it I tried:
return $response->withStatus( 404 )->withRedirect( '/message' );
or
return $response->withRedirect( '/message', 404 );
but the response returned always has code 200.
How to enforce 404 ?
You cannot redirect with 404 status code. Only 3xx is valid for redirection. When browser receives a Location: header it makes a new request to the given url. This means you could however redirect to a route which returns 404.
$app->get("/test", function ($request, $response, $arguments) {
return $response->withRedirect("/message");
});
$app->get("/message", function ($request, $response, $arguments) {
return $response->write("Oh noes!")->withStatus(404);
});
Above code will redirect you to response with 404 status code.
$ curl --include --location http://0.0.0.0:8080/test
HTTP/1.1 302 Found
Host: 0.0.0.0:8080
Date: Sun, 26 Mar 2017 04:53:05 +0000
Connection: close
X-Powered-By: PHP/7.1.2
Content-Type: text/html; charset=UTF-8
Location: /message
HTTP/1.1 404 Not Found
Host: 0.0.0.0:8080
Date: Sun, 26 Mar 2017 04:53:05 +0000
Connection: close
X-Powered-By: PHP/7.1.2
Content-Type: text/html; charset=UTF-8
Oh noes!

Google oAuth2 unauthorized_client by refresh_token

i'm trying to use an refresh token from the oAuth2 web redirect auth in my console perl script. The client id is the same and correct client id i used in my javascript an i checked it 5 times that it ist the same as i have in my google API's Console.
The client secret is checked two and it is correct.
The refresh token was created with approval_prompt=force&access_type=offline
Here is my perl sample code i use:
# -----------------------------------------------------------------------------------
my $CLIENT_ID = 'XXXXX.apps.googleusercontent.com';
my $CLIENT_SECRET = 'YYYYYYYYYYY';
# -----------------------------------------------------------------------------------
# TESTING
my $refresh_token = '1/is_5_minutes_old';
# -----------------------------------------------------------------------------------
my $string = '';
$string .= 'grant_type=refresh_token';
$string .= '&client_id=' . $CLIENT_ID;
$string .= '&client_secret=' . $CLIENT_SECRET;
$string .= '&refresh_token=' . $refresh_token;
$ua = LWP::UserAgent->new;
my $req =
HTTP::Request->new( POST => 'https://accounts.google.com/o/oauth2/token' );
$req->content_type('application/x-www-form-urlencoded');
$req->content($string);
print $string . "\n";
my $res = $ua->request($req);
print $res->as_string;
The response of it:
HTTP/1.1 400 Bad Request
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Connection: close
Date: Mon, 02 Sep 2013 10:50:26 GMT
Pragma: no-cache
Server: GSE
Content-Type: application/json
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Alternate-Protocol: 443:quic
Client-Date: Mon, 02 Sep 2013 10:50:26 GMT
Client-Peer: 74.125.136.84:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/O=Google Inc/CN=Google Internet Authority G2
Client-SSL-Cert-Subject: /C=US/ST=California/L=Mountain View/O=Google Inc/CN=accounts.google.com
Client-SSL-Cipher: RC4-SHA
Client-SSL-Warning: Peer certificate not verified
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
{
"error" : "unauthorized_client"
}
i hope you have an idea to help.
greatings
Invalid client usually means that the client ID and client secret don't match, or there is a typo in one of them (though you mention you've double checked this!). Nothing in your code looks wrong.
When you retrieve the refresh token, could you try putting the access token that comes along with it into the tokeinfo endpoint and making sure the values for client ID there match the ones you've configured with: https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=
It might be worth dumping out the request to make sure there isn't a mistake in that (e.g. too short a content-length header or similar).
Dumping the request is key. To help , here is one I baked earlier..
==POST==
https://accounts.google.com/o/oauth2/token
refresh_token=1/_PEzU2m71wertwertwerJUtrtrytrytryf3trytryoCo
&client_id=612222222225
&client_secret=Q7334534543534yKLu
&grant_type=refresh_token
Are you using the short form of the client id, ie. just the number?

adding a response header to 302 response using perl

i am trying to write a perl page that returns an http 302 response to a different location and adds a custom header to that response.
so my desired http response should be something like this:
HTTP/1.1 302 Moved
Date: Sun, 15 Apr 2012 10:59:02 GMT
Server: Apache
Location: http://www.google.com
Content-Length: 396
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
CUSTOM_HEADER: CUSTOM_VALUE
i tried using CGI:
#!/bin/perl
use strict;
use APR::Request::Apache2;
my $r = shift;
$r->content_type('text/html; charset=utf-8');
$r->headers_out()->add("CUSTOM_HEADER", "CUSTOM_VALUE");
$r->headers_out()->add("Location", "http://www.google.com");
$r->status(302);
and i do get 302 response to google but no CUSTOM_HEADER. once i change the status to 200 by $r->status(200); i do get the CUSTOM_HEADER.
so whats up with this behavior ? how can i add my header to the 302 response ?
Use $r->err_headers_out->set or $r->err_headers_out->add
my $r = shift;
$r->content_type('text/html; charset=utf-8');
$r->err_headers_out->set(Location => "http://www.google.com");
$r->status(302);
You should use err_headers_out(). These will be printed even on errors and redirects.

How can I get both the GET and POST request params, on a POST request?

I'm creating a facebook app with a Perl backend. The problem is that since Facebook sends the request to my web app as a POST request I'm having a problem getting the GET parameters that were also part of the base URL for the application -- in effect I'm only getting the POST params from $CGI->Vars.
See CGI/MIXING POST AND URL PARAMETERS.
Short version: use $CGI->param() for post paramenters and $CGI->url_param() for query string parameters.
Dump CGI in favour of a better interface. Plack's param method returns GET and POST parameters mixed.
plackup -MPlack::Request -e 'sub {
my ($env) = #_;
my $r = Plack::Request->new($env);
return [200, ["Content-Type" => "text/plain"], [join "\n", $r->param("foo")]];
}'
> lwp-request -m POST -USe 'http://localhost:5000/fnord?foo=bar;baz=quux'
Please enter content (application/x-www-form-urlencoded) to be POSTed:
foo=123;baz=456
␄
POST http://localhost:5000/fnord?foo=bar;baz=quux
User-Agent: lwp-request/6.03 libwww-perl/6.03
Content-Length: 16
Content-Type: application/x-www-form-urlencoded
200 OK
Date: Thu, 27 Oct 2011 21:27:46 GMT
Server: HTTP::Server::PSGI
Content-Length: 7
Content-Type: text/plain
Client-Date: Thu, 27 Oct 2011 21:27:46 GMT
Client-Peer: 127.0.0.1:5000
Client-Response-Num: 1
bar
123
Just set $CGI::APPEND_QUERY_STRING = 1;