POSTDATA is not the correct answer. I have read the docs and still don't see how I can get the data.
I want to receive this request:
POST /cgi-bin/myscript.cgi HTTP/1.1
Host: myhost.com
Content-Length: 3
Content-Type: application/x-www-form-urlencoded
255
and have the server respond
You sent the string "255"
Please assist, I am a Perl beginner and have gotten a bunch of seemingly wrong and useless answers to this seemingly simple request.
CGI will automatically parse form data, so you need to hide that what you got is form data (or at least claims to be).
use CGI qw( );
$ENV{CONTENT_TYPE} = 'application/octet-stream';
my $cgi = CGI->new();
my $post_data = $cgi->param('POSTDATA');
Better solution: Have the requester use a correct content type (e.g. application/octet-stream), or have the requester actually send form data (e.g. data=255).
Unique solution for me, was change of ContentType on client's petition to 'application/octet-stream'
Module CGI CPAN says:
If POSTed data is not of type application/x-www-form-urlencoded or
multipart/form-data, then the POSTed data will not be processed, but
instead be returned as-is in a parameter named POSTDATA.
So if you can't change on clients petition to other ContentType, it won't be processed.
CGI (in recent versions at least) will stuff incorrectly encoded x-www-form-urlencoded params into a parameter named keywords. Better to send a proper content type though, then the POSTDATA works exactly as the docs say:
If POSTed data is not of type application/x-www-form-urlencoded or
multipart/form-data, then the POSTed data will not be processed...
use strictures;
use CGI::Emulate::PSGI;
use Plack::Test;
use HTTP::Request::Common;
use Test::More;
my $post = POST "/non-e-importa",
"Content-Length" => 5,
"Content-Type" => "application/x-www-form-urlencoded",
Content => "ohai\n";
my $cgis = CGI::Emulate::PSGI->handler( sub {
use CGI "param", "header";
my $incorrectly_encoded_body = param("keywords");
print header("text/plain"), $incorrectly_encoded_body;
});
test_psgi $cgis, sub {
my $cb = shift;
my $res = $cb->($post);
is $res->content, "ohai", "Soopersek437 param: keywords";
};
done_testing();
__END__
prove so-16846138 -v
ok 1 - Soopersek437 param: keywords
1..1
ok
All tests successful.
Result: PASS
Related
I want to send xml request using HTTP streaming protocol . where transfer-encoding is "chunked". Currently i am using LWP::UserAgent to send the xml transaction.
my $userAgent = LWP::UserAgent->new;
my $starttime = time();
my $response = $userAgent->request(POST $url,
Content_Type => 'application/xml',
Transfer_Encoding => 'Chunked',
Content => $xml);
print "Response".Dumper($response);
But i am getting http status code 411 Length Required. Which means "client error response code indicates that the server refuses to accept the request without a defined "
How we can handle this while sending a request in chunked ?
LWP::UserAgent's API isn't designed to send a stream, but it is able to do so with minimal hacking.
use strict;
use warnings qw( all );
use HTTP::Request::Common qw( POST );
use LWP::UserAgent qw( );
my $ua = LWP::UserAgent->new();
# Don't provide any content.
my $request = POST('http://stackoverflow.org/junk',
Content_Type => 'application/xml',
);
# POST() insists on adding a Content-Length header.
# We need to remove it to get a chunked request.
$request->headers->remove_header('Content-Length');
# Here's where we provide the stream generator.
my $buffer = 'abc\n';
$request->content(sub {
return undef if !length($buffer); # Return undef when done.
return substr($buffer, 0, length($buffer), ''); # Return a chunk of data otherwise.
});
my $response = $ua->request($request);
print($response->status_line);
Using a proxy (Fiddler), we can see this does indeed send a chunked request:
There's no point in using a chunked request if you already have the entire document handy like in the example you gave. Instead, let's say wanted to upload the output of some external tool as it produced its output. To do that, you could use the following:
open(my $pipe, '-|:raw', 'some_tool');
$request->content(sub {
my $rv = sysread($pipe, my $buf, 64*1024);
die $! if !defined($rv);
return undef if !$rv;
return $buf;
});
But i am getting http status code 411 Length Required.
Not all servers understand a request with a chunked payload even though this is standardized in HTTP/1.1 (but not in HTTP/1.0). For example nginx only supports chunking within a request since version 1.3.9 (2012), see Is there a way to avoid nginx 411 Content-Length required errors?. If the server does not understand a request with chunked encoding there is nothing you can do from the client side, i.e. you simply cannot use chunked transfer encoding then. If you have control over the server make sure that the server actually supports it.
I've also never experienced browsers send such requests, probably since they cannot guarantee that the server will support such request. I've only seen mobile apps used where the server and app is managed by the same party and thus support for chunked requests can be guaranteed.
I've checked the documentation https://metacpan.org/pod/Furl
but can't found how can I get sites base URI while using Furl?
With LWP it's easy:
my $res = $ua->get($url);
my $base_uri = $res->base;
The base function try to get values from this header fields
my $base = (
$self->header('Content-Base'), # used to be HTTP/1.1
$self->header('Content-Location'), # HTTP/1.1
$self->header('Base'), # HTTP/1.0
)[0];
But I couldn't do the same with Furl.
First: it seems you want to do an anonymous array at $base, thus, it should be:
my $base = [
$res->header('header1'),
$res->header('header2'),
$res->header('header3')
];
Because the code you had just saved the first header (in your case, Content-Base) and did nothing with the last two, you can check that with Data::Dumper.
Maybe that's why it didn't work.
Second: But, after reading through the code of the Furl module, I found out there's no exposed method for getting an url's base, so unless you are also checking in your own code for the <base> html tag and the uri you used to request your response (even after redirects), your code might break with some oldish sites. HTTP::Response does this checking, and that's what LWP uses.
Citation for hierarchy of base URIs: HTTP::Response - HTTP style response message
I need a simple CGI based Perl script to receive a POST (directly, not from another HTML page) with Content-Type being application/x-www-form-urlencoded and to echo back
I received: (encoded string)
(and if possible)
decoded, the string is: (decoded string)
I am new to CGI Perl, and this is a one off request for testing a product (I'm a sysadmin. not a programmer). I intend to learn Perl more deeply in the future, but in this case I'm hoping for a gimme.
To start off, I will quickly skim some of the basics.
Following is the package for PERL/CGI application:
use CGI;
To create CGI object:
my $web = CGI->new;
Make sure you set and then write HTTP headers to outstream, before flushing out any CGI data to outstream. Otherwise you would end up in 500 error.
To set the headers:
print $web->header();
print $web->header('application/x-www-form-urlencoded');
To receive any post data from HTML, say for example,
http://example.com?POSTDATA=helloworld
you may use param() function:
my $data = $web->param('POSTDATA');
scalar $data would be set with "helloworld".
It is advisable to check if $web->param('POSTDATA') is defined before you assign to a scalar.
I'm querying a webserver for a document and I want to capture the both the document and the related server response headers (esp. Content-Type: ...). I have trouble finding out how to read the headers. Here are some sniplets from my Perl script, I left error checking out for clarity:
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent( 'requiredCustomUserAgent' ); # I'm required to set a custom user agent
$imageData = $response->content; # This is the received document
So at this point I can retrieve the web document, but I want to know what Content-Type the server sent with it. Unfortunately this is not always the same as the mime type found by the bash 'file' command. This file-method fails in case of .js or .css documents.
http://search.cpan.org/perldoc?HTTP::Response
use LWP::UserAgent;
my $ua = new LWP::UserAgent;
my $response = $ua->get("http://google.co.uk");
print $response->headers()->as_string;
print $response->header('content-type');
the thing that request returns contains a HTTP::Headers object, so look at the docs for HTTP::Headers to see how to use it. For instance
my $response = $ua->request($req);
my $headers = $response->headers();
my #header_field_names = $headers->header_field_names();
$logger->info("$_: ".$headers->header($_)) for grep {/Hogwarts/} #header_field_names;
I'm trying to write a Perl CGI script to handle XML-RPC requests, in which an XML document is sent as the body of an HTTP POST request.
The CGI.pm module does a great job at extracting named params from an HTTP request, but I can't figure out how to make it give me the entire HTTP request body (i.e. the XML document in the XML-RPC request I'm handling).
If not CGI.pm, is there another module that would be able to parse this information out of the request? I'd prefer not to have to extract this information "by hand" from the environment variables. Thanks for any help.
You can get the raw POST data by using the special parameter name POSTDATA.
my $q = CGI->new;
my $xml = $q->param( 'POSTDATA' );
Alternatively, you could read STDIN directly instead of using CGI.pm, but then you lose all the other useful stuff that CGI.pm does.
The POSTDATA trick is documented in the excellent CGI.pm docs here.
Right, one could use POSTDATA, but that only works if the request Content-Type has not been set to 'multipart/form-data'.
If it is set to 'multipart/form-data', CGI.pm does its own content processing and POSTDATA is not initialized.
So, other options include $cgi->query_string and/or $cgi->Dump.
The $cgi->query_string returns the contents of the POST in a GET format (param=value&...), and there doesn't seem to be a way to simply get the contents of the POST STDIN as they were passed in by the client.
So to get the actual content of the standard input of a POST request, if modifying CGI.pm is an option for you, you could modify around line 620 to save the content of #lines somewhere in a variable, such as:
$self->{standard_input} = join '', #lines;
And then access it through $cgi->{standard_input}.
To handle all cases, including those when Content-Type is multipart/form-data, read (and put back) the raw data, before CGI does.
use strict;
use warnings;
use IO::Handle;
use IO::Scalar;
STDIN->blocking(1); # ensure to read everything
my $cgi_raw = '';
{
local $/;
$cgi_raw = <STDIN>;
my $s;
tie *STDIN, 'IO::Scalar', \$s;
print STDIN $cgi_raw;
tied(*STDIN)->setpos(0);
}
use CGI qw /:standard/;
...