Guys I need this curl request be translated to LWP::UserAgent HTTP Request
echo 'test{test="test"} 3' | curl -v --data-binary #- http://localhost:9090/api/metrics
What I've tried is this :
my $ua = LWP::UserAgent->new;
my $res = $ua->post('http://localhost:9090/api/metrics', ['test{test="test"}' => 3]);
die Dumper $res
But the response says
'_rc' => '400',
'_msg' => 'Bad Request',
'_content' => 'text format parsing error in line 1: unexpected end of input stream
You can try use the following POST request:
use feature qw(say);
use strict;
use warnings;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new();
my $res = $ua->post('http://localhost:9090/api/metrics', Content => 'test{test="test"} 3');
if ($res->is_success) {
say $res->decoded_content;
}
else {
die $res->status_line;
}
And, since you didn't ask, here's a Mojo example:
use v5.10;
use Mojo::UserAgent;
my $ua = Mojo::UserAgent->new();
my $tx = $ua->post(
'http://httpbin.org/post',
=> 'test{test="test"} 3'
);
if ($tx->result->is_success) {
say $tx->result->body;
}
else {
die $tx->result->code;
}
It's basically the same as LWP except that Mojo returns a transaction object so you can play with the request too. It's something I wanted in LWP even before Mojo existed.
Related
I'm trying to debug a weird warning that is showing up in server logs when a Plack::Request is being parsed. In some cases, a broken UserAgent will send a Content-Length header that looks something like "6375, 6375", which is obviously wrong.
To fix this properly, I need to be able to reproduce the warning. I'd like to include this in a unit test so that I can ensure there are no regressions after the warning is silenced. However, I'm having trouble doing this with Perl. I know this can be done using netcat and socat, but I don't want the unit test to have to rely on other binaries to be installed.
Here is what I've tried:
#!/usr/bin/env perl
use strict;
use warnings;
use JSON::XS qw( encode_json );
use WWW::Mechanize;
my $mech = WWW::Mechanize->new;
$mech->add_handler(
request_prepare => sub {
my ( $req, $ua, $h ) = #_;
$req->headers->header( 'Content-Length' => 9999 );
return;
}
);
my $json = encode_json( { foo => 'bar' } );
$mech->post(
'http://example.com'/url,
'Content-Length' => 999,
Content => $json
);
Output is:
Content-Length header value was wrong, fixed at /opt/perl5.16.3/lib/site_perl/5.16.3/LWP/Protocol/http.pm line 260.
200
That's entirely too helpful for me. :)
If I use HTTP::Request and LWP::UserAgent, it's the same end result.
So, I tried HTTP::Tiny.
#!/usr/bin/env perl
use strict;
use warnings;
use DDP;
use HTTP::Tiny;
use JSON::XS qw( encode_json );
my $http = HTTP::Tiny->new;
my $json = encode_json( { foo => 'bar' } );
my $response = $http->request(
'POST',
'http://example.com'/url',
{ headers => { 'Content-Length' => 999, },
content => $json,
}
);
p $response;
The output is:
{ content => "Content-Length missmatch (got: 13 expected: 999)
",
headers => {
content
-length => 49,
content-type => "text/plain",
},
reason => "Internal Exception",
status => 599,
success => "",
url => "http://example.com'/url",
}
Again, too helpful. At this point, I could use a few suggestions.
Seems like the higher level API's are fixing your error; Here's an example using raw sockets that overcomes this;
#!/usr/bin/env perl
use strict 'vars';
use warnings;
use Socket;
# initialize host and port
my $host = 'www.example.com';
my $port = 80;
# contact the server
open_tcp(F, $host, $port)
or die 'Could not connect to server';
# Send request data
while ( my $request = <DATA> ) {
print F $request;
}
# Get Response
while ( my $response = <F> ) {
print "Response:> $response";
}
close(F);
# TCP Helper
sub open_tcp
{
# get parameters
my ($FS, $dest, $port) = #_;
my $proto = getprotobyname('tcp');
socket($FS, PF_INET, SOCK_STREAM, $proto);
my $sin = sockaddr_in($port,inet_aton($dest));
connect($FS,$sin);
my $old_fh = select($FS);
$| = 1; # don't buffer output
select($old_fh);
}
__DATA__
GET / HTTP/1.1
Host: example.com
Content-Length: 999
-END-
I want to print the redirected url in perl.
Input url : http://pricecheckindia.com/go/store/snapdeal/52517?ref=velusliv
output url : http://www.snapdeal.com/product/vox-2-in-1-camcorder/1154987704?utm_source=aff_prog&utm_campaign=afts&offer_id=17&aff_id=1298&source=pricecheckindia
use LWP::UserAgent qw();
use CGI qw(:all);
print header();
my ($url) = "http://pricecheckindia.com/go/store/snapdeal/52517?ref=velusliv";
my $ua = LWP::UserAgent->new;
my $req = new HTTP::Request(GET => $url);
my $res = $ua->request($req);
print $res->request;
How to get this done in perl?
You need to examine the HTTP response to find the URL. The documentation of HTTP::Response gives full details of how to do this, but to summarise, you should do the following:
use strict;
use warnings;
use feature ':5.10'; # enables "say"
use LWP::UserAgent;
my $url = "http://pricecheckindia.com/go/store/snapdeal/52517?ref=velusliv";
my $ua = LWP::UserAgent->new;
my $req = new HTTP::Request(GET => $url);
my $res = $ua->request($req);
# you should add a check to ensure the response was actually successful:
if (! $res->is_success) {
say "GET failed! " . $res->status_line;
}
# show the base URI for the response:
say "Base URI: " . $res->base;
You can view redirects using HTTP::Response's redirects method:
if ($res->redirects) { # are there any redirects?
my #redirects = $res->redirects;
say join(", ", #redirects);
}
else {
say "No redirects.";
}
In this case, the base URI is the same as $url, and if you examine the contents of the page, you can see why.
# print out the contents of the response:
say $res->decoded_contents;
Right near the bottom of the page, there is the following code:
$(window).load(function() {
window.setTimeout(function() {
window.location = "http://www.snapdeal.com/product/vox-2-in-1-camcorder/1154987704?utm_source=aff_prog&utm_campaign=afts&offer_id=17&aff_id=1298&source=pricecheckindia"
}, 300);
});
The redirect is handled by javascript, and so is not picked up by LWP::UserAgent. If you want to get this URL, you will need to extract it from the response contents (or use a different client that supports javascript).
On a different note, your script starts off like this:
use LWP::UserAgent qw();
The code following the module name, qw(), is used to import particular subroutines into your script so that you can use them by name (instead of having to refer to the module name and the subroutine name). If the qw() is empty, it's not doing anything, so you can just omit it.
To have LWP::UserAgent follow redirects, just set the max_redirects option:
use strict;
use warnings;
use LWP::UserAgent qw();
my $url = "http://pricecheckindia.com/go/store/snapdeal/52517?ref=velusliv";
my $ua = LWP::UserAgent->new( max_redirect => 5 );
my $res = $ua->get($url);
if ( $res->is_success ) {
print $res->decoded_content; # or whatever
} else {
die $res->status_line;
}
However, that website is using a JavaScript redirect.
$(window).load(function() {
window.setTimeout(function() {
window.location = "http://www.snapdeal.com/product/vox-2-in-1-camcorder/1154987704?utm_source=aff_prog&utm_campaign=afts&offer_id=17&aff_id=1298&source=pricecheckindia"
}, 300);
});
This will not work unless you use a framework that enables JavaScript, like WWW::Mechanize::Firefox.
It will throw you an error for the last line $res - > request since it is returning hash and content from the response. So below is the code:
use LWP::UserAgent qw();
use CGI qw(:all);
print header();
my ($url) = "http://pricecheckindia.com/go/store/snapdeal/52517?ref=velusliv";
my $ua = LWP::UserAgent->new;
my $req = new HTTP::Request(GET => $url);
my $res = $ua->request($req);
print $res->content;
In a certain script I tried to write this:
my $ua = LWP::UserAgent->new;
my $res = $ua->post($url, Content => $data);
and got "400 Bad Request".
After some reading I tried this:
my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new( 'POST', $url );
$req->content( $data );
my $res = $ua->request( $req );
and it worked, but I thought these two should do the same. What am I missing here?
Am I misunderstanding something in the documentation of HTTP::Request and LWP::UserAgent?
Is there a way to ask LWP::UserAgent to print what it is doing?
Here's one way to do it:
#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
{
no strict "refs";
no warnings "redefine";
my $orig_sub = \&LWP::UserAgent::send_request;
*{"LWP::UserAgent::send_request"} = sub {
my ($self, $request) = #_;
print $request->as_string . "\n";
my $response = $orig_sub->(#_);
print $response->as_string . "\n";
return $response;
};
}
my $a = LWP::UserAgent->new;
my $response = $a->get("http://google.com");
It will print out all the requests and responses that LWP::UserAgent does.
I am attempting to request a token from https://launchpad.net, according to the docs all it wants is a POST to /+request-token with the form encoded values of oauth_consumer_key, oauth_signature, and oauth_signature_method. Providing those items via curl works as expected:
curl --data "oauth_consumer_key=test-app&oauth_signature=%26&oauth_signature_method=PLAINTEXT" https://launchpad.net/+request-token
However, when i attempt to do it through my perl script it is giving me a 401 unauthorized error.
#!/usr/bin/env perl
use strict;
use YAML qw(DumpFile);
use Log::Log4perl qw(:easy);
use LWP::UserAgent;
use Net::OAuth;
$Net::OAuth::PROTOCOL_VERSION = Net::OAuth::PROTOCOL_VERSION_1_0A;
use HTTP::Request::Common;
use Data::Dumper;
use Browser::Open qw(open_browser);
my $ua = LWP::UserAgent->new;
my ($home) = glob '~';
my $cfg = "$home/.lp-auth.yml";
my $access_token_url = q[https://launchpad.net/+access-token];
my $authorize_path = q[https://launchpad.net/+authorize-token];
sub consumer_key { 'lp-ua-browser' }
sub request_url {"https://launchpad.net/+request-token"}
my $request = Net::OAuth->request('consumer')->new(
consumer_key => consumer_key(),
consumer_secret => '',
request_url => request_url(),
request_method => 'POST',
signature_method => 'PLAINTEXT',
timestamp => time,
nonce => nonce(),
);
$request->sign;
print $request->to_url;
my $res = $ua->request(POST $request->to_url, Content $request->to_post_body);
my $token;
my $token_secret;
print Dumper($res);
if ($res->is_success) {
my $response =
Net::OAuth->response('request token')->from_post_body($res->content);
$token = $response->token;
$token_secret = $response->token_secret;
print "request token ", $token, "\n";
print "request token secret", $token_secret, "\n";
open_browser($authorize_path . "?oauth_token=" . $token);
}
else {
die "something broke ($!)";
}
I tried both with $request->sign and without it as i dont think that is required during the request token phase. Anyway any help with this would be appreciated.
Update, switched to LWP::UserAgent and had to pass in both POST and Content :
my $res = $ua->request(POST $request->to_url, Content $request->to_post_body);
Thanks
Sorry I'm not able to verify from my tablet but with recent Perl you should install and use
use LWP::Protocol::https;
http://blogs.perl.org/users/brian_d_foy/2011/07/now-you-need-lwpprotocolhttps.html
I'm trying to get Code Closure to work, but unfortunately, there's always an error thrown.
Here's the code:
use LWP::UserAgent;
use HTTP::Request::Common;
use HTTP::Response;
my $name = 'test.js';
my $agent = new LWP::UserAgent();
$agent->agent("curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18");
$res = $agent->request(POST 'http://closure-compiler.appspot.com/compile',
content_type => 'multipart/form-data',
content => [
output_info => 'compiled_code',
compilation_level => 'SIMPLE_OPTIMIZATIONS',
output_format => 'text',
js_code => [File::Spec->rel2abs($name)]
]);
if ($res->is_success) {
$minified = $res->decoded_content;
print $minified;die;
}
I get the following error:
Error(13): No output information to produce, yet compilation was requested.
Here's the api reference I used:
http://code.google.com/intl/de-DE/closure/compiler/docs/api-ref.html
Hope anyone knows what's going wrong here. Thanks.
#!/usr/bin/perl
use strict; use warnings;
use File::Slurp;
use LWP::UserAgent;
my $agent = LWP::UserAgent->new;
my $script = 'test.js';
my $response = $agent->post(
'http://closure-compiler.appspot.com/compile',
content_type => 'application/x-www-form-urlencoded',
content => [
compilation_level => 'SIMPLE_OPTIMIZATIONS',
output_info => 'compiled_code',
output_format => 'text',
js_code => scalar read_file($script),
],
);
if ($response->is_success) {
my $minified = $response->decoded_content;
print $minified;
}
Output:
C:\Temp> cat test.js
// ADD YOUR CODE HERE
function hello(name) {
alert('Hello, ' + name);
}
hello('New user');
C:\Temp> t
function hello(a){alert("Hello, "+a)}hello("New user");
Pass as js_code the actual code to compile. Try (removing the form-data content_type header):
use File::Slurp "read_file";
...
js_code => scalar( read_file($name) ),
I see you are trying to use POST's file upload feature; what in the API documentation do you see that makes you think that would work? If there is something there, I don't see it.