I call in this mode:
my $ua = new LWP::UserAgent;
my $response= $ua->post('www.example.com', {param1=>'val1',param2=>'val2'...} );
Can I call the above in the same way passing the values in GET form?:
my $response= $ua->post('www.example.com?param=val1¶m2=val2' );
It is because I'm using Firebug and when I go to Net tab under the "POST" tab it shows individual parameters as well as a GET string for POST submitted requests.
So I was wondering if I use GET string in this function call.
Parametersapplication/x-www-form-urlencoded
Itemid 4 option com_search
searchword dsd task search Source
Content-Type:
application/x-www-form-urlencoded
Content-Length: 53
searchword=dsd&task=search&option=com_search&Itemid=4
In short you can pass GET strings yes, but if your end code does not accept GET METHOD it will fail.
Also you might still need to specify some parameters since the post method asks for post(url,array_with_parameters).
sub post {
require HTTP::Request::Common;
my($self, #parameters) = #_;
my #suff = $self->_process_colonic_headers(\#parameters, (ref($parameters[1]) ? 2 : 1));
return $self->request( HTTP::Request::Common::POST( #parameters ), #suff );
}
Using along with HTTP::Request you can specify it at the content in the way you prefer:
# Create a user agent object
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent("MyApp/0.1 ");
# Create a request
my $req = HTTP::Request->new(POST => 'http://www.example.com');
$req->content_type('application/x-www-form-urlencoded');
$req->content('searchword=dsd&task=search&option=com_search&Itemid=4');
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
# Check the outcome of the response
if ($res->is_success) {
print $res->content;
} else {
print $res->status_line, "\n";
}
Read more...
Related
I have a simple code like this:
use LWP::UserAgent;
use HTTP::Cookies;
use HTTP::Request;
my $cookies = HTTP::Cookies->new();
my $browser = LWP::UserAgent->new();
$browser->agent(' ... ');
$browser->cookie_jar($cookies);
my $request = HTTP::Request->new();
my $response;
my $url;
my $referer;
$referer = '';
$url = 'https:// ...'; # url #1
$request->url($url);
$request->method('GET');
$request->header('Referer' => $referer);
$response = $browser->request($request);
print $response->request()->uri() . "\n\n" .
$response->headers()->as_string . "\n\n" .
$response->content . "\n\n";
$referer = $response->request()->uri();
$url = 'https:// ... '; # url #2
$request->url($url);
$request->method('GET');
$request->header('Referer' => $referer);
$response = $browser->request($request);
print $response->request()->uri() . "\n\n" .
$response->headers()->as_string . "\n\n" .
$response->content . "\n\n";
Now, I want to see full HTTP request headers as well, not just response headers.
How can I do it? What has to be added to this code?
I think you almost have it in your existing code. You are accessing the request URI with $response->request()->uri(). The ->request() is your HTTP::Request object. I believe that you can use $response->request->headers->as_string to get what you want.
print $response->request->as_string
This will show you requests as well as responses.
use LWP::UserAgent;
use LWP::ConsoleLogger::Easy qw( debug_ua );
my $browser = LWP::UserAgent->new();
debug_ua( $browser );
$request->headers->as_string and $response->request->headers->as_string will you get you the headers of the first and last request passed to Net::HTTP by LWP[1], but these aren't quite what Net::HTTP sends. For example, Net::HTTP can add a Content-Length header, a TE header, and/or a number of others.
Net::HTTP doesn't keep a record of the headers it actually sends. You will need a wire sniffer (e.g. tcpdump) or a debugging proxy (e.g. Fiddler) for that. You could also use a debugger or trace statements to view the request prepared in Net::HTTP::Methods's format_request. The most convenient, however, might be to wrap Net::HTTP::Methods's format_request.
These are the same unless the initial request was redirected. To get all the requests (and responses), you can use:
while ($response) {
my $request = $response->request;
...
$response = $response->previous;
}
When I try to submit a POST request with Perl, it often ends in a 301 redirect to the homepage. Here is the code :
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
# This does not work
my $url = 'http://www.opensubtitles.org/en/search2';
my $req = HTTP::Request->new(POST => $url);
$req->content('MovieName=the+terminator+(1996)');
# Pass request to the user agent and get a response back
print $req->as_string."\n";;
my $res = $ua->request($req);
if (!$res->is_success) {
print $res->status_line, "\n";
}
else {
print "Success in posting search\n";
}
In order to make it work, I have to manually use Firefox, go to the url (!). Then the script works. However, using a GET request works flawlessly :
# This works
my $url = 'http://www.opensubtitles.org/en/search2?MovieName=the+terminator+(1996)';
my $req = HTTP::Request->new(GET => $url);
Why is that ?
The site doesn't expect a POST to that URL, so it redirects you to back to the search page.
Firefox will use GET, not POST, if you just put the URL into the address line, that's why it works.
I have a CGI server side script that accepts GET and POST, with login parameters.
I want to test it to make sure it is not vulnerable. So the plan is to use Perl LWP, and send login parameters in GET and POST, and compare the results. the interface has been changed, so that only in POST we can send user-name and password in session cookies ( not sure if that is a great idea ) , so how do i test it ? Here is what i have so far:
#!/usr/bin/perl
use LWP;
print "This is libwww-perl-$LWP::VERSION\n";
# Create a user agent object
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent("MyApp/0.1 ");
# Create a request
#my $req = HTTP::Request->new(POST => 'http://search.cpan.org/search');
#my $req = HTTP::Request->new(GET => 'https://qa.co.net:443/cgi-bin/n-cu.cgi');
my $req = HTTP::Request->new(GET => 'https://qa.co.net:443/cgi-bin/n-cu.cgi?mode=frameset&JScript=1&remote_user&login=foo&password=foo HTTP/1.1');
$req->content_type('application/x-www-form-urlencoded');
$req->content('query=libwww-perl&mode=dist');
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
# Check the outcome of the response
if ($res->is_success) {
print $res->content;
#print $res->code;
#print $res->message;
}
else {
print $res->status_line, "\n";
}
This is not going to do it, since it does not have the session cookie stuff. But might be a good start though. Is this the right way to test the GET and POST ?
Here is what was implemented in the cgi:
#cr_login for POST && login for GET -- leave GET param as it used to be.
if ($m eq 'GET' && defined($req->param('login'))) {
$msg = 'parameter "login" is invalid for this request type.';
+ my $seclog = $event_logging_directory . '/invalid_request.log';
+ open(S, ">>$seclog") or die $!;
+ my $logmsg = sprintf("%4d-%02d-%02d %02d:%02d:%02d",Today_and_Now())
+ . "|mode:" . $req->param('mode')
+ . "|login:" . $req->param('login')
+ . "|remote_addr:" . $ENV{REMOTE_ADDR}
+ . "|$msg\n";
+ print S $logmsg;
and :
POST request to n-cu.cgi should use parameter "cr_login". If the parameter "login" is passed in a post request, it should throw error and return to login screen.
GET request to n-cu.cgi should use the parameter "login". If the parameter "cr_login" is passed in a post request, it should throw error and return to login screen.
so here is how we do it:
Keep the session cookie and context alive :
my $browser = LWP::UserAgent->new(keep_alive => 10);
$browser->cookie_jar( {} );
$browser->agent('Mozilla/8.0');
#$browser->ssl_opts({ verify_hostname => 0 });
$browser->show_progress(1);
and later: print the response
print "Cookies:\n", Dumper($browser->cookie_jar()), "\n\n";
my $content = $response->as_string;
print "$content\n";
Sending password in a cookie? Nope.
Disallow GET for /login.
POST username and password to /login, over SSL.
In CGI, the GET/POST is indicated via the REQUEST_METHOD environment variable.
You cannot stop determined people from issuing a GET request to your server, but you can refuse to process it like so (untested code - you have to fill in details):
if ($ENV{REQUEST_METHOD} ne 'POST') {
# issue a redirect to a suitable error page, then return.
}
my $q = CGI->new();
my $user = $q->params('username');
my $password = $q->params('password');
my $encrypted_password = my_password_encryptor($password);
unless ( can_log_in($user, $encrypted_password) ) {
# issue an error message - redirect&return or fall-through...
}
else {
$session->set_user_logged_in();
}
Most people do not roll their own authentication or session handling. They mostly use one from CPAN, or one included with the larger app framework. If you're doing CGI, you can use CGI::Session.
You might give CGI::Application and/or its offspring a look. Those authors have already solved a bunch of the problems that you're encountering.
We have created a WEB API (in .NET framework 4.0) and gave the endpoint info to one of our clients. They created a program in Perl that posts to our endpoint.
Every post they have made so far arrives into our endpoint as null. When we initially started programming, we had that same issue in JQuery when posting by means of $.ajax. We solved it by adding a '=' at the beginning of the post data.
The Perl code they have submitted is the following:
sub _postPackages {
my ($self,$dataToSend) = #_;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent("integrationapp/1.0 ");
# Create a request
my $req = HTTP::Request->new(POST => $self->{postAddress} );
$req->content_type("application/json;charset=utf-8");
$req->content($dataToSend->{data});
#print Data::Dumper->Dump([$req]);
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
where postAddress is our endpoint and $dataToSend is the message data. Is it possible that they need to add the '=' at the beginning of the $dataToSend message.
Any help will be greatly appreciated.
This is a bit of pseudo code here..
But I'm guessing you want to do something like this:
# some post sub
my ($self, $data) = #_;
my $ua = $self->get_user_agent();
my $json_xs = $self->get_json_xs();
my $json_encoded = $json_xs->utf8->encode($data);
$self->set_post_data($json_encoded);
$self->set_api_call();
my $response_body = $ua->post(
$self->get_api_call(),
'Content' => $self->get_post_data(),
'Content-type' => "application/json;charset=UTF-8"
);
print STDERR "POSTING NEW RESOURCE: " . Dumper($self);
I am having some difficulties getting results from a form via Perl. I believe I have successfully found the form and submitted the value I want to the appropriate field, but am unsure of how to turn the response object into something useful (If I print it out it shows up as the following).
HTTP::Request=HASH(0x895b8ac)
Here is the relevant code (assume $url is correct)
my $ua = LWP::UserAgent->new;
my $responce = $ua->get($url);
my #form = HTML::Form->parse($responce);
my $chosen = $form[0];
$chosen->value('netid', $user);
my $ro = $chosen->click('Search');
What can I do to make $ro useful?
Thanks!
To quote the HTML::Form docs on click:
The result of clicking is an HTTP::Request object that can then be passed to LWP::UserAgent if you want to obtain the server response.
So you can do:
my $ua = LWP::UserAgent->new;
my $response = $ua->get($url);
my #form = HTML::Form->parse($response);
my $chosen = $form[0];
$chosen->value('netid', $user);
my $ro = $chosen->click('Search');
# If you want to see what you're sending to the server:
print $ro->as_string;
# Fetch the server's response:
$response = $ua->request($ro);
What you do with $response next depends on what you're trying to do.
P.S. "responce" is usually spelled without a C. But HTTP does have a history of misspellings. (I'm looking at you, "Referer".)