Parse HTTP:Response object - perl

I am having some difficulties getting results from a form via Perl. I believe I have successfully found the form and submitted the value I want to the appropriate field, but am unsure of how to turn the response object into something useful (If I print it out it shows up as the following).
HTTP::Request=HASH(0x895b8ac)
Here is the relevant code (assume $url is correct)
my $ua = LWP::UserAgent->new;
my $responce = $ua->get($url);
my #form = HTML::Form->parse($responce);
my $chosen = $form[0];
$chosen->value('netid', $user);
my $ro = $chosen->click('Search');
What can I do to make $ro useful?
Thanks!

To quote the HTML::Form docs on click:
The result of clicking is an HTTP::Request object that can then be passed to LWP::UserAgent if you want to obtain the server response.
So you can do:
my $ua = LWP::UserAgent->new;
my $response = $ua->get($url);
my #form = HTML::Form->parse($response);
my $chosen = $form[0];
$chosen->value('netid', $user);
my $ro = $chosen->click('Search');
# If you want to see what you're sending to the server:
print $ro->as_string;
# Fetch the server's response:
$response = $ua->request($ro);
What you do with $response next depends on what you're trying to do.
P.S. "responce" is usually spelled without a C. But HTTP does have a history of misspellings. (I'm looking at you, "Referer".)

Related

how to send a http patch request with Lwp::Useragent?

I am working against the salesforce rest api with lwp::useragent.
I have to use the http patch request.
For get and post requests we get use the following code:
require LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->env_proxy;
my $get_response = $ua->get('http://search.cpan.org/',x=>'y');
my $post_response = $ua->post('http://search.cpan.org/',x=>'y');
Unfortunately this does not work
my $patch_response = $ua->patch('http://search.cpan.org/',x=>'y');
I don't find how to do it with this module.
There is a workaround to this problem like explained here How do I send a request using the PATCH method for a Salesforce update?
This works but this is not a nice solution.
I saw that with python it is possible to make explicitly patch requests How do I make a PATCH request in Python? so i assume that there is also an option with perl.
my $request = HTTP::Request->new(PATCH => $url);
... Add any necessary headers and body ...
my $response = $ua->request($request);
This has recently got a whole lot easier. PATCH is now implemented (like POST) in HTTP::Message.
First, update the HTTP::Message module (to 6.13 or later).
Then
my %fields = ( title => 'something', body => something else');
my $ua = LWP::UserAgent->new();
my $request = HTTP::Request::Common::PATCH( $url, [ %fields ] );
my $response = $ua->request($request);

Perl HTTP request : POST fails while GET succeeds

When I try to submit a POST request with Perl, it often ends in a 301 redirect to the homepage. Here is the code :
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
# This does not work
my $url = 'http://www.opensubtitles.org/en/search2';
my $req = HTTP::Request->new(POST => $url);
$req->content('MovieName=the+terminator+(1996)');
# Pass request to the user agent and get a response back
print $req->as_string."\n";;
my $res = $ua->request($req);
if (!$res->is_success) {
print $res->status_line, "\n";
}
else {
print "Success in posting search\n";
}
In order to make it work, I have to manually use Firefox, go to the url (!). Then the script works. However, using a GET request works flawlessly :
# This works
my $url = 'http://www.opensubtitles.org/en/search2?MovieName=the+terminator+(1996)';
my $req = HTTP::Request->new(GET => $url);
Why is that ?
The site doesn't expect a POST to that URL, so it redirects you to back to the search page.
Firefox will use GET, not POST, if you just put the URL into the address line, that's why it works.

Get redirected url in perl

I want to get last of redirect URL.
like
url_1 : http://on.fb.me/4VGeu
url_2 : https://www.facebook.com/
I want to get url_2 by url_1 in perl.
Previous source is below.
sub get_redirect_location
{
my ($url) = #_;
my $ua = LWP::UserAgent->new;
$ua->proxy('http', 'SAMPLE_PROXY');
my $req = new HTTP::Request(GET => $url);
my $res = $ua->request($req);
return $res->headers_as_string;
}
Thanks in advance.
You can find the request that lead to a response using
$response->request()
You can get the previous response in the chain using
$response->previous()
All together:
while ($response) {
say $response->request()->uri();
$response = $response->previous();
}
You could look at WWW::Mechanize. I have used it before to do something like this.
http://search.cpan.org/~jesse/WWW-Mechanize-1.72/lib/WWW/Mechanize.pm#$mech->redirect_ok()
You may also find this post helpful:
Perl WWW::Mechanize (or LWP) get redirect url

Posting to a WEB API from Perl results in a null value in [FromBody]

We have created a WEB API (in .NET framework 4.0) and gave the endpoint info to one of our clients. They created a program in Perl that posts to our endpoint.
Every post they have made so far arrives into our endpoint as null. When we initially started programming, we had that same issue in JQuery when posting by means of $.ajax. We solved it by adding a '=' at the beginning of the post data.
The Perl code they have submitted is the following:
sub _postPackages {
my ($self,$dataToSend) = #_;
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent("integrationapp/1.0 ");
# Create a request
my $req = HTTP::Request->new(POST => $self->{postAddress} );
$req->content_type("application/json;charset=utf-8");
$req->content($dataToSend->{data});
#print Data::Dumper->Dump([$req]);
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
where postAddress is our endpoint and $dataToSend is the message data. Is it possible that they need to add the '=' at the beginning of the $dataToSend message.
Any help will be greatly appreciated.
This is a bit of pseudo code here..
But I'm guessing you want to do something like this:
# some post sub
my ($self, $data) = #_;
my $ua = $self->get_user_agent();
my $json_xs = $self->get_json_xs();
my $json_encoded = $json_xs->utf8->encode($data);
$self->set_post_data($json_encoded);
$self->set_api_call();
my $response_body = $ua->post(
$self->get_api_call(),
'Content' => $self->get_post_data(),
'Content-type' => "application/json;charset=UTF-8"
);
print STDERR "POSTING NEW RESOURCE: " . Dumper($self);

Can I pass GET string in UserAgent post method

I call in this mode:
my $ua = new LWP::UserAgent;
my $response= $ua->post('www.example.com', {param1=>'val1',param2=>'val2'...} );
Can I call the above in the same way passing the values in GET form?:
my $response= $ua->post('www.example.com?param=val1&param2=val2' );
It is because I'm using Firebug and when I go to Net tab under the "POST" tab it shows individual parameters as well as a GET string for POST submitted requests.
So I was wondering if I use GET string in this function call.
Parametersapplication/x-www-form-urlencoded
Itemid 4 option com_search
searchword dsd task search Source
Content-Type:
application/x-www-form-urlencoded
Content-Length: 53
searchword=dsd&task=search&option=com_search&Itemid=4
In short you can pass GET strings yes, but if your end code does not accept GET METHOD it will fail.
Also you might still need to specify some parameters since the post method asks for post(url,array_with_parameters).
sub post {
require HTTP::Request::Common;
my($self, #parameters) = #_;
my #suff = $self->_process_colonic_headers(\#parameters, (ref($parameters[1]) ? 2 : 1));
return $self->request( HTTP::Request::Common::POST( #parameters ), #suff );
}
Using along with HTTP::Request you can specify it at the content in the way you prefer:
# Create a user agent object
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
$ua->agent("MyApp/0.1 ");
# Create a request
my $req = HTTP::Request->new(POST => 'http://www.example.com');
$req->content_type('application/x-www-form-urlencoded');
$req->content('searchword=dsd&task=search&option=com_search&Itemid=4');
# Pass request to the user agent and get a response back
my $res = $ua->request($req);
# Check the outcome of the response
if ($res->is_success) {
print $res->content;
} else {
print $res->status_line, "\n";
}
Read more...