How do I access a value of a nested Perl hash? - perl

I am new to Perl and I have a problem that's very simple but I cannot find the answer when consulting my Perl book.
When printing the result of
Dumper($request);
I get the following result:
$VAR1 = bless( {
'_protocol' => 'HTTP/1.1',
'_content' => '',
'_uri' => bless( do{\(my $o = 'http://myawesomeserver.org:8081/counter/')}, 'URI::http' ),
'_headers' => bless( {
'user-agent' => 'Mozilla/5.0 (X11; U; Linux i686; en; rv:1.9.0.4) Gecko/20080528 Epiphany/2.22 Firefox/3.0',
'connection' => 'keep-alive',
'cache-control' => 'max-age=0',
'keep-alive' => '300',
'accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'accept-language' => 'en-us,en;q=0.5',
'accept-encoding' => 'gzip,deflate',
'host' => 'localhost:8081',
'accept-charset' => 'ISO-8859-1,utf-8;q=0.7,*;q=0.7'
}, 'HTTP::Headers' ),
'_method' => 'GET',
'_handle' => bless( \*Symbol::GEN0, 'FileHandle' )
}, 'HTTP::Server::Simple::Dispatched::Request' );
How can I access the values of '_method' ('GET') or of 'host' ('localhost:8081').
I know that's an easy question, but Perl is somewhat cryptic at the beginning.

Narthring has it right as far as the brute force method. Nested hashes are addressed by chaining the keys like so:
$hash{top_key}{next_key}{another_key}; # for %hash
# OR
$hash_ref->{top_key}{next_key}{another_key}; # for refs.
However since both of these "hashes" are blessed objects. It might help reading up on HTTP::Server::Simple::Dispatched::Request, which can tell you that it's a HTTP::Request object and looking at HTTP::Request section on the header and method methods, tells you that the following do the trick:
my $method = $request->method();
my $host = $request->header( 'host' );
Really, I recommend you get the firefox search plugin called Perldoc Module::Name and when you encounter Dumper output that says "bless ... 'Some::Module::Name'" you can just copy and paste it into the search plugin and read the documentation on CPAN.

Related

Parsing this kind of data

I have written a wrapper for an API. Previously I'd worked on simple string-based GET requests to PHP scripts using Perl.
As part of analysing the response, I have to analyse the following data which appears to be an object. Unfortunately, I'm not sure how to extract usable data from this.
print Dumper on the data returns this:
$VAR1 = bless( {
'_rc' => '200',
'_request' => bless( {
'_uri_canonical' => bless( do{\(my $o = 'http://example.com/?list=1&token=h_DQ-3lru6uy_Zy0w-KXGbPm_b9llY3LAAAAALSF1roAAAAANxAtg49JqlUAAAAA')}, 'URI::http' ),
'_content' => '',
'_uri' => $VAR1->{'_request'}{'_uri_canonical'},
'_method' => 'GET',
'_headers' => bless( {
'accept-charset' => 'iso-8859-1,*,utf-8',
'accept' => 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*',
'cookie' => 'GUID=cHoW3DLOljP4K9LzposM',
'user-agent' => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20041107 Firefox/1.0',
'authorization' => 'Basic YWRtaW46bmljb2xl',
'cookie2' => '$Version="1"',
'::std_case' => {
'cookie' => 'Cookie',
'cookie2' => 'Cookie2'
},
'accept-language' => 'en-US'
}, 'HTTP::Headers' )
}, 'HTTP::Request' ),
'_headers' => bless( {
'client-peer' => 'myip:8085',
'content-type' => 'text/plain',
'cache-control' => 'no-cache',
'connection' => 'keep-alive',
'client-date' => 'Sat, 18 Jul 2015 12:41:00 GMT',
'::std_case' => {
'client-response-num' => 'Client-Response-Num',
'set-cookie2' => 'Set-Cookie2',
'client-date' => 'Client-Date',
'client-peer' => 'Client-Peer',
'set-cookie' => 'Set-Cookie'
},
'client-response-num' => 1,
'content-length' => '8684'
}, 'HTTP::Headers' ),
'_msg' => 'OK',
'_protocol' => 'HTTP/1.1',
'_content' => '{"build":30470,"torrents": [
["043CC5FA0C741CDAD9D2E5CC20DF64A4A400FA34",136,"Epi.S01E03.720p.HDTV.x264-IMMERSE[rarbg]",690765843,39,26951680,671744,24,0,0,0,"",0,1454,0,114,2436,1,663814163,"","","Stopped","512840d7",1437022635,0,"","/mydir/Epi.S01E03.720p.HDTV.x264-IMMERSE[rarbg]",0,"0368737A",false],
["097AA60280AE3E4BA8741192CB015EE06BD9F992",200,"Epi.S01E04.HDTV.x264-KILLERS[ettv]",221928759,1000,221928759,8890308649,40059,0,0,0,"",0,1461,0,4395,65536,-1,0,"","","Queued Seed","512840d8",1437022635,1437023190,"","/mydir/Epi.S01E04.HDTV.x264-KILLERS[ettv]",0,"8F52310A",false]],
"label": [],"torrentc": "350372445"
,"rssfeeds": []
,"rssfilters": []
}
',
'_msg' => 'OK',
'_protocol' => 'HTTP/1.1'
}, 'HTTP::Response' );
I would like to extract each of the following strings from the returned object
097AA60280AE3E4BA8741192CB015EE06BD9F992
200
Epi.S01E04.HDTV.x264-KILLERS[ettv]
Unfortunately, my understanding of objects in Perl is very elementary.
The original code which returns this data looks like this:
my $ua = LWP::UserAgent->new();
my $response = $ua->get( $url, #ns_headers );
print Dumper($response);
How can I work on the strings that of interest?
If you read the documentation for HTTP::Response, you will see that there is a content method, which will return the content of your HTTP message, and a decoded_content method that does the same but also decompresses the data if it happens to be compressed (in your case the data is uncompressed.)
In this case it looks like the content is encoded as JSON data, so you will also need to load the JSON module to decode it into a Perl data structure
For example
use JSON 'from_json';
my $content = from_json $response->decoded_content;
my $torrents = $content->{torrents};
for my $torrent ( #$torrents ) {
say for #$torrent[0,1,2];
say '';
}
output
043CC5FA0C741CDAD9D2E5CC20DF64A4A400FA34
136
Epi.S01E03.720p.HDTV.x264-IMMERSE[rarbg]
097AA60280AE3E4BA8741192CB015EE06BD9F992
200
Epi.S01E04.HDTV.x264-KILLERS[ettv]

Facebook status update using WWW::Mechanize

I am trying to update status on facebook using Mechanize.I am able to login using the script but unable to update.I verified the id of form for status update is "u_0_w".
But selecting the form_id method says "There is no form with ID "u_0_w"".
My script is this:
use WWW::Mechanize;
use strict;
use warnings;
use Data::Dumper;
use HTTP::Cookies::Netscape;
my $cookiesfilename='/home/xxx/xxx/cookies.txt';
my $out;
my $mech = WWW::Mechanize->new( cookie_jar => HTTP::Cookies::Netscape->new( file => $cookiesfilename ) );
$mech->get("https://www.facebook.com/login.php");
my $response=$mech->submit_form(
fields => {
email => 'xxxx#xxxx.com',
pass => 'xxxxx',
}
);
#my $array=$mech->forms();
#$mech->get('/home.php');
print Dumper($mech->forms());
#$mech->form_id("u_0_w");
$mech->submit_form(
fields => {
xhpc_message_text=>'Why so serious'
}
);
print $response->status_line;
open($out, ">", "output_page.html") or die "Can't open output_page.html: $!";
print $out $response->decoded_content;
Then I tried to print all the forms on the page using Dumper the output is:
$VAR1 = bless( {
'default_charset' => 'UTF-8',
'enctype' => 'application/x-www-form-urlencoded',
'accept_charset' => 'UNKNOWN',
'action' => bless( do{\(my $o = 'https://www.facebook.com/search/web/direct_search.php')}, 'URI::https' ),
'method' => 'GET',
'attr' => {
'method' => 'get'
},
'inputs' => [
bless( {
'tabindex' => '-1',
'value' => '1',
'class' => '_42ft _42fu _4w98',
'type' => 'submit'
}, 'HTML::Form::SubmitInput' ),
bless( {
'/' => '/',
'autocomplete' => 'off',
'tabindex' => '1',
'name' => 'q',
'aria-label' => 'Search Facebook',
'value_name' => '',
'class' => 'inputtext _586f',
'type' => 'text',
'id' => 'u_0_b',
'role' => 'combobox',
'placeholder' => 'Search Facebook'
}, 'HTML::Form::TextInput' )
]
}, 'HTML::Form' );
It means it is not detecting the status update form it is detecting only Facebook search form.
What may be the problem for mechanize not detecting all the form elements?
The form contains <button type="submit">. Do Mechanize support it?
Why do you have to use Mechanize for this? There's already a module available for this on CPAN.
Take a look at WWW::Facebook::API.
Also see a related question: How do I use Perl's WWW::Facebook::API to publish to a user's newsfeed?
Synopsis:
use WWW::Facebook::API;
my $facebook = WWW::Facebook::API->new(
desktop => 0,
api_key => $fb_api_key,
secret => $fb_secret,
session_key => $query->cookie($fb_api_key.'_session_key'),
session_expires => $query->cookie($fb_api_key.'_expires'),
session_uid => $query->cookie($fb_api_key.'_user')
);
my $response = $facebook->stream->publish(
message => qq|Test status message|,
);

Perl and LWP not authenticating

I'm trying to get an LWP request working to an https server. I have been given a user & pass, advised to use basic authentication. I've tried various chunks of code, and all seem to get an authentication error. My current code is...
use warnings;
use strict;
use Data::Dumper;
use LWP;
my $ua = LWP::UserAgent->new( keep_alive => 1 );
##also tried by $ua->credentials('domain','','user','pass');
##not sure if I need 'realm' or how I get it, as no popup on screen.
my $request = HTTP::Request->new( GET => "https://my.url.com/somepath/" );
$request->authorization_basic('myuser','mypass');
$request->header( 'Cache-Control' => 'no-cache' );
print $response->content;
print Dumper $response;
The server gives a security error, but if I look at a dump of $response, I see the following...
'_rc' => '401',
'_headers' => bless( { .... lots of stuff
'title' => 'Security Exception',
'client-warning' => 'Missing Authenticate header',
'client-ssl-socket-class' => 'IO::Socket::SSL',
...
'expires' => '-1'
}, 'HTTP::Headers' ),
'_msg' => 'Unauthorized',
'_request' => bless( {
'_content' => '',
'_uri' => bless( do{\(my $o = 'https:theurlabove')}, 'URI::https' ),
'_method' => 'GET',
'_uri_canonical' => $VAR1->{'_request'}{'_uri'}
'_headers' => bless( {
'user-agent' => 'libwww-perl/6.04',
'cache-control' => 'no-cache',
'authorization' => 'Basic dzx..........'
}, 'HTTP::Headers' ),
I'm trying to understand whats happening, it looks like in the original request, it has the headers in there, but in the response, its saying I'm 'Missing Authenticate Header'.
Is there something amiss with the code, or something I'm misunderstanding with the request/respinse ?
Thanks.
The "Missing Authenticate header" message is coming from LWP itself. This means that it couldn't find an authenticate header in the target response. This might mean that your proxy settings are misconfigured, if you have anything like that.
I don't know if this is what you are looking for but I came across the same problem trying to authenticate to a webpage and had to solve it with WWW::Mechanize. I had to go to the first page and login then request the page I wanted.
use WWW::Mechanize;
my $loginPage = "http://my.url.com/login.htm"; # Authentication page
my $mech = WWW::Mechanize->new(); # Create new brower object
$mech->get($loginPage); # Go to login page
$mech->form_name('LogonForm'); # Search form named LogonForm
$mech->field("username", myuser); # Fill out username field
$mech->field("password", mypass); # Fill out password field
$mech->click("loginloginbutton"); # submit form
$mech->get("http://my.url.com/somepath/"); # Get webpage
# Some more code here with $mech->content()

Logging into Jenkins via Perl script

I am able to change a build's description with the following program. This will change the build's description to "FOO FOO FOO". Unfortunately, my login doesn't work. Right now, this test Jenkins build server has no security on it. However, on our regular Jenkins server, you need to be logged in to change a build's description.:
#! /usr/bin/env perl
use 5.12.0;
use warnings;
use LWP::UserAgent;
use HTTP::Request::Common qw(POST);
use Data::Dumper;
use constant {
JENKINS_BASE => "http://build.vegibank.com/",
USER_ID => "buildguy",
PASSWORD => "swordfish",
};
use constant {
LOGIN_URL => JENKINS_BASE . '/j_acegi_security_check',
JOB_URL => JENKINS_BASE . '/job',
SUBMIT_DESCRIPTION => 'submitDescription',
};
my $job_number = 4;
my $job_name = "proginator-2.0";
my $description = "FOO FOO FOO";
my $user_agent = LWP::UserAgent->new or die qq(Can't get User Agent);
#
# My Login Stuff (but it doesn't do anything w/ security off
#
my $response = $user_agent->request (
POST LOGIN_URL,
[
j_username => USER_ID,
j_password => PASSWORD,
],
);
$response = $user_agent->request (
POST "#{[JOB_URL]}/$job_name/$job_number/#{[SUBMIT_DESCRIPTION]}",
[
description => "$description",
],
);
I'm trying to connect to the Jenkins login session, but I don't believe I'm doing it quite right. When I attempt to login, I get a 302 response and the following dump of my response object:
$VAR1 = bless( {
'_protocol' => 'HTTP/1.1',
'_content' => '',
'_rc' => '302',
'_headers' => bless( {
'connection' => 'close',
'client-response-num' => 1,
'set-cookie' => 'JSESSIONID=1D5DF6FAF8714B2ACA4D496FBFE6E983; Path=/jenkins/; HttpOnly',
'location' => 'http://build.vegicorp.com/;jsessionid=1D5DF6FAF8714B2ACA4D496FBFE6E983',
'date' => 'Mon, 13 May 2013 20:02:35 GMT',
'client-peer' => '10.10.21.96:80',
'content-length' => '0',
'client-date' => 'Mon, 13 May 2013 20:02:35 GMT',
'content-type' => 'text/plain; charset=UTF-8',
'server' => 'Apache-Coyote/1.1'
}, 'HTTP::Headers' ),
'_msg' => 'Moved Temporarily',
'_request' => bless( {
'_content' => 'j_username=buildguy&j_password=swordfish',
'_uri' => bless( do{\(my $o = 'http://build.vegicorp.com/j_acegi_security_check')}, 'URI::http' ),
'_headers' => bless( {
'user-agent' => 'libwww-perl/6.03',
'content-type' => 'application/x-www-form-urlencoded',
'content-length' => 42
}, 'HTTP::Headers' ),
'_method' => 'POST',
'_uri_canonical' => $VAR1->{'_request'}{'_uri'}
}, 'HTTP::Request' )
}, 'HTTP::Response' );
I figure I must be hitting a valid page since I'm getting a 302 code, but my fields might not be correct (or I'm going to the wrong page).
Can anyone help?
My authorization is failing because ...what is the technical term? Oh yeah... "doing it all wrong."
After Googling and getting a lot of unhelpful junk, I, on a lark, decided to see if the Jenkins website might have something on this. And, it did right under a page called Authenticating scripted clients. In fact, they even give a Perl LWP example for a scripted client.
Ha ha, I was trying too hard. It seems that Jenkins will use the basic HTTP authentication mechanism, and I don't have to go through conniptions trying to figure out how their login form works. Apparently, Jenkins is simplifying the basic authentication mechanism for you even if your authentication mechanism is far from basic -- like a good program should do.
So, all I had to do was use the basic authentication mechanism.
my $browser = LWP::UserAgent->new or die qq(Cannot get User Agent);
my $request = HTTP::Request->new;
$request->authorization_basic(USER_ID, PASSWORD);
$request->method("GET");
$request->url("$jenkins_url");
my $response = $browser->request($request);
if ( not $response->is_success ) {
die qq(Something went horribly wrong...);
}
I've seen the redirect when the login is successful -- it sets the session cookie and redirects you to the main page.
Your post might be failing because the UA object isn't persisting the session cookie. Per the documentation, 'The default is to have no cookie_jar, i.e. never automatically add "Cookie" headers to the requests.' Try:
my $ua = LWP::UserAgent->new( cookie_jar => HTTP::Cookies->new() );
To store and reuse your session for the description change post.
(Also, credentials are visible in your header dump, may want to edit... Edit: I'm an idiot, they're in your constants too and're likely fake.)

POST web authentication of Nest Thermostat site in Perl (attempting to convert from Ruby)

there is a bit of code I'm trying to replicate in Perl using either LWP::UserAgent or WWW::Mechanize from an existing script.
The original script actually does more than I'm looking to do. I'd just like to log into the Nest website (the part I need help with) and then parse out some data for historical logging (I'm good there).
My current script I would expect to work, but I'm not sure if the authResult/access_token from the Ruby example us actually understood/used by either Perl module.
My code in Perl:
#!/usr/bin/perl
use WWW::Mechanize;
#use HTTP::Request::Common qw(POST);
use HTTP::Cookies;
use LWP::UserAgent;
use Data::Dumper;
use CGI;
my $email; #stores our mail
my $password; #stores our password
my $user_agent = 'Nest/1.1.0.10 CFNetwork/548.0.4';
$email = "email#email";
$password = "mypassword";
my #headers = (
'User-Agent' => 'Nest/1.1.0.10 CFNetwork/548.0.4',
'X-nl-user-id' => $email,
'X-nl-protocol-version' => '1',
'Accept-Language' => 'en-us',
'Connection' => 'keep-alive',
'Accept' => '*/*'
);
# print "Content-type: text/html\n\n";
my $cookie = HTTP::Cookies->new(file => 'cookie',autosave => 1,);
my $browser = WWW::Mechanize->new(cookie_jar => $cookie, autocheck => 1,);
# tell it to get the main page
$browser->get("https://home.nest.com/user/login");
print Dumper($browser->forms);
# okay, fill in the box with the name of the
# module we want to look up
$browser->form_number(1);
$browser->field("username", $email);
$browser->field("password", $password);
$browser->submit();
print $browser->content();
When I submit the form, I just get the same page returned back to me, and I don't know what exactly is causing Nest to not like what I'm submitting. There are two additional fields in the form on their log-in page:
'inputs' => [
bless( {
'maxlength' => '75',
'/' => '/',
'value_name' => 'E-mail address',
'name' => 'username',
'id' => 'id_username',
'type' => 'text'
}, 'HTML::Form::TextInput' ),
bless( {
'/' => '/',
'value_name' => 'Password',
'name' => 'password',
'id' => 'id_password',
'type' => 'password',
'minlength' => '6'
}, 'HTML::Form::TextInput' ),
bless( {
'readonly' => 1,
'/' => '/',
'value_name' => '',
'value' => '',
'name' => 'next',
'type' => 'hidden'
}, 'HTML::Form::TextInput' ),
bless( {
'readonly' => 1,
'/' => '/',
'value_name' => '',
'value' => 'dbbadca7910c5290a13d30785ac7fb79',
'name' => 'csrfmiddlewaretoken',
'type' => 'hidden'
}, 'HTML::Form::TextInput' )
Do I need to use the csrfmiddlewaretoken value in each submission? It appears to change. I thought getting a cookie upon a successful login would be enough.
Any suggestions on what I'm doing wrong?
Shot in the blue:
perl -E'use warnings; $email = "email#email"; say "<$email>"'
Possible unintended interpolation of #email in string at -e line 1.
<email>
I suspect it fails because the form gets the wrong user name, print it out to confirm. Always enable the pragmas strict and warnings to make many common mistakes visible.