Why does CGI::Session new and load fail ( couldn't thaw() )? - perl

I tried using the CGI::Session Library but for some reason my code won't keep a persistant session ... this is using Perl Moose for OOP, and is using Moose builders to instantiate the _cgi and _sss (session) parameters of a My::Session object...
UPDATED CODE
My::Role::PersistantData
package My::Role::PersistsData;
use Moose::Role;
use namespace::autoclean;
has '_cgi' => (
is => 'rw',
isa => 'Maybe[CGI]',
builder => '_build_cgi'
);
has '_sss' => (
is => 'rw',
isa => 'Maybe[CGI::Session]',
builder => '_build_sss'
);
My::Session
package My::Session;
use Moose;
use namespace::autoclean;
with 'My::Role::PersistsData';
use CGI;
use CGI::Session ('-ip_match');
use CGI::Carp qw/fatalsToBrowser warningsToBrowser/;
sub start{
my($self) = #_;
my $cgi = $self->cgi();
$self->log("Session Started!");
}
sub cgi{
my($self) = #_;
$self->_cgi = $self->_build_cgi() unless $self->_cgi;
return ($self->_cgi);
}
sub _build_cgi{
my($self) = #_;
my $cgi = CGI->new();
if(!$cgi){
#print "mising cgi";
}
return ( $cgi );
}
sub _build_sss{
my($self) = #_;
my $cgi = $self->cgi();
my $sid = $cgi->cookie("CGISESSID") || $cgi->param('CGISESSID') || undef;
$self->log("Session ID Initial is: ".($sid?$sid:"undef"));
my $sss = CGI::Session->new(undef, $cgi, {Directory=>'tmp'}) or die CGI::Session->errstr;
my $cookie = $cgi->cookie(CGISESSID => $sss->id() );
$self->log("Resulting Session ID is: ".$sid." cookie is: ".$cookie);
print $cgi->header( -cookie=>$cookie );
return ( $sss );
}
main.pl
use Data::Dumper;
$Data::Dumper::Sortkeys = 1;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use My::Session;
$| = 1;
$, = " ";
$\ = "\n <br />";
my $sss = My::Session->new();
$sss->start();
print Dumper($sss);
It's pretty weird because the first time I run this I get an actual CGISESSION ID and I am able to carry it over on a page refresh...
however if I load the page again, suddenly the $sss (session) comes back as undefined, when it should return a new Session object:
$sss = new CGI::Session("driver:File", $sid, {Directory=>'/tmp'})
for some reason $sss is coming back as undefined, which means it didnt initiate a new Session. A few tweaks to my code revealed this error:
new(): failed: load(): couldn't thaw() data using CGI::Session::Serialize::default:thaw(): couldn't thaw. syntax error at (eval 253) line 2, near "/>"
I also snooped around in CGI::Session.pm and found where this error was being thrown at, I guess it's not able to parse _DATA or even read it...because of some strange characters... "/>"
CGI::Session.pm
....
$self->{_DATA} = $self->{_OBJECTS}->{serializer}->thaw($raw_data);
unless ( defined $self->{_DATA} ) {
#die $raw_data . "\n";
return $self->set_error( "load(): couldn't thaw() data using $self->{_OBJECTS}->{serializer} :" .
$self->{_OBJECTS}->{serializer}->errstr );
}
Any idea why this isnt working?

Most likely this is due to a different session cookie being sent (been there, hit that wall with head. HARD).
Please print the session cookie value being used to store the session initially as well as session cookie value supplied by subsequent request.
If they are indeed different, you have 2 options:
Investigate why different session cookie is sent by the browser in subsequent requests and fix that issue somehow.
I was never able to find conclusive answer but my app consisted of a frame with internal <iframe> so I suspect it was due to that.
If like me you can't find the root cause, you can also work around this.
My workaround: explicitly STORING the original session cookie value as a form variable being passed around 100% of your code pieces.
Then re-initialize session object with correct cookie value before your server side code requests session data.
Not very secure, annoying, hard to get right. But works. I wouldn't recommend it except as a uber-last-resort hack

Perhaps you could try (or at least look at the code to see how it works) for some stateful webapp module. I have used Continuity, very cool stuff.

For some reason you can't use Data::Dumper or other HTML tags with CGI::Session
Answer found here and here
Removing Dumper and HTML output fixed this problem -- kind of --
updated
Apparently you have to use escapes
$cgi->escapeHTML ( Dumper($session) );
and that FINALLY resolves this problem.
Perl is a pain!

Related

Reuse LWP Useragent object with HTTP POST query in a for/while loop

I am using LWP Useragent to make multiple POST calls with basic Authorization, wherein POST URL parameters are read from a CSV file. Here is my code:
use strict;
use warnings;
use LWP::UserAgent;
use JSON 'from_json';
use MIME::Base64 'encode_base64';
use Data::Dumper;
my #assets;
my %data;
my $response;
my $csvfile = 'ScrappedData_Coins.csv';
my $dir = "CurrencyImages";
open (my $csv, '<', "$dir/$csvfile") || die "cant open";
foreach (<$csv>) {
chomp;
my #currencyfields = split(/\,/);
push(#assets, \#currencyfields);
}
close $csv;
my $url = 'https://example.org/objects?';
my %options = (
"username" => 'API KEY',
"password" => '' ); # Password field is left blank
my $ua = LWP::UserAgent->new(keep_alive=>1);
$ua->agent("MyApp/0.1");
$ua->default_header(
Authorization => 'Basic '. encode_base64( $options{username} . ':' . $options{password} )
);
my $count =0;
foreach my $row (#cryptoassets) {
$response = $ua->post(
$url,
Content_Type => 'multipart/form-data',
Content => {
'name'=>${$row}[1],
'lang' => 'en',
'description' => ${$row}[6],
'parents[0][Objects][id]' => 42100,
'Objects[imageFiles][0]' =>[${$row}[4]],
}
);
if ( $response->is_success ) {
my $json = eval { from_json( $response->decoded_content ) };
print Dumper $json;
}
else {
$response->status_line;
print $response;
}
}
sleep(2);
}
Basically, I want to reuse the LWP object. For this, I am creating the LWP object, its headers, and response objects once with the option of keep_alive true, so that connection is kept open between server and client. However, the response from the server is not what I want to achieve. One parameter value ('parents[0][Objects][id]' => 42100) seems to not get passed to the server in HTTP POST calls. In fact, its behavior is random, sometimes the parentID object value is passed, and sometimes not, while all other param values are passing correctly. Is this a problem due to the reusing of the LWP agent object or is there some other problem? Because when I make a single HTTP POST call, all the param values are passed correctly, which is not the case when doing it in a loop. I want to make 50+ POST calls.
Reusing the user-agent object would not be my first suspicion.
Mojo::UserAgent returns a complete transaction object when you make a request. It's easy for me to inspect the request even after I've sent it. It's one of the huge benefits that always annoyed my about LWP. You can do it, but you have to break down the work to form the request first.
In this case, create the query hash first, then look at it before you send it off. Does it have everything that you expect?
Then, look at the request. Does the request match the hash you just gave it?
Also, when does it go wrong? Is the first request okay but the second fails, or several are okay then one fails?
Instead of testing against your live system, you might try httpbin.org. You can send it requests in various ways
use Mojo::UserAgent;
use Mojo::Util qw(dumper);
my $hash = { ... };
say dumper( $hash );
my $ua = Mojo::UserAgent->new;
$ua->on( prepare => sub { ... } ); # add default headers, etc
my $tx = $ua->post( $url, form => $hash );
say "Request: " . $tx->req->to_string;
I found the solution myself. I was passing form parameter data (key/value pairs) using hashref to POST method. I changed it to arrayref and the problem was solved. I read how to pass data to POST method on CPAN page. Thus, reusing LWP object is not an issue as pointed out by #brian d foy.
CPAN HTTP LWP::UserAgent API
CPAN HTTP Request Common API

Perl Web Login Script With CGI::Session

i'm on the same problem since almost two week ago.
i'm a newbie with Perl and Web :/
i followed the CGI::Session tutorial and Cookbook, the code seems to be good but... not working.
index.cgi
#!/usr/bin/perl
use CGI;
use CGI::Cookie;
use HTML::Template;
use strict;
use warnings;
use CGI::Session;
use CGI::Carp qw(fatalsToBrowser);
require "cgi-bin/web_base.pl";
require "cgi-bin/login.pl";
my $cgi = new CGI;
my $session = new CGI::Session("driver:File", undef, {Directory=>'/tmp'}) or die CGI::Session->errstr;
my $CGISESSID = $session->id();
print header();
print $cgi->header();
print my_topbar();
login_attempt($session, $cgi);
if ( $session->param("~login-trials") >= 3 ) {
print error("You failed 3 times in a row.\n" .
"Your session is blocked. Please contact us with ".
"the details of your action");
exit(0);
}
unless ( $session->param("~logged-in") ) {
print login_form($cgi, $session);
exit(0);
}
print footer();
login.cgi
#!/usr/bin/perl
use strict;
use warnings;
use CGI;
use CGI::Cookie;
use HTML::Template;
use CGI::Session;
use CGI::Carp qw(fatalsToBrowser);
use Fcntl;
my $cgi = new CGI;
my $session = new CGI::Session(undef, $cgi, {Directory=>'/tmp'});
sub login_attempt {
my ($session, $cgi) = #_;
if ( $session->param("~logged-in") ) {
return 1; # Verify if user is not logged.
}
my $username = $cgi->param("username") or return;
my $password=$cgi->param("password") or return;
# Form submited. Try to load profile.
if ( my $profile = load_profile($username, $password) ) {
$session->param("~profile", $profile);
$session->param("~logged-in", 1);
print "YOUPIIIII";
$session->clear(["~login-trials"]);
$session->redirect("dashboard.cgi");
return 1;
}
# Failed to login, wrong credentials.
my $trials = $session->param("~login-trials") || 0;
return $session->param("~login-trials", ++$trials);
}
return 1;
sub load_profile {
my ($username, $password) = #_;
local $/ = "\n";
unless (sysopen(PROFILE, "profile.txt", O_RDONLY) ) {
die ("Couldn't open profile.txt: $!");
}
while ( <PROFILE> ) {
/^(\n|#)/ and next;
chomp;
my ($n, $p) = split "\s+";
if ( ($n eq $username) && ($p eq $password) ) {
my $p_mask = "x" . length($p);
return {username=>$n, password=>$p_mask};
}
}
close(PROFILE);
return undef;
}
profile.txt
Formget 123
When i try to login, nothing happen, even when i try wrong crendentials it should block me after 3 attemps but it is not.
Can someone really help me on this ? i can't take it anymooooore.
feel free for any questions :)
EDIT :
-login_attempt() corrected
-load-profile wasn't working, made a new one, but still need improvement.
-Last Problem is session init
Are you sure that you don't get any errors? Have you checked the web server error log?
You call login_attempt() with two parameters ($session and $cgi) but in login.cgi, that subroutine is defined like this:
sub login_attempt() {
...
}
You're (probably accidentally) using a prototype on that subroutine, telling Perl that it takes no parameters. So I'd be surprised if you don't get an error saying:
Too many arguments for main::login_attempt
Remove the parentheses from that definition.
sub login_attempt {
...
}
Update: I think you're missing one very important step here. From the CGI::Session Tutorial:
There is one small, but very important thing your application needs to perform after creating CGI::Session object as above. It needs to drop Session ID as an HTTP cookie into the user's computer. CGI::Session will use this cookie to identify the user at his/her next request and will be able to load his/her previously stored session data.
To make sure CGI::Session will be able to read your cookie at next request you need to consult its name() method for cookie's suggested name:
$cookie = $query->cookie( -name => $session->name,
-value => $session->id );
print $query->header( -cookie=>$cookie );
name() returns CGISESSID by default. If you prefer a different cookie name, you can change it as easily too, but you have to do it before CGI::Session object is created:
CGI::Session->name("SID");
$session = CGI::Session->new();
Baking the cookie wasn't too difficult, was it? But there is an even easier way to send a cookie using CGI::Session:
print $session->header();
The above will create the cookie using CGI::Cookie and will return proper http headers using CGI.pm's CGI method. Any arguments to CGI::Session will be passed to CGI::header().
Without this, you'll be creating a brand new session for each request.

Extracting cookies from a Mojolicious user agent response

I started using the Mojolicious library for testing and everything was working fine in until I tried to extract cookies from a response.
I've tried several variants of:
$ua = Mojo::UserAgent->new();
$ua->on( error => sub { my ($ua, $error) = #_; say "This looks bad: $error"; } );
$ua->max_redirects(1)->connect_timeout(10)->request_timeout(20);
$ua->cookie_jar(Mojo::CookieJar->new);
# ... later ...
my $tx = $ua->get($url);
my $jar = $ua->cookie_jar->extract($tx); # This is undef
I can however extract the cookies via LWP::UserAgent. However, LWP has several different issues that make that option unworkable for now. Just for a comparison here is the LWP code that does extract the cookies.
my $lwp = LWP::UserAgent->new(cookie_jar => {}, timeout => 20, max_redirect => 1);
push #{ $lwp->requests_redirectable }, 'POST';
my $response = $lwp->get($url);
die $response->status_line unless $response->is_success;
$lwp->cookie_jar->scan(\&ScanCookies);
sub ScanCookies {
my ($version, $key, $value) = #_;
say "$key = $value";
}
So I know that I have the $url etc. correct.
Edit: I should mention that i'm using strawberry 5.14
Edit2: I should also mention that the cookies are getting into the user agent for sure, because the session ID is getting handled properly. Unfortunately, I have a need to access another cookie (for testing the site) and I just don't seem to be able the get the right incantation to access them... saying that I believe this to be a programmer problem and nothing more.
Use this:
$tx->res->cookies

HTTP request not going through proxy

I have written this code to fire a http request through a proxy.
But the request does not seem to use proxy. Even though I give a wrong proxy, it is returning OK.
Is there any way I can check, whether the HTTP request went via proxy?
What is the issue in this code which makes it not use proxy?
sub fire_http_request_through_proxy()
{
my $proxy = $_;
my $ua = LWP::UserAgent->new;
$ENV{HTTP_PROXY} = $proxy;
$ua->env_proxy; # initialize from environment variables
$ua->timeout(20);
my $response = $ua->get('http://www.google.com');
delete $ENV{HTTP_PROXY};
if ($response->is_success)
{
print $response->decoded_content . "\n";
}
else
{
die $response->status_line;
}
}
Sebastian and oalders have already solved your problem, but I'd just like to note that you don't need to mess around with $ENV{HTTP_PROXY} anyway — you can just use $ua->proxy(), like this:
$ua->proxy( http => 'http://1.1.1.1' );
or even:
$ua->proxy( ['http', 'https', 'ftp'] => 'http://1.1.1.1' );
Ps. If you really want to check which proxy was used by LWP for a particular request, you can peek at $response->request->{proxy}, which should be a URI object. However, as far as I know, this property is undocumented (I found out about it by reading the source) and thus subject to change in later versions of LWP. Use at your own risk!
Are you sure that $_ has a true value? This dies appropriately for me:
#!/usr/bin/env perl
use strict;
use warnings;
use LWP::UserAgent;
fire_http_request_through_proxy();
sub fire_http_request_through_proxy {
my $ua = LWP::UserAgent->new;
local $ENV{HTTP_PROXY} = 'http://1.1.1.1';
$ua->env_proxy; # initialize from environment variables
$ua->timeout( 20 );
my $response = $ua->get( 'http://www.google.com' );
delete $ENV{HTTP_PROXY};
if ( $response->is_success ) {
print $response->decoded_content . "\n";
}
else {
die $response->status_line;
}
}
So, maybe $_ isn't what you think it is. If it's not defined, then no proxy will be used. Having said that, $_ is probably not the variable you want to use here. You could either declare a variable for use in this case, pass a variable right to the subroutine or actually set an ENV variable outside of the script.
One other point. Rather than setting and deleting the ENV var in your script, just declare the change with local and it will only take effect inside this block of code. That way you don't have to clean up after yourself and you don't risk overriding vars which may have been set elsewhere.
Take a look at your code sub fire_http_request_through_proxy(), especially the last two characters... This is a prototype. Basically you are saying "I don't take any arguments during compile-time".
I guess you are simply invoking the method before its declaration -> Always use warnings:
main::fire_http_request_through_proxy() called too early to check
prototype at test.pl line ...
So either change it to fire_http_request_through_proxy or change it to fire_http_request_through_proxy($) and invoke it after its declaration.
More about prototyping in perlsub.
Be sure
to read parameter as $_[0] or pop, not $_
to not include () in sub definition
Script:
sub fire_http_request_through_proxy {
my $proxy = $_[0];
my $timeout = 20;
my $url = 'http://www.google.com';
my $ua = LWP::UserAgent->new;
$ua->proxy(['http', 'https', 'ftp'] => $proxy);
$ua->timeout($timeout);
my $response = $ua->get($url);
if ($response->is_success) {
print $response->decoded_content . "\n";
}
else {
die $response->status_line;
}
}
Test:
To make it work, parameter of proxy has to be in correct format (http://host:port)
fire_http_request_through_proxy('http://176.34.248.142:9050');

Perl: Problems with WWW:Mechanize and a form

i am trying to write a script that will navigate through a soccer website to the player of my choice and scrape their info for me. I have the scraping part working by just hard coding an individual player's page in, but trying to implement the navigation is giving me some problems. The website in question is http://www.soccerbase.com.
I have to fill in a form present at the top of the page with the player's name, then submit it for the search. I have tried it two different ways(commenting out one of them) based on info i found online but to no avail. I am an absolute novice when it comes to Perl so any help would be greatly appreciated! Thanks in advance. here is my code:
#!/usr/bin/perl
use strict;
require WWW::Mechanize;
require HTML::TokeParser;
my $player = 'Luis Antonio Valencia';
#die "Must provide a player's name" unless $player ne 1;
my $agent = WWW::Mechanize->new();
$agent->get('http://www.soccerbase.com/players/home.sd');
$agent->form_name('headSearch');
$agent->set_fields('searchTeamField', $player);
$agent->click_button(name=>"Search");
#$agent->submit_form(
# form_number => 1,
# fields => { => 'Luis Antonio Valencia', }
# );
my $stream = HTML::TokeParser->new(\$agent->{content});
my $player_name;
$stream->get_tag("strong");
$player_name = $stream->get_trimmed_text("/strong");
print "\n", "Player Name: ", $player_name, "\n";
It's a bit tricky because the form action plays switcharoo with Javascript, but HTML::Form is able to handle that perfectly fine:
#!/usr/bin/env perl
use WWW::Mechanize qw();
use URI qw();
my $player = 'Luis Antonio Valencia';
my $agent = WWW::Mechanize->new;
$agent->get('http://www.soccerbase.com/players/home.sd');
my $form = $agent->form_id('headSearch');
{
my $search_uri = $agent->uri;
$search_uri->path('/players/search.sd');
$form->action($search_uri);
# requires absolute URI
}
$agent->submit_form(
fields => {
search => $player,
type => 'player',
}
);
Easier way is to look at the HTTP request it makes, for instance:
http://www.soccerbase.com/players/search.sd?search=kkkk&type=player
'kkkk' is the player name, use LWP::UserAgent to make that request, and it will give you the result, change the 'kkk' to the name of the player you are looking to get info for, and that will do the job, using Mech for that is an overkill, if you ask me, make sure that if the player name has spaces,etc encode it.
It looks like the form elements do not have name attributes and I am assuming the query string is formed by some other means by translating the id attributes to yield:
http://www.soccerbase.com/players/search.sd?search=Luis+Antonio+Valencia&type=player
You'd think the following would work, but it doesn't suggesting that there is some other JavaScript goodness(!) happening behind the scenes.
#!/usr/bin/env perl
use strict;
use warnings;
use HTML::TableExtract;
use LWP::Simple qw(get);
use URI;
my $player = 'Luis Antonio Valencia';
my $uri = URI->new('http://www.soccerbase.com/players/home.sd');
$uri->query_form(
search => $player,
type => 'player',
);
my $content = get "$uri";
die "Failed to get '$uri'\n" unless defined $content;
my $te = HTML::TableExtract->new(
attribs => { class => 'clubInfo' },
);
$te->parse($content);
die unless $te->tables;
my ($table) = $te->tables;
my ($row) = $table->rows;
print $row->[1], "\n";