I am trying to automate sms sending through Way2sms in Perl LWP. I am able to login successfully. Then I click the Send SMS link in the browser to go the sms-sending page. From there, based upon the page url and the url of the iframe within which the sms fields are located, I try to construct the absolute URL of the page to which the form should be posted, and post it with the correct parameters (you can see it all in the image). However, the sms isn't being sent. Can anybody tell me what am I doing wrong here? (There is a similar module in CPAN which accomplishes this through Mechanize, I am trying a different approach)
use LWP::UserAgent;
use HTTP::Cookies;
my $cookie_jar = HTTP::Cookies->new(
file => "cookies.txt",
autosave => 1,
);
my $ua = LWP::UserAgent->new(
agent =>
'Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1',
cookie_jar => $cookie_jar,
);
my $response = $ua->post(
'http://site2.way2sms.com/contentt/bar/Login1.action',
{
username => $mob,
password => $pass,
}
);
if ( $response->is_redirect ) {
$response = $ua->get( $response->header('Location') );
print 5 if $response->decoded_content =~ /Kaustav Mukherjee/i; #prints it, showing that the login is successful
}
my $smsresp = $ua->post("http://site5.way2sms.com/jsp/quicksms.action",[MobNo=>$mob,textArea=>'Hello World']);
if ( $smsresp->is_redirect ) {
$smsresp = $ua->post($smsresp->header('Location'),[MobNo=>$mob,textArea=>'Hello World']);
}
Here is an indirect answer that may help you out:
If you have to use this website, try using something like CasperJS. There may be some JavaScript magic that needs to happen.
Maybe you could use something like Google Voice or some other smarter service to automate send text messages. Looks like someone on CPAN already has a sweet Google::Voice module for this.
Related
OK, this is driving me mad. I currently run a background script on our server that logs in to our SagePay account every 5 minutes and updates currently logged in users on our local database.
SagePay rolled out a new admin system some time ago but due to many complaints kept the old version running alongside so I never bothered to update my script. SagePay have now decided to decommission the old version so I have no choice.
The following code is just the first part of script that logs in to SagePay:
use LWP::UserAgent;
use HTTP::Cookies;
use HTTP::Request;
use HTTP::Headers;
use HTTP::Request::Common qw(POST);
my $request = "";
my $content = "";
my $req = "";
my $cookie_jar = HTTP::Cookies->new();
my $ua = LWP::UserAgent->new();
$ua->cookie_jar($cookie_jar);
$ua->agent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)');
# LOGIN TO SAGEPAGY
$req = (POST 'https://portal.sagepay.com/myoldskool/loginpage.asp',
["vendorname" => "MYVENDORNAME",
"username" => "MYUSERNAME",
"password" => "MYPASSWORD",
"clickedButton" => "login"]);
$request = $ua->request($req);
$content = $request->content;
print $content;
This code works perfectly every time and prints the HTML of the SagePay dashboard as it would if you logged in.
The new POST url is https://portal.sagepay.com/mysagepay/j_spring_security_check and the parameters that need to be passed are pretty much the same, so technically I should just need to make the following alterations to the code and it should work:
$req = (POST 'https://portal.sagepay.com/mysagepay/j_spring_security_check',
["inp_vspvendorname" => "MYVENDORNAME",
"inp_username" => "MYUSERNAME",
"inp_password" => "MYPASSWORD",
"CSRFToken" => ""]);
When I change this code in my script $content prints nothing and I cannot for the life of me figure out why. I've even tried using HttpRequester Firefox addon and sent the same params to the same URL and that returns the dashboard HTML as expected.
$request->status_line is '302 Found'.
Obviously I can't give you my login details for SagePay but even just getting it to return the HTML for an invalid login, at least I'm making progress. I've been stuck on this for a few days now, I'm hoping someone can help.
You're receiving an HTTP 302 response, which is a redirect. By default, LWP::UserAgent doesn't follow redirects with POST requests. To change this, alter the user agent's requests_redirectable array before sending the request:
push #{ $ua->requests_redirectable }, 'POST';
my $response = $ua->request($req);
This question already has answers here:
Python Mechanize log into Facebook cookie error
(2 answers)
Closed 6 years ago.
I have a Perl script that I run in Strawberry Perl in which I login to Facebook using WWW::Mechanize. For a short background, I use my script to automate an occasional "leaderboard" for a fun little game I invented to be played among my group of friends. I assign points based upon certain posts, and the objective is to earn the most points. Very simple. This script is complete, and it's worked for me in the past (I last ran it 4 days ago). However, I am unable to login to Facebook today. I dumped the content after the form submission to an HTML file, and it appears that I've returned back to the login screen with a red error box:
Cookies Required
Cookies are not enabled on your browser. Please enable cookies in your browser preferences to continue.
From what I understand, WWW::Mechanize features automatic cookies -- in other words by the default constructor, the bot should be accepting cookies. Cookies should be enabled by the browser. I should not see this screen, right?
I've read and experimented with WWW:Mechanize and HTTP::Cookies, but no matter what I try (fooling around with the cookie jar every which way), I cannot get past this "cookies required" error.
This is my code without any experimental fluff, very simple.
use WWW::Mechanize;
my $mech = WWW::Mechanize->new();
$mech->get("https://www.facebook.com/login.php");
$mech->submit_form(
fields => {
email => '<my email here>',
pass => '<my password here>',
}
);
open($out, ">", "output_page.html") or die "Can't open output_page.html: $!";
print $out $mech->content;
Capture a session of you logging in with your browser (if you're using firefox you can use firebug, and check what HTTP headers are set - how-to. Set the same headers in your script, including the user agent. Check if the same cookies are set. You may end up with something like the below:
use strict;
use warnings;
use WWW::Mechanize;
use HTTP::Cookies;
my $cookie_jar = HTTP::Cookies->new(
hide_cookie2 => 1,
);
my $mech = WWW::Mechanize->new(
cookie_jar => $cookie_jar,
);
$mech->agent_alias('Windows Mozilla');
$mech->add_header( 'Connection' => 'keep-alive' );
$mech->add_header( 'Accept-Language' => 'en-GB,en;q=0.5' );
$mech->add_header( 'Accept-Encoding' => 'gzip, deflate' );
$mech->add_header( 'Accept' =>
'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' );
Somewhat inexperienced programmer here trying to write a program to log into my courses site and download all the content (lectures homeworks etc). Obviously it is a password protected site so I have to give it that. I understand LWP::UserAgent and the likes well enough, and that I need to use credentials. What I cannot figure out is how to get to the next page. I can go to the log-in, but how does perl get the result of my log-in?
code example (i pulled out the log-info obviously):
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
my $url = 'login URL';
$ua -> credentials(
$url,
'',
'user',
'pass'
);
my $response = $ua ->get($url);
print $response->content;
the content from response is the same content as what I would have got as if I had not passed any credentials. Obviously I'm missing something here....
Oh one other thing my own courses site does not have a unique url as far as I know.
You probably want to be using WWW::Mechanize, a subclass of LWP::UserAgent designed to act more like a browser, allowing you to navigate through pages of a website with cookie storage already taken care of for you.
You only use credentials if the site uses HTTP basic auth, in which case you don't "log in", you just pass the credentials with every request.
If the site has a form based login system, then you need to use cookie_jar and request the form's action URI with whatever data it expects.
#!/usr/bin/perl
use LWP::UserAgent;
use HTTP::Cookies;
my $ua=LWP::UserAgent->new(timeout => 20);
$ua->agent('Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.1.8) Gecko/20100202 MRA 5.5 (build 02842) Firefox/3.5.8');
$ua->requests_redirectable(0);
my $cook = HTTP::Cookies->new;
$ua->cookie_jar($cook);
print = requester('http://urlexample/login.php', 'login=yourlogin&password=pass' )->as_string;
sub requester
{
my $type = 'GET';
if($_[1]){$type = 'POST'}
my $req = HTTP::Request->new($type => $_[0]);
$req->content_type('application/x-www-form-urlencoded; charset=UTF-8');
if($_[1]){$req->content($_[1])}
my $res = $ua->request($req);
return $res;
}
I am attempting to login to Youtube with WWW:Mechanize and use forms() to print out all the forms on the page after logging in. My script is logging in successfully, and also successfully navigating to Youtube.com/inbox; However, for some reason Mechanize can not see any forms at Youtube.com/inbox. It just returns blank. Here is my code:
#!"C:\Perl64\bin\perl.exe" -T
use strict;
use warnings;
use CGI;
use CGI::Carp qw/fatalsToBrowser/;
use WWW::Mechanize;
use Data::Dumper;
my $q = CGI->new;
$q->header();
my $url = 'https://www.google.com/accounts/ServiceLogin?uilel=3&service=youtube&passive=true&continue=http://www.youtube.com/signin%3Faction_handle_signin%3Dtrue%26nomobiletemp%3D1%26hl%3Den_US%26next%3D%252Findex&hl=en_US<mpl=sso';
my $mechanize = WWW::Mechanize->new(autocheck => 1);
$mechanize->agent_alias( 'Windows Mozilla' );
$mechanize->get($url);
$mechanize->submit_form(
form_id => 'gaia_loginform',
fields => { Email => 'myemail',Passwd => 'mypassword' },
);
die unless ($mechanize->success);
$url = 'http://www.youtube.com/inbox';
$mechanize->get($url);
$mechanize->form_id('comeposeform');
my $page = $mechanize->content();
print Dumper($mechanize->forms());
Mechanize is unable to see any forms at youtube.com/inbox, however, like I said, I can print all of the forms from the initial link, no matter what I change it to...
Thanks in advance.
As always, one of the best debugging approaches is to print what you get and check if it is what you were expecting. This applies to your problem too.
In your case, if you print $mechanize->content() you'll see that you didn't get the page you're expecting. YouTube wants you to follow a JavaScript redirect in order to complete your cross-domain login action. You have multiple options here:
parse the returned content manually – i.e. /location\.replace\("(.+?)"/
try to have your code parse JavaScript (have a look at WWW::Scripter)
[recommended] use YouTube API for managing your inbox
I would like to write a script to login to a web application and then move to other parts
of the application:
use HTTP::Request::Common qw(POST);
use LWP::UserAgent;
use Data::Dumper;
$ua = LWP::UserAgent->new(keep_alive=>1);
my $req = POST "http://example.com:5002/index.php",
[ user_name => 'username',
user_password => "password",
module => 'Users',
action => 'Authenticate',
return_module => 'Users',
return_action => 'Login',
];
my $res = $ua->request($req);
print Dumper(\$res);
if ( $res->is_success ) {
print $res->as_string;
}
When I try this code I am not able to login to the application. The HTTP status code returned is 302 that is found, but with no data.
If I post username/password with all required things then it should return the home page of the application and keep the connection live to move other parts of the application.
You may be able to use WWW::Mechanize for this purpose:
Mech supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited.
I'm guessing that LWP isn't following the redirect:
push #{ $ua->requests_redirectable }, 'POST';
Any reason why you're not using WWW::Mechanize?
I've used LWP to log in to plenty of web sites and do stuff with the content, so there should be no problem doing what you want. Your code looks good so far but two things I'd suggest:
As mentioned, you may need to make the requests redirectable
You may also need to enable cookies:
$ua->cookie_jar( {} );
Hope this helps