Response Codes Net::Twitter:Stream - perl

I'm new in this forum and I'm having some problems with the perl library Net::Twitter:Stream. I'm following the example in this link Net::Twitter:Stream.
But it is missing the part when I get a bad response code(another than 200) and I have to stop my algorithm. So, what can I do in this case? I'm afraid to use it so much and enter into the twitter black list...
I'm basing in this code below:
use Net::Twitter::Stream;
Net::Twitter::Stream->new ( user => $username, pass => $password,
callback => \&got_tweet,
track => 'perl,tinychat,emacs',
follow => '27712481,14252288,972651' );
sub got_tweet {
my ( $tweet, $json ) = #_; # a hash containing the tweet
# and the original json
print "By: $tweet->{user}{screen_name}\n";
print "Message: $tweet->{text}\n";
}

I think you'll want to add connection_closed_cb=>\&bad_response, see this stackoverflow questions last answer. I'm not sure why that ability isn't documented but it is available if you check the source code. I also couldn't find that module in CPAN.

Related

Adding language variable into WWW::Mailchimp (subscription)

I'm trying to work out how I can use WWW::Mailchimp ( http://search.cpan.org/~arcanez/WWW-Mailchimp/ ) to sign someone up to our list, but also assign the language of the person (i.e english, french, german, spanish, etc).
Here is what I have thus far:
my $mailchimp = WWW::Mailchimp->new(apikey => 'xxxx' );
$mailchimp->listSubscribe( id => "xxx", email_address => $in->{Email}, merge_vars => [ FNAME => $name[0], LNAME => $name[1], mc_language => "fr", LANG => "fr", LANGUAGE => "fr" ] );
mc_language => "fr", LANG => "fr", LANGUAGE => "fr" doesn't seem to do anything (been trying all the params I see laying around, in the vain hope one of them works!)
While it works (and asks you to confirm your subscription), all the language variables are ignored. Looking at their documents, I'm a bit confused as to what to use:
https://apidocs.mailchimp.com/api/2.0/lists/subscribe.php
The code "fr" is ok, but I'm unsure what params to pass along to it.
Has anyone had any experience with this before? Apart from the language, it works fine (but I need to be able to send the confirmation emails in their own language, and then also filter down when doing mailings)
UPDATE: Ok, so it looks like its not going to be a simple case of updating to the newer API. I've been looking into the v3.0 API, and its a total overhaul of the older one (new function names, new ways of sending requests, etc). What I'm going to do is look into a "Curl" method, so we can at least get it going with that. Once I've got that going, I'll probably have a look at coding something to work with LWP::UserAgent, as that'd be cleaner than doing lots of curl requests. Shame there isn't anything out there already for Perl and MailChimp (with the new API, or even version 2.0!)
From looking at the source, it defaults to API 1.3:
has api_version => (
is => 'ro',
isa => Num,
lazy => 1,
default => sub { 1.3 },
);
The documentation for that shows you need to use MC_LANGUAGE:
string MC_LANGUAGE Set the member's language preference. Supported
codes are fully case-sensitive and can be found here.
It looks like the module just shoves whatever data structure you provide into JSON and POSTs it to Mailchimp, so the appropriate Mailchimp API doc version for the API you target should be referenced as a primary source.
Ok, so I got there in the end! I have been talking with MailChimp support, and they were very helpful. Turns out it was a double issue.
1) Auto-Translate needed to be enabled for the list in question. This was their answer around that:
After taking a look at the call, it appears to be set up properly now, so you are all good on that front. That being said, I am seeing
that the Auto-translate option doesn't seem to be enabled for any of
your lists. In order for the Confirmation and all other response
emails to automatically translate, this will need to be enabled for
all of the lists being used.
We have a bit of additional information on that, here, if you'd like to check that out:
http://kb.mailchimp.com/lists/signup-forms/translate-signup-forms-and-emails#Auto-Translate-Forms
2) When making the request via the API, you need to specifically set the Accept-Language: xx value. For example, en, fr, es, de, etc.
Here is a working function for anyone who needs it in the future. Just be sure to update the apikey,listId and endpoint URL.
do_register_email_list('foo#bar.com','Andrew Test',"en")
sub do_register_email_list {
# (email,name,lang)
use WWW::Curl::Easy;
use Digest::MD5;
use JSON;
my #name = split /\s+/, $_[1];
my $apikey = 'xxxx-us6';
my $listid = 'xxxx';
my $email = $_[0];
my $endpoint = "https://us6.api.mailchimp.com/3.0/lists";
my $lang = $_[2]||'en';
my $json = JSON::encode_json({
'email_address' => $email,
'status' => 'pending',
'language' => $lang,
'merge_fields' => {
'FNAME' => $name[0]||'',
'LNAME' => $name[1]||''
}
});
my $curl = WWW::Curl::Easy->new;
my $url = "$endpoint/$listid/members/" . Digest::MD5::md5(lc($email));
$curl->setopt(CURLOPT_HEADER,1);
$curl->setopt(CURLOPT_URL, $url);
# $curl->setopt(CURLOPT_VERBOSE, 1);
$curl->setopt(CURLOPT_USERPWD, 'user:' . $apikey);
$curl->setopt(CURLOPT_HTTPHEADER, ['Content-Type: application/json',"Accept-Language: $lang"]);
$curl->setopt(CURLOPT_TIMEOUT, 10);
$curl->setopt(CURLOPT_CUSTOMREQUEST, 'PUT');
$curl->setopt(CURLOPT_SSL_VERIFYPEER, 0);
$curl->setopt(CURLOPT_POSTFIELDS, $json);
# A filehandle, reference to a scalar or reference to a typeglob can be used here.
my $response_body;
$curl->setopt(CURLOPT_WRITEDATA,\$response_body);
# Starts the actual request
my $retcode = $curl->perform;
#print "FOO HERE";
# Looking at the results...
if ($retcode == 0) {
print "Transfer went ok\n";
my $response_code = $curl->getinfo(CURLINFO_HTTP_CODE);
print "Received response: $response_body\n";
} else {
# Error code, type of error, error message
print "An error happened: $retcode ".$curl->strerror($retcode)." ".$curl->errbuf."\n";
}
}
Hopefully this saves someone else from all the grief I had with it :) (the MailChimp support lady also said she will get their team to add something about this in the developer notes, so its made a bit clearer!)

Upload media to twitter using perl

Has anyone successfully uploaded media to Twitter, ie posted a tweet with an image using Perl?
I would like to upload images from my blog without doing it all manually.
update_with_media(status, media[]) is deprecated, and crashes. Twitter says to use plain update(), passing a media id. However it is first necessary to upload the media to get the id, and I can find no code examples.
Any experience in this area?
Cheers,
Peter
I thought I'd add an update here even though the thread is quite old. I too was looking for an answer to how to use Perl to upload media to twitter.
Net::Twitter is perfectly capable of sending PNG images up to Twitter, though the documentation is poor. The OP is correct that update_with_media is deprecated and crashed for us.
You do need to use the $nt->upload AND $nt->update methods combined. You upload the RAW PNG file encoded with base64, I could not get the RAW PNG file upload to work but no issues when base64 encoded. The upload method returns, on success, a JSON structure which has the media_ids in. These id's are then passed with the $nt->update method. Here's some actual code
use Net::Twitter;
use File::Slurp;
use MIME::Base64;
use Data::Dumper;
my $nt = Net::Twitter->new(
ssl => 1,
traits => [qw/API::RESTv1_1/],
consumer_key => $config->{twitter}{api_key},
consumer_secret => $config->{twitter}{api_secret},
access_token => $config->{twitter}{access_token},
access_token_secret => $config->{twitter}{access_token_secret},
);
my $filename = "<somelink to a PNG file>";
my $file_contents = read_file ($filename , binmode => ':raw');
my $status = $nt->upload(encode_base64($file_contents));
print "SendTweet: status = ".Dumper($status);
my $status2;
eval {
$status2 = $nt->update({status => $s , media_ids => $status->{media_id} });
print "status2 = ".Dumper($status2);
};
if ($#)
{
print "Error: $#\n";
}
The code is pulled directly from our working test code so should work. This code is purely proof of concept so you will need to add in all your twitter authentication etc.
We have only done PNG files but I see no reason why video etc shouldn't work fine as we simply followed the Twitter docs.
Rob
In the end I used readpipe with twurl. Had I known about twurl in the first place I likely would not have bothered with Net::Twitter! twurl works great, and returns a full json string to tell you what went wrong, if anything.
Pre-requisite: you need to get the Oauth keys for your twitter account. (See here - https://developer.twitter.com/en/docs/basics/authentication/guides/access-tokens.html). There are some step by step online exmaples elsewhere also that can help.
Here's the code I ended up with.
First call the tweet module (and these are not valid keys by the way - just insert yours)
use Jimtweet;
my $tweet=Jimtweet->new();
$tweet->consumer_key('KvfevhjwkKJvinvalidkeycvhejwkKJVCwe');
$tweet->consumer_secret('KvfevhjwkKJvcvnvalidkeyhejwkKJVCwe');
$tweet->oauth_token('KvfevhjwkKJvcvhenvalidkeyjwkKJVCwKvfevhjwkKJvcvhejwkKJVCweVU');
$tweet->oauth_token_secret('KvfevhjwkKJvcvhejwnvalidkeykKJVCwe');
my $res = $tweet->update_status($message, $ENV{DOCUMENT_ROOT}.$li);
Jimtweet is a free module I found online (I forget where), but I had to modify it to do the image upload. It follows below.
$message is the text status message to send
$li contains the local path to the file I want to upload. This is a file local on the server. $ENV{DOCUMENT_ROOT} contains the server path to the public HTML files on my website.
$res contains a JSON reply from twitter you can look at if you need to.
If you want to use this, cut & paste everything from 'package Jimtweet;' and on into a file called Jimtweet.pm which the above code should use. If your perl installation can't find the module try adding the line use lib "/home/your/path/to/jimtweet/directory;" before the use Jimtweet; line.
Twitter requires an image to be uploaded, it then returns a media_id, then you include the media_id in a regular message you want to post. See Jimtweet package below:
package Jimtweet;
#####JimTweet 0.1 By James Januszka 2010
#####Email:jimjanuszka#gmail.com
#####Twitter:#jamesjanuszka
#####Modifications by John Bell to include image upload. Twitter: #BellUkcbajr
use strict;
use warnings;
use LWP;
use HTTP::Headers;
use URI::Encode qw(uri_encode);
use URI::Escape qw(uri_escape);
use Digest::HMAC_SHA1;
####Constructor####
sub new {
my $self={};
$self->{OAUTH_VERSION}=uri_escape("1.0");
$self->{OAUTH_SIGNATURE_METHOD}=uri_escape("HMAC-SHA1");
$self->{OAUTH_TIMESTAMP}=undef;
$self->{OAUTH_NONCE}=undef;
$self->{AGENT}="jimtweet/0.1";
#####################
#Use this for status updates
$self->{URLx}="https://api.twitter.com/1.1/statuses/update.json";
#####################
#Use this for image upload
$self->{URL}="https://upload.twitter.com/1.1/media/upload.json";
$self->{BROWSER}=LWP::UserAgent->new(agent =>$self->{AGENT});
$self->{CONSUMER_KEY}=undef;
$self->{CONSUMER_SECRET}=undef;
$self->{OAUTH_TOKEN}=undef;
$self->{OAUTH_TOKEN_SECRET}=undef;
$self->{STATUS}=undef;
$self->{MEDIAurl}=undef;
bless($self);
return $self;
}
sub consumer_key{
my $self=shift;
if (#_) { $self->{CONSUMER_KEY}=uri_escape(shift) }
return $self->{CONSUMER_KEY};
}
sub consumer_secret{
my $self = shift;
if (#_) { $self->{CONSUMER_SECRET}=uri_escape(shift) }
return $self->{CONSUMER_SECRET};
}
sub oauth_token{
my $self = shift;
if (#_) { $self->{OAUTH_TOKEN}=uri_escape(shift) }
return $self->{OAUTH_TOKEN};
}
sub oauth_token_secret{
my $self = shift;
if (#_) { $self->{OAUTH_TOKEN_SECRET}=uri_escape(shift) }
return $self->{OAUTH_TOKEN_SECRET};
}
sub update_status(#){
sleep(2);
my $self = shift;
if (#_) { $self->{STATUS}=uri_escape(shift) }
if (#_) { $self->{MEDIAurl}=shift }
#Got parameters. Now create the POST to upload an image
my $seconds = time();
$self->{OAUTH_TIMESTAMP}=uri_escape($seconds);
$self->{OAUTH_NONCE}=$self->{OAUTH_TIMESTAMP};
#####################
#Use this for uploads. Parameters have to be in alphabetical order!
my $query=qq(oauth_consumer_key=$self->{CONSUMER_KEY}&oauth_nonce=$self->{OAUTH_NONCE}&oauth_signature_method=$self->{OAUTH_SIGNATURE_METHOD}&oauth_timestamp=$self->{OAUTH_TIMESTAMP}&oauth_token=$self->{OAUTH_TOKEN}&oauth_version=$self->{OAUTH_VERSION});
my $sig="POST&";
$sig .=uri_encode($self->{URL},1);
$sig .="&";
$sig .=uri_encode($query,1);
my $sig_key=$self->{CONSUMER_SECRET};
$sig_key .="&";
$sig_key .=$self->{OAUTH_TOKEN_SECRET};
my $hmac = Digest::HMAC_SHA1->new($sig_key);
$hmac->add($sig);
my $oauth_signature_base64=$hmac->b64digest;
$oauth_signature_base64 .="=";
my $utf8_oauth_signature_base64=uri_escape($oauth_signature_base64);
my $header=qq(OAuth oauth_nonce="$self->{OAUTH_NONCE}", oauth_signature_method="$self->{OAUTH_SIGNATURE_METHOD}", oauth_timestamp="$self->{OAUTH_TIMESTAMP}", oauth_consumer_key="$self->{CONSUMER_KEY}", oauth_token="$self->{OAUTH_TOKEN}", oauth_signature="$utf8_oauth_signature_base64", oauth_version="$self->{OAUTH_VERSION}");
my $res = $self->{BROWSER}->post(
$self->{URL},
'Authorization' => $header,
'content-type' => 'form-data',
'Content' => [ media => ["$self->{MEDIAurl}"] ]
);
use JSON;
my $response = decode_json ($res->content);
my $media_id = $response->{'media_id'};
$seconds = time();
$self->{OAUTH_TIMESTAMP}=uri_escape($seconds);
$self->{OAUTH_NONCE}=$self->{OAUTH_TIMESTAMP}
my $queryx=qq(media_ids=$media_id&oauth_consumer_key=$self->{CONSUMER_KEY}&oauth_nonce=$self->{OAUTH_NONCE}&oauth_signature_method=$self->{OAUTH_SIGNATURE_METHOD}&oauth_timestamp=$self->{OAUTH_TIMESTAMP}&oauth_token=$self->{OAUTH_TOKEN}&oauth_version=$self->{OAUTH_VERSION}&status=$self->{STATUS});
my $sigx="POST&";
$sigx .=uri_encode($self->{URLx},1);
$sigx .="&";
$sigx .=uri_encode($queryx,1);
my $hmacx = Digest::HMAC_SHA1->new($sig_key);
$hmacx->add($sigx);
my $oauth_signature_base64x=$hmacx->b64digest;
$oauth_signature_base64x .="=";
my $utf8_oauth_signature_base64x=uri_escape($oauth_signature_base64x);
my $headerx=qq(OAuth oauth_nonce="$self->{OAUTH_NONCE}", oauth_signature_method="$self->{OAUTH_SIGNATURE_METHOD}", oauth_timestamp="$self->{OAUTH_TIMESTAMP}", oauth_consumer_key="$self->{CONSUMER_KEY}", oauth_token="$self->{OAUTH_TOKEN}", oauth_signature="$utf8_oauth_signature_base64x", oauth_version="$self->{OAUTH_VERSION}");
#And done generating content. Now to POST to twitter.
$res = $self->{BROWSER}->post(
$self->{URLx},
'Authorization' => $headerx,
'content-type' => 'application/x-www-form-urlencoded',
'Content' => qq(media_ids=$media_id&status=$self->{STATUS})
);
return $res;
}
####Footer####
1; #so the require or use succeeds

WWW::Mechanize text field issue

I'm trying to submit a form by post method using WWW::Mechanize perl module.
use WWW::Mechanize;
my $mech = WWW::Mechanize->new();
...
$mech->get($url);
...
my $response = $mech->submit_form(
form_name => $name,
fields => {
$field_name => $field_value
},
button => 'Button'
);
$field_name is generally speaking a text field (though the type is not specified explicitly in the form), which has a preset value.
$field_name => $field_value in $mech->submit_form on whatever reason does not replace the value, instead $field_value is added into the form after the original value:
{submitted_field_value} = {original_value},{provided_value}
How to replace {original_value} with {provided_value} in the form to be submitted ?
What happens if you add this single line to your code before calling $mech->submit_form():
$mech->field( $name, [$field_value], 1 );
This makes sure that the first value is added, or overwritten if it already exists.
1 is the number parameter (or position index)
See the documentation of WWW::Mechanize:
$mech->field( $name, \#values, $number )
Given the name of a field, set its value to the value specified. [...]
The optional $number parameter is used to distinguish between two
fields with the same name. The fields are numbered from 1.
It's important to remember WWW::Mechanize is better thought of as a 'headless browser' as opposed to say LWP or curl, which only handle all the fiddly bits of http requests for you. Mech keeps its state as you do things.
You'll need to get the form by using $mech->forms or something similar (its best to decide from the documentation. I mean there so many ways to do it.), and then set the input field you want to change, using the field methods.
I guess the basic way to do this comes out as so:
$mech->form_name($name);
$mech->field($field_name, $field_value);
my $response = $mech->click('Button');
Should work. I believe it will also work if you get the field and directly use that (ie my $field = $mech->form_name($name); then use $field methods instead of $mech.
I managed to make it working at my will. Thanks Timbus and knb for your suggestions. Though my case may not be completely general (I know the preset value) but I'd share what I've found (by trails & errors).
my $mech = WWW::Mechanize->new();
$mech->get($url);
$mech->form_name( $name );
my $fields = $mech->form_name($name);
foreach my $k ( #{$fields->{inputs}}){
if ($k->{value} eq $default_value){
$k->{value}=$field_value;
}
}
my $response = $mech->click('Button_name');

Perl -- 'Not a HASH reference' error when using JSON::RPC::Client

I'm a newbie in Perl.
I have a JSON-RPC server running at http://localhost:19000 and I need to call checkEmail() method.
use JSON::RPC::Client;
my $client = new JSON::RPC::Client;
my $url = 'http://localhost:19000';
my $callobj = {
method => 'checkEmail',
params => [ 'rprikhodchenko#gmail.com' ],
};
my $res = $client->call($url, $callobj);
if($res) {
if ($res->is_error) {
print "Error : ", $res->error_message;
}
else {
print $res->result;
}
}
else {
print $client->status_line;
}
When I try to launch it it tells following:
perl ./check_ac.pl
Not a HASH reference at /usr/local/share/perl/5.10.1/JSON/RPC/Client.pm line 193.
UPD:
Full stack-trace:
perl -MCarp::Always ./check_ac.pl
Not a HASH reference at /usr/local/share/perl/5.10.1/JSON/RPC/Client.pm line 193
JSON::RPC::ReturnObject::new('JSON::RPC::ReturnObject', 'HTTP::Response=HASH(0x9938d48)', 'JSON=SCALAR(0x96f1518)') called at /usr/local/share/perl/5.10.1/JSON/RPC/Client.pm line 118
JSON::RPC::Client::call('JSON::RPC::Client=HASH(0x944a818)', 'http://localhost:19000', 'HASH(0x96f1578)') called at ./check_ac.pl line 11
This error means that your JSON-RPC server is not actually one, inasmuch as it does not satisfy requirement 7.3. The error is triggered when JSON::RPC::Client assumes the document returned by the JSON-RPC service is well-formed (i.e., a JSON Object), and this assumptions turns out to have been in error. A bug report to the author of JSON::RPC::Client would be an appropriate way to request better error messaging.
I would attack this sort of problem by finding out what the server was returning that was causing JSON::RPC::Client to choke. Unfortunately, JRC fails to provide adequate hookpoints for finding this out, so you'll have to be a little bit tricky.
I don't like editing external libraries, so I recommend an extend-and-override approach to instrumenting traffic with the JSON-RPC server. Something like this (in check_ac.pl):
use Data::Dumper qw();
package JSON::RPC::InstrumentedClient;
use base 'JSON::RPC::Client';
# This would be better done with Module::Install, but I'm limiting dependencies today.
sub _get {
my ($self, #args) = #_;
return $self->_dump_response($self->SUPER::_get(#args));
}
sub _post {
my ($self, #args) = #_;
return $self->_dump_response($self->SUPER::_post(#args));
}
sub _dump_response {
my ($self, $response) = #_;
warn Data::Dumper::Dump([$response->decoded_content], [qw(content)]);
return $response;
}
package main;
my $client = JSON::RPC::InstrumentedClient->new();
my $url = 'http://localhost:19000';
... # rest of check_ac.pl
This wraps the calls to _get and _post that JSON::RPC::Client makes internally in such a way as to let you examine what the web server actually said in response to the request we made. The above code dumps the text content of the page; this might not be the right thing in your case and will blow up if an error is encountered. It's a debugging aid only, to help you figure out from the client code side what is wrong with the server.
That's enough caveats for now, I think. Good luck.
It seems to be a bug in method new of JSON::RPC::ReturnObject.
sub new {
my ($class, $obj, $json) = #_;
my $content = ( $json || JSON->new->utf8 )->decode( $obj->content );
#...
# line 193
$content->{error} ? $self->is_success(0) : $self->is_success(1);
#...
}
$content's value will be something returned from a JSON::decode() call. But looking at the documentation, it seems that JSON->decode() returns a scalar which could be a number, a string, an array reference, or a hash reference.
Unfortunately, JSON::RPC::ReturnObject->new() doesn't check what sort of thing JSON->decode() returned before trying to access it as a hashref. Given your error, I'm going to go ahead and assume what it got in your case was not one. :-)
I don't know if there's a way to force a fix from your code. I'd recommend contacting the author and letting him know about the issue, and/or filing a bug.

How can I log in to YouTube using Perl?

I am trying to write a Perl script to connect to me YouTube account but it doesnt seem to work. Basically I just want to connect to my account but apparently it is not working. I don't even have an idea on how I could debug this! Maybe it is something related to https protocol?
Please enlighten me! Thanks in advance.
use HTTP::Request::Common;
use LWP::UserAgent;
use strict;
my $login="test";
my $pass = "test";
my $res = "";
my $ua = "";
# Create user agent, make it look like FireFox and store cookies
$ua = LWP::UserAgent->new;
$ua->agent("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20051213 Firefox/1.0.7");
$ua->cookie_jar ( {} );
# Request login page
$res = $ua->request(GET "https://www.google.com/accounts/ServiceLogin?service=youtube&hl=en_US&passive=true&ltmpl=sso&uilel=3&continue=http%3A//www.youtube.com/signup%3Fhl%3Den_US%26warned%3D%26nomobiletemp%3D1%26next%3D/index");
die("ERROR1: GET http://www.youtube.com/login\n") unless ($res->is_success);
# Now we login with our user/pass
$res = $ua->request(
POST "https://www.google.com/accounts/ServiceLoginAuth?service=youtube",
Referer => "http://www.youtube.com/login",
Content_Type => "application/x-www-form-urlencoded",
Content => [
currentform => "login",
next => "/index",
username => $login,
password => $pass,
action_login => "Log+In"
]
);
# YouTube redirects (302) to a new page when login is success
# and returns OK (200) if the login failed.
#die("ERROR: Login Failed\n") unless ($res->is_redirect());
print $res->content;
what i am doing is learning the web features of perl, so i dont want to use any library except wwwlib or mechanize to get the job done.
how can i just connect to my account using a perl script? this is my objective for now
hope someone can post a script or correct mine.
thank you guys for you help.
i am testing Webscarab now..
What data are you trying to grab? Why not just using an existing implementation like WebService::YouTube
Some comments on your code: I always avoided the shortcut $ua->request(GET/POST) method since I always ended up needing more flexibility that only the use of HTTP::Request and HTTP::Response allowed. I always felt the code was cleaner that way too.
Why is your code not working? Who knows.
Make sure your cookiejar is adding your cookies to the outgoing HTTP::Request. I'd suggest dumping all your headers when you do it in a browser and compare with the headers and data that libwww is sending. There may be some additional fields that they are checking for that vary for every hit. They may be checking for your UserAgent string. If you are just looking to learn libwww I'd suggest using a different site as a target as I'm sure YouTube has all sort of anti-scripting hardening.
Are you using YouTube's stable documented API?
Use an HTTP proxy such as WebScarab to watch the data flow.
Trey's suggestion to use somebody else's CPAN package for the mechanics is a good idea too.
Right right by and large, what you want to do is define a cookiejar for most of these websites that have a redirection login. This is what the package has done. Also the package tunes a lot of the lookups and scrapes based on the youtube spec.
Ajax content for example will be rough since its not there when your scraping
You just picked a somewhat rough page to start out with.
Enjoy
I'm actually working on this issue myself. Before, I would suggest read over this the API guide from Google as a good starting reference. If I'm reading it correctly, one begins with passing user credentials through a REST interface to get a Authentication Token. To handle that, I'm using the following:
sub getToken {
my %parms = #_;
my $response = LWP::UserAgent->new->post(
'https://www.google.com/youtube/accounts/ClientLogin',
[
Email => $parms{'username'},
Passwd => $parms{'password'},
service => "youtube",
source => "<<Your Value Here>>",
]
);
my $content = $response->content;
my ($auth) = $content =~ /^Auth=(.*)YouTubeUser(.*)$/msg
or die "Unable to authenticate?\n";
my ($user) = $content =~ /YouTubeUser=(.*)$/msg
or die "Could not extract user name from response string. ";
return ($auth, $user);
}
And I call that from the main part of my program as such:
## Get $AuthToken
my ($AuthToken, $GoogleUserName) = getToken((
username => $email, password => $password
));
Once I have these two things -- $AuthToken and $GoogleUserName, I'm still testing the LWP Post. I'm still writing this unit:
sub test {
my %parms = #_;
## Copy file contents. Use, foy's three param open method.
my $fileSize = -s $parms{'File'};
open(VideoFile, '<', "$parms{'File'}") or die "Can't open $parms{'File'}.";
binmode VideoFile;
read(VideoFile, my $fileContents, $fileSize) or die "Can't read $parms{'File'}";
close VideoFile;
my $r = LWP::UserAgent->new->post(
"http://uploads.gdata.youtube.com/feeds/api/users/$parms{'user'}/uploads",
[
Host => "uploads.gdata.youtube.com",
'Authorization' => "AuthSub token=\"$parms{'auth'}\"",
'GData-Version' => "2",
'X-GData-Key' => "key=$YouTubeDeveloperKey",
'Slug' => "$parms{'File'}",
'Content-Type' => "multipart/related; boundary=\"<boundary_string>\"",
'Content-Length' => "<content_length>",
'video_content_type'=> "video/wmv",
'Connection' => "close",
'Content' => $fileContents
]
);
print Dumper(\$r->content)
}
And that is called as
&test((auth=>$Auth, user=>$user, File=>'test.wmv'));