Downloading a image via web scraping with a Perl script - perl

Question

I'm a noob in perl, trying to download a IMDB movie poster image via perl script with the help of Mechanize framework. I'm not getting the 'id' attribute for 'td' tags so that I can find the specific place for the image. This is how the HTML of the image portion of IMDB page looks like:
<table id="title-overview-widget-layout" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td id="img_primary" rowspan="2">
<div class="image">
<a href="/media/rm419297536/tt2338151?ref_=tt_ov_i">
<img width="214" height="317" itemprop="image" src="http://ia.media-imdb.com/images/M/MV5BMTYzOTE2NjkxN15BMl5BanBnXkFtZTgwMDgzMTg0MzE#._V1_SY317_CR2,0,214,317_AL_.jpg" title="PK (2014) Poster" alt="PK (2014) Poster">
</a>
</div>
<div class="pro-title-link text-center">
</td>
<td id="overview-top">
</tr>
<tr>
</tbody>
</table>
And here is the perl script I'm trying to download with:
use strict;
use warnings;
use WWW::Mechanize;
use HTML::TokeParser;
#create a new instance of mechanize
my $agent = WWW::Mechanize->new();
#get the page we want.
$agent->get("http://www.imdb.com/title/tt2338151/");
#supply a reference to that page to TokeParser
my $stream = HTML::TokeParser->new(\$agent->{content});
my $c = 0;#to store the count of images and give the images names
#loop through all the td's
while (my $tag1 = $stream->get_tag("td")) {
$tag1->[1]->{id} ||= 'none';
my $asd = $tag1->[1]->{id};
print "$asd\n"; #shows none for all of the td's
if ($asd && $asd eq 'img_primary') {
while(my $tag = $stream->get_tag("div"))
{
# $tag will contain this array => [$tag, $attr, $attrseq, $text]
#get the class of the div tag from attr
my $cls = $tag->[1]{class};
#we're looking for div's with the class gallery-img2
if($cls && $cls eq "image") {
#get the content of the src tag
my $image = $stream->get_tag('img')->[1]{src};
#create a new mechanize to download the image
my $imgDown = WWW::Mechanize->new();
#give the image url and the local path to mechanize
$imgDown->get($image, ":content_file" => ".//image".$c.".jpg");
#update the count
$c++;
}
}
}
}
print "Total images scraped $c\n";
Any help will be much appropriated.

Answer 1

When JavaScript is involved, it's best to use a real browser to visit pages and query their contents.
You can do this with Selenium::Remote::Driver.