Finding broken links with Selenium Remote Driver - perl

Question

I have a site with login and i want to test all links present in that site.
I tried with finding links and click on each to verify with Selenium Remote Driver. But one problem i have is coming back to previous URL and selecting next link. This testing should be recursive.
How can we do this with Selenium Remote Driver?
Following program i tried to check broken links
sub traverse {
my ($self) = #_;
my $links = find_links("//a");
foreach my $index (1..$#$links) {
my $url = $links->[$index]->get_attribute('href');
my $result = $links->[$index]->click();
if ($result) {
traverse();
} else {
print "url is broken $url\n";
}
}
}

Answer 1

I know it's possible to do in C# by checking the returned status code. So you don't actually click on the link, but you are retrieving the header of the response that link is going to give. In this header you can find the HTTP Status Code which you can check to see if the link is giving a valid response or not. Plus you're not leaving the current site!
In C#, a possible method to get the status code will look like this (The checking of the HTTP status code is not included):
private static HttpStatusCode GetStatusCode(string url)
{
var result = default(HttpStatusCode);
var request = WebRequest.Create(url);
request.Method = "HEAD";
HttpWebResponse response;
try {
response = request.GetResponse() as HttpWebResponse;
} catch (WebException) {
return HttpStatusCode.NotFound;
}
if (response != null)
{
result = response.StatusCode;
response.Close();
response.Dispose();
}
return result;
}
Altough this is no Perl code, I hope this helps

Answer 2

Why are you not trying to use some tool, because your site can has over 9000+ urls, it's a lot of time and job, you can use Xenu
Install
In option check use Cookie
Run IE and login thorugh it
Run Xenu
P.S. To test privete part of your site, you must login thorugh IE because Xenu uses only IE cookie

Answer 3

Hmm, I've crossed this bridge before and here is how I solved it. Now I should say that I crossed this bridge before WebDriver :) so this is using WWW::Selenium instead of S:R:D but the concept is the same and still applies.
One of the most tedious tasks, IMO, for a test engineer, is manually verifying links. We can automate most of the process and as long as we have the URL's for where we are expected to land after clicking the link, we can verify this functionality using Selenium and a little bit of JS.
In the below example we first navigate to our desired website and then use Selenium's getEval() function to execute JavaScript that gathers all the links on the page (anchors) and saves them in a comma separated list. This list then gets split and pushed into an array. We then iterate through the list of links in the array clicking on each one and then navigating back to the starting page using go_back.
use strict;
use warnings;
use Time::HiRes qw(sleep);
use Test::WWW::Selenium;
use Test::More "no_plan";
my $sel = Test::WWW::Selenium->new( host => "localhost",
port => 4444,
browser => "*iexplore",
browser_url => "http://www.google.com/");
$sel->open_ok("/", "true");
$sel->set_speed("1000");
my $javascript = "var allLinks = this.browserbot.getCurrentWindow().document.getElementsByTagName('a');
var separator = ',';
var all_links_texts = '';
for(var i = 0; i < allLinks.length; i++) {
all_links_texts = all_links_texts+separator+allLinks[i].href;
}
all_links_texts;";
# Get all of the links in the page and, using a comma to separate each one, add them to the all_links_texts var.
my $link_list = $sel->get_eval($javascript);
my #link_array = split /,/ , $link_list;
my $count = 0;
# Click on each link contained in the array and then go_back
# You can add other logic here like capture and store a screenshot for example
foreach my $link_name (#link_array) {
unless ($link_name =~ /^$/){
$sel->click_ok("css=a[href $= $link_name]");
$sel->wait_for_page_to_load("30000");
print "Clicked Link href: $link_name \n";
$sel->go_back();
$count++;
}
}
print "Clicked $count URL's";
pass;
This can be easily modified to do much more than just click on the links. And of course nothing beats a good pair of eyes on the intended landing pages for the links clicked. Implementing a similar solution in your organization might ease with the manual testing. Here is how I have done it in the past:
Not everything can be automated, but we can certainly make it much easier to review large amounts of links. The above logic can be easily extended to capture a screen shot and add it to a queue of "to be reviewed" images. These properly tagged [by the software] images are what you use in the final phase of the test; visual verification phase.
With this approach you'll know right away if a link is broken or not (assuming you update the logic above to also include this, again this example can be easily extended to include that functionality). As well you will have the capability of visually verifying the screen shots of the intended link landing pages.
I actually have a blog post about this very same issue here: get all links and click on each one
Hope that helps.