Perl + recursive subroutine + accessing variable defined outside of subroutine - perl

Question

I am pulling bitbucket repo list using Perl. The response from bitbucket will contain only 10 repositories and a marker for the next page where there will be another 10 repos and so on ... (they call it paging response)
So, I wrote a recursive subroutine which calls itself if the next page marker is present. This will continue until it reaches the last page.
Here is my code:
#!/usr/bin/perl
use warnings;
use strict;
use Data::Dumper;
use LWP::UserAgent;
use JSON;
my #array;
recursive("my_bitbucket_url");
foreach ( #array ) { print $_."\n"; }
sub recursive
{
my $url = $_[0];
### here goes my LWP::UserAgent code which connects to bitbucket and pulls back the response in a JSON as $response->decoded_content
### hence, removing this code for brevity
my $hash = decode_json $response->decoded_content;
#print Dumper ($hash);
foreach my $a ( #{$hash->{values}} )
{
push #array, $a->{links}->{self}->{href};
}
if ( defined $hash->{next})
{
print "Next page Exists \n";
print "Recursing with $hash->{next} \n";
recursive( $hash->{next} );
}
else
{
print "Last page reached. No more recursion \n"
}
}
Now, my code works fine and it lists all the repos.
Question:
I am not sure about the way I have used the variable my #array; above. I have defined it outside the subroutine, However, I am accessing it directly from the subroutine. Somehow, I feel this is not right.
So, how to append to an array using a recursive subroutine in such cases. Does my code obey Perl ethics or is it something really absurd (yet correct because it works) ?
UPDATE
After following suggestions from #ikegami, #Sobrique and #Hynek -Pichi- Vychodil, I have come with below code which uses while loop and avoids recusrsion.
Here is my thought process:
Define an array #array.
Call the subroutine call_url with initial bitbucket URL and save the response in $hash
Check the $hash for the next page marker
If next page marker exists, then push the elements to #array and call call_url with the new marker. This will be done with the while loop.
If the next page marker does NOT exists, then push the elements to #array. Period.
Print #array content.
And here is my code:
my #array;
my $hash = call_url("my_bitbucket_url ");
if (defined $hash->{next})
{
while (defined $hash->{next})
{
foreach my $a ( #{$hash->{values}} )
{
push #array, $a->{links}->{self}->{href};
}
$hash = call_url($hash->{next});
}
}
foreach my $a ( #{$hash->{values}} )
{
push #array, $a->{links}->{self}->{href};
}
foreach (#array) { print $_."\n"; }
sub call_url
{
### here goes my LWP::UserAgent code which connects to bitbucket and pulls back the response in a JSON as $response->decoded_content
### hence, removing this code for brevity
my $hash = decode_json $response->decoded_content;
#print Dumper ($hash);
return $hash;
}
Would definitely like to hear whether this looks OK or is there still a room for improvement.

Answer 1

Using global variables to return values demonstrates high coupling, something to be avoided.
You're asking if the following is acceptable:
my $sum;
sum(4, 5);
print("$sum\n");
sub sum {
my ($x, $y) = #_;
$sum = $x + $y;
}
The fact that the sub is recursive is completely irrelevant; it just makes your example larger.
Problem fixed:
sub recursive
{
my $url = $_[0];
my #array;
my $hash = ...;
foreach my $a ( #{$hash->{values}} )
{
push #array, $a->{links}->{self}->{href};
}
if ( defined $hash->{next})
{
print "Next page Exists \n";
print "Recursing with $hash->{next} \n";
push #array, recursive( $hash->{next} );
}
else
{
print "Last page reached. No more recursion \n"
}
return #array;
}
{
my #array = recursive("my_bitbucket_url");
foreach ( #array ) { print $_."\n"; }
}
Recursion removed:
sub recursive
{
my $url = $_[0];
my #array;
while (defined($url)) {
my $hash = ...;
foreach my $a ( #{$hash->{values}} )
{
push #array, $a->{links}->{self}->{href};
}
$url = $hash->{next};
if ( defined $url)
{
print "Next page Exists \n";
print "Recursing with $url\n";
}
else
{
print "Last page reached. No more recursion \n"
}
}
return #array;
}
{
my #array = recursive("my_bitbucket_url");
foreach ( #array ) { print $_."\n"; }
}
Clean up of the latest code you posted:
my $url = "my_bitbucket_url";
my #array;
while ($url) {
my $hash = call_url($url);
for my $value ( #{ $hash->{values} } ) {
push #array, $value->{links}{self}{href};
}
$url = $hash->{next};
}
print("$_\n") for #array;

Answer 2

Yes, using a global variable is bad habit even it is lexical scoped variable.
Each recursive code can be rewritten into its imperative loop version and vice versa. It is because all of this is implemented on CPU which doesn't know anything about recursion at all. Thre are only jumps. All calls and returns are just jumps with some stack manipulation so you can rewrite your recursion algorithm into loop. If it is not obvious and simple as in this case you can even emulate stack and behaviour as it is done in your favourite language interpreter or compiler. In this case it's very simple:
my #array = with_loop("my_bitbucket_url");
foreach ( #array ) { print $_."\n"; }
sub with_loop
{
my $url = $_[0];
my #array;
while(1)
{
### here goes my LWP::UserAgent code which connects to bitbucket and
### pulls back the response in a JSON as $response->decoded_content
### hence, removing this code for brevity
my $hash = decode_json $response->decoded_content;
#print Dumper ($hash);
foreach my $a ( #{$hash->{values}} )
{
push #array, $a->{links}->{self}->{href};
}
unless ( defined $hash->{next})
{
print "Last page reached. No more recursion \n";
last
};
print "Next page Exists \n";
print "Recursing with $hash->{next} \n";
$url = $hash->{next};
};
return #array;
}
But when you would like to stick with recursion you can but it is a little bit more tricky. First of all, there is not tail call optimization so you don't have to try write tail call code as your original version does. So you can do this:
my #array = recursion("my_bitbucket_url");
foreach ( #array ) { print $_."\n"; }
sub recursion
{
my $url = $_[0];
### here goes my LWP::UserAgent code which connects to bitbucket and
### pulls back the response in a JSON as $response->decoded_content
### hence, removing this code for brevity
my $hash = decode_json $response->decoded_content;
#print Dumper ($hash);
# this map version is same as foreach with push but more perlish
my #array = map $_->{links}->{self}->{href}, #{$hash->{values}};
if (defined $hash->{next})
{
print "Next page Exists \n";
print "Recursing with $hash->{next} \n";
push #array, recursive( $hash->{next} );
}
else
{
print "Last page reached. No more recursion \n"
}
return #array;
}
But this version is not very efficient so there is way how to write tail call recursive version in perl which is a little bit tricky.
my #array = tail_recursive("my_bitbucket_url");
foreach ( #array ) { print $_."\n"; }
sub tail_recursive
{
my $url = $_[0];
my #array;
return tail_recursive_inner($url, \#array);
# url is mutable parameter
}
sub tail_recursive_inner
{
my $url = $_[0];
my $array = $_[1];
# $array is reference to accumulator #array
# from tail_recursive function
### here goes my LWP::UserAgent code which connects to bitbucket and
### pulls back the response in a JSON as $response->decoded_content
### hence, removing this code for brevity
my $hash = decode_json $response->decoded_content;
#print Dumper ($hash);
foreach my $a ( #{$hash->{values}} )
{
push #$array, $a->{links}->{self}->{href};
}
if (defined $hash->{next})
{
print "Next page Exists \n";
print "Recursing with $hash->{next} \n";
# first parameter is mutable so its OK to assign
$_[0] = $hash->{next};
goto &tail_recursive_inner;
}
else
{
print "Last page reached. No more recursion \n"
}
return #$array;
}
And if you are interested in some real perl trickery
print $_."\n" for tricky_tail_recursion("my_bitbucket_url");
sub tricky_tail_recursion {
my $url = shift;
### here goes my LWP::UserAgent code which connects to bitbucket and
### pulls back the response in a JSON as $response->decoded_content
### hence, removing this code for brevity
my $hash = decode_json $response->decoded_content;
#print Dumper ($hash);
push #_, $_->{links}->{self}->{href} for #{$hash->{values}};
if (defined $hash->{next}) {
print "Next page Exists \n";
print "Recursing with $hash->{next} \n";
unshift #_, $hash->{next};
goto &tricky_tail_recursion;
} else {
print "Last page reached. No more recursion \n"
};
return #_;
}
See also: LWP::UserAgent docs.

Answer 3

A variable defined outside any closures is available to the whole program. It works fine, there's nothing to worry about. Some might call it 'bad style' in certain cases (mostly around program length and action at distance) but that's not a hard constraint.
I'm not sure I necessarily see the advantage of recursion here though - your problem doesn't seem to warrant it. That's not a problem per-se, but it can be a little confusing for future maintenance programmers ;).
I'd be thinking something along the lines of (non recursive):
my $url = "my_bitbucket_url";
while ( defined $url ) {
##LWP Stuff;
my $hash = decode_json $response->decoded_content;
foreach my $element ( #{ $hash->{values} } ) {
print join( "\n", #{ $element->{links}->{self}->{href} } ), "\n";
}
$url = $hash->{next}; #undef if it doesn't exist, so loop breaks.
}