Passing arrays in Perl - perl

All right, I'll try to explain what I've done so far. I'm using a Parellel::ForkManager to grab data from an array of URLs, which is then stored in variables (value1, value2, value3).
I then collect the data from all of those processes, and display the data with $pm->run_on_finish.
#...
my $pm = new Parallel::ForkManager(10);
$pm->run_on_finish (
sub {
my #info = #{$data_structure_reference};
print $info[0];
print $info[1];
print $info[2];
}
);
for my $var (#urls) {
$pm->start and next;
#...
#returned = &something($var);
#...
$pm->finish(0, \#returned);
}
sub something {
#... getting data from each URL and storing it in variables
my #array = (
$value1,
$value2,
$value3
);
return #array;
}
Now, what I want to do, is to pass an array, #value4, as well, and then only display that data if there is something in the array. So, I want it to look like this:
sub something {
#... getting data from each URL and storing it in variables
my #array = (
$value1,
$value2,
$value3,
#value4
);
return #array;
}
And then I want it to print that array, only if there is something in it.
Unfortunately, I'm not entirely sure how to go about doing that.

I assume that what you are asking is how to return an array along with the three scalars returned from the something() sub, and print it?
I also assume that those three scalars are what's referred to as being in #info.
The simplest way seems to me to be to simply tack them to the end of the array you return, use the three first values, and if there's anything left, print that too.
$pm->run_on_finish (
sub {
my #info = #{$data_structure_reference};
print splice #info, 0, 3;
print #info if (#info);
}
);
sub something {
return (
$value1,
$value2,
$value3,
#value4
);
}
As you'll notice, you do not need to fill a dummy array for the return value, simply return values inside the parens. You do not need to dereference the array, since you can use the #info array straight up if you splice off the first three values.
I like it simple. If it works.

I've previously provided a simple solution to this problem. It can use threads (use threads;) or processes (use forks;).
use threads; # or: use forks;
use Thread::Queue qw( );
use constant NUM_WORKERS => 10;
my $request_q = Thread::Queue->new();
my $response_q = Thread::Queue->new();
# Create the workers.
for (1..NUM_WORKERS) {
(async {
while (defined(my $request = $request_q->dequeue())) {
$response_q->enqueue(process_request($request));
}
})->detach();
}
# Submit work to workers.
$request_q->enqueue(#requests);
# Signal the workers they are done.
$request_q->enqueue(undef) for 1..NUM_WORKERS;
# Collect the results.
my $working = NUM_WORKERS;
while ($working) {
my $result = $response_q->dequeue();
if (!defined($result)) {
--$working;
next;
}
process_response($result);
}
The work to be done in the children is done by process_request.
sub process_request {
# ...
return [
$value1,
$value2,
$value3,
\#value4,
];
}
The results are passed to process_response in the parent.
sub process_response {
my ($value1, $value2, $value3, $value4) = #{ $_[0] };
...
}

I am not totally sure what you are asking, but in order to pass multiple arrays to a function in Perl one must pass by reference.
my #array1 = (1, 2, 3);
my #array2 = ('a', 'b', 'c');
&fn(\#array1, \#array2);
In order to print an array only in the case when it has value, one must simply check that it has value and print it:
print "#array" if #array;
Though, the nice function of "#array" is that if #array has no value then "#array" evaluates to "". This allows you to reduce the previous statement to a simple:
print "#array"

Related

Extract data hash reference from perl array

I am new to perl. I am returning an array of references from a function. However I am lost as how I loop over the data.
sub whatever{
my %product;
my %resolution;
my #data = ();
push #data, \%product;
push #data, \%resolution;
return #data;
}
In the control sub module.
my #results = $whatever($dt_id);
$c->app->log->debug(Dumper(#results));
results
$VAR1 = {'IOP' => 'IOP'};
$VAR2 = {'4km' => '4km','9km' => '9km'};
I get the two hashes but how do I loop over them.
To return the two hashes seperately:
return (\%product, \%resolution);
To map over them:
my ($product, $resolution) = whatever(...);
for my $key (%$product) {
# do something with $product->{$key};
}
try this:
for my $hash_ref (#results) {
for my $key (keys %$hash_ref) {
say "key '$key' has value '$hash_ref->{$key}'";
}
}

Automatically call hash values that are subroutine references

I have a hash with a few values that are not scalar data but rather anonymous subroutines that return scalar data. I want to make this completely transparent to the part of the code that looks up values in the hash, so that it doesn't have to be aware that some of the hash values may be anonymous subroutines that return scalar data rather than just plain old scalar data.
To that effect, is there any way to have the anonymous subroutines executed when their keys are accessed, without using any special syntax? Here's a simplified example that illustrates the goal and the problem:
#!/usr/bin/perl
my %hash = (
key1 => "value1",
key2 => sub {
return "value2"; # In the real code, this value can differ
},
);
foreach my $key (sort keys %hash) {
print $hash{$key} . "\n";
}
The output I would like is:
perl ./test.pl
value1
value2
Instead, this is what I get:
perl ./test.pl
value1
CODE(0x7fb30282cfe0)
As noted by Oleg, it's possible to do this using various more or less arcane tricks like tie, overloading or magic variables. However, this would be both needlessly complicated and pointlessly obfuscated. As cool as such tricks are, using them in real code would be a mistake at least 99% of the time.
In practice, the simplest and cleanest solution is probably to write a helper subroutine that takes a scalar and, if it's a code reference, executes it and returns the result:
sub evaluate {
my $val = shift;
return $val->() if ref($val) eq 'CODE';
return $val; # otherwise
}
and use it like this:
foreach my $key (sort keys %hash) {
print evaluate($hash{$key}) . "\n";
}
I don't believe that the words that others have written in disapproval of the tie mechanism are warranted. None of the authors seem to properly understand how it works and what core library backup is available
Here's a tie example based on Tie::StdHash
If you tie a hash to the Tie::StdHash class then it works exactly as a normal hash. That means there's nothing left to write except for methods that you may want to override
In this case I've overridden TIEHASH so that I could specify the initialisation list in the same statement as the tie command, and FETCH, which calls the superclass's FETCH and then makes a call to it if it happens to be a subroutine reference
Your tied hash will work as normal except for the change that you have asked for. I hope it is obvious that there is no longer a direct way to retrieve a subroutine reference if you have stored it as a hash value. Such a value will always be replaced by the result of calling it without any parameters
SpecialHash.pm
package SpecialHash;
use Tie::Hash;
use base 'Tie::StdHash';
sub TIEHASH {
my $class = shift;
bless { #_ }, $class;
}
sub FETCH {
my $self = shift;
my $val = $self->SUPER::FETCH(#_);
ref $val eq 'CODE' ? $val->() : $val;
}
1;
main.pl
use strict;
use warnings 'all';
use SpecialHash;
tie my %hash, SpecialHash => (
key1 => "value1",
key2 => sub {
return "value2"; # In the real code, this value can differ
},
);
print "$hash{$_}\n" for sort keys %hash;
output
value1
value2
Update
It sounds like your real situation is with an existing hash that looks something like this
my %hash = (
a => {
key_a1 => 'value_a1',
key_a2 => sub { 'value_a2' },
},
b => {
key_b1 => sub { 'value_b1' },
key_b2 => 'value_b2',
},
);
Using tie on already-populated variables isn't so neat as tying then at the point of declaration and then inserting the values as the data must be copied to the tied object. However the way I have written the TIEHASH method in the SpecialHash class makes this simple to do in the tie statement
If possible, it would be much better to tie each hash before you put data into it and add it to the primary hash
This program ties every value of %hash that happens to be a hash reference. The core of this is the statement
tie %$val, SpecialHash => ( %$val )
which functions identically to
tie my %hash, SpecialHash => ( ... )
in the previous code but dereferences $val to make the syntax valid, and also uses the current contents of the hash as the initialisation data for the tied hash. That is how the data gets copied
After that there is just a couple of nested loops that dump the whole of %hash to verify that the ties are working
use strict;
use warnings 'all';
use SpecialHash;
my %hash = (
a => {
key_a1 => 'value_a1',
key_a2 => sub { 'value_a2' },
},
b => {
key_b1 => sub { 'value_b1' },
key_b2 => 'value_b2',
},
);
# Tie all the secondary hashes that are hash references
#
for my $val ( values %hash ) {
tie %$val, SpecialHash => ( %$val ) if ref $val eq 'HASH';
}
# Dump all the elements of the second-level hashes
#
for my $k ( sort keys %hash ) {
my $v = $hash{$k};
next unless ref $v eq 'HASH';
print "$k =>\n";
for my $kk ( sort keys %$v ) {
my $vv = $v->{$kk};
print " $kk => $v->{$kk}\n"
}
}
output
a =>
key_a1 => value_a1
key_a2 => value_a2
b =>
key_b1 => value_b1
key_b2 => value_b2
There's a feature called "magic" that allows code to be called when variables are accessed.
Adding magic to a variable greatly slows down access to that variable, but some are more expensive than others.
There's no need to make access to every element of the hash magical, just some values.
tie is an more expensive form of magic, and it's not needed here.
As such, the most efficient solution is the following:
use Time::HiRes qw( time );
use Variable::Magic qw( cast wizard );
{
my $wiz = wizard(
data => sub { my $code = $_[1]; $code },
get => sub { ${ $_[0] } = $_[1]->(); },
);
sub make_evaluator { cast($_[0], $wiz, $_[1]) }
}
my %hash;
$hash{key1} = 'value1';
make_evaluator($hash{key2}, sub { 'value2#'.time });
print("$hash{$_}\n") for qw( key1 key2 key2 );
Output:
value1
value2#1462548850.76715
value2#1462548850.76721
Other examples:
my %hash; make_evaluator($hash{key}, sub { ... });
my $hash; make_evaluator($hash->{$key}, sub { ... });
my $x; make_evaluator($x, sub { ... });
make_evaluator(my $x, sub { ... });
make_evaluator(..., sub { ... });
make_evaluator(..., \&some_sub);
You can also "fix up" an existing hash. In your hash-of-hashes scenario,
my $hoh = {
{
key1 => 'value1',
key2 => sub { ... },
...
},
...
);
for my $h (values(%$hoh)) {
for my $v (values(%$h)) {
if (ref($v) eq 'CODE') {
make_evaluator($v, $v);
}
}
}
Yes you can. You can either tie hash to implementation that will resolve coderefs to their return values or you can use blessed scalars as values with overloaded mehods for stringification, numification and whatever else context you want to resolve automatically.
One of perl's special features for just such a use case is tie. This allows you to attach object oriented style methods, to a scalar or hash.
It should be used with caution, because it can mean that your code is doing really strange things, in unexpected ways.
But as an example:
#!/usr/bin/env perl
package RandomScalar;
my $random_range = 10;
sub TIESCALAR {
my ( $class, $range ) = #_;
my $value = 0;
bless \$value, $class;
}
sub FETCH {
my ($self) = #_;
return rand($random_range);
}
sub STORE {
my ( $self, $range ) = #_;
$random_range = $range;
}
package main;
use strict;
use warnings;
tie my $random_var, 'RandomScalar', 5;
for ( 1 .. 10 ) {
print $random_var, "\n";
}
$random_var = 100;
for ( 1 .. 10 ) {
print $random_var, "\n";
}
As you can see - this lets you take an 'ordinary' scalar, and do fruity things with it. You can use a very similar mechanism with a hash - an example might be to do database lookups.
However, you also need to be quite cautious - because you're creating action at a distance by doing so. Future maintenance programmers might well not expect your $random_var to actually change each time you run it, and a value assignment to not actually 'set'.
It can be really useful for e.g. testing though, which is why I give an example.
In your example - you could potentially 'tie' the hash:
#!/usr/bin/env perl
package MagicHash;
sub TIEHASH {
my ($class) = #_;
my $self = {};
return bless $self, $class;
}
sub FETCH {
my ( $self, $key ) = #_;
if ( ref( $self->{$key} ) eq 'CODE' ) {
return $self->{$key}->();
}
else {
return $self->{$key};
}
}
sub STORE {
my ( $self, $key, $value ) = #_;
$self->{$key} = $value;
}
sub CLEAR {
my ($self) = #_;
$self = {};
}
sub FIRSTKEY {
my ($self) = #_;
my $null = keys %$self; #reset iterator
return each %$self;
}
sub NEXTKEY {
my ($self) = #_;
return each %$self;
}
package main;
use strict;
use warnings;
use Data::Dumper;
tie my %magic_hash, 'MagicHash';
%magic_hash = (
key1 => 2,
key2 => sub { return "beefcake" },
);
$magic_hash{random} = sub { return rand 10 };
foreach my $key ( keys %magic_hash ) {
print "$key => $magic_hash{$key}\n";
}
foreach my $key ( keys %magic_hash ) {
print "$key => $magic_hash{$key}\n";
}
foreach my $key ( keys %magic_hash ) {
print "$key => $magic_hash{$key}\n";
}
This is slightly less evil, because future maintenance programmers can use your 'hash' normally. But dynamic eval can shoot the unwary in the foot, so still - caution is advised.
And alternative is to do it 'proper' object oriented - create a 'storage object' that's ... basically like the above - only it creates an object, rather than using tie. This should be much clearer for long term usage, because you won't get unexpected behaviour. (It's an object doing magic, which is normal, not a hash that 'works funny').
You need to identify when a code ref is present, then execute it as an actual call:
foreach my $key (sort keys %hash) {
if (ref $hash{$key} eq 'CODE'){
print $hash{$key}->() . "\n";
}
else {
print "$hash{$key}\n";
}
}
Note that you may consider making all of the hash values subs (a true dispatch table) instead of having some that return non-coderefs and some that return refs.
However, if you define the hash as such, you don't have to do any special trickery when it comes time to use the hash. It calls the sub and returns the value directly when the key is looked up.
key2 => sub {
return "value2";
}->(),
No, not without some ancillary code. You are asking for a simple scalar value and a code reference to behave in the same way. The code that would do that is far from simple and also injects complexity between your hash and its use. You might find the following approach simpler and cleaner.
You can make all values code references, making the hash a dispatch table, for uniform invocation
my %hash = (
key1 => sub { return "value1" },
key2 => sub {
# carry on some processing ...
return "value2"; # In the real code, this value can differ
},
);
print $hash{$_}->() . "\n" for sort keys %hash;
But of course there is a minimal overhead to this approach.

Use of reference to elements in #_ to avoid duplicating code

Is it safe to take reference of elements of #_ in a subroutine in order to avoid duplicating code? I also wonder if the following is good practice or can be simplified. I have a subroutine mod_str that takes an option saying if a string argument should be modified in-place or not:
use feature qw(say);
use strict;
use warnings;
my $str = 'abc';
my $mstr = mod_str( $str, in_place => 0 );
say $mstr;
mod_str( $str, in_place => 1 );
say $str;
sub mod_str {
my %opt;
%opt = #_[1..$#_];
if ( $opt{in_place} ) {
$_[0] =~ s/a/A/g;
# .. do more stuff with $_[0]
return;
}
else {
my $str = $_[0];
$str =~ s/a/A/g;
# .. do more stuff with $str
return $str;
}
}
In order to avoid repeating/duplicating code in the if and else blocks above, I tried to improve mod_str:
sub mod_str {
my %opt;
%opt = #_[1..$#_];
my $ref;
my $str;
if ( $opt{in_place} ) {
$ref = \$_[0];
}
else {
$str = $_[0]; # make copy
$ref = \$str;
}
$$ref =~ s/a/A/g;
# .. do more stuff with $$ref
$opt{in_place} ? return : return $$ref;
}
The "in place" flag changes the function's interface to the point where it should be a new function. It will simplify the interface, testing, documentation and the internals to have two functions. Rather than having to parse arguments and have a big if/else block, the user has already made that choice for you.
Another way to look at it is the in_place option will always be set to a constant. Because it fundamentally changes how the function behaves, there's no sensible case where you'd write in_place => $flag.
Once you do that, the reuse becomes more obvious. Write one function to do the operation in place. Write another which calls that on a copy.
sub mod_str_in_place {
# ...Do work on $_[0]...
return;
}
sub mod_str {
my $str = $_[0]; # string is copied
mod_str_in_place($str);
return $str;
}
In the absence of the disgraced given I like using for as a topicalizer. This effectively aliases $_ to either $_[0] or the local copy depending on the value of the in_place hash element. It's directly comparable to your $ref but with aliases, and a lot cleaner
I see no reason to return a useless undef / () in the case that the string is modified in place; the subroutine may as well return the new value of the string. (I suspect the old value might be more useful, after the fashion of $x++, but that makes for uglier code!)
I'm not sure whether this is readable code to anyone but me, so comments are welcome!
use strict;
use warnings;
my $ss = 'abcabc';
printf "%s %s\n", mod_str($ss), $ss;
$ss = 'abcabc';
printf "%s %s\n", mod_str($ss, in_place => 1), $ss;
sub mod_str {
my ($copy, %opt) = #_;
for ( $opt{in_place} ? $_[0] : $copy ) {
s/a/A/g;
# .. do more stuff with $_
return $_;
}
}
output
AbcAbc abcabc
AbcAbc AbcAbc

Nested modular subroutines where the inner subroutine changes

I have a sub that reads a FASTA text file in chunks.
sub reader {
foreach my $line (<IN>) { # read line by line
chomp $line;
if ($line =~ m/^>/) { # if it's a title
&initiator($title, $seq) unless $firsttitle == 1;
$firsttitle = 0;
($title = $line) =~ s/^>//; # title without > at start
$seq = ''; # new seq
} else {
$seq = $seq . $line; # append seq lines
}
}
&initiator($title, $seq); # Do the thing for the last seq.
}
In the middle of several loops, &initiator is called. I'd like to have this in a module that I can "use" but substitute &initiator with other subs from other modules. These subs will need to have their own inputs as well. Would something like the following work or is there a more elegant solution?
use Reader qw(reader);
use Othersub qw(subroutine);
my #par = ('Mary', 'Lamb');
my %functions = (foo => \&Othersub::subroutine);
&reader($file_to_read, $functions{'foo'}($par[0], $par[1]));
Note: Final file structure is Othersub.pm, Reader.pm and the script that uses both modules.
Perl allows you to create references to things, and that includes both subroutines and arrays.
If you've got differing arguments to pass, then I would suggest you want to do so via array reference rather than what you're doing. A bit like this:
use strict;
use warnings;
sub variable_args {
my ( $code_ref, $array_ref ) = #_;
#dereference code ref;
#dereference array ref;
&$code_ref( #$array_ref, "optional", "extra", "arg" );
}
sub foo_func {
foreach (#_) {
print "Foo $_\n";
}
}
sub bar_func {
print "BAR: ", join( ":", #_ ), "\n";
}
#could inline the functions as anonymous subs. I would avoid doing that
#unless they're pretty short/clear.
my %functions = (
'foo' => \&foo_func,
'bar' => \&bar_func,
);
my %args_to_pass = (
'foo' => [ "Mary", "Lamb" ],
'bar' => [ "Some", "Fish", "Pie" ],
);
for my $thing ( "foo", "bar" ) {
variable_args( $functions{$thing}, $args_to_pass{$thing} );
}
Note - in the example above, you call &initiator. You shouldn't do this. It's deprecated syntax from Perl 4, and is redundant (and may have some undesired consequences in certain scenarios).
But I would suggest doing it this way rather than the way you've got. You could get this to work:
&reader($file_to_read, $functions{'foo'}($par[0], $par[1]));
But what will happen when you try and do that is you'll (potentially) just run your function immediately, and pass it's result into reader.
E.g.:
variable_args ( &{$functions{'foo'}}("Mary", "Lamb"), ["more stuff"] );
Won't work, because you're 'running' it immediately, and then sending the result - which'll make your $code_ref whatever the result of the subroutine was.
However you could make an anonymous sub, and pass that:
variable_args( sub {
&{ $functions{'foo'} }( "Special", "Argument", #_ )
},
$args_to_pass{'foo'} );
I would suggest you're getting needlessly convoluted by that point though :)
As far as I can guess, you want to pass a function (reference) as parameter.
Something like this should work for you
# Script
use Reader qw(reader);
use Othersub qw(subroutine);
my #par = .... ;
reader( $file_to_read , \&subroutine , #par);
# Reader.pm
sub reader {
my $file = shift;
my $initiator = shift;
my #par = #_;
...
$initiator->( $file , #par)
...
}
Remark: In the last line of your code, you are not passing the function subroutine to reader, as you might have intendend; instead, you invoke it and pass the result of soubroutine, given the paramers par to reader.

"Passing arguments to subroutine"-question?

Is routine2 ok too or shouldn't I do this? (I don't need a copy of #list in the subroutine)
#!/usr/bin/perl
use 5.012;
use warnings;
my #list = 0 .. 9;
sub routine1 {
my $list = shift;
for (#$list) { $_++ };
return $list
}
my $l = routine1( \#list );
say "#$l";
sub routine2 {
for (#list) { $_++ };
}
routine2();
say "#list";
If it works for you, then it's ok too. But the first sub can do the job for any array you pass to it which makes it more general.
P.S. Remember that #_ contains aliases for the parameters passed to the function. So you could also use this:
sub increment { $_++ for #_ }
increment(#list);
If you're worried about making the syntax look nice, try this:
sub routine3 (\#) {
for (#{$_[0]}) { $_++ }
}
my #list = (0 .. 9);
routine3(#list);
say "#list"; # prints 1 .. 10
This declares routine3 with a prototype - it takes an array argument by reference. So $_[0] is a reference to #list, no rather unsightly \ needed by the caller. (Some people discourage prototypes, so take this as you will. I like them.)
But unless this is a simplification for what your actual routine does, what I'd do is this:
my #list = 0 .. 9;
my #new_list = map { $_ + 1 } #list;
say "#new_list";
Unless routine is actually really complicated, and it's vital somehow that you modify the original array, I'd just use map. Especially with map, you can plug in a subroutine:
sub complex_operation { ... }
my #new_list = map { complex_operation($_) } #list;
Of course, you could prototype complex_operation with (_) and then just write map(complex_operation, #list); but I like the bracket-syntax personally.