How can I inject multiple lines with Devel::Declare? - perl

I want to use Devel::Declare to inject multiple lines of Perl code. However, Devel::Declare::set_linestr() cannot deal with multiple lines.
Normally I would join multiple statements together as a single line. These statements must be on separate lines to preserve their line numbers for error reporting purposes. This is to solve this bug in Method::Signatures and this related bug. I'm open to alternative solutions.
For example, Method::Signatures currently turns this code...
use Method::Signatures;
func hello(
$who = "World",
$greeting = get_greeting($who)
) {
die "$greeting, $who";
}
...into this...
func \&hello; sub hello { BEGIN { Method::Signatures->inject_scope('') }; my $who = (#_ > 0) ? ($_[0]) : ( get_greeting($who)); my $greeting = (#_ > 1) ? ($_[1]) : ( "Hello"); Method::Signatures->too_many_args_error(2) if #_ > 2;
die "$greeting, $who";
}
die $who then reports line 4 instead of line 7.
I would like it to instead be this (or perhaps something involving #line).
func \&hello; sub hello { BEGIN { Method::Signatures->inject_scope('') };
my $who = (#_ > 0) ? ($_[0]) : ( "World");
my $greeting = (#_ > 1) ? ($_[1]) : ( get_greeting($who));
Method::Signatures->too_many_args_error(2) if #_ > 2;
die "$greeting, $who";
}
Not only does this faithfully reproduce the line numbers, should get_greeting croak it will report having been called from the correct line.

As per your own answer, the following works:
sub __empty() { '' }
sub parse_proto {
my $self = shift;
return q[print __LINE__."\n"; Foo::__empty(
);print __LINE__."\n"; Foo::__empty(
);print __LINE__."\n";];
}
But introduces unacceptable overhead because the __empty() function must be called for every parameter. The overhead can be eliminated by calling __empty() conditionally using a condition which will never evaluate to true.
sub __empty() { '' }
sub parse_proto {
my $self = shift;
return q[print __LINE__."\n"; 0 and Foo::__empty(
);print __LINE__."\n"; 0 and Foo::__empty(
);print __LINE__."\n";];
}

I figured out a way to do it, but it's hacky and slow.
I noticed you could inject strings with literal newlines in them and they'd work. I reckon the parser knows enough to keep going past a newline in a string. I figured one can exploit that to trick the parser to keep going.
package Foo;
use strict;
use warnings;
use v5.12;
use parent "Devel::Declare::MethodInstaller::Simple";
sub import {
my $class = shift;
my $caller = caller;
$class->install_methodhandler(
into => $caller,
name => 'method'
);
}
sub parse_proto {
my $self = shift;
return q[print __LINE__."\n"; my $__empty = q{
};print __LINE__."\n"; $__empty = q{
};print __LINE__."\n";];
}
1;
And it works... except __LINE__ doesn't get incremented. Glitch in the Perl parser? I tried a regex with a newline in it, that didn't increment the line either.
But a subroutine does work!
sub __empty() { '' }
sub parse_proto {
my $self = shift;
return q[print __LINE__."\n"; Foo::__empty(
);print __LINE__."\n"; Foo::__empty(
);print __LINE__."\n";];
}
And it's a constant subroutine call, Perl should optimize it out, right? Alas, no. It seems the newline in the call fools the optimizer. On the upside, this avoids a "Useless use of a constant in void context" warning. On the down side, it introduces a subroutine call for each parameter and that's an unacceptable amount of overhead to add to every subroutine call.
Maybe someone else can come up with a clever way to squeak a newline into Perl syntax?

Related

Should a subroutine always return explicitly?

If perlcritic says "having no returns in a sub is wrong", what is the alternative if they really aren't needed?
I've developed two apparently bad habits:
I explicitly assign variables to the '$main::' namespace.
I then play with those variables in subs.
For example, I might do..
#!/usr/bin/perl
use strict;
use warnings;
#main::array = (1,4,2,6,1,8,5,5,2);
&sort_array;
&push_array;
&pop_array;
sub sort_array{
#main::array = sort #main::array;
for (#main::array){
print "$_\n";
}
}
sub push_array{
for ( 1 .. 9 ){
push #main::array, $_;
}
}
sub pop_array {
for ( 1 .. 3 ){
pop #main::array;
}
}
I don't do this all the time. But in the above, it makes sense, because I can segregate the operations, not have to worry about passing values back and forth and it generally looks tidy to me.
But as I said, perl critic says its wrong - because there's no return..
So, is anyone able to interpret what I'm trying to do and suggest a better way of approaching this style of coding in perl? eg. am I sort of doing OOP?
In short - yes, you're basically doing OO, but in a way that's going to confuse everyone.
The danger of doing subs like that is that you're acting at a distance. It's a bad coding style to have to look somewhere else entirely for what might be breaking your code.
This is generally why 'globals' are to be avoided wherever possible.
For a short script, it doesn't matter too much.
Regarding return values - Perl returns the result of the last expression by default. (See: return)
(In the absence of an explicit return, a subroutine, eval, or do FILE automatically returns the value of the last expression evaluated.)
The reason Perl critic flags it is:
Require all subroutines to terminate explicitly with one of the following: return, carp, croak, die, exec, exit, goto, or throw.
Subroutines without explicit return statements at their ends can be confusing. It can be challenging to deduce what the return value will be.
Furthermore, if the programmer did not mean for there to be a significant return value, and omits a return statement, some of the subroutine's inner data can leak to the outside.
Perlcritic isn't always right though - if there's good reason for doing what you're doing, then turn it off. Just as long as you've thought about it and are aware of the risks an consequences.
Personally I think it's better style to explicitly return something, even if it is just return;.
Anyway, redrafting your code in a (crude) OO fashion:
#!/usr/bin/perl
use strict;
use warnings;
package MyArray;
my $default_array = [ 1,4,2,6,1,8,5,5,2 ];
sub new {
my ( $class ) = #_;
my $self = {};
$self -> {myarray} = $default_array;
bless ( $self, $class );
return $self;
}
sub get_array {
my ( $self ) = #_;
return ( $self -> {myarray} );
}
sub sort_array{
my ( $self ) = #_;
#{ $self -> {myarray} } = sort ( #{ $self -> {myarray} } );
for ( #{ $self -> {myarray} } ) {
print $_,"\n";
}
return 1;
}
sub push_array{
my ( $self ) = #_;
for ( 1 .. 9 ){
push #{$self -> {myarray}}, $_;
}
return 1;
}
sub pop_array {
my ( $self ) = #_;
for ( 1 .. 3 ){
pop #{$self -> {myarray}};
}
return 1;
}
1;
And then call it with:
#!/usr/bin/perl
use strict;
use warnings;
use MyArray;
my $array = MyArray -> new();
print "Started:\n";
print join (",", #{ $array -> get_array()} ),"\n";
print "Reshuffling:\n";
$array -> sort_array();
$array -> push_array();
$array -> pop_array();
print "Finished:\n";
print join (",", #{ $array -> get_array()} ),"\n";
It can probably be tidied up a bit, but hopefully this illustrates - within your object, you've got an internal 'array' which you then 'do stuff with' by making your calls.
Result is much the same (I think I've replicated the logic, but don't trust that entirely!) but you have a self contained thing going on.
If the function doesn't mean to return anything, there's no need to use return!
No, you don't use any aspects of OO (encapsulation, polymorphism, etc). What you are doing is called procedural programming. Nothing wrong with that. All my work for nuclear power plants was written in that style.
The problem is using #main::array, and I'm not talking about the fact that you could abbreviate that to #::array. Fully-qualified names escape strict checks, so they are far, far more error-prone. Mistyped var name won't get caught as easily, and it's easy to have two pieces of code collide by using the same variable name.
If you're just using one file, you can use my #array, but I presume you are using #main::array because you are accessing it from multiple files/modules. I suggest placing our #array in a module, and exporting it.
package MyData;
use Exporter qw( import );
our #EXPORT = qw( #array );
our #array;
1;
Having some kind of hint in the variable name (such as a prefix or suffix) indicating this is a variable used across many modules would be nice.
By the way, if you wanted do create an object, it would look like
package MyArray;
sub new {
my $class = shift;
my $self = bless({}, $class);
$self->{array} = [ #_ ];
return $self;
}
sub get_elements {
my ($self) = #_;
return #{ $self->{array} };
}
sub sort {
my ($self) = #_;
#{ $self->{array} } = sort #{ $self->{array} };
}
sub push {
my $self = shift;
push #{ $self->{array} }, #_;
}
sub pop {
my ($self, $n) = #_;
return splice(#{ $self->{array} }, 0, $n//1);
}
my $array = MyArray->new(1,4,2,6,1,8,5,5,2);
$array->sort;
print("$_\n") for $array->get_elements();
$array->push_array(1..9);
$array->pop_array(3);
I improved your interface a bit. (Sorting shouldn't print. Would be nice to push different things and to pop other than three elements.)

Perl: Syntactical Sugar for Latter Coderef Arguments?

Using sub prototypes, we can define our own subs that look like map or grep. That is, the first coderef argument has shorter syntax than a normal anonymous sub. For example:
sub thunked (&) { $_[0] }
my $val = thunked { 2 * 4 };
Works great here, since the first argument is the coderef. For latter arguments however, it simple won't parse properly.
I made a with sub designed to make writing GTK2 code cleaner. It's meant to look like this (untested since it's hypothetical code):
use 5.012;
use warnings;
use Gtk2 '-init';
sub with ($&) {
local $_ = $_[0];
$_[1]->();
$_;
}
for (Gtk2::Window->new('toplevel')) {
$_->set_title('Test Application');
$_->add(with Gtk2::VBox->new {
my $box = $_;
$box->add(Gtk2::Button->new("Button $_")) for (1..4);
});
$_->show_all;
}
Gtk2->main;
It doesn't work because with needs to take the block as a first argument for the nice syntax to work. Is there any way to pull it off?
The module Devel::Declare contains tools for extending Perl's syntax in a relatively safe way.
Using Devel::Declare you would create a hook on the with token, which will stop the parser when it reaches that word. From there, you have control over the parser and you can read ahead until you reach a { symbol. At that point, you have what you need to work with, so you rewrite it into valid Perl, and pass it back to the parser.
in the file With.pm:
package With;
use warnings;
use strict;
use Devel::Declare;
sub import {
my $caller = caller;
Devel::Declare->setup_for (
$caller => {with => {const => \&parser}}
);
no strict 'refs';
*{$caller.'::with'} = sub ($&) {
$_[1]() for $_[0];
$_[0]
}
}
our $prefix = '';
sub get {substr Devel::Declare::get_linestr, length $prefix}
sub set { Devel::Declare::set_linestr $prefix . $_[0]}
sub parser {
local $prefix = substr get, 0, length($_[0]) + $_[1];
my $with = strip_with();
strip_space();
set "scalar($with), sub " . get;
}
sub strip_space {
my $skip = Devel::Declare::toke_skipspace length $prefix;
set substr get, $skip;
}
sub strip_with {
strip_space;
my $with;
until (get =~ /^\{/) {
(my $line = get) =~ s/^([^{]+)//;
$with .= $1;
set $line;
strip_space;
}
$with =~ s/\s+/ /g;
$with
}
and to use it:
use With;
sub Window::add {say "window add: ", $_[1]->str}
sub Window::new {bless [] => 'Window'}
sub Box::new {bless [] => 'Box'}
sub Box::add {push #{$_[0]}, #_[1..$#_]}
sub Box::str {"Box(#{$_[0]})"}
sub Button::new {"Button($_[1])"}
with Window->new {
$_->add(with Box->new {
for my $num (1 .. 4) {
$_->add(Button->new($num))
}
})
};
Which prints:
window add: Box(Button(1) Button(2) Button(3) Button(4))
A completely different approach would be to skip the with keyword altogether and write a routine to generate constructor subroutines:
BEGIN {
for my $name (qw(VBox)) { # and any others you want
no strict 'refs';
*$name = sub (&#) {
use strict;
my $code = shift;
my $with = "Gtk2::$name"->new(#_);
$code->() for $with;
$with
}
}
}
and then your code could look like
for (Gtk2::Window->new('toplevel')) {
$_->set_title('Test Application');
$_->add(VBox {
my $box = $_;
$box->add(Gtk2::Button->new("Button $_")) for (1..4);
});
$_->show_all;
}
One way that you could deal with it is to add a fairly useless keyword:
sub perform(&) { $_[0] }
with GTK2::VBox->new, perform { ... }
where perform is really just a sugarier alternative to sub.
Another way is to write a Devel::Declare filter or a Syntax::Keyword:: plugin to implement your with, as long as you have some way to tell when you're done parsing the with argument and ready to start parsing the block — balanced parentheses would do (so would an opening curly brace, but then hashes become a problem). Then you could support something like
with (GTK2::VBox->new) { ... }
and let the filter rewrite it to something like
do {
local $_ = GTK2::VBox->new;
do {
...;
};
$_;
}
which, if it works, has the advantage of not actually creating a sub, and thus not interfering with #_, return, and a few other things. The two layers of do-age I think are necessary for being able to install an EndOfScope hook in the proper place.
The obvious disadvantages of this are that it's tricky, it's hairy, and it's a source filter (even if it's a tame one) which means there are problems you have to solve if you want any code using it to be debuggable at all.

Is it ok to use Attribute::Handlers for implementing retry logic

Is it ok to use Attribute::Handlers for implementing retry logic
I have almost 50+ subroutine like verifyXXXX. I need to implement the retry logic for all these subs. I want to write this retry logic where the sub is actually implemented. If the return value of sub is false/undef then it will retry again.
subs will be called in regular way, so that the caller will not know about the retry logic, something like.
verify_am_i_doing_good()
or die('sorry you are not doing as expected.');
verify_am_i_fine()
or die ('sorry you are not find.');
:
:
the actual implementation of these functions is something like this in the package.
use Attribute::Handlers;
use constant RETRY_LIMIT => 4;
use constant RETRY_DELAY => 2;
sub verify_am_i_doing_good : __retry
{
return 1 if ($x == $y);
return;
}
sub __retry : ATTR(CODE) {
my ($pkg, $sym, $code) = #_;
my $name = *{ $sym }{NAME};
no warnings 'redefine';
*{ $sym } = sub
{
my $self = $_[0];
my $result;
logMsg (INFO, "Executing subroutine $name with retry limit " . RETRY_LIMIT);
for (my $retryCount = 1; $retryCount <= RETRY_LIMIT; $retryCount++)
{
logMsg (INFO, "Executing subroutine $name with retry count $retryCount");
my $result = $code->( #_ );
if ($result)
{
logMsg (INFO, "Expected result observed in retry count $retryCount");
return wantarray ? #$result : $result;
}
else
{
logMsg (INFO, "Expected result is NOT observed in retry count $retryCount");
logMsg (INFO, "Retrying again by updating uixml");
sleep RETRY_DELAY;
$self->updateState();
}
}
logMsg (WARN, "Failed to verify expected result for subroutine $name with retry limit " . RETRY_LIMIT);
return;
};
}
The reason to use Attribute::Handlers, inplace of Attribute::Attempts is that in the case of failure, I need to call another subroutine updateState() before retrying (re-executing) the subroutine.
I got this idea of writing the retry logic from following post http://www.perl.com/pub/2007/04/12/lightning-four.html
My main concern is that since I am using this __retry attribute for almost 50+ subs. Is it a good practice to do in this way or is there anything simple I can do?
You help will be highly appreciated.
You don't need attributes to do a sub wrapper. There was Memoize long before there was Memoize::Attrs (or Attribute::Memoize for that matter). You can just take a look at how Memoize handles it.
Quite recently, I was writing some Perl for functions called in another interface. All the arguments passed to the Perl function from this interface would be passed in a funky-but-universal format used by my division. Rather than deal with this everywhere, I wrote a logic wrapper like so
sub external (#) {
my ( $subname, $code ) = #_;
...
my $wrapped
= sub {
my $count = 5;
while ( --$count and not my #results = &$code ) {
adjust_stuff();
}
return #results;
};
{ no strict 'refs'; # my special "no-block"
*$subname = $wrapped;
}
return;
}
And used it like this (some people don't like this use of the "fat comma")
external something_I_want_to_do => sub {
my #regular_old_perl_args = #_;
...
};
The prototype (#) helps a sub act as an operator and need not always be called with parenthesis.
But by all means if you like method attributes and it works and you can get it not to bite you, use them. But you don't have to. You should probably read up on the caveats though.

Why is Perl's Math::Complex taking up so much time when I try acos(1)?

While trying to profile my Perl program, I find that Math::Complex is taking up a lot of time with what looks like some kind of warning.
Also, my code shouldn't have any complex numbers being generated or used, so I am not sure what it is doing in Math::Complex, anyway. Here's the FastProf output for the most expensive lines:
/usr/lib/perl5/5.8.8/Math/Complex.pm:182 1.55480 276232: _cannot_make("real part", $re) unless $re =~ /^$gre$/;
/usr/lib/perl5/5.8.8/Math/Complex.pm:310 1.01132 453641: sub cartesian {$_[0]->{c_dirty} ?
/usr/lib/perl5/5.8.8/Math/Complex.pm:315 0.97497 562188: sub set_cartesian { $_[0]->{p_dirty}++; $_[0]->{c_dirty} = 0;
/usr/lib/perl5/5.8.8/Math/Complex.pm:189 0.86302 276232: return $self;
/usr/lib/perl5/5.8.8/Math/Complex.pm:1332 0.85628 293660: $self->{display_format} = { %display_format };
/usr/lib/perl5/5.8.8/Math/Complex.pm:185 0.81529 276232: _cannot_make("imaginary part", $im) unless $im =~ /^$gre$/;
/usr/lib/perl5/5.8.8/Math/Complex.pm:1316 0.78749 293660: my %display_format = %DISPLAY_FORMAT;
/usr/lib/perl5/5.8.8/Math/Complex.pm:1335 0.69534 293660: %{$self->{display_format}} :
/usr/lib/perl5/5.8.8/Math/Complex.pm:186 0.66697 276232: $self->set_cartesian([$re, $im ]);
/usr/lib/perl5/5.8.8/Math/Complex.pm:170 0.62790 276232: my $self = bless {}, shift;
/usr/lib/perl5/5.8.8/Math/Complex.pm:172 0.56733 276232: if (#_ == 0) {
/usr/lib/perl5/5.8.8/Math/Complex.pm:316 0.53179 281094: $_[0]->{'cartesian'} = $_[1] }
/usr/lib/perl5/5.8.8/Math/Complex.pm:1324 0.48768 293660: if (#_ == 1) {
/usr/lib/perl5/5.8.8/Math/Complex.pm:1319 0.44835 293660: if (exists $self->{display_format}) {
/usr/lib/perl5/5.8.8/Math/Complex.pm:1318 0.40355 293660: if (ref $self) { # Called as an object method
/usr/lib/perl5/5.8.8/Math/Complex.pm:187 0.39950 276232: $self->display_format('cartesian');
/usr/lib/perl5/5.8.8/Math/Complex.pm:1315 0.39312 293660: my $self = shift;
/usr/lib/perl5/5.8.8/Math/Complex.pm:1331 0.38087 293660: if (ref $self) { # Called as an object method
/usr/lib/perl5/5.8.8/Math/Complex.pm:184 0.35171 276232: $im ||= 0;
/usr/lib/perl5/5.8.8/Math/Complex.pm:181 0.34145 276232: if (defined $re) {
/usr/lib/perl5/5.8.8/Math/Complex.pm:171 0.33492 276232: my ($re, $im);
/usr/lib/perl5/5.8.8/Math/Complex.pm:390 0.20658 128280: my ($z1, $z2, $regular) = #_;
/usr/lib/perl5/5.8.8/Math/Complex.pm:391 0.20631 128280: if ($z1->{p_dirty} == 0 and ref $z2 and $z2->{p_dirty} == 0) {
Thanks for any help!
Lines 182 and 185 do a warning unless a regex matches. So the slowness is probably not from outputting warnings but rather doing the regular expression matching.
If the profiler says those lines are being executed, they are. If you aren't calling them directly, a module that you load may be calling them indirectly. If you use a more advanced profiler (such as Devel::NYTProf), you'll be able to see the call graph to determine what part of your code is ultimately causing the call to the slow library code.
Edit Math/Complex.pm and put
use Carp;
Carp::cluck("in Math/Complex.pm");
around one of the lines listed above. This will print out a stack trace and you will see exactly how you are getting into this module.
According to the perl debugger doc, you can set up a handler for the INT signal. Then while it's running you can type Ctrl-C, and it will enter the debugger. Then type T to get a backtrace, and it should show you exactly why it is in the Complex.pm code. Since you are spending a lot of time in that code, you will probably land in it, but if you don't, try it again until you do.

How can I code in a functional style in Perl?

How do you either:
have a sub return a sub
or
execute text as code
in Perl?
Also, how do I have an anonymous function store state?
A sub returns a sub as a coderef:
# example 1: return a sub that is defined inline.
sub foo
{
return sub {
my $this = shift;
my #other_params = #_;
do_stuff();
return $some_value;
};
}
# example 2: return a sub that is defined elsewhere.
sub bar
{
return \&foo;
}
Arbitrary text can be executed with the eval function: see the documentation at perldoc -f eval:
eval q{print "hello world!\n"};
Note that this is very dangerous if you are evaluating anything extracted from user input, and is generally a poor practice anyway as you can generally define your code in a coderef as in the earlier examples above.
You can store state with a state variable (new in perl5.10), or with a variable scoped higher than the sub itself, as a closure:
use feature 'state';
sub baz
{
state $x;
return ++$x;
}
# create a new scope so that $y is not visible to other functions in this package
{
my $y;
sub quux
{
return ++$y;
}
}
Return a subroutine reference.
Here's a simple example that creates sub refs closed over a value:
my $add_5_to = add_x_to(5);
print $add_5_to->(7), "\n";
sub add_x_to {
my $x = shift;
return sub { my $value = shift; return $x + $value; };
}
You can also work with named subs like this:
sub op {
my $name = shift;
return $op eq 'add' ? \&add : sub {};
}
sub add {
my $l = shift;
my $r = shift;
return $l + $r;
}
You can use eval with an arbitrary string, but don't do it. The code is hard to read and it restarts compilation, which slows everything down. There are a small number of cases where string eval is the best tool for the job. Any time string eval seems like a good idea, you are almost certainly better off with another approach.
Almost anything you would like to do with string eval can be achieved with closures.
Returning subs is easy by using the sub keyword. The returned sub closes over the lexical variables it uses:
#!/usr/bin/perl
use strict; use warnings;
sub mk_count_from_to {
my ($from, $to) = #_;
return sub {
return if $from > $to;
return $from ++;
};
}
my $c = mk_count_from_to(-5, 5);
while ( defined( my $n = $c->() ) ) {
print "$n\n";
}
5.10 introduced state variables.
Executing text as Perl is accomplished using eval EXPR:
the return value of EXPR is parsed and executed as if it were a little Perl program. The value of the expression (which is itself determined within scalar context) is first parsed, and if there weren't any errors, executed in the lexical context of the current Perl program, so that any variable settings or subroutine and format definitions remain afterwards. Note that the value is parsed every time the eval executes
Executing arbitrary strings will open up huge gaping security holes.
You can create anonymous subroutines and access them via a reference; this reference can of course be assigned to a scalar:
my $subref = sub { ... code ... }
or returned from another subroutine
return sub { ... code ... }
If you need to store states, you can create closures with lexical variables defined in an outer scope like:
sub create_func {
my $state;
return sub { ... code that can refer to $state ... }
}
You can run code with eval