Weakening captures using Sub::Quote - perl

I'd like to weaken captured variables in the code generated by Sub::Quote. For example, here's the non-quoted alternative:
use 5.10.0;
use Scalar::Util qw[ weaken ];
{
my $s = 'foo';
my $x = sub { say $s };
weaken( my $y = $x );
my $bar = sub { &$y };
&$bar;
$x = undef;
&$bar
}
and the output:
foo
Can't use an undefined value as a subroutine reference [...]
And here's my Sub::Quote attempt:
use 5.10.0;
use Sub::Quote;
use Scalar::Util qw[ weaken ];
{
my $s = 'foo';
my $x = sub { say $s };
weaken( my $y = $x );
my $bar = quote_sub( '&$y', { '$y' => \$y } );
&$bar;
$x = undef;
&$bar;
}
and the output:
foo
foo
Obviously the captured $y isn't weakened. Is there a way of altering the generated code to weaken captured variables?
The documentation is sparse, and the Sub::Quote implementation is complex; I'm fairly convinced this isn't possible with the current code, but I'd love to be shown to be wrong.

my $bar = quote_sub( '&$y', { '$y' => \$y } );
is roughly the same as
my $bar = eval(q{ my $y = $y; sub { &$y } });
(It does more, but those bits are irrelevant to this question). As you can see, that creates a new strong reference to the sub[1].
As a workaround, you could add a layer of indirection:
my $bar = eval(q{ my $y_ref = \$y; sub { &{ $$y_ref } } });
This can be achieved by using:
my $bar = quote_sub( '&{$$y_ref}', { '$y_ref' => \\$y } );
There wouldn't be any problems if the $y created by Sub::Quote was an alias for your $y. This can be achieved using Data::Alias or an experimental feature introduced in 5.22.
This can be demonstrated using the following:
{
package Sub::Quote;
my $sub = sub {
my ($from, $captures, $indent) = #_;
join(
'',
"use feature qw( refaliasing );\n",
"no warnings qw( experimental::refaliasing );\n",
map {
/^([\#\%\$])/
or croak "capture key should start with \#, \% or \$: $_";
(' ' x $indent).qq{\\my ${_} = \\${1}{${from}->{${\quotify $_}}};\n};
} keys %$captures
)
};
no warnings qw( redefine );
*capture_unroll = $sub;
}
my $bar = quote_sub( '&$y', { '$y' => \$y } );
You could talk to the module's maintainer about adding an option that would cause the use of aliasing.
When you create a copy of a (strong or weak) reference, it's a strong reference.

Related

Deep cloning of inside-out Perl classes - how to use methods from copied objects?

I have 3 classes declared as inside-out Perl classes using Class::Std. In one of these 3, there's a hash reference stored in $basket{ident $self} that looks like so (output of Data::Dumper):
$VAR1 = {
'auto' => {
'items' => {
'abc' => bless( do{\(my $o = undef)}, 'Item' )
},
'obj' => bless( do{\(my $o = undef)}, 'Category' )
}
};
I need to take this hash reference and create everything in it again (deep cloning). I tried to use dclone from Storable like so:
my $new_basket = dclone $basket{ident $self};
When I print the hashes, I get different memory addresses:
print $new_basket, "\n";
print $basket{ident $self}, "\n";
print $new_basket->{auto}->{items}, "\n";
print $basket{ident $self}{auto}->{items}, "\n";
print $new_basket->{auto}->{items}->{abc}, "\n";
print $basket{ident $self}{auto}->{items}->{abc}, "\n";
this will output:
HASH(0x55d325306a20)
HASH(0x55d325245298)
HASH(0x55d323b35ca8)
HASH(0x55d3243dd968)
Item=SCALAR(0x55d323b45190)
Item=SCALAR(0x55d325306588)
When I don't use dclone and use my $new_basket = $basket{ident $self} instead, I get the same memory addresses. When I use my $new_basket = { %{ $basket{ident $self} } }, I get different addresses only on the first level, which should be a shallow copy. All this seems fine and expected.
So, to me it seems that dclone actually deep-copied everything because the addresses are different. But when I try to use a method inside Item like so:
print $new_basket->{auto}->{items}->{abc}->get_added_on();
print $basket{ident $self}{auto}->{items}->{abc}->get_added_on();
I get:
Use of uninitialized value in print at lib/Basket.pm line 231.
2020-05-30
clearly that dclone works differently than I naively thought.
How should I deep-copy this whole structure? I'd appreciate some help or reference to some article/doc where I can read what's going on here.
One solution is to create the whole structure again using constructors, but I thought I'd save some space and use dclone. That obviously didn't turn out very well.
EDIT: I've been asked to provide a minimal runnable demonstration, here it is:
#!/usr/bin/env perl
use strict;
use warnings;
{
package A;
use Class::Std;
use Data::Dumper;
use Storable qw(dclone);
my %basket :ATTR;
sub BUILD {
my ($self, $ident, $args_ref) = #_;
$basket{$ident}->{auto} = {};
my $c = C->new({ date => q{2020-05-30} });
$basket{$ident}->{auto}->{items}->{abc} = $c;
return;
}
sub deep_clone {
my $self = shift;
print Dumper $basket{ident $self};
# the next line prints "2020-05-30" as expected
print $basket{ident $self}->{auto}->{items}->{abc}->get_added_on();
my $new_basket = dclone $basket{ident $self};
# "Use of uninitialized value in print at ./deep-clone.pl line 35."
print $new_basket->{auto}->{items}->{abc}->get_added_on();
}
}
{
package C;
use Class::Std;
my %added_on :ATTR( :get<added_on> );
sub BUILD {
my ($self, $ident, $args_ref) = #_;
$added_on{$ident} = $args_ref->{date};
return;
}
}
####
my $a = A->new();
$a->deep_clone();
The newly created "C" object was never added to %added_on.
Your classes will have to provide custom handlers for Storable to handle them.
Added to "A":
sub STORABLE_freeze {
my ($self, $cloning) = #_;
my $ident = ident($self);
return "", {
basket => $basket{$ident},
# Other attributes...
};
}
sub STORABLE_thaw {
my ($self, $cloning, $serialized, $inner) = #_;
my $ident = ident($self);
$basket{$ident} = $inner->{basket};
# Other attributes...
}
Added to "C":
sub STORABLE_freeze {
my ($self, $cloning) = #_;
my $ident = ident($self);
return "", {
added_on => $added_on{$ident},
# Other attributes...
};
}
sub STORABLE_thaw {
my ($self, $cloning, $serialized, $inner) = #_;
my $ident = ident($self);
$added_on{$ident} = $inner->{added_on};
# Other attributes...
}
Then you can use freeze/thaw/dclone without problem.
sub deep_clone {
my $self = shift;
#print Dumper $basket{ident $self};
CORE::say $basket{ ident $self }{auto}{items}{abc}->get_added_on();
my $clone = dclone($self);
#print Dumper $basket{ident $self};
CORE::say $basket{ ident $clone }{auto}{items}{abc}->get_added_on();
}

Perl calling subroutine reference with explicit additional scope as cleanly as possible

I'd like to be able to write something like the following...
call_with_scope({
x => 47,
}, sub {
printf "$x\n";
printf "$y\n";
});
Where $y is bound in the environment containing the expression (either lexically or dynamically depending on the symbol).
I've found a way to do it, but it requires no strict "vars" to be in effect in the expression containing call_with_scope(...) and the implementation of call_with_scope uses eval to create local bindings before transferring control to the callback.
Is there a way to avoid either requiring no strict "vars" at the call site or refer to and change the value of a local variable without resorting to eval?
For completeness, the code snippet below implements call_with_scope and prints 47 and then 48.
#!/usr/bin/env perl
use strict;
use warnings;
sub call_with_scope {
my ($env, $func) = #_;
my %property;
my #preamble;
foreach my $k (keys %$env) {
$property{$k} = $env->{$k};
# deliberately omitted: logic to ensure that ${$k} is a well-formed variable
push #preamble, "local \$$k = \$property{'$k'};";
}
# force scalar context
do {
my $str = join('', 'no strict "vars";', #preamble, '$_[1]->();');
return scalar(eval($str));
};
}
do {
no strict 'vars';
local $x;
my $y = 48;
call_with_scope(
{
x => 47,
},
sub {
printf "$x\n";
printf "$y\n";
}
);
};
I'm trying to write something kind of like Test::LectroTest ... except that instead of using a source filter and comments like in Property { ##[ x <- Int, y <- Int ]## <body> } ... I want to write something like Property({x => gen_int, y => gen_int}, sub { <body> }) where $x and $y inside body get their values when an "instantiation" of a property test is performed.
You can do this by defining $x and $y as globals in the caller's package.
no strict 'refs';
my $caller = caller;
for my $var (keys %$properties) {
*{$caller.'::'.$var} = $properties->{$var};
}
$code->();
But this can't be easily localized. And polluting the caller's namespace with globals potentially leads to mysterious data leaking between tests. In general, use as little magic as possible in a test library; the user will have enough of their own weird magic to debug.
Instead, provide a function which returns the properties. For example, p.
package LectroTest;
use Exporter qw(import);
our #EXPORT = qw(test p);
our $P;
sub test {
my($props, $test) = #_;
local $P = $props;
$test->();
}
sub p {
return $P;
}
And the test looks like:
use LectroTest;
test(
{ x => 42 }, sub { print p->{x} }
);
The problem is that the anon sub is compiled before call_with_scope is called, so there's no chance for call_with_scope to declare variables for that sub.
Any reason you're not using arguments like any other sub?
call_with_scope([ 47 ], sub {
my ($x) = #_;
printf "%s\n", $x;
printf "%s\n", $y;
});
It's not any longer!
Here's an alternative if you're ok in declaring $x outside of the sub.
use strict;
use warnings;
use PadWalker qw( closed_over );
sub call_with_scope {
my ($inits, $cb) = #_;
my $captures = closed_over($cb);
for my $var_name_with_sigil (keys(%$captures)) {
my ($var_name) = $var_name_with_sigil =~ /^\$(.*)/s
or next;
$inits->{$var_name}
or next;
${ $captures->{$var_name_with_sigil} } = $inits->{$var_name};
}
return $cb->();
}
{
my $x;
my $y = 48;
call_with_scope({
x => 47,
}, sub {
printf "%s\n", $x;
printf "%s\n", $y;
});
}
This works because variables are created at compile-time and cleared on scope exit.
It even works if sub was compiled in a different scope and package than the call to call_with_scope.
{
my $sub = do {
my $x;
my $y = 48;
sub {
printf "%s\n", $x;
printf "%s\n", $y;
}
};
call_with_scope({ x => 47 }, $sub);
}
But do you really want that kind of magic in your program?

Can I associate a CODE reference with a HASH reference that contains it in Perl?

I want to create a hash reference with code references mapped to scalars (strings) as its members.
So far I have a map reference that looks something like this:
my $object;
$object = {
'code1' => sub {
print $_[0];
},
'code2' => sub {
return 'Hello, World!';
},
'code3' => sub {
$object->{code1}->($object->{code2}->());
}
};
$object->{code3}->();
I would like to be able to "bless" the 'code3' reference in $object with $object, so I can do something like:
my $object;
$object = {
'code1' => sub {
print $_[0];
},
'code2' => sub {
return 'Hello, World!';
},
'code3' => sub {
$self = shift;
$self->{code1}->($self->{code2}->());
}
};
$object->{code3}->();
However, bless only works with packages, rather than hash tables.
Is there a way to do this in Perl 5 version 22?
Note: now that I think of it, it's better to pass $object to the method explicitly, as it solves JavaScript's "this" problem. I am just too used to Java's "this" which makes sense in Java where everything is a class and therefore all methods have a "this", but in scripting, it really helps to know if the "this" is actually passed, or is it just called as a function(and you end up accidentally polluting global scope or triggering strict warning) passing $self explicitly makes it clear that you are not calling it as a function, but as a method.
You are doing sub calls (not method calls), so you simply forgot to pass $self as a parameter.
my $object = {
code1 => sub {
print $_[0];
},
code2 => sub {
return 'Hello, World!';
},
code3 => sub {
my $self = shift;
$self->{code1}->( $self, $self->{code2}->($self) );
}
};
$object->{code3}->($object);
But I think you're trying to create JavaScript-like objects. You can start with the following:
package PrototypeObject;
sub new {
my $class = shift;
my $self = bless({}, $class);
%$self = #_;
return $self;
}
sub AUTOLOAD {
my $self = shift;
( my $method = our $AUTOLOAD ) =~ s/^.*:://s;
return $self->{$method}->($self, #_);
}
1;
use PrototypeObject qw( );
my $object = PrototypeObject->new(
code1 => sub {
print $_[1];
},
code2 => sub {
return 'Hello, World!';
},
code3 => sub {
my $self = shift;
$self->code1( $self->code2() );
}
);
$object->code3();
Note that this will slow down your method calls as it must call AUTOLOAD before calling your method. This could be addressed by overloading the method call operator.
Check on CPAN. Someone might already have a more complete implementation.
This is not the exact syntax you want, but Perl 5 supports many ways of making method calls, including method calls via strings. So you could say:
#!/usr/bin/perl
{ package Foo;
use strict;
use warnings;
sub new { bless {}, shift }
sub code1 { my $self = shift; print "$_[0]\n" };
sub code2 { "Hello, World!" }
sub code3 {
my $self = shift;
my $method1 = "code1";
my $method2 = "code2";
$self->$method1($self->$method2);
}
}
use strict;
use warnings;
my $o = Foo->new;
print "normal call\n";
$o->code3;
print "via string\n";
my $method = "code3";
$o->$method;
Also, remember that a package's symbol table is a hash: %Foo::, so you can always go spelunking in there yourself:
#!/usr/bin/perl
{ package Foo;
use strict;
use warnings;
sub new { bless {}, shift }
sub code1 { my $self = shift; print "$_[0]\n" };
sub code2 { "Hello, World!" }
sub code3 {
my $self = shift;
my $method1 = "code1";
my $method2 = "code2";
$self->$method1($self->$method2);
}
}
use strict;
use warnings;
print $Foo::{code2}->(), "\n";
However, I would suggest having a really code reason for these techniques as it can make maintenance a nightmare (eg imaging trying to find all of the code calling Foo::approved, you can't just grep for "->approved" because the actual call is ->$state()).
I just read the comments and noticed you said
my concern with packages is that I can't seem to create packages at runtime, but I can create hash tables at runtime
Perl 5 does allow you to create packages at runtime. In fact, depending on how you define runtime, you can do anything at runtime with string eval as it reenters compile time when it is called. But there is also a pure-runtime method of manipulating the symbol tables with typeglobs:
#!/usr/bin/perl
{ package Foo;
use strict;
use warnings;
sub new { bless {}, shift }
}
use strict;
use warnings;
my $o = Foo->new;
# here we add functions at runtime to the package Foo
{
no warnings "once";
*Foo::code1 = sub { my $self = shift; print "$_[0]\n" };
*Foo::code2 = sub { "Hello, World!" };
*Foo::code3 = sub {
my $self = shift;
my $method1 = "code1";
my $method2 = "code2";
$self->$method1($self->$method2);
};
}
$o->code3;
Because Perl 5 is object oriented (and not object based like JavaScript) these methods are attached to all Foo objects. If you want individual objects have their own symbol tables, then I am there are certainly ways to do that. Off the top of my head, AUTOLOAD comes to mind:
#!/usr/bin/perl
{ package Foo;
use strict;
use Carp;
use warnings;
sub new {
bless {
symtab => {}
}, shift
}
sub AUTOLOAD {
my $self = shift;
our $AUTOLOAD;
my $method = $AUTOLOAD =~ s/.*:://r;
my (undef, $file, $line) = caller();
die "$method does not exist at $file line $line"
unless exists $self->{symtab}{$method};
$self->{symtab}{$method}->($self, #_);
}
sub DESTROY {} # prevent DESTROY method from being hijacked by AUTOLOAD
}
use v5.22;
use warnings;
my $o1 = Foo->new;
my $o2 = Foo->new;
$o1->{symtab}{inc} = sub { my $self = shift; $self->{i}++; };
$o1->inc;
$o1->inc;
$o1->inc;
say "inc called on o1 $o1->{i} times";
$o2->inc; #dies because we haven't defined inc for $o2 yet
Perl 5 is very flexible and will let you do just about anything you want (after all the motto is TIMTOWTDI), but you should always keep in mind the future programmer tasked with maintaining your code who may want to hunt you down and wear your skin for doing some of these tricks.
This question has a definite XY problem feel. It seems like you are trying to solve a problem in Perl 5 the same way you would have solved it in JavaScript. While Perl 5 will let you do that (as I have demonstrated), there may be a more idiomatic way of achieving the same effect. Can you describe what you are trying to do (not how you want to do it) in a different question and we can suggest the ways in which we would solve your problem.

Equivalent of "shift" for a hash to create a $class->next() method

I almost feel like saying "it's me again!".
Anyway, here we go.
I like using while $object->next() style constructs. They appeal to me and seem "neat".
Now, when the thing I'm iterating over is an array, it's straightforward ("shift #ary or return undef")
sub next {
my ( $self, $args ) = #_;
my $next = shift #{ $self->{list_of_things} } or return undef;
my ( $car, $engine_size, $color )
= split( /\Q$opts->{fieldsep}/, $next );
$self->car = $host;
$self->engine_size = $engine_size;
$self->color = $color;
}
In this example I use AUTOLOAD to create the getters and setters and then have those instance variables available in my object during the while loop.
I'd like to do something similar but with the "list_of_things" being a %hash.
Here's a non-OO example that doesn't make it into the first iteration. Any ideas why?
(The total "list_of_things" is not that big - maybe 100 entries - so to do a keys(%{$hash}) every time doesn't seem too wasteful to me).
use strict;
use warnings;
use Data::Dumper;
my $list_of_things = {
volvo => {
color => "red",
engine_size => 2000,
},
bmw => {
color => "black",
engine_size => 2500,
},
mini => {
color => "british racing green",
engine_size => 1200,
}
};
sub next {
my $args = $_;
my #list = keys( %{$list_of_things} );
return undef if scalar #list == "0";
my $next = $list_of_things->{ $list[0] };
delete $list_of_things->{ $list[0] };
return $next;
}
while ( next()) {
print Dumper $_;
print scalar keys %{ $list_of_things }
}
Is there a better way of doing this? Am I doing something crazy?
EDIT:
I tried Ikegami's suggestion. Of course, Ikegami's example works flawlessly. When I try and abstract a little, so that all that is exposed to the object is a next->() method, I get the same "perl-going-to-100%-cpu" problem as in my original example.
Here's a non-OO example:
use Data::Dumper qw( Dumper );
sub make_list_iter {
my #list = #_;
return sub { #list ? shift(#list) : () };
}
sub next {
make_list_iter( keys %$hash );
}
my $hash = { ... };
while ( my ($k) = next->() ) {
print Dumper $hash->{$k};
}
It does not seem to get past the first step of the while() loop.
I am obviously missing something here...
If you don't want to rely on the hash's builtin iterator (used by each, keys and values), there's nothing stopping you from making your own.
use Data::Dumper qw( Dumper );
sub make_list_iter {
my #list = #_;
return sub { #list ? shift(#list) : () };
}
my $list_of_things = { ... };
my $i = make_list_iter(keys %$list_of_things);
while (my ($k) = $i->()) {
local $Data::Dumper::Terse = 1;
local $Data::Dumper::Indent = 0;
say "$k: " . Dumper($list_of_things->{$k});
}
The each operator is a builtin that iterates over hashes. It returns undef when it runs out of elements to return. So you could so something like
package SomeObject;
# creates new object instance
sub new {
my $class = shift;
return bless { hash_of_things => { #_ } }, $class
}
sub next {
my $self = shift;
my ($key,$value) = each %{ $self->{hash_of_things} };
return $key; # or return $value
}
Calling keys on the hash will reset the each iterator. It's good to know this so you can reset it on purpose:
sub reset {
my $self = shift;
keys %{ $self->{hash_of_things} }
}
and so you can avoid resetting it on accident.
The section on tie'ing hashes in perltie also has an example like this.
Here's how List::Gen could be used to create an iterator from a list:
use strict;
use warnings;
use List::Gen 'makegen';
my #list_of_things = ( # This structure is more suitable IMO
{
make => 'volvo',
color => 'red',
engine_size => 2000,
},
{
make => 'bmw',
color => 'black',
engine_size => 2500,
},
{
make => 'mini',
color => 'british racing green',
engine_size => 1200,
}
);
my $cars = makegen #list_of_things;
print $_->{make}, "\n" while $cars->next;
Well, if you don't need $list_of_things for later, you can always do something like
while(keys %$list_of_things)
{
my $temp=(sort keys %$list_of_things)[0];
print "key: $temp, value array: " . join(",",#{$list_of_things->{$temp}}) . "\n";
delete $list_of_things->{$temp};
}
And if you do need it, you can always assign it to a temporary hash reference and perform the same while loop on it.

How can I serialize a closure in Perl?

I think this might be best asked using an example:
use strict;
use warnings;
use 5.010;
use Storable qw(nstore retrieve);
local $Storable::Deparse = 1;
local $Storable::Eval = 1;
sub sub_generator {
my ($x) = #_;
return sub {
my ($y) = #_;
return $x + $y;
};
}
my $sub = sub_generator(1000);
say $sub->(1); # gives 1001
nstore( $sub, "/tmp/sub.store" );
$sub = retrieve("/tmp/sub.store");
say $sub->(1); # gives 1
When I dump /tmp/sub.store I see:
$VAR1 = sub {
package Storable;
use warnings;
use strict 'refs';
my($y) = #_;
return $x + $y;
}
But $x is never defined in this sub. I would expect that the sub generated by sub_generator will have $x replaced with its actual value upon generation. How should I solve this?
Note this question relates to this one.
Unfortunately I don't think Storable works with closures. However there are other CPAN modules that will serialise a closure. For eg. Data::Dump::Streamer
use 5.012;
use warnings;
use Data::Dump::Streamer;
sub sub_generator {
my ($x) = #_;
return sub {
my ($y) = #_;
return $x + $y;
};
}
my $sub = sub_generator(1000);
say $sub->(1); # gives 1001
my $serialised = Dump( $sub )->Out;
my $copy = do {
my $CODE1 = undef;
eval $serialised;
$CODE1;
};
say $copy->(2); # gives 1002
say $sub->(1); # still gives 1001
This is what the serialised code looks like when printed here, say Dump $sub;:
my ($x);
$x = 1000;
$CODE1 = sub {
use warnings;
use strict 'refs';
BEGIN {
$^H{'feature_unicode'} = q(1);
$^H{'feature_say'} = q(1);
$^H{'feature_state'} = q(1);
$^H{'feature_switch'} = q(1);
}
my($y) = #_;
return $x + $y;
};
Update
See this thread Storable and Closures on the Perl5 porters mailing list. It confirms what I thought about Storable and closures.
/I3az/