I'm trying to log data structures in an old and big Perl project. In order to do so, I use Data::Dumper, however, some structures are a bit too large and spam the log. So I'm looking for a way to log them in a less verbose manner.
Now Dumper's doc mentions $Data::Dumper::Freezer = <method_name> variable that can be used to fix that. I've tried using that.
Adding a serializer method that returns "shortened" value results in nothing, though the method gets called. Making the serializer method act on $_[0] causes the needed effect, but spoils the original data structure.
I'm confused. What am I doing wrong? How can I fix it?
Here's a refined sample code:
#!/usr/bin/env perl
use strict;
use warnings;
use Data::Dumper;
$Data::Dumper::Indent = 0;
$\="\n";
my $x = Foo->new ( answer => 42, use => "force" );
my $y = { foo => $x };
print "initial plain:\n\t", Dumper( $x );
print "initial compound:\n\t", Dumper( $y );
{
local $Data::Dumper::Freezer = 'freeze_pure';
print "still not abbreviated data:\n\t", Dumper( $y );
};
{
local $Data::Dumper::Freezer = 'freeze_replace';
print "abbreviated data:\n\t", Dumper( $y );
};
print "initial data is still intact:\n\t", Dumper( $x );
print "compound data is corrupted:\n\t", Dumper( $y );
package Foo;
sub new {
my $class = shift;
return bless { #_ }, $class;
};
sub freeze_pure {
my $self = $_[0];
warn "# In freeze_pure";
return bless {
values => join ",", values %$self
}, (ref $self) . "::short";
};
sub freeze_replace {
my $self = $_[0];
warn "# In freeze_replace";
$_[0] = bless {
values => join ",", values %$self
}, (ref $self) . "::short";
return;
};
And output:
initial plain:
$VAR1 = bless( {'use' => 'force','answer' => 42}, 'Foo' );
initial compound:
$VAR1 = {'foo' => bless( {'use' => 'force','answer' => 42}, 'Foo' )};
# In freeze_pure at dumper-freezer.pl line 36.
still not abbreviated data:
$VAR1 = {'foo' => bless( {'use' => 'force','answer' => 42}, 'Foo' )};
# In freeze_replace at dumper-freezer.pl line 42.
abbreviated data:
$VAR1 = {'foo' => bless( {'values' => 'force,42'}, 'Foo::short' )};
initial data is still intact:
$VAR1 = bless( {'use' => 'force','answer' => 42}, 'Foo' );
compound data is corrupted:
$VAR1 = {'foo' => bless( {'values' => 'force,42'}, 'Foo::short' )};
Although the documentation is a bit sparse, the intended use of freezer/toaster is data serialization/de-serialization, not prettification of debugging output.
So, Data::Dumper calls the freezer method, but doesn't use the return value. The idea is probably that if you're going to serialize an object, you won't be messing with it again until you de-serialize it, so there's no problem with changing the object itself.
Here's the relevant section of code from the Data::Dumper source:
# Call the freezer method if it's specified and the object has the
# method. Trap errors and warn() instead of die()ing, like the XS
# implementation.
my $freezer = $s->{freezer};
if ($freezer and UNIVERSAL::can($val, $freezer)) {
eval { $val->$freezer() };
warn "WARNING(Freezer method call failed): $#" if $#;
}
If you just want to reduce the size of the output in your logs, you can remove newlines and indentation by setting $Data::Dumper::Indent to zero:
use Data::Dumper;
use WWW::Mechanize;
$Data::Dumper::Indent = 0;
my $mech = WWW::Mechanize->new;
print Dumper $mech;
Output:
$VAR1 = bless( {'headers' => {},'ssl_opts' => {'verify_hostname' => 1},'forms' => undef,'page_stack' => [],'text' => undef,'requests_redirectable' => ['GET','HEAD','POST'],'timeout' => 180,'onerror' => sub { "DUMMY" },'current_form' => undef,'links' => undef,'max_redirect' => 7,'quiet' => 0,'images' => undef,'noproxy' => 0,'stack_depth' => 8675309,'show_progress' => undef,'protocols_forbidden' => undef,'no_proxy' => [],'handlers' => {'request_prepare' => bless( [{'owner' => 'LWP::UserAgent::cookie_jar','callback' => sub { "DUMMY" },'line' => '/home/foo/perl5/perlbrew/perls/perl-5.16.3/lib/site_perl/5.16.3/LWP/UserAgent.pm:705'}], 'HTTP::Config' ),'response_header' => bless( [{'owner' => 'LWP::UserAgent::parse_head','callback' => sub { "DUMMY" },'m_media_type' => 'html','line' => '/home/foo/perl5/perlbrew/perls/perl-5.16.3/lib/site_perl/5.16.3/LWP/UserAgent.pm:684'}], 'HTTP::Config' ),'response_done' => bless( [{'owner' => 'LWP::UserAgent::cookie_jar','callback' => sub { "DUMMY" },'line' => '/home/foo/perl5/perlbrew/perls/perl-5.16.3/lib/site_perl/5.16.3/LWP/UserAgent.pm:708'}], 'HTTP::Config' )},'onwarn' => sub { "DUMMY" },'protocols_allowed' => undef,'use_eval' => 1,'local_address' => undef,'autocheck' => 1,'title' => undef,'def_headers' => bless( {'user-agent' => 'WWW-Mechanize/1.75'}, 'HTTP::Headers' ),'cookie_jar' => bless( {'COOKIES' => {}}, 'HTTP::Cookies' ),'proxy' => {},'max_size' => undef}, 'WWW::Mechanize' );
This is still a lot of output, but it's certainly more compact than:
$VAR1 = bless( {
'headers' => {},
'ssl_opts' => {
'verify_hostname' => 1
},
'forms' => undef,
'page_stack' => [],
'text' => undef,
'requests_redirectable' => [
'GET',
'HEAD',
'POST'
],
'timeout' => 180,
'onerror' => sub { "DUMMY" },
'current_form' => undef,
'links' => undef,
'max_redirect' => 7,
'quiet' => 0,
'images' => undef,
'noproxy' => 0,
'stack_depth' => 8675309,
'show_progress' => undef,
'protocols_forbidden' => undef,
'no_proxy' => [],
'handlers' => {
'request_prepare' => bless( [
{
'owner' => 'LWP::UserAgent::cookie_jar',
'callback' => sub { "DUMMY" },
'line' => '/home/foo/perl5/perlbrew/perls/perl-5.16.3/lib/site_perl/5.16.3/LWP/UserAgent.pm:705'
}
], 'HTTP::Config' ),
'response_header' => bless( [
{
'owner' => 'LWP::UserAgent::parse_head',
'callback' => sub { "DUMMY" },
'm_media_type' => 'html',
'line' => '/home/foo/perl5/perlbrew/perls/perl-5.16.3/lib/site_perl/5.16.3/LWP/UserAgent.pm:684'
}
], 'HTTP::Config' ),
'response_done' => bless( [
{
'owner' => 'LWP::UserAgent::cookie_jar',
'callback' => sub { "DUMMY" },
'line' => '/home/foo/perl5/perlbrew/perls/perl-5.16.3/lib/site_perl/5.16.3/LWP/UserAgent.pm:708'
}
], 'HTTP::Config' )
},
'onwarn' => sub { "DUMMY" },
'protocols_allowed' => undef,
'use_eval' => 1,
'local_address' => undef,
'autocheck' => 1,
'title' => undef,
'def_headers' => bless( {
'user-agent' => 'WWW-Mechanize/1.75'
}, 'HTTP::Headers' ),
'cookie_jar' => bless( {
'COOKIES' => {}
}, 'HTTP::Cookies' ),
'proxy' => {},
'max_size' => undef
}, 'WWW::Mechanize' );
Alternatively, you could try Data::Dump, which allows you to filter the output using Data::Dump::Filtered. I prefer Data::Dump to Data::Dumper anyway because I think it has more sensible defaults (e.g. outputting escape sequences for whitespace other than spaces).
I haven't used the filtering feature yet, but brian d foy wrote a nice article about it with several examples.
Related
Here hash2 attribute is dependent on hash1. infact, hash2 is driven by hash1. for example,
hash1 -> key1 => value1, key2 => value2 etc..
hash2 -> key1 => 6, key2 => 6 etc. it is length(value from hash1, going to hash2)
Tried something like below, but not helpful.
has 'hash1' => (
is => 'rw',
isa => 'HashRef[Str]',
default => sub { {} },
handles => {
map { $_ . '_hash1' => $_ } #hash_delegations
},
);
has 'hash2' => (
is => 'rw',
isa => 'HashRef',
builder => '_filter_hash1',
handles => {
map { $_ . 'hash2' => $_ } #hash_delegations
},
);
sub _filter_hash1 {
my $self = shift;
for my $alias ($self->keys_hash1()) {
return {$alias, length($alias)};
}
}
Hash1 is going to set over time, not sure how to make sure that how should I capture the event on hash1 to update the entry in the hash2. Any idea how can I achieve this ?
Are you trying to create a cache of value lengths? In practice, length is so fast you don't need to cache it, but it might be just a simplified example of something more complex. I'd use a trigger, plus a trait on the first hash to enforce setting the value via a method. Changing the hash value directly wouldn't trigger the change in the other attribute.
#!/usr/bin/perl
use warnings;
use strict;
use feature qw{ say };
{ package My::Object;
use Moose;
my #hash_delegations = qw( keys );
has 'hash1' => (
is => 'rw',
isa => 'HashRef[Str]',
default => sub { {} },
trigger => \&_update_hash2,
traits => ['Hash'],
handles => { set_hash1 => 'set' },
);
has 'hash2' => (
is => 'ro',
writer => '_set_hash2',
isa => 'HashRef',
);
sub _update_hash2 {
my ($self, $new, $old) = #_;
$self->_set_hash2({ map { $_ => length $self->hash1->{$_} }
keys %{ $self->hash1 }});
}
}
my $o = 'My::Object'->new(hash1 => {a => 42, b => 'Universe'});
say $o->hash2->{$_} for qw( a b );
$o->set_hash1(c => '0123456789');
say $o->hash2->{c};
$o->hash1->{c} = ""; # Wrong!
say $o->hash2->{c}; # Didn't change :-(
Here's an example which uses read-only hashes with triggers and method modifiers...
package MyApp;
use Z qw( Dumper );
use Hash::Util qw( unlock_ref_keys lock_ref_keys );
class '::My::Object' => sub {
my %common = (
is => 'rw',
isa => HashRef[Str],
trigger => sub { lock_ref_keys($_[1]) },
default => sub { lock_ref_keys(my $ref = {}); $ref },
handles_via => 'Hash',
);
has hash1 => (
%common,
handles => [
'set_hash1' => 'set',
'get_hash1' => 'get',
],
);
has hash2 => (
%common,
isa => HashRef[Int],
handles => [
'set_hash2' => 'set',
'get_hash2' => 'get',
],
);
around set_hash1 => sub {
my ( $next, $self, $key, $val ) = ( shift, shift, #_ );
unlock_ref_keys( $self->hash1 );
unlock_ref_keys( $self->hash2 );
my $r = $self->$next( #_ );
$self->set_hash2( $key, length($val) );
lock_ref_keys( $self->hash1 );
lock_ref_keys( $self->hash2 );
return $r;
};
method BUILD => sub {
my ( $self, $args ) = #_;
if ( my $h1 = $args->{hash1} ) {
$self->set_hash1( $_, length $h1->{$_} ) for keys %$h1;
}
};
};
my $obj = 'My::Object'->new(
hash1 => { foo => 'xyzzy' },
);
$obj->set_hash1('bar', 'quux');
print Dumper($obj);
I have this Perl subroutine:
sub ask_for_lease {
my $url = '/sp/api/v1/lease';
my $formdata = '{"classname":"lease",}';
my $c = REST::Client->new();
$c->setHost($wizhost);
$c->PUT (
$url
, $formdata
, $headers
);
my $r = from_json($c->responseContent());
#print Dumper($r);
#my #results = $r->{'results'};
my #items = %{#{$r->{'results'}}[0]}->{'items'};
print Dumper(#items);
for my $item (#items) {
print "=============\n";
print Dumper($item);
print "=============\n";
}
}
It produces this output:
$VAR1 = [
{
'owner' => undef,
'notes' => 'Simulation One',
'version' => undef,
'status' => 'Ready',
'name' => 'One',
'lease' => '7070',
'_oid' => '1'
},
{
'owner' => undef,
'notes' => 'Simulation Two',
'version' => undef,
'status' => 'Ready',
'name' => 'Two',
'lease' => '2',
'_oid' => '2'
},
{
'owner' => undef,
'notes' => 'Simulation Three',
'version' => undef,
'status' => 'Ready',
'name' => 'Three 2012',
'lease' => '3',
'_oid' => '3'
},
...
];
=============
$VAR1 = [
{
'owner' => undef,
'notes' => 'Simulation One',
'version' => undef,
'status' => 'Ready',
'name' => 'One',
'lease' => '7070',
'_oid' => '1'
},
{
'owner' => undef,
'notes' => 'Simulation Two',
'version' => undef,
'status' => 'Ready',
'name' => 'Two',
'lease' => '2',
'_oid' => '2'
},
{
'owner' => undef,
'notes' => 'Simulation Three',
'version' => undef,
'status' => 'Ready',
'name' => 'Three 2012',
'lease' => '3',
'_oid' => '3'
},
...
];
=============
What I want to be able to do is iterate over the items array and print out the status and the name, but I am not sure I am dereferencing $r correctly.
The line
my #items = %{#{$r->{'results'}}[0]}->{'items'}
is very suspicious. You are extracting the first element of the array referred to by $r->{results}, dereferencing that as a hash, and using that hash in reference syntax. You should have got
Using a hash as a reference is deprecated
if you have use strict and use warnings in place as you should.
It is best to extract complex nested data in layers. In this case you can get the reference to the items array into a scalar variable and use that.
my $items= $r->{results}[0]{items};
for my $item ( #$items ) {
printf "name: %s, $item->{name};
printf "status: %s, $item->{status};
print "--\n";
}
If you post your JSON data then we will be able to help much better
I'm a non-programmer attemting to retrieve useful info from our InfoBlox DHCP boxes. I've installed the Perl API and can make some use of it.
I've got an output from the Data::Dumper "thingie" that appears to have some of the info I want. I'd like to directly reference some of that data but I'm unsure how.
print Dumper(\$object)
Here is part of the Data::Dumper output;
$VAR1 = \bless( {
'network' => '10.183.1.0/24',
'override_lease_scavenge_time' => 'false',
'enable_ifmap_publishing' => 'false',
'low_water_mark_reset' => '10',
'use_lease_time' => 0,
'use_enable_option81' => 0,
'network_container' => '/',
'override_ddns_ttl' => 'false',
'rir' => 'NONE',
'network_view' => bless( {
<snip> --------------------------------------
'extattrs' => {
'Use' => bless( {
'value' => 'Voip'
}, 'Infoblox::Grid::Extattr' )
},
<snip> --------------------------------------
'members' => [
bless( {
'ipv4addr' => '10.85.9.242',
'name' => 'ig3-app3.my.net'
}, 'Infoblox::DHCP::Member' ),
bless( {
'ipv4addr' => '10.85.9.210',
'name' => 'ig3-app1.my.net'
}, 'Infoblox::DHCP::Member' ),
bless( {
'ipv4addr' => '10.85.9.226',
'name' => 'ig3-app2.my.net'
}, 'Infoblox::DHCP::Member' )
],
'override_ignore_client_identifier' => 'false',
'email_list' => undef,
'rir_registration_status' => '??
}, 'Infoblox::DHCP::Network' );
How do I view the elements? ie ...
print $object{members->name};
print $object{members->ipv4addr};
print $object{extattrs->Use->value};
I've found the API dox insufficiant for my skill level:) The data I'd like to pull remains just out of reach.
my #retrieved_objs = $session->search (
object => "Infoblox::DHCP::Network",
network => '.*\.*\.*\..*',
);
foreach $object ( #retrieved_objs ) {
my $network = $object->network;
my $comment = $object->comment;
my $extattrs = $object->extattrs;
my $options = $object->options;
print $network, " network ", $comment, " ", $extattrs, " ", $options, "\n";
}
-------- output ---
10.183.2.0/24 network HASH(0x6a2f038) ARRAY(0x1d20eb0)
10.192.1.0/24 network HASH(0x9df6540) ARRAY(0x9df5468)
10.192.2.0/24 network HASH(0xa088fc8) ARRAY(0xa089718)
You shouldn't try to access the internal values of an object directly. The module - in this case Infoblox::DHCP::Network will provide methods that allow you to read or manipulate the values properly.
Can I do this in Moose?
package SomeClass;
use Moose;
has start => (
isa => 'Int',
is => 'ro',
lazy => 1,
default => sub { $_[0]->end },
);
has end => (
isa => 'Int',
is => 'ro',
lazy => 1,
default => sub { $_[0]->start },
);
...
In other words, I want two attributes called "start" and "end", and if only one of them is specified, I want the other one to be set to the same thing. Not specifying either one is an error.
Does this mutually-dependent setup work?
Yes, if you remove the possibility of infinite recursion by verifying that at least one of these values is specified:
has start => (
...
predicate => 'has_start',
);
has end => (
...
predicate => 'has_end',
);
sub BUILD
{
my $self = shift;
die "Need to specify at least one of 'start', 'end'!" if not $self->has_start and not $self->has_end;
}
Alternatively, you could delay the check to the default subs:
has start => (
...
predicate => 'has_start',
default => sub {
my $self = shift;
die "Need to specify at least one of 'start', 'end'!" if not $self->has_end;
$self->end;
},
);
has end => (
...
predicate => 'has_end',
default => sub {
my $self = shift;
die "Need to specify at least one of 'start', 'end'!" if not $self->has_start;
$self->start;
},
);
Personally, I'd take advantage of laziness to ensure that I didn't get caught in an infinite recursion:
has start => (
is => 'ro',
isa => 'Int',
lazy => 1,
default => sub { shift->end },
predicate => 'has_start',
);
has end => (
is => 'ro',
isa => 'Int',
lazy => 1,
default => sub { shift->start },
predicate => 'has_end',
);
sub BUILD {
my $self = shift;
die "Need to specify at least one of 'start', 'end'!"
unless $self->has_start || $self->has_end;
}
This could seem like an obviously hopeless case, but is there a trick to create a cyclic graph of immutable objects in Perl? Something like this:
package Node;
use Moose;
has [qw/parent child/] => (is => 'ro', isa => 'Node');
package main;
my $a = Node->new;
my $b = Node->new(parent => $a);
Now if I wanted $a->child to point to $b, what can I do?
You could play games with lazy initialization:
package Node;
use Moose;
has parent => (
is => 'ro',
isa => 'Node',
lazy => 1,
init_arg => undef,
builder => '_build_parent',
);
has _parent => (
is => 'ro',
init_arg => 'parent',
);
has child => (
is => 'ro',
isa => 'Node',
lazy => 1,
init_arg => undef,
builder => '_build_child',
);
has _child => (
is => 'ro',
init_arg => 'child',
predicate => undef,
);
has name => is => 'ro', isa => 'Str';
Generate the builders and predicates on the fly:
BEGIN {
for (qw/ parent child /) {
no strict 'refs';
my $squirreled = "_" . $_;
*{"_build" . $squirreled} = sub {
my($self) = #_;
my $proto = $self->$squirreled;
ref $proto eq "REF" ? $$proto : $proto;
};
*{"has" . $squirreled} = sub {
my($self) = #_;
defined $self->$squirreled;
};
}
}
This allows
my $a = Node->new(parent => \my $b, name => "A");
$b = Node->new(child => $a, name => "B");
for ($a, $b) {
print $_->name, ":\n";
if ($_->has_parent) {
print " - parent: ", $_->parent->name, "\n";
}
elsif ($_->has_child) {
print " - child: ", $_->child->name, "\n";
}
}
Its output is
A:
- parent: B
B:
- child: A
The code could be more elegant with η-conversion, but Moose won't pass parameters to builder methods.
I had to go and look at how really immutable languages do something like
this, and I think the following is probably a reasonable attempt.
use 5.10.0;
{
package Node;
use Moose;
has [qw(parent child)] => ( isa => 'Node', is => 'ro' );
sub BUILD {
my ( $self, $p ) = #_;
return unless exists $p->{_child};
my $child = Node->new( parent => $self, %{ delete $p->{_child} }, );
$self->meta->get_attribute('child')->set_value( $self, $child );
}
}
say Node->new( _child => {} )->dump
Basically instead of trying to build the objects separately, you have
the parent auto-vivify the child based on passing in it's arguments. The
output for this is, which is I believe the structure you were wanting.
$VAR1 = bless( {
'child' => bless( {
'parent' => $VAR1
}, 'Node' )
}, 'Node' );
I'm still very new to Moose, but would a trigger work?
use Modern::Perl;
package Node;
use Moose;
has 'parent' => (
is => 'ro',
isa => 'Node',
trigger => sub{
my ($self, $parent) = #_;
$parent->{child} = $self unless defined $parent->child;
}
);
has 'child' => (
is => 'ro',
isa => 'Node',
trigger => sub{
my ($self, $child) = #_;
$child->{parent} = $self unless defined $child->parent;
}
);
package main;
my $p = Node->new;
my $c = Node->new(parent => $p);
say $p, ' == ', $c->parent;
say $c, ' == ', $p->child;