Tie Reading and Writing Perl Config Files - perl

I'm using the PerlMonk example I found on:
Reading and Writing Perl Config Files
Configuration.pl:
%CFG = (
'servers' => {
'SRV1' => {
'IP' => 99.32.4.0,
'user' => 'aname',
'pswd' => 'p4ssw0rd',
'status' => 'unavailable'
},
'SRV2' => {
'IP' => 129.99.10.5
'user' => 'guest',
'pswd' => 'guest'
'status' => 'unavailable'
}
},
'timeout' => 60,
'log' => {
'file' => '/var/log/my_log.log',
'level' => 'warn',
},
'temp' => 'remove me'
);
It is working great, but the only issue is when reading and writing the HASH like configuration is being 'out of order'.
Is there a way to keep it TIED?
This important since the configuration file will be also edited manually, so I want the keys and values in the same order.

You could tie config variable before using it, so later hash keys will stay in same order as before,
use strict;
use warnings;
use Tie::IxHash;
tie my %CFG, 'Tie::IxHash';
%CFG = (
'servers' => {
'SRV1' => {
'IP' => '99.32.4.0',
'user' => 'aname',
'pswd' => 'p4ssw0rd',
'status' => 'unavailable'
},
'SRV2' => {
'IP' => '129.99.10.5',
'user' => 'guest',
'pswd' => 'guest',
'status' => 'unavailable'
}
},
'timeout' => 60,
'log' => {
'file' => '/var/log/my_log.log',
'level' => 'warn',
},
'temp' => 'remove me'
);
use Data::Dumper;
print Dumper \%CFG;

If you use JSON then you have the advantage that your software is safe from a malicious attack (or perhaps accidental corruption). JSON also has a simpler syntax than Perl data structures, and it is easier to recover from syntax errors.
Setting the canonical option will create the data with the keys in sorted order, and so generate the same output for the same Perl data every time. If you need the data in a specific order other than alphabetical then you can use the Tie::IxHash module as #mpapec describes in his answer.
Alternatively you can use the sort_by method from the Pure Perl version of the module that lets you pass a collation subroutine. That would let you prescribe the order of your keys, and could be as simple as using a hash that relates all the possible key values with a numerical sort order.
This program uses the sort_by method to reconstruct the JSON in the same order as the keys appear in your original hash. That is unlikely to be the order you want, but the mechanism is there. It works by looking up each key in a hash table to determine how they should be ordered. Any keys (like SVR1 and SVR2 here) that don't appear in the hash are sorted in alphabetical order by default.
use strict;
use warnings;
use JSON::PP ();
my %CFG = (
'servers' => {
'SRV1' => {
'IP' => '99.32.4.0',
'user' => 'aname',
'pswd' => 'p4ssw0rd',
'status' => 'unavailable'
},
'SRV2' => {
'IP' => '129.99.10.5',
'user' => 'guest',
'pswd' => 'guest',
'status' => 'unavailable'
}
},
'timeout' => 60,
'log' => {
'file' => '/var/log/my_log.log',
'level' => 'warn',
},
'temp' => 'remove me'
);
my %sort_order;
my $n = 0;
$sort_order{$_} = ++$n for qw/ servers timeout log temp /;
$sort_order{$_} = ++$n for qw/ IP user pswd status /;
$sort_order{$_} = ++$n for qw/ file level /;
my $json = JSON::PP->new->pretty->sort_by(\&json_sort);
print $json->encode(\%CFG);
sub json_sort {
my ($aa, $bb) = map $sort_order{$_}, $JSON::PP::a, $JSON::PP::b;
$aa and $bb and $aa <=> $bb or $JSON::PP::a cmp $JSON::PP::b;
}
generates this output
{
"servers" : {
"SRV1" : {
"IP" : "99.32.4.0",
"user" : "aname",
"pswd" : "p4ssw0rd",
"status" : "unavailable"
},
"SRV2" : {
"IP" : "129.99.10.5",
"user" : "guest",
"pswd" : "guest",
"status" : "unavailable"
}
},
"timeout" : 60,
"log" : {
"file" : "/var/log/my_log.log",
"level" : "warn"
},
"temp" : "remove me"
}
which can simply be saved to a file and similarly restored.

Related

How can I redact some values in the dump of a Perl hash?

supposed I have these hashes:
my $hash1 = {
firstname => 'john',
lastname => 'doe',
};
my $hash2_nested = {
name => {
firstname => 'jean',
lastname => 'doe',
}
};
Note: hashes can be nested x times deeply.
I want to use Data::Dumper where I can print the copy of those hashes, but with hidden lastname.
means, it should print out:
$VAR1 = {
'firstname' => 'john'
'lastname' => '***',
};
and this:
$VAR1 = {
'name' => {
'firstname' => 'john'
'lastname' => '***',
}
};
is there any Perl library where it search for a hash key recursively and replace its value dynamically? something like:
replace_hash_value($hash1, 'lastname', '***');
There are several things to consider here. Mostly, you don't want to reinvent what is already out there. Also remember that any Personal Identifying Information (PII) in your program has a way to leak out despite your best efforts, but that's not the programming question at hand.
First, you don't want to operate on the original data, and since you have nested structures, you can't simply make a copy because that only copies the top level and still shares references at the lower level:
my %copy = %original; # shallow copy!
But, the core module Storable can make a deep copy that is completely disconnected, new copy that shares no references:
use Storable qw(dclone);
my $deep_copy = dclone $hash1;
Now you can play with $deep_copy without changing $hash1. You want to find all the last_name keys and remove their value. Grinnz suggested the Data::Walk module (an example of the Visitor design pattern). It's like File::Find for data structures. It's going to handle all the business of finding the hashes for you. In your wanted subroutine, skip everything that's not interesting, then change the nodes that are interesting. You don't worry about how you find or are given the nodes:
use Data::Walk;
walk \&wanted, $deep_copy;
sub wanted {
return unless ref $_ eq ref {};
return unless exists $_->{last_name};
$_->{last_name} = '****';
}
Now, put that all together. Here's a mix of nested things, with some odd cases thrown in, including an object that uses a hash:
use v5.10;
use Hash::AsObject;
my $data = {
first_name => 'Amelia',
last_name => 'Camel',
friends => [
q(last_name => 'REDACTED BY POLICY'),
{
first_name => 'Camelia',
last_name => 'Butterfly',
},
{
first_name => 'Larry',
last_name => 'Llama',
associate => {
first_name => 'Vicky',
last_name => 'Vicuna',
}
},
],
name => {
first_name => 'Andy',
last_name => 'Alpaca',
},
object => bless {
first_name => 'Peter',
last_name => 'Python',
}, 'FooBar',
};
use Storable qw(dclone);
my $deep_copy = dclone( $data );
use Data::Walk;
walk \&wanted, $deep_copy;
use Data::Dumper;
say Dumper( $deep_copy );
sub wanted {
return unless ref $_ eq ref {};
return unless exists $_->{last_name};
$_->{last_name} = '****';
}
And, here's the output from Data::Dumper (which you can prettify with some of its settings):
$VAR1 = {
'object' => bless( {
'first_name' => 'Peter',
'last_name' => 'Python'
}, 'Hash::AsObject' ),
'first_name' => 'Amelia',
'last_name' => '****',
'friends' => [
'last_name => \'REDACTED BY POLICY\'',
{
'last_name' => '****',
'first_name' => 'Camelia'
},
{
'last_name' => '****',
'first_name' => 'Larry',
'associate' => {
'first_name' => 'Vicky',
'last_name' => '****'
}
}
],
'name' => {
'first_name' => 'Andy',
'last_name' => '****'
}
};
Notice that it finds the hashes in the array reference, it doesn't touch the object, and it doesn't touch the literal data that has last_name => in it.
If you don't like those behaviors, then you can modify what you do in wanted to account for what you'd like to happen. Suppose you want to look at certain objects too, like that Hash::AsObject object. One (polymorphic) way to do that is look for objects that let you call a last_name method (although this assumes you can give it an argument to change the last name):
sub wanted {
if( ref $_ eq ref {} and exists $_->{last_name} ) {
$_->{last_name} = '****';
}
# merely one way to do this
elsif( eval { $_->can('last_name') } ) {
$_->last_name( '****' );
}
}
Now the last_name member in the object is also redacted:
$VAR1 = {
'first_name' => 'Amelia',
'friends' => [
'last_name => \'REDACTED BY POLICY\'',
{
'last_name' => '****',
'first_name' => 'Camelia'
},
{
'first_name' => 'Larry',
'associate' => {
'first_name' => 'Vicky',
'last_name' => '****'
},
'last_name' => '****'
}
],
'last_name' => '****',
'name' => {
'first_name' => 'Andy',
'last_name' => '****'
},
'object' => bless( {
'first_name' => 'Peter',
'last_name' => '****'
}, 'Hash::AsObject' )
};
That wanted is as flexible as you'd like it to be, and it's pretty simple.
Why not to code such subroutine yourself?
use strict;
use warnings;
use feature 'say';
my $hash1 = {
firstname => 'john',
lastname => 'doe'
};
my $hash2_nested = {
name => {
firstname => 'jean',
lastname => 'doe'
}
};
my $mask = 'lastname';
hash_mask($hash1,$mask);
hash_mask($hash2_nested,$mask);
sub hash_mask {
say "\$VAR = {";
hash_mask_x(shift, shift, 1);
say "};";
}
sub hash_mask_x {
my $hash = shift;
my $mask_k = shift;
my $depth = shift;
my $indent = ' ' x 8;
my $space = $indent x $depth;
while( my($k,$v) = each %{$hash} ) {
if (ref $v eq 'HASH') {
say $space . "$k => {";
hash_mask_x($v,$mask_k,$depth+1);
say $space . "}";
} elsif( $k eq $mask_k ) {
say $space . "'$k' => '*****'";
} else {
say $space . "'$k' => '$v'";
}
}
}
Output
$VAR = {
'lastname' => '*****'
'firstname' => 'john'
};
$VAR = {
name => {
'lastname' => '*****'
'firstname' => 'jean'
}
};

Perl DBM::Deep - add/delete in an arrayref of hashrefs

I've been working with DBM::Deep and so far, it's been easy to Read and Update the keys in the DB but when it comes to adding or deleting entities it gets a little complicated and I can't see how it could be done.
I've imported an XML file with XML::Hash and then copied on a DBM::Deep object. So the result is somehow complicated ... The objective of course is to be able to recreate the XML file easily.
So this code:
use DBM::Deep;
use List::Util qw(first);
use Data::Dumper;
my $db = DBM::Deep->new('foo.db');
my $devices = $db->{foo}->{devices}->{device};
(my $match) = grep { $_->{hostname} eq 'myfoo' } #$devices;
print Dumper ($match);
print Dumper($devices);
Gives the following output for the first print:
$VAR1 = bless( {
'enable' => '0',
'hostname' => 'myfoo',
'auth' => 'myauth',
'ip' => 'myip',
'protocol' => 'ssh'
}, 'DBM::Deep::Hash' );
The second print shows:
$VAR1 = bless( [
bless( {
'enable' => '0',
'hostname' => 'myfoo',
'auth' => 'myauth',
'ip' => 'myip',
'protocol' => 'ssh'
}, 'DBM::Deep::Hash' ),
bless( {
'ip' => 'myotherip',
'hostname' => 'myotherfoo',
'auth' => 'myauth',
'protocol' => 'telnet'
}, 'DBM::Deep::Hash' ),
and so on.
Can someone please help me to understand how to Create and Delete in this data structure?

Config::IniFiles hash behaves different than manually written hash

I am loading a config file, which ends up as an embedded hash, with Config::IniFiles. After that, I want to modify the resulting hash by, for some keys, bringing its values one level up. In the example below, I am aiming for this as a result:
$VAR1 = {
'max_childrensubtree' => '7',
'port' => '1984',
'user' => 'someuser',
'password' => 'somepw',
'max_width' => '20',
'host' => 'localhost',
'attrs' => {
'subattr2' => 'cat',
'topattr1' => 'cat',
'subattr2_1' => 'pt',
'subattr1' => 'rel'
},
'max_descendants' => '1000'
};
So for the keys params and basex at the highest level, I want to move its contents (key-value pairs) to the highest level - and remove the items themselves. In short:
(
a => {
'key1' => 'ok',
'key2' => 'hello'
}
)
turns into
(
'key1' => 'ok',
'key2' => 'hello'
)
The strange thing is that what I am trying to do does not work on a hash built from a read INI file, but it does work with a manually inserted hash. In other words, this works:
#!/usr/bin/perl
use utf8;
use strict;
use warnings;
use Data::Dumper;
my %ini = (
'params' => {
'max_width' => '20',
'max_childrensubtree' => '7',
'max_descendants' => '1000'
},
'attrs' => {
'topattr1' => 'cat',
'subattr1' => 'rel',
'subattr2' => 'cat',
'subattr2_1' => 'pt',
},
'basex' => {
'host' => 'localhost',
'port' => '1984',
'user' => 'someuser',
'password' => 'somepw'
}
);
&_parse_ini(\%ini);
sub _parse_ini {
my $ref = shift;
foreach (('params', 'basex')) {
foreach my $k (keys %{$ref->{$_}}) {
$ref->{$k} = $ref->{$_}->{$k};
}
delete $ref->{$_};
}
print Dumper($ref);
}
But this does not:
#!/usr/bin/perl
use utf8;
use strict;
use warnings;
use Data::Dumper;
use Config::IniFiles;
# Load config file
tie my %ini, 'Config::IniFiles', (-file => $ARGV[0]);
&_parse_ini(\%ini);
sub _parse_ini {
my $ref = shift;
foreach (('params', 'basex')) {
foreach my $k (keys %{$ref->{$_}}) {
$ref->{$k} = $ref->{$_}->{$k};
}
delete $ref->{$_};
}
print Dumper($ref);
}
The input ini file for this example would be:
[params]
max_width = 20
max_childrensubtree = 7
max_descendants = 1000
[attrs]
topattr1 = cat
subattr1 = rel
subattr2 = cat
subattr2_1 = pt
[basex]
host = localhost
port = 1984
user = admin
password = admin
I have been looking in the documentation and on SO for similar issues but have found none. It appears that the hashes are identical (Config::IniFiles doesn't seem to add something specific), so I have no idea why it works for 'manual' hashes, and not for read-in ones.
The two hashes are not identical at all, although they may appear to be from the point of view of the data they contain.
The first one is a regular hash. You can do whatever you like with it.
The second one is a tied hash. It becomes an object of Config::IniFiles, but with a hash like interface. So whilst it appears to be a hash, the package can override the methods for storing or fetching information in the hash however it likes.
In this particular case, it looks like Config::IniFiles will only store a new key value in the hash if the value is hash ref. So you can't flatten out the tied hash as you want. Instead you'll have to create a new hash and copy the data in to it to do what you want.

Perl : parse a file and grab blocks

impossible to parse the file below and grab the blocks in an hash table or simple tab.
I would like to have an hash table with for example
[serv-test] => parent=PRODUCTION.Windows,host=1.1.1.1
Problem is I can delimit the start of a block (with /\[.*\]/) but impossible to delimit the end. The end of my blocks is the start of another.
My file:
authreq=false
default.secured=false
port=3181
protocol=TCP
seclevel=2
secured=false
[serv-test]
parent=PRODUCTION.Windows
host=1.1.1.1
[citrix]
parent=PRODUCTION.Windows
host=1.1.1.2
[cluster-serv]
parent=PRODUCTION.Unix._INFRA
host=1.1.1.3
port=3182
Instead of worrying about getting a hash, be satisfied with getting the data. If you give the top a section name, you have an INI File:
[Default]
authreq=false
default.secured=false
port=3181
protocol=TCP
seclevel=2
secured=false
[serv-test]
parent=PRODUCTION.Windows
host=1.1.1.1
[citrix]
parent=PRODUCTION.Windows
host=1.1.1.2
[cluster-serv]
parent=PRODUCTION.Unix._INFRA
host=1.1.1.3
port=3182
Now you can use Config::IniFiles:
use v5.10;
use Config::IniFiles;
my $cfg = Config::IniFiles->new(
-file => "test.ini"
) or die "#Config::IniFiles::errors";
say "Port is ", $cfg->val( 'Default', 'port' );
say "Cluster host is ", $cfg->val( 'cluster-serv', 'host' );
If you really want the hash, that's not so hard:
use Config::IniFiles;
use Data::Dumper;
my $cfg = Config::IniFiles->new(
-file => "test.ini"
) or die "#Config::IniFiles::errors";
my %hash;
foreach my $section ( $cfg->Sections ) {
foreach my $parameter ( $cfg->Parameters( $section ) ) {
$hash{$section}{$parameter} = $cfg->val( $section, $parameter );
}
}
say Dumper \%hash;
Now you have:
$VAR1 = {
'citrix' => {
'parent' => 'PRODUCTION.Windows',
'host' => '1.1.1.2'
},
'Default' => {
'secured' => 'false',
'port' => '3181',
'protocol' => 'TCP',
'default.secured' => 'false',
'authreq' => 'false',
'seclevel' => '2'
},
'serv-test' => {
'host' => '1.1.1.1',
'parent' => 'PRODUCTION.Windows'
},
'cluster-serv' => {
'port' => '3182',
'parent' => 'PRODUCTION.Unix._INFRA',
'host' => '1.1.1.3'
}
};
Don't reinvent the wheel. There are plenty of existing modules for working with INI-style files, including Config::Tiny, Config::INI, and Config::IniFiles, just to name a few.

How do I access values in the data structure returned by XML::Simple?

Hi everyone,
This is very simple for perl programmers but not beginners like me,
I have one xml file and I processed using XML::Simple like this
my $file="service.xml";
my $xml = new XML::Simple;
my $data = $xml->XMLin("$file", ForceArray => ['Service','SystemReaction',
'Customers', 'Suppliers','SW','HW'],);
Dumping out $data, it looks like this:
$data = {
'Service' => [{
'Suppliers' => [{
'SW' => [
{'Path' => '/work/service.xml', 'Service' => 'b7a'},
{'Path' => '/work/service1.xml', 'Service' => 'b7b'},
{'Path' => '/work/service2.xml', 'Service' => 'b5'}]}
],
'Id' => 'SKRM',
'Customers' =>
[{'SW' => [{'Path' => '/work/service.xml', 'Service' => 'ASOC'}]}],
'Des' => 'Control the current through the pipe',
'Name' => ' Control unit'
},
{
'Suppliers' => [{
'HW' => [{
'Type' => 'W',
'Path' => '/work/hardware.xml',
'Nr' => '18',
'Service' => '1'
},
{
'Type' => 'B',
'Path' => '/work/hardware.xml',
'Nr' => '7',
'Service' => '1'
},
{
'Type' => 'k',
'Path' => '/work/hardware.xml',
'Nr' => '1',
'Service' => '1'
}]}
],
'Id' => 'ADTM',
'Customers' =>
[{'SW' => [{'Path' => '/work/service.xml', 'Service' => 'SDCR'}]}],
'Des' => 'It delivers actual motor speed',
'Name' => ' Motor Drivers and Diognostics'
},
# etc.
],
'Systemreaction' => [
# etc.
],
};
How to access each elements in the service and systemReaction(not provided). because I am using "$data" in further processing. So I need to access each Id,customers, suppliers values in each service. How to get particular value from service to do some process with that value.for example I need to get all Id values form service and create nodes for each id values.
To get Type and Nr value I tried like this
foreach my $service (#{ $data->{Service}[1]{Suppliers}[0]{HW}[0] }) {
say $service->{Nr};
}
foreach my $service (#{ $data->{Service}[1]{Suppliers}[0]{HW}[0] }) {
say $service->{Type};
}
can you help me how to get all Nr and Type values from Supplier->HW.
I suggest reading perldocs Reference Tutorial and References and Nested Data Structures. They contain an introduction and full explanation of how to access data like that.
But, for example, you can access the service ID by doing:
say $data->{Service}[0]{Id} # prints SKRM
You could go through all the services, printing their ID, with a loop:
foreach my $service (#{ $data->{Service} }) {
say $service->{Id};
}
In response to your edit
$data->{Service}[1]{Suppliers}[0]{HW}[0] is an hash reference (you can check this quickly by either using Data::Dumper or Data::Dump on it, or just the ref function). In particular, it is { Nr => 18, Path => "/work/hardware.xml", Service => 1, Type => "W" }
In other words, you've almost got it—you just went one level too deep. It should be:
foreach my $service (#{ $data->{Service}[1]{Suppliers}[0]{HW} }) {
say $service->{Nr};
}
Note the lack of the final [0] that you had.