how to assign lines of variable length to a variable in perl? - perl

I have a file I want to read that has a variable number of ids for each location that looks like this:
loc1 id1 id4 id5 id9
loc2 id2
loc3 id1 id11 id23
I would like to store this as follows locs(loc) = {all ids belonging to that location}
So that later, when I read another file I can do something like
if (grep id, locs(loc)){do something}
I tried to do this using a hash, but this is not working. I tried:
open my $loclist, '<', $ARGV[0];
my %locs;
while (<$loclist>) {
my #loclist_rec = split;
my $loclist_loc = #rlist_rec[0];
$locs{$loclist_loc} = #loclist_rec;
}
but this isnt working.
I new to perl and still trying to understand the different datatypes.
Any ideas? Thanks a lot!

This should do what you want.
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
open my $loclist, '<', "test.txt" or die $!;
my %locs;
while (<$loclist>) {
my ($loclist_loc, #loclist_rec) = split;
$locs{$loclist_loc} = \#loclist_rec;
}
print Dumper \%locs;
OUTPUT:
$ perl test.pl
$VAR1 = {
'loc2' => [
'id2'
],
'loc1' => [
'id1',
'id4',
'id5',
'id9'
],
'loc3' => [
'id1',
'id11',
'id23'
]
};

Also a possible choice would be a hash of hashes. When you want to look up an id, you could say if ($locs{$loc}{$id}) {do something}. The data structure would be
$VAR1 = {
'loc2' => {
'id2' => 1
},
'loc1' => {
'id1' => 1,
'id5' => 1,
'id4' => 1,
'id9' => 1
},
'loc3' => {
'id1' => 1,
'id11' => 1,
'id23' => 1
}
};

Related

Get key details from the value through grep

I am trying to find the key name as output by matching $country_value variable in grep through the hash I have.
#!/usr/bin/perl -w
use strict;
use warnings;
my $country_value = 1;
my $country = {
'IN' => [
1,
5
],
'US' => [
2,
6
],
'UK' => [
3,
7
]
};
my $country_details = grep { $_ eq $country_value } values %{$country};
print $country_details;
print "\n";
As per the hash, I need to get the output as IN because the value of IN is 1 and the $country_value is 1, which is what I am trying to find out.
But, I get the output as 0 instead of IN.
Can someone please help?
In your code, values returns a reference to an array. You need to dereference that to get a list for grep.
use warnings;
use strict;
my $country_value = 1;
my $country = {
'IN' => [
1,
5
],
'US' => [
2,
6
],
'UK' => [
3,
7
]
};
my $country_details;
for my $name (keys %{$country}) {
if (grep { $_ == $country_value } #{ $country->{$name} }) {
$country_details = $name;
last;
}
}
print $country_details, "\n";
Prints:
IN

Perl XML::Simple output nested node

As input I have
$XML = {'node1' => {'node2' => {'node3' => {'node4'}}}};
then I generate the out xml
print(XMLout($XML, KeepRoot => 1));
and I get
<node1>
<node2 name="node3" node4="" />
</node1>
How can I get this as output
<node1>
<node2>
<node3>
<node4></node4>
</node3>
</node2>
</node1>
perl -MXML::Simple -e '$XML = {"n1" => {"n2" => [{"n3" => [{"n4"=>[{}]}]}]}};
print(XMLout($XML, KeepRoot=>1));'
Gives
<n1>
<n2>
<n3>
<n4></n4>
</n3>
</n2>
</n1>
The use of XML::Simple is discouraged. https://metacpan.org/pod/XML::Simple
XML::Simple isn't actually simple to use. You are better off with XML::LibXML.
Here is an example that shows how you can find how the data structure should look.
use warnings;
use strict;
use XML::Simple;
use Data::Dumper;
my $sample =
"<node1>
<node2>
<node3>
<node4></node4>
</node3>
</node2>
</node1>";
my $xs = XML::Simple->new(ForceArray => 1, KeepRoot => 1);
my $ref = $xs->XMLin($sample);
print "\n---------------\n", Dumper($ref), "\n--------------\n";
my $xml = $xs->XMLout($ref);
print "\n----------------\n";
print $xml;
The output produced from Data::Dumper shows the data structure.
---------------
$VAR1 = {
'node1' => [
{
'node2' => [
{
'node3' => [
{
'node4' => [
{}
]
}
]
}
]
}
]
};
--------------
----------------
<node1>
<node2>
<node3>
<node4></node4>
</node3>
</node2>
</node1>
Without ForceArray it looks like the output you received.

Turning a set of parent-child relationships into a hierarchical structure

I have an LDAP directory that I'm querying using Net::LDAP. This gives me a set of parent-child relationships.
It's a directory of people - and includes a 'manager' DN (which is another field within the directory).
I'm having real trouble turning this manager->person set of records into a hierarchical structure.
What I've got so far is:
#!/usr/bin/env perl
use strict;
use warnings;
use Net::LDAP;
use Data::Dumper;
my %people;
my $ldap = Net::LDAP->new('my_ldap_server');
my $result = $ldap->bind('bind_dn');
die if $result->code;
my $search = $ldap->search(
base => 'ou=yaddayadda',
scope => 'subtree',
filter => 'objectClass=person',
attrs => ['manager'],
);
foreach my $found ( $search->entries ) {
my $mgr = $found->get_value('manager');
my $dn = $result->dn;
push( #{ $people{$mgr} }, $dn );
}
What this gives me is a hash of managers and the people who work for them (using DN, which is unique).
An entry from %people looks like:
$VAR1 = {
'cn=Firstname Lastname,ou=OrgUnit' => [
'cn=Personame Lastname,ou=OrgUnit',
'cn=AnotherPerson NameHere,ou=OrgUnit',
],
'cn=AnotherPerson NameHere,ou=OrgUnit' => [
'cn=Someone Else,ou=OrgUnit',
]
};
But I'm having trouble with turning that parent-child mapping into a hierarchical structure.
e.g.:
'ceo' => [
'pa' => [],
'head_of_dept' => [
'person' => [],
'person_with_staff' => [ 'person3', 'person4' ]
]
]
I'm at something of a loss for how to accomplish this. It seems it shouldn't be too hard to do, given that each person is unique within the organisation structure.
NB - in the above, I've got cn=AnotherPerson NameHere,ou=OrgUnit who has a subordinate, and I'm after making a nested mapping out of this:
e.g.:
$VAR1 = {
'cn=Firstname Lastname,ou=OrgUnit' => [
'cn=Personame Lastname,ou=OrgUnit',
'cn=AnotherPerson NameHere,ou=OrgUnit',
[
'cn=Someone Else,ou=OrgUnit'
]
]
};
What you need is a directed graph, and I suggest using the Graph::Directed module, whose methods are documented in Graph
This program will build the graph for you, but without any data I couldn't test it beyond making sure it compiles
use strict;
use warnings 'all';
use feature 'say';
use Net::LDAP;
use Graph::Directed;
use Data::Dumper;
my $ldap = Net::LDAP->new('my_ldap_server');
my $result = $ldap->bind('bind_dn');
die if $result->code;
my $search = $ldap->search(
base => 'ou=yaddayadda',
scope => 'subtree',
filter => 'objectClass=person',
attrs => ['manager'],
);
my $g = Graph::Directed->new;
for my $found ( $search->entries ) {
my $mgr = $found->get_value('manager');
my $dn = $result->dn;
$g->add_edge($mgr, $dn);
}
say $g;
The resulting Graph::Directed object has a stringification overload so you can examine it superficially by simply printing it, but when you want to interrogate the structure further you will need to know some of the terms of graph theory. For instance, $g->source_vertices will return a list of all nodes that have descendants but no parents—in this case, a list of senior management, or $g->is_cyclic will return true if your data has any loops anywhere
Here's an example of a program that uses your brief sample data to display a hierarchical tree of nodes
use strict;
use warnings 'all';
use Graph::Directed;
my $data = {
'cn=Firstname Lastname,ou=OrgUnit' => [
'cn=Personame Lastname,ou=OrgUnit',
'cn=AnotherPerson NameHere,ou=OrgUnit',
],
'cn=AnotherPerson NameHere,ou=OrgUnit' =>
[ 'cn=Someone Else,ou=OrgUnit', ]
};
my $g = Graph::Directed->new;
for my $mgr ( keys %$data ) {
$g->add_edge($mgr, $_) for #{ $data->{$mgr} };
}
dump_tree($_) for $g->source_vertices;
sub dump_tree {
my ($node, $level) = ( #_, 0);
print ' ' x $level, $node, "\n";
dump_tree($_, $level+1) for $g->successors($node);
}
output
cn=Firstname Lastname,ou=OrgUnit
cn=AnotherPerson NameHere,ou=OrgUnit
cn=Someone Else,ou=OrgUnit
cn=Personame Lastname,ou=OrgUnit
#Hunter McMillen unfortunately deleted his very good but slightly off answer. Here is my attempt to augment his code by turning the relationship from underling -> boss towards boss -> underlings.
To simulate the LDAP responses, I created a simple Moose class.
package Person;
use Moose;
has name => ( is => 'ro' );
has boss => ( is => 'ro', predicate => 'has_boss' );
package main;
use strict;
use warnings;
use Data::Printer;
# make a randomized list of people
my %people = map { $_->name => $_ }
map {
Person->new(
name => $_->[0], ( $_->[1] ? ( boss => $_->[1] ) : () )
)
} (
[qw( ceo )], [qw( head_of_dept ceo)],
[qw( person head_of_dept)], [qw( person_with_staff head_of_dept )],
[qw( person3 person_with_staff )], [qw( person4 person_with_staff )],
);
my %manages;
foreach my $p (values %people) {
push #{ $manages{ $p->boss } }, $p->name if $p->has_boss;
}
# this part shamelessly stolen from #HunterMcMillen's deleted answer
sub build_tree {
my ($person) = #_;
my #subtrees;
foreach my $managee ( #{ $manages{$person} } ) {
push #subtrees, build_tree($managee);
}
return { $person => \#subtrees };
}
p build_tree 'ceo';
Here's the output.
\ {
ceo [
[0] {
head_of_dept [
[0] {
person []
},
[1] {
person_with_staff [
[0] {
person4 []
},
[1] {
person3 []
}
]
}
]
}
]
}
This should be more or less what you want.

Perl: overwrite structure member value

I am creating $input with this code:
push(#{$input->{$step}},$time);, then I save it in an xml file, and at the next compiling, I read it from that file. When i print it, i get the structure bellow.
if(-e $file)
my $input =XMLin($file...);
print Dumper $input;
and I get this structure
$VAR1 = {
'opt' => {
'step820' => '0',
'step190' => '0',
'step124' => '0',
}
};
for each step with it's time..
push(#{$input->{$step}},$time3);
XmlOut($file, $input);
If I run the program again, I get this structure:
$VAR1 = {
'opt' => {
'step820' => '0',
'step190' => '0',
'step124' => '0',
'opt' => {
'step820' => '0',
'step190' => '0',
'step124' => '0'
}
}
I just need to overwrite the values of steps(ex:$var1->opt->step820 = 2). How can i do that?
I just need to overwrite the values of steps(ex:$var1->opt->step820 = 2). How can i do that?
$input->{opt}->{step820} = 2;
I'm going to say what I always do, whenever someone posts something asking about XML::Simple - and that is that XML::Simple is deceitful - it isn't simple at all.
Why is XML::Simple "Discouraged"?
So - in your example:
#!/usr/bin/env perl
use strict;
use warnings;
use XML::Twig;
my $xml= XML::Twig->new->parsefile($file);
$xml -> get_xpath('./opt/step820',0)->set_text("2");
$xml -> print;
The problem is that XML::Simple is only any good for parsing the type of XML that you didn't really need XML for in the first place.
For more simple examples - have you considered using JSON for serialisation? As it more directly reflects the hash/array structure of native perl data types.
That way you can instead:
print {$output_fh} to_json ( $myconfig, {pretty=>1} );
And read it back in:
my $myconfig = from_json ( do { local $/; <$input_fh> });
Something like:
#!/usr/bin/env perl
use strict;
use warnings;
use JSON;
my $input;
my $time = 0;
foreach my $step ( qw ( step820 step190 step124 ) ) {
push(#{$input->{$step}},$time);
}
print to_json ( $input, {pretty=>1} );
Giving resultant JSON of:
{
"step190" : [
0
],
"step820" : [
0
],
"step124" : [
0
]
}
Although actually, I'd probably:
foreach my $step ( qw ( step820 step190 step124 ) ) {
$input->{$step} = $time;
}
print to_json ( $input, {pretty=>1} );
Which gives;
{
"step190" : 0,
"step124" : 0,
"step820" : 0
}
JSON uses very similar conventions to perl - in that {} denote key value pairs (hashes) and [] denote arrays.
Look at the RootName option of XMLout. By default, when "XMLout()" generates XML, the root element will be named 'opt'. This option allows you to specify an alternative name.
Specifying either undef or the empty string for the RootName option will produce XML with no root elements.

Populating an array of hashes with arrays of hashes

I am currently developing a piece of monitoring software that takes an input file of server names and ip addresses and creates a rudimentary database of information. I want to default some values as it processes the config file and it works fine for the first time round the loop but any subsequent entries get created with weird (well weird to me was the best way to describe it as it is probably correct and the code is wrong, as in the code is doing exactly what i have asked it to do but not necessarily what i want it to do).
the output from the code below looks like:
$VAR1 = [
{
'IPAddress' => '196.8.150.163',
'Boxname' => 'MPLRDFDSOAK1',
'CurrentStatusInfo' => {
'LineHandlersRunning' => [
{
'NumberOfGaps' => 0,
'LineHandlerName' => 'DEFAULT',
'NumberOfCommLinkDowns' => 0,
'LineHandlerUpTime' => 0,
'MemoryUsage' => 0
}
]
},
'PreviousStatusInfo' => {
'LineHandlersRunning' =>
$VAR1->[0]{'CurrentStatusInfo'}{'LineHandlersRunning'}[0]
]
}
},
{
'IPAddress' => '196.8.150.164',
'Boxname' => 'MPLRDFDSOAK2',
'CurrentStatusInfo' => {
'LineHandlersRunning' =>
$VAR1->[0]{'CurrentStatusInfo'}{'LineHandlersRunning'}
},
'PreviousStatusInfo' => {
'LineHandlersRunning' =>
$VAR1->[0]{'PreviousStatusInfo'}{'LineHandlersRunning'}
}
}
];
The following is the code:
#######################################################################################
# Version History #
#######################################################################################
# example of the ini file
#box=>MPLRDFDSOAK1;ip=>196.8.150.163
#box=>MPLRDFDSOAK2;ip=>196.8.150.164
use strict;
use warnings;
# include the library to allow easy access to command line arguments
use Getopt::Long;
# include the data dumper utility
use Data::Dumper;
my $usageInstructions = "Some instructions\n";
my $showMeTheInstructions = "";
my $iniFileToReadIn = "";
my #boxes;
# read in the command line arguments
GetOptions( "ini=s" => \$iniFileToReadIn,
"H|h|?!" => \$showMeTheInstructions);
if ($showMeTheInstructions)
{
print $usageInstructions;
exit 0;
}
readInINIFileIn($iniFileToReadIn, \#boxes) if ($iniFileToReadIn ne "");
print Dumper(\#boxes);
print "\n\#\n\# END OF DATA DUMP\n\#\n\n";
exit 0;
#######################################################################################
# subroutine to read in the ini file and create the empty records for the boxes
# specified
sub readInINIFileIn
{
my ($iniFile, $pointerToBoxes) = #_;
my $noCRLFOnString = "";
# open the file
open (ConfigFile, "<$iniFile") || die $!;
# read in all the lines into an array
my #configurationItems = <ConfigFile>;
# close the file
close (ConfigFile);
# temporary record storage
my %tempRecord;
# create the defaults for all boxes
my #LineHandlersRunning;
my %tmpLineHandlerRunning = ( LineHandlerName => "DEFAULT",
LineHandlerUpTime => 0,
NumberOfCommLinkDowns => 0,
NumberOfGaps => 0,
MemoryUsage => 0 );
push (#LineHandlersRunning, {%tmpLineHandlerRunning});
my %CurrentStatusInfo;
my %PreviousStatusInfo;
push #{ $CurrentStatusInfo{'LineHandlersRunning'} }, #LineHandlersRunning;
push #{ $PreviousStatusInfo{'LineHandlersRunning'} }, #LineHandlersRunning;
# loop through the config file and create the defaults for the database of boxes
foreach my $configLine (#configurationItems)
{
my #TokenisedLineFromFileItems = ();
my #TokenisedLineFromFileNameValuePairs = ();
# store parameters
# each line will be ; separated then => separated, as in each one will have a number of items separated by ;'s and
# each item will be be a name & value pair separated by =>'s
#TokenisedLineFromFileItems = split(/;/,$configLine);
# remove quote marks around the outside of each element of the newly created array
s/^"|"$//g foreach #TokenisedLineFromFileItems;
# create information in database record to add to boxes
foreach my $NameValuePair (#TokenisedLineFromFileItems)
{
#TokenisedLineFromFileNameValuePairs = split(/=>/,$NameValuePair);
$noCRLFOnString = $TokenisedLineFromFileNameValuePairs[1];
$noCRLFOnString =~ s/(\n|\r)//g;
$tempRecord{'Boxname'} = $noCRLFOnString if ($TokenisedLineFromFileNameValuePairs[0] eq "box");
$tempRecord{'IPAddress'} = $noCRLFOnString if ($TokenisedLineFromFileNameValuePairs[0] eq "ip");
}
# add all other defaults as blank
$tempRecord{'CurrentStatusInfo'} = {%CurrentStatusInfo};
$tempRecord{'PreviousStatusInfo'} = {%PreviousStatusInfo};
push(#$pointerToBoxes, {%tempRecord});
}
}
I don't have the patience to wade through all of your code, but I'll bet your problem is related to this aspect of the Data::Dumper output:
$VAR1->[0]{'CurrentStatusInfo'}{'LineHandlersRunning'}[0]
In other words, your data structure contains a reference to other parts of the structure.
Perhaps you think you are making a copy of part of the data structure, but instead you are getting a shallow copy rather than a deep copy? For example, I'm suspicious of this code:
$tempRecord{'CurrentStatusInfo'} = {%CurrentStatusInfo};
$tempRecord{'PreviousStatusInfo'} = {%PreviousStatusInfo};
If indeed the problem is related to shallow copying, the Clone module might help.
Use lexical filehandles, declare variables in the smallest possible scope. I do not know what your problem is, but it is most likely caused by some variable persisting longer than you think it does.
I'm guessing it's because these two lines end up pushing the same hash reference into two locations - so if you alter the hashref contents in one location, the other will change as well which is probably not what you want for default values.
As FM pointed out, this is why you have the circular reference in your Dumper output.
If someone I'm waiting to get off the phone takes long enough i'll refactor your code for you.
Update: ok, so without knowing the full scenario it's hard to say if this is a sensible approach. certainly you should look at the various INI parsing modules in CPAN, but here is a very quick tweak of your code, leaving your existing logic structure in place:
use strict;
use warnings;
use Getopt::Long;
use Data::Dumper;
my $cmd_help = "Some instructions\n";
my $show_help = "";
my $ini_file_path = "";
# read in the command line arguments
GetOptions( "ini=s" => \$ini_file_path,
"H|h|?!" => \$show_help );
if ($show_help) {
print $cmd_help;
exit 0;
}
if (! -f $ini_file_path) {
die "File '$ini_file_path' doesn't seem to exist.";
}
my $boxes = read_ini_file($ini_file_path);
print Dumper($boxes);
exit 0;
=head2 read_ini_file
read in the ini file and create the empty records for the boxes
=cut
sub read_ini_file {
my ($ini_file) = #_;
my #boxes;
my #config_lines;
{
# consider using File::Slurp
open (my $ini_fh, '<', $ini_file_path) || die $!;
#config_lines = <$ini_fh>;
chomp #config_lines; # remove \r\n
# file handle will close when $ini_fh goes out of scope
}
# create the defaults for all boxes
my %line_handlers_running_defaults = ( LineHandlerName => "DEFAULT",
LineHandlerUpTime => 0,
NumberOfCommLinkDowns => 0,
NumberOfGaps => 0,
MemoryUsage => 0 );
# loop through the config file and create the defaults for the database of boxes
foreach my $line (#config_lines) {
my %record;
my #token_pairs = map { s/^"//; s/^$//; $_ } split(/;/,$line);
# create information in database record to add to boxes
foreach my $pair (#token_pairs) {
my ($key, $val) = split(/=>/,$pair);
$record{Boxname} = $val if $key eq "box";
$record{IPAddress} = $val if $key eq "ip";
}
# add all other defaults as blank
$record{CurrentStatusInfo} = { LineHandlersRunning => [{%line_handlers_running_defaults}] };
$record{PreviousStatusInfo} = { LineHandlersRunning => [{%line_handlers_running_defaults}] };
push #boxes, \%record;
}
return \#boxes;
}
gives this output:
$VAR1 = [
{
'IPAddress' => '196.8.150.163',
'CurrentStatusInfo' => {
'LineHandlersRunning' => [
{
'NumberOfGaps' => 0,
'LineHandlerName' => 'DEFAULT',
'NumberOfCommLinkDowns' => 0,
'LineHandlerUpTime' => 0,
'MemoryUsage' => 0
}
]
},
'Boxname' => 'MPLRDFDSOAK1',
'PreviousStatusInfo' => {
'LineHandlersRunning' => [
{
'NumberOfGaps' => 0,
'LineHandlerName' => 'DEFAULT',
'NumberOfCommLinkDowns' => 0,
'LineHandlerUpTime' => 0,
'MemoryUsage' => 0
}
]
}
},
{
'IPAddress' => '196.8.150.164',
'CurrentStatusInfo' => {
'LineHandlersRunning' => [
{
'NumberOfGaps' => 0,
'LineHandlerName' => 'DEFAULT',
'NumberOfCommLinkDowns' => 0,
'LineHandlerUpTime' => 0,
'MemoryUsage' => 0
}
]
},
'Boxname' => 'MPLRDFDSOAK2',
'PreviousStatusInfo' => {
'LineHandlersRunning' => [
{
'NumberOfGaps' => 0,
'LineHandlerName' => 'DEFAULT',
'NumberOfCommLinkDowns' => 0,
'LineHandlerUpTime' => 0,
'MemoryUsage' => 0
}
]
}
}
];