I have two arrays roles added and roles deleted, want to merge them and write in this order in perl - text-processing

I have another scenario.
My file has the following fields:
Username | Roles | Type |date |
abc|admin |added | 01072015
abc|developer |deleted |01072015
abc|deploy |added |01072015
xyz |admin |deleted |01072015
xyz| deploy|deleted|01072015
cdf|deploy|added|01072015
Note here, the date is going to be the same day, so no change
now, I want this to be printed as
username |roles_added |roles deleted |date
abc |admin,deploy |developer |01072015
xyz ||admin,deploy |01072015
cdf |deploy||01072015
I tried the below approach given, but didn't work out for me. Please guide me.
#!/usr/bin/perl
open(FIL,"report.txt") or die("$!");
my %k=();
while (my $line=<FIL>) {
my ($user,$roles,$type,$dt)=split(/\|/,$line);
$k{$user}{$roles}=1;
$k{$user}{$type}=1;
}
my #names=(sort keys(%k));
foreach my $name (#names) {
foreach my $value ( (keys(%{$k{$name}})) ){
print "$value ";
}
print "$name\n";
}
print " i am here \n";
while( my ($k, $v) = each %$k ) {
print "key: $k, value: $v.\n";
}

When dealing with complex data structures, Data::Dumper is your friend. Try adding use Data::Dumper to your code and then print Dumper \%k just after you've finished building your %k hash. You'll see something like this:
$VAR1 = {
'xyz' => {
' deploy' => 1,
'deleted' => 1
},
'Username ' => {
' Type ' => 1,
' Roles ' => 1
},
'xyz ' => {
'deleted ' => 1,
'admin ' => 1
},
'cdf' => {
'deploy' => 1,
'added' => 1
},
'abc' => {
'deleted ' => 1,
'deploy ' => 1,
'developer ' => 1,
'added ' => 1,
'admin ' => 1
}
};
See how the keys in the sub-hashes are names after two types of things. Half of them are the role names and half of them are "added" or "deleted". It's going to be really hard to get anything useful out of that data structure, so let's try something different.
Where you had:
$k{$user}{$roles}=1;
$k{$user}{$type}=1;
Try this instead:
push #{$k{$user}{type}}, $role;
Now our data structure looks like this:
$VAR1 = {
'xyz' => {
'deleted' => [
' deploy'
]
},
'xyz ' => {
'deleted ' => [
'admin '
]
},
'cdf' => {
'added' => [
'deploy'
]
},
'abc' => {
'deleted ' => [
'developer '
],
'added ' => [
'admin ',
'deploy '
]
}
};
I think you can see that it's far easier to get the information you want out of this data structure. It's basically:
foreach (#names) {
print join ',', #{$k{$_}{added}};
print join ',', #{$k{$_}{deleted}};
}
You'll need to change the code a little to get exactly what you want.
Oh, and please get into the habit of adding use strict and use warnings to all of your code. They would have shown you why the debugging output at the end of your original code wasn't working.
Update: I threw together a complete solution.
#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
my %k;
<>; # Skip header
while (<>) {
chomp;
my ($user, $roles, $type, $dt) = split(/\s*\|\s*/);
push #{$k{$user}{$type}}, $roles;
$k{$user}{date} = $dt;
}
say 'username |roles_added |roles deleted |date';
foreach my $name (sort keys %k) {
say "$name |",
join(',',#{$k{$name}{added} || []}), ' |',
join(',',#{$k{$name}{deleted} || []}), ' |',
$k{$name}{date};
}
You need to pass it the name of the input file as a command line argument.

Related

Issue in Printing data from a hash table in perl

I am trying to process the data in a single file . i have to read the file and create a hash structure,get the value of fruitname append it to fruitCount and fruitValue and delete the line fruitName and write the entire output after the change is done.Given below is the content of file.
# this is a new file
{
date 14/07/2016
time 11:15
end 11:20
total 30
No "FRUITS"
Fruit_class
{
Name "fruit 1"
fruitName "apple.fru"
fruitId "0"
fruitCount 5
fruitValue 6
}
{
Name "fruit 2"
fruitName "orange.fru"
fruitId "1"
fruitCount 10
fruitValue 20
}
}
I tried with following code :
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my %hash_table;
my $name;
my $file = '/tmp/fruitdir/fruit1.txt';
open my $fh, "<", $file or die "Can't open $file: $!";
while (<$fh>) {
chomp;
if (/^\s*fruitName/) {
($name) = /(\".+\")/;
next;
}
s/(fruitCount|fruitValue)/$name\.$1/;
my ($key, $value) = split /\s+/, $_, 2;
$hash_table{$key} = $value;
}
print Dumper(\%hash_table);
This is not working . I need to append the value of fruitname and print the the entire file content as output. Any help will be appreciated.Given below is the output that i got.
$VAR1 = {
'' => undef,
'time' => '11:15 ',
'date' => '14/07/2016',
'{' => undef,
'#' => 'this is a new file',
'total' => '30 ',
'end' => '11:20 ',
'No' => '"FRUITS"',
'Fruit_class' => undef,
'}' => undef
};
Expected hash as output:
$VAR1 = {
'Name' => '"fruit 1"',
'fruitId' => '"0" ',
'"apple_fru".fruitValue' => '6 ',
'"apple_fru".fruitCount' => '5'
'Name' => '"fruit 2"',
'fruitId' => '"0" ',
'"orange_fru".fruitValue' => '10 ',
'"orange_fru".fruitCount' => '20'
};
One word of advice before I continue:
Document your code
There are several logic errors in your code which I think you would have recognized if you wrote down what you thought each line was supposed to do. First, write down the algorithm that you would like to implement, then document how each step in the code implements a step in the algorithm. At the end you'll be able to see what you missed, or what part is not working.
Here are the errors that I see
You aren't ignoring lines that you shouldn't be parsing. For example, you're grabbing the '}' and '{' lines.
You aren't actually storing the name of the fruit. You grab it, but immediately start the next loop without storing it.
You're not keeping track of each structure. You need to start a new structure for each fruit.
Do you really want to keep the double quotes in the values?
Other things to worry about:
Are you guaranteed that the list of attributes is in that order? For example, can Name come last?
Here's some code which does what I think you want.
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my %hash_table;
my $name;
my #fruit;
my $file = '/tmp/fruitdir/fruit1.txt';
open my $fh, "<", $file or die "Can't open $file: $!";
while (<$fh>) {
chomp;
# save hash table if there's a close bracket, but
# only if it has been filled
if ( /^\s*}\s*$/ ) {
next unless keys %hash_table;
# save COPY of hash table
push #fruit, { %hash_table };
# clear it out for the next iteration
%hash_table = ();
}
# only parse lines that start with Name or fruit
next unless
my ( $key, $value ) =
/^
# skip any leading spaces
\s*
# parse a line beginning with Name or fruitXXXXX
(
Name
|
fruit[^\s]+
)
# need space between key and value
\s+
# everything that follows is a value. clean up
# double quotes in post processing
(.*)
/x;
# remove double quotes
$value =~ s/"//g;
if ( $key eq 'Name' ) {
$name = $value;
}
else {
$key = "${name}.${key}";
}
$hash_table{$key} = $value;
}
print Dumper \#fruit;
and here's the output:
$VAR1 = [
{
'fruit 1.fruitValue' => '6',
'fruit 1.fruitName' => 'apple.fru',
'Name' => 'fruit 1',
'fruit 1.fruitCount' => '5',
'fruit 1.fruitId' => '0'
},
{
'fruit 2.fruitName' => 'orange.fru',
'fruit 2.fruitId' => '1',
'fruit 2.fruitCount' => '10',
'fruit 2.fruitValue' => '20',
'Name' => 'fruit 2'
}
];

Printing only parts of a hash

So I have a hash in perl setup that has multiple values for each field, i used this to push data into the hash:
push #{$user{$infoName}}, $information;.
eg. a user my have 3 favourite shows all stored as
{'favourite_TV_shows' => [ 'A Country Practice', 'All Saints', 'Falling Skies' ], 'weight' => [ '53kg' ]}
Some of this user info is private such as weight so the fields i want to show are stored in the array #fieldsToPrint = ['username','favourite_TV_shows']
how can i write a foreach loop to print only the fields that are in the feildsToPrint array.
The following is my attempt so far...
foreach ($user{$infoName} == #fieldsToPrint){
#print
}
Just iterate on your #fieldsToPrint array, skipping those keys that don't have a value:
use strict;
use warnings;
my #fieldsToPrint = ( 'username', 'favourite_TV_shows' );
my %user = (
'favourite_TV_shows' => [ 'A Country Practice', 'All Saints', 'Falling Skies' ],
'weight' => ['53kg'],
);
for my $key (#fieldsToPrint) {
next if !$user{$key};
print "$key = ", join(', ', #{ $user{$key} }), "\n";
}
Outputs:
favourite_TV_shows = A Country Practice, All Saints, Falling Skies

Reverse a linked list in perl

So I know that there are hundred examples on Stack overflow, and in fact i have used all the information from there - so this is what i have
use strict;
use warnings;
use Data::Dumper;
my $head= undef;
my $tail=\$head;
open FILE, "<datastored.txt" or die $!;
while (<FILE>){
my $node = {
"data" => $_ ,
"next" => undef
};
$$tail=$node;
$tail = \$node->{"next"};
};
print Dumper $head; #before reversing
$head = reverse_list($head);
print Dumper $head; #after reversing
sub reverse_list{
my ($list) =#_[0];
my $previous = undef;
while ($list->{next}){
$forward = $list->{next};
$list->{next}= $previous;
$previous = $list;
$list=$forward;
};
return $previous;
};
and this is the output I get
#this is the output before reversing (normal linked list)
$VAR1 = {
'next' => {
'next' => {
'next' => {
'next' => undef,
'data' => 'line 4
'
},
'data' => 'line 3
'
},
'data' => 'line 2
'
},
'data' => 'line 1
'
};
#this is the linked list after reversing (WITHOUT THE LAST DATA VARIABLE - "line 4")
$VAR1 = {
'next' => {
'next' => {
'next' => undef,
'data' => 'line 1
'
},
'data' => 'line 2
'
},
'data' => 'line 3
'
};
Note - the content of the file datastored.txt is simply
line 1
line 2
line 3
line 4
So my question is where is the data "line 4" gone and what should i change to ACTUALLY reverse the linked list without losing any value.
Your reversal sub-routine is almost correct. However, it misses the last entry (i.e. adding it in the final reversed list) because of the condition you are using. You have two options:
Change the while ($list->{next}) to while ($list) and make the code more idiomatic.
Add a $list->{next}= $previous; after the end of the while loop to add back the last remaining node to your reversed list. (Think of a list of two elements and see what your code does).

In Perl, how can I skip an empty key when traversing a hash?

This is my problem, I'm not very knowledgeable in Perl, and I have this function that needs to be fixed.
When this function deviceModelMenu() is called, the CLI displays the following text:
The following models are available
==================================================
1.
2. Cisco1240
3. Catalyst3750
4. Catalyst3650
5. HP2524
The first item is empty, which is wrong, and I need to fix that, the piece of code that displays this menu is:
my $features = shift;
print "=" x 50, "\n";
print "The following models are available\n";
print "=" x 50, "\n";
my $i=1;
foreach (keys %{$features->{features}[0]->{deviceModel}})
{
print "$i. $_ \n";
$i++;
}
If I add the following line:
warn Dumper($features->{features}[0]->{deviceModel});
It dumps this:
$VAR1 = {
'deviceModel' => {
'' => {
'cfg' => []
},
'Cisco1240' => {
'cfg' => [
'cisco1240feature.cfg'
]
},
'Catalyst3750' => {
'cfg' => [
'catalyst3750feature.cfg'
]
},
'Catalyst3650' => {
'cfg' => [
'catalyst3650feature.cfg'
]
},
'HP2524' => {
'cfg' => [
'hp2524feature.cfg'
]
}
}
};
As you may notice, the first item is indeed empty. I added the following line to skip it, and just print the rest of the info:
if ($_ eq '') {
shift;
}
But it doesn't seem to work do what I want. I want to skip the item if it's empty.
Well, shifting #ARGV (implicit argument to shift in main program) nor shifting #_ (implicit argument of shift in a function) are not going to help you, because you are not printing either of them.
You can either:
Not add the '' entry in the first place (depends on how it's generated)
Remove the '' entry before printing:
delete $features->{features}[0]->{deviceModel}->{''};
Don't print the entry:
if($_ eq '') {
next;
}
or
if($_ ne '') {
print "$i. $_ \n";
$i++;
}
foreach (keys %{$features->{features}[0]->{deviceModel}})
{
next unless length($_);
print "$i. $_ \n";
$i++;
}
#!/usr/bin/env perl
use strict; use warnings;
my $devices = {
'deviceModel' => {
'' => { 'cfg' => [] },
'Cisco1240' => { 'cfg' => ['cisco1240feature.cfg' ] },
'Catalyst3750' => { 'cfg' => [ 'catalyst3750feature.cfg' ]},
'Catalyst3650' => { 'cfg' => [ 'catalyst3650feature.cfg' ]},
'HP2524' => { 'cfg' => [ 'hp2524feature.cfg' ]},
}
};
{
my $item = 1;
for my $d (grep length, keys %{ $devices->{deviceModel} }) {
printf "%2d. %s\n", $item++, $d;
}
}
Output:
1. Catalyst3750
2. Cisco1240
3. Catalyst3650
4. HP2524

Perl - Removing unwanted elements from an arrayref

I'm writing a script that parses the "pure-ftpwho -s" command to get a list of the current transfers. But when a user disconnects from the FTP and reconnects back and resumes a transfer, the file shows up twice. I want to remove the ghosted one with Perl. After parsing, here is what the arrayref looks like (dumped with Data::Dumper)
$VAR1 = [
{
'status' => 'DL',
'percent' => '20',
'speed' => '10',
'file' => 'somefile.txt',
'user' => 'user1',
'size' => '14648'
},
{
'status' => 'DL',
'percent' => '63',
'speed' => '11',
'file' => 'somefile.txt',
'user' => 'user1',
'size' => '14648'
},
{
'status' => 'DL',
'percent' => '16',
'speed' => '60',
'file' => 'somefile.txt',
'user' => 'user2',
'size' => '14648'
}
];
Here user1 and user2 are downloading the same file, but user1 appears twice because the first one is a "ghost". What's the best way to check and remove elements that I don't need (in this case the first element of the arrayref). The condition to check is that - If the "file" key and "user" key is the same, then delete the hashref that contains the smaller value of "percent" key (if they're the same then delete all except one).
If order in the original arrayref doesn't matter, this should work:
my %users;
my #result;
for my $data (#$arrayref) {
push #{ $users{$data->{user}.$data->{file}} }, $data;
}
for my $value (values %users) {
my #data = sort { $a->{percent} <=> $b->{percent} } #$value;
push #result, $data[-1];
}
This can definitely be improved for efficiency.
The correct solution in this case would have been to use a hash when parsing the log file. Put all information into a hash, say %log, keyed by user and file:
$log{$user}->{$file} = {
'status' => 'DL',
'percent' => '20',
'speed' => '10',
'size' => '14648'
};
etc. Latter entries in the log file would overwrite earlier ones. Alternatively, you can choose to overwrite entries with lower percent completed with ones that have higher completion rates.
Using a hash would get rid of a lot of completely superfluous code working around the choice of the wrong data structure.
For what it's worth, here's my (slightly) alternative approach. Again, it doesn't preserve the original order:
my %most_progress;
for my $data ( sort { $b->{percent} <=> $a->{percent} } #$data ) {
next if exists $most_progress{$data->{user}.$data->{file}};
$most_progress{$data->{user}.$data->{file}} = $data;
}
my #clean_data = values %most_progress;
This will preserve order:
use strict;
use warnings;
my $data = [ ... ]; # As posted.
my %pct;
for my $i ( 0 .. $#{$data} ){
my $r = $data->[$i];
my $k = join '|', $r->{file}, $r->{user};
next if exists $pct{$k} and $pct{$k}[1] >= $r->{percent};
$pct{$k} = [$i, $r->{percent}];
}
#$data = #$data[sort map $_->[0], values %pct];
my %check;
for (my $i = 0; $i <= $#{$arrayref}; $i++) {
my $transfer = $arrayref->[$i];
# check the transfer for user and file
my $key = $transfer->{user} . $transfer->{file};
$check{$key} = { } if ( !exists $check{$key} );
if ( $transfer->{percent} <= $check{$key}->{percent} ) {
# undefine this less advanced transfer
$arrayref->[$i] = undef;
} else {
# remove the other transfer
$arrayref->[$check{$key}->{index}] = undef if exists $check{$key}->{index};
# set the new standard
$check{$key} = { index => $i, percent => $transfer->{percent} }
}
}
# remove all undefined transfers
$arrayref = [ grep { defined $_ } #$arrayref ];
Variation on the theme with Perl6::Gather
use Perl6::Gather;
my #cleaned = gather {
my %seen;
for (sort { $b->{percent} <=> $a->{percent} } #$data) {
take unless $seen{ $_->{user} . $_->{file} }++;
}
};