the analysis of my $file = ${$chainro->{$ro}->{$id}}[$i]; - perl
I have a two level hash %chainro , each key of $chainro{$ro}{$id}points to an array. The following code is to iterate through the first level of hash, $chainro->{$ro}. I can guess what
does my $file = ${$chainro->{$ro}->$id}}[$i]; aim to perform. However, I do not know why ${$chainro->{$ro}->{$id}}was written this way? In specific, why do we need to add ${ } to wrap the $chainro->${ro}->{$id}
foreach my $id (keys %{$chainro->{$ro}})
{
$size = $#{$chainro->{$ro}->{$id}};
for ($i=0; $i<$size; $i++)
{
my $file = ${$chainro->{$ro}->{$id}}[$i];
}
}
${ EXPR1 }[ EXPR2 ]
is an alternate way of writing
EXPR1->[ EXPR2 ]
so
${ $chainro->{$ro}->{$id} }[$i]
can be written as
$chainro->{$ro}->{$id}->[$i]
or even as
$chainro->{$ro}{$id}[$i]
Cleaned up:
for my $id (keys %{ $chainro->{$ro} }) {
my $files = $chainro->{$ro}{$id};
for my $i (0..$#$files) {
my $file = $files->[$i];
...
}
}
Or if you don't need $i:
for my $id (keys %{ $chainro->{$ro} }) {
my $files = $chainro->{$ro}{$id};
for my $file (#$files) {
...
}
}
It is to dereference a reference to something.
The something is an array here.
Related
How to access a local variable with in a foreach loop outside of its scope?
I have two arrays of string data type and i am comparing those using foreach loops and raising the counter when a match is found
my #array1 = ('India');
my #array2 = ('India', 'America');
foreach my $a (#array1) {
foreach my $b (#array2) {
my $count=0;
if($a eq $b) {
$count++;
}
}
}
Now I want to use this count variable outside of its scope
if ($count > 0) {
call_some_function();
}
Where am I going wrong?
$count is declared into the foreach loop, it doesn't exist outside this loop, if you want to use $count outside the loop, simply put my $count=0 before the first foreach and remove the one into the foreach
here's what you want:
my #array1=('India');
my #array2=('India','America');
my $count=0;
foreach my $country1(#array1)
{
foreach my $country2(#array2)
{
if($country1 eq $country2)
{
$count++;
}
}
}
Declare variable outside of the loops, you've also got a typo in $count:
my $count = 0;
foreach $a (#array1) {
foreach $b (#array2) {
if ( $a eq $b ) {
$count++;
}
}
}
Perl: How to access an array thats inside of 3 hashes passed to subroutine by reference
I have a code that is something like this:
foreach $item (#total_data)
{
setinfo($item);
} # #total_data contains an array of references to hashes (\%hash1 ... \%hashN)
In the subrutine goes something like this:
sub setinfo
{
my ($argument) = #_;
my $i = 0;
#inside original hash $argument{"data"}{"fulldraw"} there is an [array]
#that contains numbers of the form XYYZ and I want to split them into
#the following pairs XY YY YZ but that code works ok#
foreach $item (${$argument{"data"}{"fulldraw"}})
{
my $match;
my $matchedstr;
if ($item =~ /^\d{4}$/)
{
...
}
else
{
print STDERR "DISCARDED: $item\n";
}
}
}
I know I am probably making the mistake in how I am dereferencing it, but couldn't figure it out with all the articles I've read on the internet.
Thanks!
#{ ... } # dereference
Maybe $argument is a hashref; you need to use
foreach $item (#{ $argument->{data}->{fulldraw} })
Just use dereference #{ ... }:
foreach $item (#{ $argument->{data}{fulldraw} })
You can use Data::Dumper to visualize complex structures:
use Data::Dumper;
print Dumper($argument);
Yaml input perl eternal looping issue
$description is Input from a yaml file of format
main_key:
-
key1:value2
key2:value2
-
key1:value1
key2:value2
Basically that is a hash of array of hashes.
I input $description and process the inner hash as follows:
while ( my ( $mod, $defined ) = each %{ $description } ) {
my $index = 0;
foreach ( #{ $defined } ) {
while ( my ( $key, $value ) = each %{ $_ } ) {
process ( $key, $mod, $description, $index );
}
$index = $index + 1;
}
}
When certain 'keyword' is used as a key I replace add more key,value pairs to the inner hash
function1() and function2() return a hash pointer.
sub process {
my ( $key, $mod, $description, $index ) = #_;
my $parameters;
if ( $key eq 'keyword' ) {
$parameters = function1( );
}
else {
$parameters = function2( );
}
$description->{$mod}[$index] = { %$parameters, %{$description->{$mod}[$index]} };
}
The issue here is that "while ( my ( $key, $value ) = each %{ $_ } )" in the main code runs forever, using the same key and value over and over again.
Yeah. Don't do that. Never modify a hash while looping over it. From perldoc -f each: If you add or delete a hash's elements while iterating over it, entries may be skipped or duplicated--so don't do that. The general pattern is to build up the list of modifications, and then make them after the end of the loop. You can, of course, embed that sequence in an outer loop that iterates over the hash until there are no more modifications that need to be made.
This refactoring of your code works fine. I have rewritten process to do all that is necessary for the innermost hashes. I have named these $item as I don't know what they are supposed to represent. Please amend this to something more descriptive.
There never was any reason to pass all those parameters, as the values of $description, $mod, and $index were only used to locate the hash in question using $description->{$mod}[$index] so it may as well have been passed directly as a reference, which is what I do. In addition, because process now loops over the array contents there is no need to pass $key either, so the subroutine now has just one parameter.
Each element of $item is examined, and the new hash of data to be added for that element is obtained from function1 or function2 as appropriate and pushed onto #params instead of being inserted straight away.
Once all the new values have been established, they are all added into $item and the process is complete.
for my $defined (values %$description) {
process($_) for #$defined;
}
sub process {
my ($item) = #_;
my #params;
for my $key (keys %$item) {
push #params, $key eq 'keyword' ? function1() : function2();
}
for my $params (#params) {
#{$item}{keys %$params} = values %{$params};
}
}
Powershell / Perl : Merging multiple CSV files into one?
I have the following CSV files, I want to merge these into a single CSV
01.csv
apples,48,12,7
pear,17,16,2
orange,22,6,1
02.csv
apples,51,8,6
grape,87,42,12
pear,22,3,7
03.csv
apples,11,12,13
grape,81,5,8
pear,11,5,6
04.csv
apples,14,12,8
orange,5,7,9
Desired output:
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,87,42,12,81,5,8,,,
pear,17,16,2,22,3,7,11,5,6,,,
orange,22,6,1,,,,,,5,7,9
Can anyone provide guidance on how to achieve this? Preferably using Powershell but open to alternatives like Perl if that's easier.
Thanks Pantik, your code's output is close to what I want:
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,87,42,12,81,5,8
orange,22,6,1,5,7,9
pear,17,16,2,22,3,7,11,5,6
Unfortunately I need "placeholder" commas in place for when the entry is NOT present in a CSV file, e.g. orange,22,6,1,,,,,,5,7,9 rather than orange,22,6,1,5,7,9
UPDATE: I would like these parsed in order of the filenames, e.g.:
$myFiles = #(gci *.csv) | sort Name
foreach ($file in $myFiles){
regards
ted
Here is my Perl version:
use strict;
use warnings;
my $filenum = 0;
my ( %fruits, %data );
foreach my $file ( sort glob("*.csv") ) {
$filenum++;
open my $fh, "<", $file or die $!;
while ( my $line = <$fh> ) {
chomp $line;
my ( $fruit, #values ) = split /,/, $line;
$fruits{$fruit} = 1;
$data{$filenum}{$fruit} = \#values;
}
close $fh;
}
foreach my $fruit ( sort keys %fruits ) {
print $fruit, ",", join( ",", map { $data{$_}{$fruit} ? #{ $data{$_}{$fruit} } : ",," } 1 .. $filenum ), "\n";
}
Which gives me:
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,
So do you have a typo for grape or i have misunderstood something?
Ok, gangabass solution works, and is cooler than mine, but I'll add mine anyway. It is slightly stricter, and preserves a data structure that can be used as well. So, enjoy. ;)
use strict;
use warnings;
opendir my $dir, '.' or die $!;
my #csv = grep (/^\d+\.csv$/i, readdir $dir);
closedir $dir;
# sorting numerically based on leading digits in filename
#csv = sort {($a=~/^(\d+)/)[0] <=> ($b=~/^(\d+)/)[0]} #csv;
my %data;
# To print empty records we first need to know all the names
for my $file (#csv) {
open my $fh, '<', $file or die $!;
while (<$fh>) {
if (m/^([^,]+),/) {
#{ $data{$1} } = ();
}
}
close $fh;
}
# Now we can fill in values
for my $file (#csv) {
open my $fh, '<', $file or die $!;
my %tmp;
while (<$fh>) {
chomp;
next if (/^\s*$/);
my ($tag,#values) = split (/,/);
$tmp{$tag} = \#values;
}
for my $key (keys %data) {
unless (defined $tmp{$key}) {
# Fill in empty values
#{$tmp{$key}} = ("","","");
}
push #{ $data{$key} }, #{ $tmp{$key} };
}
}
&myreport;
sub myreport {
for my $key (sort keys %data) {
print "$key," . (join ',', #{$data{$key}}), "\n";
}
}
Powershell:
$produce = "apples","grape","orange","pear"
$produce_hash = #{}
$produce | foreach-object {$produce_hash[$_] = #(,$_)}
$myFiles = #(gci *.csv) | sort Name
foreach ($file in $myFiles){
$file_hash = #{}
$produce | foreach-object {$file_hash[$_] = #($null,$null,$null)}
get-content $file | foreach-object{
$line = $_.split(",")
$file_hash[$line[0]] = $line[1..3]
}
$produce | foreach-object {
$produce_hash[$_] += $file_hash[$_]
}
}
$ofs = ","
$out = #()
$produce | foreach-object {
$out += [string]$produce_hash[$_]
}
$out | out-file "outputfile.csv"
gc outputfile.csv
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,
Should be easy to modify for additional items. Just add them to the $produce array.
Second Powershell solution (as requested)
$produce = #()
$produce_hash = #{}
$file_count = -1
$myFiles = #(gci 0*.csv) | sort Name
foreach ($file in $myFiles){
$file_count ++
$file_hash = #{}
get-content $file | foreach-object{
$line = $_.split(",")
if ($produce -contains $line[0]){
$file_hash[$line[0]] += $line[1..3]
}
else {
$produce += $line[0]
$file_hash[$line[0]] = #(,$line[0]) + (#($null) * 3 * $file_count) + $line[1..3]
}
}
$produce | foreach-object {
if ($file_hash[$_]){$produce_hash[$_] += $file_hash[$_]}
else {$produce_hash[$_] += #(,$null) * 3}
}
}
$ofs = ","
$out = #()
$produce_hash.keys | foreach-object {
$out += [string]$produce_hash[$_]
}
$out | out-file "outputfile.csv"
gc outputfile.csv
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,
you have to parse the files, I don't see easier way hot to do it
solution in powershell:
UPDATE: ok, adjusted a bit - hopefully understandable
$items = #{}
$colCount = 0 # total amount of columns
# loop through all files
foreach ($file in (gci *.csv | sort Name))
{
$content = Get-Content $file
$itemsToAdd = 0; # columns added by this file
foreach ($line in $content)
{
if ($line -match "^(?<group>\w+),(?<value>.*)")
{
$group = $matches["group"]
if (-not $items.ContainsKey($group))
{ # in case the row doesn't exists add and fill with empty columns
$items.Add($group, #())
for($i = 0; $i -lt $colCount; $i++) { $items[$group] += "" }
}
# add new values to correct row
$matches["value"].Split(",") | foreach { $items[$group] += $_ }
$itemsToAdd = ($matches["value"].Split(",") | measure).Count # saves col count
}
}
# in case that file didn't contain some row, add empty cols for those rows
$colCount += $itemsToAdd
$toAddEmpty = #()
$items.Keys | ? { (($items[$_] | measure).Count -lt $colCount) } | foreach { $toAddEmpty += $_ }
foreach ($key in $toAddEmpty)
{
for($i = 0; $i -lt $itemsToAdd; $i++) { $items[$key] += "" }
}
}
# output
Remove-Item "output.csv" -ea 0
foreach ($key in $items.Keys)
{
"$key,{0}" -f [string]::Join(",", $items[$key]) | Add-Content "output.csv"
}
Output:
apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,
Here is a more consise way how to do it. However, it still doesn't add the commas when the item is missing.
Get-ChildItem D:\temp\a\ *.csv |
Get-Content |
ForEach-Object -begin { $result=#{} } -process {
$name, $otherCols = $_ -split '(?<=\w+),'
if (!$result[$name]) { $result[$name] = #() }
$result[$name] += $otherCols
} -end {
$result.GetEnumerator() | % {
"{0},{1}" -f $_.Key, ($_.Value -join ",")
}
} | Sort
Where does $_ come from in this Perl foreach loop?
I found this in Mail::IMAPClient. Where does the $_ in $SEARCH_KEYS{ uc($_) } come from?
sub _quote_search {
my ( $self, #args ) = #_;
my #ret;
foreach my $v (#args) {
if ( ref($v) eq "SCALAR" ) {
push( #ret, $$v );
}
elsif ( exists $SEARCH_KEYS{ uc($_) } ) {
push( #ret, $v );
}
elsif ( #args == 1 ) {
push( #ret, $v ); # <3.17 compat: caller responsible for quoting
}
else {
push( #ret, $self->Quote($v) );
}
}
return #ret;
}
That looks to me like a typo where the author converted an anonymous for loop foreach (#args) to one with an explicit iterator variable foreach my $v (#args) and forgot to convert all the incidences of $_ to $v. You should probably file a bug against the distribution on CPAN.
Even though this is probably a bug, lets consider how this code behaves.
The value of $_ will be determined by the current dynamic scope. What this means is that $_ will have whatever value (the dynamically scoped copy of) $_ has in the calling subroutine.
So for example if I have:
for (1 .. 5 ) {
foo();
bar();
}
sub foo {
print "\$_ = $_\n";
}
sub bar {
for ( 'a' .. 'c' ) {
foo();
}
}
You get output like:
$_ = 1
$_ = a
$_ = b
$_ = c
$_ = 2
$_ = a
$_ = b
$_ = c
...
It gets a little weirder in Perl 5.10 and up, where a lexical $_ exists.
for (1 .. 5 ) {
foo();
bar();
}
sub foo {
print "\$_ = $_\n";
}
sub bar {
my $_;
for ( 'a' .. 'c' ) {
foo();
}
}
Run this and get:
$_ = 1
$_ = 1
$_ = 1
$_ = 1
$_ = 2
$_ = 2
$_ = 2
$_ = 2
As you can see, if this isn't a bug, it's probably a bad idea.