Perl Hash + File + While - perl

well, the idea is to remove a file a direction with their description and store it in a hash
this is content in file /home/opmeitle/files-pl/bookmarks2
}, {
"date_added": "12989744094664781",
"id": "1721",
"name": "Perl DBI - dbi.perl.org",
"type": "url",
"url": "http://dbi.perl.org/"
}, {
"date_added": "12989744373130384",
"id": "1722",
"name": "DBD::mysql - MySQL driver for the Perl5 Database Interface (DBI) - metacpan.org",
"type": "url",
"url": "https://metacpan.org/module/DBD::mysql"
}, {
now, the code in perl.
use strict;
open(FILE, '/home/opmeitle/files-pl/bookmarks2');
my #lines = <FILE>;
my #list55;
my $count = 1;
my $n = 0;
my %hash=(); #$hash{$lines[$n]}=$lines[$n];
while ($lines[$n]) {
if ($lines[$n] =~ /(http:|https:|name)/) {
if ($lines[$n] =~ s/("|: |,|id|url|name|\n)//g) {
if ($lines[$n] =~ s/^\s+//){
if ($lines[$n] =~ /http:|https/){
$hash{$lines[$n]} = '';
}
else {
$hash{$n} = $lines[$n];
}
}
}
}
$n++;
$count++;
}
close(FILE);
# print hash
my $key;
my $value;
while( ($key,$value) = each %hash){
print "$key = $value\n";
}
result after executing the script.
http://dbi.perl.org/ =
https://metacpan.org/module/DBD::mysql =
3 = Perl DBI - dbi.perl.org
9 = DBD::mysql - MySQL driver for the Perl5 Database Interface (DBI) - metacpan.org
but i need something like this
http://dbi.perl.org/ = Perl DBI - dbi.perl.org
Perl DBI - dbi.perl.org = DBD::mysql - MySQL driver for the Perl5 Database Interface (DBI) - metacpan.org
thanks for you answers.

As #amon hinted, Chrome bookmarks are JSON format, for which there are several good modules on CPAN.
use strict;
use warnings;
use JSON;
my $file = '/home/opmeitle/files-pl/bookmarks2';
open my $fh, '<', $file or die "$file: $!\n";
my $inhash = decode_json(join '', <$fh>);
close $fh;
my %outhash = map traverse($_), values %{ $inhash->{roots} };
sub traverse
{
my $hashref = shift;
if (exists $hashref->{children}) {
return map traverse($_), #{ $hashref->{children} };
} else {
return $hashref->{url} => $hashref->{name};
}
}
Now %outhash has the data you wanted.
EDIT: to help understand what's going on here:
use Data::Dumper;
print Dumper($inhash); # pretty-print the structure returned by decode_json

As others have said, the best thing to do is to load the JSON data into a Perl datastructure. This is easily done using the JSON module. Before we can do this, we need to read in the file. There are two ways to do this. The non-CPAN way:
# always ...
use strict;
use warnings;
my $file = '/home/opmeitle/files-pl/bookmarks2';
my $text = do {
open my $fh, '<', $file or die "Cannot open $file: $!\n";
local $/; #enable slurp
<$fh>;
};
or the CPAN way
# always ...
use strict;
use warnings;
use File::Slurp;
my $text = read_file $file;
Once you have the file read in, then decode
use JSON;
my $data = decode_json $text;
Please post a whole file and a better description of what you want and I would be glad to comment on a more formal way of traversing the datastructure.

Related

function call in perl

As a part of my course work I have been learning perl programming language for the first time in last the few weeks. I have been writing small functions and making function calls. I have written a function for string matching.
use strict;
use warnings;
sub find_multi_string {
my ($file, #strings) = #_;
my $fh;
open ($fh, "<$file");
#store the whole file in an array
my #array = <$fh>;
for my $string (#strings) {
if (grep /$string/, #array) {
next;
} else {
die "Cannot find $string in $file";
}
}
return 1;
}
find_multi_string('file name', 'string1','string2','string3','string4','string 5');
In the above script I'm passing the arguments in the function call. The script works.
But I'd like to know if there is way to specify the file name and string1... string n in an array in the program itself and just make the function call.
find_multi_string();
That would be a mistake, always pass parameters and return values to your subroutines.
What you're describing is essentially using subroutines solely to subdivide and document your code. If you were to do that, it would better to just remove the subroutine entirely and include a comment before the section of code.
Overall, your code looks good as is. You probably will want to use quotemeta though, and your logic can be simplified a little:
use strict;
use warnings;
use autodie;
sub find_multi_string {
my ($file, #strings) = #_;
# Load the file
my $data = do {
open my $fh, "<", $file;
local $/;
<$fh>
};
for my $string (#strings) {
if ($data !~ /\Q$string/) {
die "Cannot find $string in $file";
}
}
return 1;
}
find_multi_string('file name', 'string1','string2','string3','string4','string 5');
A few improvements of your original code:
use autodie
use 3-args open
as you want to check anywhere in the file, just load the file as a single string
if the matching string are just text without metacharacters from regexp, just use the index function
Your question is about passing the function arguments from your program.
I suspect that you are looking for #ARGV. See perlvar.
Here is the modified code:
use strict;
use warnings;
use autodie;
sub find_multi_string {
my ($file, #strings) = #_;
my $content = do {
open my $fh, '<', $file;
local $/;
<$fh>
};
foreach (#strings) {
die "Cannot find $string in $file" unless index($content, $_) >= 0;
}
return 1;
}
find_multi_string(#ARGV);

Can't call method "network" without a package or object reference at blib/lib/NetAddr/IP.pm

I'm trying to write a Perl script to take a list of IPv4 aggregates and another list of addresses and using NetAddr::IP to take each IP and compare to the list of aggregates to see if it belongs to any of the aggregates. I need to find which ones are not part of any of the list of aggregates I have.
I finally got past all of the Perl errors and now I'm getting some kind of error with the NetAddr::IP module it appears. Can anyone assist?
Here is the error I'm getting:
Can't call method "network" without a package or object reference at blib/lib/NetAddr/IP.pm (autosplit into blib/lib/auto/NetAddr/IP/compactref.al) line 1075.
And here is the code I'm using:
#!/usr/bin/perl
use strict;
use NetAddr::IP;
my $fh = ();
my $sfile = "/home/dkenne201/ex-addresses.txt";
my $afile = "/home/dkenne201/aggs.txt";
my #space;
my #ips;
my $ip;
open($fh, "<", $sfile)
or die "Failed to open file: $!\n";
while(<$fh>) {
chomp;
push #space, $_;
}
close $fh;
open($fh, "<", $afile)
or die "Failed to open file: $!\n";
while(<$fh>) {
chomp;
push #ips, $_;
}
close $fh;
for my $netblock (NetAddr::IP::compact #space)
{
for $ip (map { new NetAddr::IP->new($_) } #ips)
{
if ($ip->within($netblock)) {
print "$ip found within $netblock\n";
}
else {
print "$ip not found within $netblock\n";
}
}
}
Here is an example of the format in the text files that contain the data as well.
Aggs example:
1.1.0.0/16
2.2.0.0/18
Addresses example:
1.1.1.1
192.168.2.3
5.2.3.4
You're calling new twice.
for $ip (map { new NetAddr::IP->new($_) } #ips)
Your code can be cleaned up to the following:
#!/usr/bin/perl
use strict;
use warnings;
use autodie;
use NetAddr::IP;
my $sfile = "/home/dkenne201/ex-addresses.txt";
my $afile = "/home/dkenne201/aggs.txt";
my #netblocks = do {
open my $fh, "<", $sfile;
my #space = <$fh>;
chomp #space;
map {NetAddr::IP->new($_)} #space;
};
open my $fh, "<", $afile;
while (<$fh>) {
chomp;
my $ip = NetAddr::IP->new($_);
if (my ($netblock) = grep {$ip->within($_)} #netblocks) {
print "$_ found within $netblock\n";
} else {
print "$_ not found\n";
}
}
close $fh;
Outputs:
1.1.1.1 found within 1.1.0.0/16
192.168.2.3 not found
5.2.3.4 not found
Thanks to Miller for the improved version of my code. I am just re-posting with the $afile and $sfile variables swapped so we are looking for ips within aggs and not aggs within ips (which broke the earlier code in my original post as well). Works perfectly for me with the below code.
#!/usr/bin/perl
use strict;
use warnings;
use autodie;
use NetAddr::IP;
my $sfile = "ex-addresses.txt";
my $afile = "aggs.txt";
my #netblocks = do {
open my $fh, "<", $afile;
my #space = <$fh>;
chomp #space;
map {NetAddr::IP->new($_)} #space;
};
open my $fh, "<", $sfile;
while (<$fh>) {
chomp;
my $ip = NetAddr::IP->new($_);
if (my ($netblock) = grep {$ip->within($_)} #netblocks) {
print "$_ found within $netblock\n";
} else {
print "$_ not found\n";
}
}
close $fh;

How to put data from CSV file to Perl hash

I have Perl and CSV file with something like:
"Name","Lastname"
"Homer","Simpsons"
"Ned","Flanders"
In this CSV file I have header in the first line and in other lines there are
data.
I want to convert this CSV file to such Perl data:
[
{
Lastname => "Simpsons",
Name => "Homer",
},
{
Lastname => "Flanders",
Name => "Ned",
},
]
I've written the function that users Text::CSV and doing what I need.
Here is the sample script:
#!/usr/bin/perl
use strict;
use warnings FATAL => 'all';
use 5.010;
use utf8;
use open qw(:std :utf8);
use Text::CSV;
sub read_csv {
my ($filename) = #_;
my #first_line;
my $result;
my $csv = Text::CSV->new ({ binary => 1, auto_diag => 1 });
open my $fh, "<:encoding(utf8)", $filename or die "$filename: $!";
while (my $row = $csv->getline ($fh)) {
if (not #first_line) {
#first_line = #{$row};
} else {
push #{$result}, { map { $first_line[$_] => $row->[$_] } 0..$#first_line };
}
}
close $fh;
return $result;
}
my $data = read_csv('sample.csv');
This works fine but this function I want to use in several scripts. I'm
greatly suprised that Text::CSV doesn't have this feature.
My question. What should I do to simplify solving such tasks in the future for
me and others?
Should I use some Perl module from CPAN, should I try to add this function to
Text::CSV, or something else?
Huh? Why so complicated? First, we fetch the header outside of the loop:
my $headers = $csv->getline($fh) or die "no header";
Assign these to be the column names:
$csv->column_names(#$headers);
Then, each call to getline_hr will provide a hashref:
while (my $hashref = $csv->getline_hr($fh)) {
push #$result, $hashref;
}
We can also use getline_hr_all:
$result = $csv->getline_hr_all($fh);
In other words, it ain't complex, most pieces are already provided by Text::CSV, and it can be done in very few lines.
Also, a module like this seems to already exist: Text::CSV::Slurp. (note: reverse dependency search through metacpan is awesome)
It's probably not a standard feature because different people will want their CSV files parsed into different data structures.
Why not create your own module that wraps this function?
package CSVRead;
use strict;
use warnings;
use 5.010;
use open qw(:std :utf8);
use Text::CSV;
require Exporter;
our #ISA = qw(Exporter);
our #EXPORT = qw(read_csv);
sub read_csv {
my ($filename) = #_;
my #first_line;
my $result;
my $csv = Text::CSV->new ({ binary => 1, auto_diag => 1 });
open my $fh, "<:encoding(utf8)", $filename or die "$filename: $!";
while (my $row = $csv->getline ($fh)) {
if (not #first_line) {
#first_line = #{$row};
} else {
push #{$result}, { map { $first_line[$_] => $row->[$_] } 0..$#first_line };
}
}
close $fh;
return $result;
}
Then, use it like this:
#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
use Data::Dumper;
use CSVRead;
my $data = read_csv('sample.csv');
say Dumper $data;

How to write a correct name using combination of variable and string as a filehandler?

I want to make a tool to classify each line in input file to several files
but it seems have some problem in naming a filehandler so I can't go ahead , how do I solve?
here is my program
ARGV[0] is the input file
ARGV[1] is the number of classes
#!/usr/bin/perl
use POSIX;
use warnings;
# open input file
open(Raw,"<","./$ARGV[0]") or die "Can't open $ARGV[0] \n";
# create a directory class to store class files
system("mkdir","Class");
# create files for store class informations
for($i=1;$i<=$ARGV[1];$i++)
{
# it seems something wrong in here
open("Class$i",">","./Class/$i.class") or die "Can't create $i.class \n";
}
# read each line and random decide which class to store
while( eof(Raw) != 1)
{
$Line = readline(*Raw);
$Random_num = ceil(rand $ARGV[1]);
for($k=1;$k<=$ARGV[1];$k++)
{
if($Random_num == $k)
{
# Store to the file
print "Class$k" $Line;
last;
}
}
}
for($h=1;$h<=$ARGV[1];$h++)
{
close "Class$h";
}
close Raw;
thanks
Later I use the advice provided by Bill Ruppert
I put the name of filehandler into array , but it seems appear a syntax bug , but I can't correct it
I label the syntax bug with ######## A syntax error but it looks quite OK ########
here is my code
#!/usr/bin/perl
use POSIX;
use warnings;
use Data::Dumper;
# open input file
open(Raw,"<","./$ARGV[0]") or die "Can't open $ARGV[0] \n";
# create a directory class to store class files
system("mkdir","Class");
# put the name of hilehandler into array
for($i=0;$i<$ARGV[1];$i++)
{
push(#Name,("Class".$i));
}
# create files of classes
for($i=0;$i<=$#Name;$i++)
{
$I = ($i+1);
open($Name[$i],">","./Class/$I.class") or die "Can't create $I.class \n";
}
# read each line and random decide which class to store
while( eof(Raw) != 1)
{
$Line = readline(*Raw);
$Random_num = ceil(rand $ARGV[1]);
for($k=0;$k<=$#Name;$k++)
{
if($Random_num == ($k+1))
{
print $Name[$k] $Line; ######## A syntax error but it looks quite OK ########
last;
}
}
}
for($h=0;$h<=$#Name;$h++)
{
close $Name[$h];
}
close Raw;
thanks
To quote the Perl documentation on the print function:
If you're storing handles in an array or hash, or in general whenever you're using any expression more complex than a bareword handle or a plain, unsubscripted scalar variable to retrieve it, you will have to use a block returning the filehandle value instead, in which case the LIST may not be omitted:
print { $files[$i] } "stuff\n";
print { $OK ? STDOUT : STDERR } "stuff\n";
Thus, print $Name[$k] $Line; needs to be changed to print { $Name[$k] } $Line;.
How about this one:
#! /usr/bin/perl -w
use strict;
use POSIX;
my $input_file = shift;
my $file_count = shift;
my %hash;
open(INPUT, "<$input_file") || die "Can't open file $input_file";
while(my $line = <INPUT>) {
my $num = ceil(rand($file_count));
$hash{$num} .= $line
}
foreach my $i (1..$file_count) {
open(OUTPUT, ">$i.txt") || die "Can't open file $i.txt";
print OUTPUT $hash{$i};
close OUTPUT;
}
close INPUT;

How to keep data marked as UTF-8 after parsing with HTML::Tree?

I wrote a script, where i slurp in UTF-8 encoded HTML-file and then parse it to tree with HTML::Tree. Problem is that after parsing the strings are not marked as UTF-8 anymore.
As _utf8_on() is not recommended way to set flag on, i am looking for proper way.
My simplified code-example:
#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
use utf8::all;
use autodie;
use HTML::Tree;
use Encode qw/is_utf8/;
my $file = shift;
my $tree;
if ($file) {
my $content = slurp_in( 'file' => $file );
$tree = html_tree('content' => $content);
} else {
die "no file";
}
my $title = $tree->look_down(_tag => 'title');
$title = $title->as_HTML('');
if ( is_utf8( $title ) ) {
say "OK: $title";
} else {
say "NOT OK: $title";
}
## SUBS
##
sub slurp_in {
my %v = #_;
open(my $fh, "<:utf8", $v{file}) || die "no $v{file}: $!";
local $/;
my $content = (<$fh>);
close $fh;
if ($content) {
return $content;
} else {
die "no content in $v{file} !";
}
}
sub html_tree {
my %v = #_;
my $tree = HTML::Tree->new();
$tree->utf8_mode(1); ## wrong call here, no such method, but no warnings on it!
$tree->parse( $v{content} );
if ($tree) {
return $tree;
} else {
die "no tree here";
}
}
Your code is overcomplicated, and you employ utf8::all and decode manually and call that strange method all at once. Rhetorically asking, what do you expect to achieve that way? I do not have the patience to find out the details what goes wrong and where, especially since you did not post any input with which your program fails to do the expected, so I drastically reduce it to a much simpler one. This works:
#!/usr/bin/env perl
use 5.010;
use strict;
use warnings FATAL => ':all';
use File::Slurp qw(read_file); # autodies on error
use HTML::Tree qw();
my $file = shift;
die 'no file' unless $file;
my $tree = HTML::Tree->new_from_content(
read_file($file, binmode => ':encoding(UTF-8)')
);
my $title = $tree->look_down(_tag => 'title');
$title->as_HTML(''); # returns a Perl string