Adding a member to zip file from a file handle in Perl - perl

I am trying to add a remote file to a local zip archive.
Currently, I am doing something like this.
use Modern::Perl;
use Archive::Zip;
use File::Remote;
my $remote = File::Remote->new(rsh => "/usr/bin/ssh", rcp => "/usr/bin/scp");
my $zip = Archive::Zip->new();
$remote->open(*FH,'host2:/file/to/add.txt');
my $fh = IO::File->new_from_fd(*FH,'r');
#this is what I want to do.
$zip->addFileHandle($fh,'add.txt');
...
Unfortunately, Archive::Zip does not have have an addFileHandle method.
Is there another way that I can do that?
Thanks.

Do something like this (copy to local path):
$remote->copy("host:/remote/file", "/local/file");
and use the addFile method provided by Archive::Zip with the local file

Archive::Zip might not have filehandle support for writing to a zip file, but Archive::Zip::SimpleZip does.
Here is a self-contained example that shows how to read from a filehandle & write directly to the zip file without any need for a temporary file.
use warnings;
use strict;
use Archive::Zip::SimpleZip;
use File::Remote;
# create a file to add to the zip archive
system "echo hello world >/tmp/hello" ;
my $remote = File::Remote->new(rsh => "/usr/bin/ssh", rcp => "/usr/bin/scp");
my $zip = Archive::Zip::SimpleZip->new("/tmp/r.zip");
$remote->open(*FH,'/tmp/hello');
# Create a filehandle to write to the zip fiule.
my $member = $zip->openMember(Name => 'add.txt');
my $buffer;
while (read(FH, $buffer, 1024*16))
{
print $member $buffer ;
}
$member->close();
$zip->close();
# dump the contents of the zipo file to stdout
system "unzip -p /tmp/r.zip" ;

Related

Perl tar file creates directory recursively

I am taring the directory contents using Archive::Tar module.
My scripts is below:
#!/usr/bin/perl -w
use strict;
use warnings;
use Archive::Tar;
use File::Find;
use Data::Dumper;
my $home_dir = "C:/Users/Documents/Vinod/Perl_Scripts/test/";
my $src_location = $home_dir."LOG_DIR";
my $dst_location = $home_dir."file.tar.gz";
my #inventory = ();
find (sub { push #inventory, $File::Find::name }, $src_location);
print "Files:".Dumper(\#inventory);
my $tar = Archive::Tar->new();
$tar->add_files( #inventory );
$tar->write( $dst_location , 9 );
Script is able to create file.tar.gz file in location C:/Users/Documents/Vinod/Perl_Scripts/test/.
But when I extract the file.tar.gz manually it creates a whole path recursively once again. So LOG_DIR contents would be visible in the location C:/Users/Documents/Vinod/Perl_Scripts/test/file.tar/file/Users/Documents/Vinod/Perl_Scripts/test/LOG_DIR/
How can I have the contents which is inside C:/Users/Documents/Vinod/Perl_Scripts/test/LOG_DIR/ in C:/Users/Documents/Vinod/Perl_Scripts/test/file.tar/file/ while extracting it.
If you don't want to recreate the full path, chdir into the home directory, and make the source dir relative:
my $home_dir = "C:/Users/Documents/Vinod/Perl_Scripts/test/";
chdir $home_dir;
my $src_location = "LOG_DIR";
my $dst_location = $home_dir."file.tar.gz";
Since you use $File::Find::name for your list, you get the absolute path to each file. That's the name that you give Archive::Tar, so that's the name that it uses. You can see the files in a tarball:
$ tar -tzf archive.tgz
There are various ways to get relative paths instead. You might do that in the wanted function. Remove the part of the path that you do not want. That's typically not going to be the directory you used for find (src_location) because you want to keep that level of structure:
my #inventory;
find(
sub {
return if /\A\.\.?\z/;
push #inventory, abs2rel( $File::Find::name, $home_dir )
}, $src_location
);
Or do it after:
#inventory = map { abs2rel($_, $home_dir) } #inventory;

How to read the properties file from perl script

I have written a perl script where I will be connecting to database, for which I'm using this statement
my $dbh = DBI->connect(
"DBI:mysql:database=test;host=localhost;mysql_socket=/var/run/mysqld/mysqld.sock",
"root", "password", {'RaiseError' => 1});
As I don't want any information to be hardcoded, I want to use properties file where I can list the above details (e.g., database, host, mysql_socket) and read the details of properties file from the script. How can I write the properties file and read the details from perl script?
There are a lot of CPAN modules that helps you to achieve this task.
I like Config::Simple, for example:
#!/usr/bin/perl
use strict;
use warnings;
use Config::Simple;
...
my $cfg = new Config::Simple('myapp.ini');
my $user = $cfg->param('database.user');
my $connection_str = $cfg->param('database.connection');
#...
and the file myapp.ini:
[database]
connection="DBI:mysql:database=test;host=localhost;mysql_socket=/var/run/mysqld/mysqld.sock"
user=root
;...
You can install the module from the terminal/command prompt using:
cpan install Config::Simple
or
yum install perl-Config-Simple
I had issues using perl-Config-Simple and decided to use Config::Properties instead. If you are experiencing the same, then you can try the following.
Make sure you have the Config::Properties installed. The following are several examples of how to install from command line, depending on the OS you are using, you'll want to use the appropriate choice:
cpan Config::Properties
cpan install Config::Properties
yum install perl-Config-Properties
The code:
#!/usr/bin/perl
use strict;
use warnings;
use Config::Properties;
open my $cfh, '<', './foo.properties' or die "unable to open property file";
my $properties = Config::Properties->new();
$properties->load($cfh);
my $dbName = $properties->getProperty('database.name');
my $dbUser = $properties->getProperty('database.user');
The property file:
database.name=somedb
database.user=someuser
Once you have the values in the variables, put them into your connection string and you should be good to go.
var temp;
if($ENV eq "Prod"){
#Prod Configurations
temp = "Prod";
}
else{
# Stage and Test Confgurations
temp = "Stage";
}
My brick and mortar solution, exports one value from a properties file:
#!/usr/bin/perl
use strict;
use warnings;
my $line;
foreach $line (<STDIN>) {
chomp($line);
if(my $match = $line =~ /^(.*)=(.*)$/){
my $key = $1;
my $value = $2;
if ($ARGV[0] eq $key) {
print "$value\n";
exit 0;
}
}
}
Usage: perl script.pl mykey < file.properties

Trouble storing files with Archive::Zip: I get empty zip files without error code

When I try to create zip archives via Archive::Zip there are no errors thrown, but the resulting zip file is broken.
use Archive::Zip;
my $zip = Archive::Zip->new();
my $file = "/a/very/long/path/with/191/characters/file.txt";
if(-f $file)
{
$zip->addFile("$file", "destinationname.txt");
print "$file added\n";
}
unless ($zip->writeToFileNamed("out.zip") == "AZ_OK") { die "error";};
Now my out.zip file is just 22B and is empty:
$> > unzip -l out.zip
Archive: out.zip
warning [out.zip]: zipfile is empty
What goes wrong?
First Update: Everything works fine when I use files with a shorter path name. Any idea for a workaround? Symlinking does not work.
Second update: This works as a workaround:
use File::Slurp;
[...]
my $text = read_file($file);
$zip->addString($text, "destinationfile.txt");
[..]
Change it to: $zip->addFile($plmxmlFile);.
$zip is already reference to your target file and by adding name of file you'd use for output, you're making Archive::Zip try read and write from same file on assembling attempt, creating a mess (and just generally doing not what your really wanted).
I cannot see why your program creates an empty zip file, but you are misusing quotation marks in several places.
In particular the value AZ_OK is a symbol for a numeric value that you can import by request.
The writeToFileNamed method will never return the string "AZ_OK" and also you should compare strings using eq instead of ==.
Fortunately (or not, depending on your point of view) these two errors together with your failure to import the value of AZ_OK and your omission of use warnings will compare the return value of writeToFileNamed with zero (the proper value of AZ_OK) and should give you the correct results.
Try this program instead.
use strict;
use warnings;
use Archive::Zip qw( :ERROR_CODES );
my $zip = Archive::Zip->new;
my $file = 'a/very/long/path/with/191/characters/file.txt';
if (-f $file) {
$zip->addFile($file, 'destinationname.txt');
print "$file added\n";
}
my $status = $zip->writeToFileNamed('out.zip');
$status == AZ_OK or die "error $status";
Update
The length of the path is unlikely to make any difference unless it is hundreds of characters long.
Try this version and tell us what you get.
use strict;
use warnings;
use Archive::Zip qw( :ERROR_CODES );
my $zip = Archive::Zip->new;
my $file = 'a/very/long/path/with/191/characters/file.txt';
unlink 'out.zip';
die qq(File "$file" not found) unless -f $file;
$zip->addFile($file, 'destinationname.txt');
print "$file added\n";
my $status = $zip->writeToFileNamed('out.zip');
$status == AZ_OK or die "error $status";
maybe i have understood what is the problem :
you use the full root a/very/long/path/with/191/characters/file.txt
so you compress all directories in you zip, your file is empty because your are note able to see the path.
use chdir
chdir 'a/very/long/path/with/191/characters/'

How can I list the contents of a ZIP file using Perl?

What I am aiming to do is pretty much what it says in the title.
I have the following line of code which simply prints out [view archive] and when I click it the browser just downloads the zip file.
print "\<a href=\"http:\/\/intranet.domain.com\/~devcvs\/view-file.cgi?file=$reviewdata{'document'}&review_id=$reviewdata{'id'}\"\>[view archive]\<\/a\>\n";
What I would love to do is to list the files contained within this zip file anywhere on the page, e.g. just underneath or even a new page which this link links to and takes the filename as a parameter.
I believe once this is done the browser should take care of the rest in terms of just clicking these files and viewing them in the browser as they will be pdfs and html files which I don't foresee any problems with.
I am sure there is a module that does this but I am unsure of how to accomplish my goal using it.
Any help is much appreciated.
Have a look at Archive::Zip :
use strict;
use warnings;
use Archive::Zip qw/ :ERROR_CODES :CONSTANTS /;
my $zipFile = 'someZip.zip';
my $zip = Archive::Zip->new();
unless ( $zip->read( $zipFile ) == AZ_OK ) { # Make sure archive got read
die 'read error';
}
my #files = $zip->memberNames(); # Lists all members in archive
print $_, "\n" for #files;
Using Archive::Zip certainly makes the code easier, and you should probably install that module if you are going to work extensively on zip files.
However, for those who prefer not to install anything, there is a way to list the content of a zip file just using the core module IO::Uncompress::Unzip (already part of any standard Perl distribution).
use strict;
use warnings;
use IO::Uncompress::Unzip qw($UnzipError);
my $zipFile = '/path/to/zipfile.zip';
my $u = IO::Uncompress::Unzip->new($zipFile)
or die "Error: $UnzipError\n";
my $status;
for ($status = 1; $status > 0; $status = $u->nextStream()) {
my $header = $u->getHeaderInfo();
my $zippedFile = $header->{Name};
if ($zippedFile =~ /\/$/) {
last if $status < 0;
next;
}
print "$zippedFile\n";
}

How do I access a hash from another subroutine?

I am trying to create some scripts for web testing and I use the following piece of code to set up variables from a config file:
package setVariables;
sub readConfig{
open(FH, "workflows.config") or die $!;
while(<FH>)
{
($s_var, $s_val) = split("=", $_);
chomp($s_var);
chomp($s_val);
$args{$s_var} = $s_val;
print "set $s_var = $s_val\n";
}
close(FH);
}
for example: var1=val1
var2=val2
var3=val3
etc...
I want to be able to pass the values set by this subroutine to a subroutine in another package. This is what I have for the package I want it passed into.
package startTest;
use setVariables;
sub startTest{
my %args = %setVariables::args;
my $s_var = $setVariables::s_var;
my $s_val = $setVariables::s_var;
setVariables::readConfig(); #runs the readConfig sub to set variables
my $sel = Test::WWW::Selenium->new( host => "localhost",
port => 4444,
browser => $args{"browser"},
browser_url => $args{"url"} );
$sel->open_ok("/index.aspx");
$sel->set_speed($args{"speed"});
$sel->type_ok("userid", $args{"usrname"});
$sel->type_ok("password", $args{"passwd"});
$sel->click_ok("//button[\#value='Submit']");
$sel->wait_for_page_to_load_ok("30000");
sleep($args{"sleep"});
}
Unfortunately its not holding on to the variables as is and I don't know how to reference them.
Thank you for any help.
Your code has some problems. Let's fix those first.
# Package names should start with upper case unless they are pragmas.
package SetVariables;
# Do this EVERYWHERE. It will save you hours of debugging.
use strict;
use warnings;
sub readConfig{
# Use the three argument form of open()
open( my $fh, '<', "workflows.config")
or die "Error opening config file: $!\n";
my %config;
# Use an explicit variable rather than $_
while( my $line = <$fh> )
{
chomp $line; # One chomp of the line is sufficient.
($s_var, $s_val) = split "=", $line;
$config{$s_var} = $s_val;
print "set $s_var = $s_val\n";
}
close $fh;
return \%config;
}
Then use like so:
use SetVariables;
my $config = SetVariables::readConfig();
print "$_ is $config->{$_}\n"
for keys %$config;
But rather than do all this yourself, check out the many, many config file modules on CPAN. Consider Config::Any, Config::IniFiles, Config::JSON.
You note in your comment that you are trying to work with multiple files, your main code and a couple of packages.
One pattern that is common is to load your config in your main code and pass it (or select elements of it) to consuming code:
package LoadConfig;
sub read_config {
my $file = shift;
my $config;
# Do stuff to read a file into your config object;
return $config;
}
1;
Meanwhile in another file:
package DoStuff;
sub run_some_tests {
my $foo = shift;
my $bar = shift;
# Do stuff here
return;
}
sub do_junk {
my $config;
my $foo = $config->{foo};
# Do junk
return;
}
1;
And in your main script:
use DoStuff;
use LoadConfig;
my $config = LoadConfig::read_config('my_config_file.cfg');
run_some_tests( $config->{foo}, $config->{bar} );
do_junk( $config );
So in run_some_tests() I extract a couple elements from the config and pass them in individually. In do_junk() I just pass in the whole config variable.
Are your users going to see the configuration file or just programmers? If it's just programmers, put your configuration in a Perl module, then use use to import it.
The only reason to use a configuration file for only programmers if you are compiling the program. Since Perl programs are scripts, don't bother with the overhead of parsing a configuration file; just do it as Perl.
Unless it's for your users and its format is simpler than Perl.
PS: There's already a module called Config. Call yours My_config and load it like this:
use FindBin '$RealBin';
use lib $RealBin;
use My_config;
See:
perldoc FindBin
perldoc Config
I would suggest using a regular format, such as YAML, to store the configuration data. You can then use YAML::LoadFile to read back a hash reference of the configuration data and then use it.
Alternatively, if you don't want to use YAML or some other configuration format with pre-written modules, you'll need for your reading routine to actually return either a hash or a a hashref.
If you need some more background information, check out perlref, perlreftut and perlintro.
all you need to do is collect the variable in a hash and return a reference to it in readConfig:
my %vars = ( var1 => val1,
var2 => val2,
var3 => val3,
);
return \%vars;
and in startTest:
my $set_vars = setVariables::readConfig();