Diff two remote files using Perl - perl

I have an array of file paths:
#files = ('/home/.../file.txt', '/home/.../file2.txt',...);
I have multiple remote machines, with a similar filestructure. How can I diff these remote files using Perl?
I thought of using Perl backticks, ssh and using diff, but I am having issues with sh (it doesn't like diff <() <()).
Is there a good Perl way of comparing at least two remote files?

Use rsync to copy the remote files to the local machine, then use diff to find out the differences:
use Net::OpenSSH;
my $ssh1 = Net::OpenSSH->new($host1);
$ssh1->rsync_get($file, 'master');
my $ssh2 = Net::OpenSSH->new($host2);
system('cp -R master remote');
$ssh2->rsync_get($file, 'remote');
system('diff -u master remote');

You can use the Perl Module on CPAN called Net::SSH::Perl to run remote commands.
Link: http://metacpan.org/pod/Net::SSH::Perl
Example from the Synopsis:
use Net::SSH::Perl;
my $ssh = Net::SSH::Perl->new($host);
$ssh->login($user, $pass);
my($stdout, $stderr, $exit) = $ssh->cmd($cmd);
You command would look something like
my $cmd = "diff /home/.../file.txt /home/.../file2.txt";
edit: The files are on different servers.
You can still use Net::SSH::Perl to read the files.
#!/bin/perl
use strict;
use warnings;
use Net::SSH::Perl;
my $host = "First_host_name";
my $user = "First_user_name";
my $pass = "First_password";
my $cmd1 = "cat /home/.../file1";
my $ssh = Net::SSH::Perl->new($host);
$ssh->login($user, $pass);
my($stdout1, $stderr1, $exit1) = $ssh->cmd($cmd1);
#now stdout1 has the contents of the first file
$host = "Second_host_name";
$user = "Second_user_name";
$pass = "Second_password";
my $cmd2 = "cat /home/.../file2";
$ssh = Net::SSH::Perl->new($host);
$ssh->login($user, $pass);
my($stdout2, $stderr2, $exit2) = $ssh->cmd($cmd2);
#now stdout2 has the contents of the second file
#write the contents to local files to diff
open(my $fh1, '>', "./temp_file1") or DIE "Failed to open file 1";
print $fh1 $stdout1;
close $fh1;
open(my $fh2, '>', "./temp_file2") or DIE "Failed to open file 2";
print $fh2 $stdout2;
close $fh2;
my $difference = `diff ./temp_file1 ./temp_file2`;
print $difference . "\n";
I haven't tested this code, but you could do something like this. Remember to download the Perl Module Net::SSH::Perl to run remote commands.
Diff is not implemented in the Perl Core Modules, but there another called Text::Diff on CPAN so maybe that would work too. Hope this helps!

Related

How do I execute a unix command containing a perl variable in perl

In the following perl code, I am tring to copy a perl variable $file from one directory to another directory with:
"system("cp $file $Output_Dir);
This command writes down the file name alright but then says:
cp: cannot stat 'tasmax_AFR-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_CLMcom-CCLM4-8-17_v1_day_19910101-19951231.nc': No such file or directory
The command
system("#sixfiles = ls $Vars[$kk]}*");
gives me the error:
sh: 1: =: not found
I wonder what is wrong with this code. Assistance will be appreciated.
#!/usr/bin/perl -w
use strict;
use warnings;
use File::Path;
use File::Copy;
my $debug = 1;
my #Vars = ("pr","tasmin","tasmax");
my $Vars;
my #sixfiles;
my $sixfiles;
my $Input_Dir = "/home/zmumba/DATA/Input_Dir";
my $Output_Dir = "/home/zmumba/DATA/Output_Dir";
for (my $kk=0; $kk < #Vars; ++$kk) {
opendir my $in_dir, $Input_Dir or die "opendir failed on $Input_Dir: $! ($^E)";
while (my $file=readdir $in_dir) {
next unless $file =~ /^$Vars[$kk]/;
next if -d $file;
print "$file\n";
print "Copying $file\n" if $debug;
my $cmd01 = "cp $file $Output_Dir";
print "Doing system ($cmd01)\n" if $debug;
system ($cmd01);
system("#sixfiles = ls $Vars[$kk]}*");
}
}
Try this:
use feature qw(say);
use strict;
use warnings;
use File::Spec;
my #Vars = ("pr","tasmin","tasmax");
my $Input_Dir = "/home/zmumba/DATA/Input_Dir";
my $Output_Dir = "/home/zmumba/DATA/Output_Dir";
opendir my $in_dir, $Input_Dir or die "opendir failed on $Input_Dir: $! ($^E)";
while (my $file=readdir $in_dir) {
next if ($file eq '.') || ($file eq '..');
next if -d $file;
next if !grep { $file =~ /^$_/ } #Vars;
say "Copying $file";
$file = File::Spec->catfile( $Input_Dir, $file );
system "cp", $file, $Output_Dir;
}
system ($cmd01);
Gives:
cp: cannot stat '<long-but-correct-file-name>': No such file or directory
This is almost certainly because you are not running the code from $Input_Dir, so that file doesn't exist in your current directory. You need to either chdir to the correct directory or add the directory path to the front of the file name variable.
system("#sixfiles = ls $Vars[$kk]}*");
This code makes no sense. The code passed to system() needs to be Unix shell code. That's the ls $Vars[$kk]}* bit (but I'm not sure where that } comes from). You can't populate a Perl array inside a shell command. You would need to capture the value returned from the ls command and then parse it somehow to separate it into a list.
You can give a try with the following code:
#!/usr/bin/env perl
use strict;
use warnings;
my $debug = 1;
my #Vars = ("pr", "tasmin", "tasmax");
my $Vars;
my $Input_Dir = "/home/zmumba/DATA/Input_Dir";
my $Output_Dir = "/home/zmumba/DATA/Output_Dir";
my $cpsrc, $cpdest = '';
print "No Write Permission: $!" unless(-w $Output_Dir);
for my $findex (0 .. $#Vars) {
$cpsrc = qq($Input_Dir/$Vars[$findex]);
print "$Vars[$findex]\n";
print "Copying $Vars[$findex]\n" if $debug;
my $cmd01 = "cp $cpsrc $Output_Dir";
print "Doing system ($cmd01)\n" if $debug;
system($cmd01);
}
You don't have to go through each file in source dir. You already know the files to copy from source.

use command lines inside a perl script

I know there is a command you can run in the command like this
perl -p -i -e 's/oldtext/newttext/g' file
I was just wondering if you could use the same command inside the script?
Yes, the script equivalent of that one liner would be the following:
local #ARGV = 'file';
local $^I = '';
while (<>) {
s/oldtext/newttext/g;
print;
}
Just search for $INPLACE_EDIT for details.
Sure, it takes a slightly different form though:
use strict;
use warnings;
my $filename = $ARGV[0]
or die "Must supply filename";
my $file;
{ # read the file into a single scalar
open(my $fh, '<', $filename)
or die "Can't open '$filename': $!\n";
local $/;
$file = <$fh>;
}
# apply your substitution
$file =~ s/oldtext/newtext/g;
{ # overwrite existing file with changes
open(my $fh, '>', $filename)
or die "Can't create '$filename': $!\n";
print {$fh} $file;
}
Perl doesn't understand bourne shell, but you can launch a shell to execute it for you using system. (For more complicated needs, modules such as IPC::Run3 and IPC::Run are very useful.)
system('sh', '-c', q{perl -p -i -e 's/oldtext/newttext/g' file});
The alternative syntax of system allows you to simplify the above to the following:
system(q{perl -p -i -e 's/oldtext/newttext/g' file});
But! Instead of telling a shell to launch perl, though, you could launch perl directly.
system('perl', '-p', '-i', '-e', 's/oldtext/newttext/g', 'file');
But! Instead of launching a second perl, you could merge the two programs together.
local #ARGV = 'file';
local $^I = '';
while (<>) {
s/oldtext/newttext/g;
print;
}

Printing cwd without full path in perl?

I have a bunch of data that is stored in sub-directories labeled by date. I have used the Cwd command to get the Current working directory so that I can then print it to the vi file that I am writing with the recovered data from the sub-directories. I am using the Cwd as a prefix to the data strings. Is there a way to print only the current directory name and not the path?
example:
Instead of printing-
/d2/aschwa/archive_data/METAR_data/20120302KDUX 121255Z.........
Is there a way to print only-
20120302KDUX 121255Z.........
Here's the code I'm using-
use strict;
use warnings;
use file::find;
use Cwd;
my #folder = ("/d2/aschwa/archive_project/METAR_data/");
open( OUT , '>', 'KDUX_METARS.txt') or die "Could not open $!";
print OUT "Station, Day/Time, Obs Type, Wind/Gust, Vis, Sky, T/Td, Alt, Rmk\n";
print STDOUT "Finding METAR files\n";
my $criteria = sub {if(-e && /^2012/) {
open(my $file,$_) or die "Could not open $_ $!\n";
my $dir = getcwd;
while(<$file>) {
print OUT $dir,$_ if /KDUX ....55Z|KDUX ....05Z/;
}
}
};
find($criteria, #folder);
close OUT;
In Perl, you can use functions basename or fileparse to extract the file name from a path.
They are included in the core module File::Basename.
Simply split, then pop.
Shamelessly stolen from perlmonks:
$ perl -e 'print pop #{[split m|/|, "/home/bin/scripts/test.pl"]};'
test.pl
Reference link: http://www.perlmonks.org/?node_id=241089
You can combing the perl module File::Basename with Cwd to get the directory without the path
perl -MCwd -MFile::Basename -e 'my $dir = cwd; print basename($dir)'
Why don't you just get the content after the last slash with a regexp like below:
$path = '/d2/aschwa/archive_data/METAR_data/20120302KDUX 121255Z.........';
$path = $1 if $path =~ m~/([^/]*)/?$~;
This is in my opinion the best way to do it. The above code is just an example, but the regexp there will do the job you want.

Unknown reason for Known error in the perl script

I am writing following script to read list of servers from the text file them ssh to them and run ldd command to fetch the version that is installed on the server.
The only problem is the error that I am seeing following error which says Bad Host name:
adev#abclnxdev:[/home/adev/perl-scripts] {63} % perl try.pl
Net::SSH: Bad host name: abclnxtest01
at try.pl line 21
when I manually do the ssh to this host. It gets connect.
Here is script :
#!/mu/bin/perl
use Net::SSH::Perl;
use warnings;
my $file = "server-list.txt";
my $usr = "user";
my $pwd = "password";
my $output_file = "GlibC-version.txt";
open(HANDLE, $file) or die("Cant open the file :( ");
#server_list = <HANDLE>;
close(HANDLE);
#debug_print_array(#server_list);
open(HANDLE, ">>$output_file"); #opening file for output.
foreach $host (#server_list)
{
my $ssh = Net::SSH::Perl->new($host);
$ssh->login($usr,$pwd,$ssh);
my($stdout, $stderr, $exit) = $ssh->cmd("ldd --version|grep ldd");
print HANDLE "----------------------------------";
print HANDLE "Hostname : $host";
print HANDLE "GLIBC Version : $stdout";
print HANDLE "----------------------------------\n\n";
}
You have a newline at the end of the server name.
Add:
chomp #server_list;
(And incidentally, it's better to use the newer 3-argument open(); see http://perlmaven.com/open-files-in-the-old-way .)

wget not working properly inside a Perl program

I am trying to download some xml files from a given URL. Below is the code which I have used for the same-
use strict;
use warnings;
my $url ='https://givenurl.com/';
my $username ='scott';
my $password='tiger';
system("wget --user=$username --password=$password $url") == 0 or die "system execution failed ($?): $!";
local $/ = undef;
open(FILE, "<index.html") or die "not able to open $!";
my $index = <FILE>;
my #childs = map /<a\s+href\=\"(AAA.*\.xml)\">/g , $index;
for my $xml (#childs)
{
system("wget --user=$username --password=$password $url/$xml");
}
But when I am running this, it gets stuck in the for-loop wget command. It seems wget is not able to fetch the files properly? Any clue or suggestion?
Thank you.
Man
You shouldn't use an external command in the first place.
Ensure that WWW::Mechanize is available, then use code like:
use strict;
use warnings;
use WWW::Mechanize;
my $mech = WWW::Mechanize->new();
...
$mech->credentials($username, $password);
$mech->get($url);
foreach my $link ($mech->find_all_links(url_regex=>qr/\bAAA/)) {
$mech->get($link);
...
}
If $url or $xml contains any shell metacharacters (? and & are common ones in URLs) then you may need to either quote them properly
system("wget --user=$username --password=$password '$url/$xml'");
system qq(wget --user=$username --password=$password "$url/$xml");
or use the LIST form of system that bypasses the shell
system( 'wget', "--user=$username", "--password=$password", "$url/$xml");
to get the command to work properly.
maybe it's because the path to wget, what if you use:
system("/usr/bin/wget --user=$username --password=$password $url")
or I guess it can be a problem with variables passed to system: ($username, $password, $url)