I am looking for ways to get file checksums in Perl but not by executing the system command cksum -- would like to do it in Perl itself because the script needs to be portable between UNIX and Windows. cksum <FILENAME> | awk '{ print $1 }' works on UNIX but obviously not in Windows. I have explored MD5 but it seems like getting a file handle is necessary and generally it doesn't seem like a very compact way to get that data (one-liner preferable).
Is there a better way?
Here are three different ways depending on which modules you have available:
use Digest::MD5 qw(md5_hex);
use File::Slurp;
print md5_hex(read_file("filename")), "\n";
use IO::All;
print md5_hex(io("filename")->all), "\n";
use IO::File;
print md5_hex(do { local $/; IO::File->new("filename")->getline }), "\n";
Not completely one-line but pretty close.
Replace Digest::MD5 with any hash algorithm you want, e.g. SHA1.
IO::File is in core and should be available everywhere, but that's the solution I personally dislike the most. Anyway, it works.
I couldn't make any of the above work for me in windows, I would always get an incorrect MD5. I got suspicious that it was being caused by differences in linebreak, but converting the file to DOS or to unix made no difference. The same code with the same file would give me the right answer on linux and the wrong one in windows. Reading the documentation, I finally found something that would work both in windows and linux:
use Digest::MD5;
open ($fh, '<myfile.txt');
binmode ($fh);
print Digest::MD5->new->addfile($fh)->hexdigest;
I hope this helps other people having difficulty in windows, I find it so weird that I didn't find any mentions to problems on windows...
This also works:
use Digest::MD5 qw(md5_base64);
...
open(HANDLE, "<", $dirItemPath);
my $cksum = md5_base64(<HANDLE>);
print "\nFile checksum = ".$cksum;
Related
I have a mailing script to send colored output, but I can't use it on a Unix machine as the MIME::Lite module is not installed on it.
Can anyone suggest an alternative to get the desired output in color?
use MIME::Lite;
my (#page_html, #sujet);
my $file = 'rpt.html';
open my $ifh, '<', $file
or die "Cannot open '$file' for reading: $!";
local $/ = "";
my $contents = <$ifh>;
close( $ifh );
my $msg = new MIME::Lite;
Output:
<tr><td>test1</td><td bgcolor=red>NOK</td></tr>
<tr><td>test2</td><td bgcolor=green>OK</td></tr>
<tr><td>test3</td><td bgcolor=green>OK</td></tr>
OK Should be in green
NOK should be red
The coloured output seems to be a complete red herring (hah!) here. You have a file which you want to send as an attachment to a MIME email.
If you're going to be using Perl, then you need to be in an environment where you can install modules from CPAN. Without CPAN, you're missing most of the power of modern Perl. I suggest that it's worth having whatever discussions you need to have in order to remove that major block to your Perl programming career.
If you really can't install modules into the system libraries, then you can install them into your home directory and use use lib or PERL5LIB to adjust the Perl library path as appropriate.
When you've solved the problem of how to use CPAN modules, can we talk about your choice of modules? MIME::Lite isn't exactly deprecated, but there have been better alternatives available for over ten years. Take a look at Email::Stuffer.
But if you're determined not to have the CPAN module installation discussion, the other approach is to use the command-line program mailx to send your message.
$ mailx -a rpt.html someone#example.com
I am looking for ways to get file checksums in Perl but not by executing the system command cksum -- would like to do it in Perl itself because the script needs to be portable between UNIX and Windows. cksum <FILENAME> | awk '{ print $1 }' works on UNIX but obviously not in Windows. I have explored MD5 but it seems like getting a file handle is necessary and generally it doesn't seem like a very compact way to get that data (one-liner preferable).
Is there a better way?
Here are three different ways depending on which modules you have available:
use Digest::MD5 qw(md5_hex);
use File::Slurp;
print md5_hex(read_file("filename")), "\n";
use IO::All;
print md5_hex(io("filename")->all), "\n";
use IO::File;
print md5_hex(do { local $/; IO::File->new("filename")->getline }), "\n";
Not completely one-line but pretty close.
Replace Digest::MD5 with any hash algorithm you want, e.g. SHA1.
IO::File is in core and should be available everywhere, but that's the solution I personally dislike the most. Anyway, it works.
I couldn't make any of the above work for me in windows, I would always get an incorrect MD5. I got suspicious that it was being caused by differences in linebreak, but converting the file to DOS or to unix made no difference. The same code with the same file would give me the right answer on linux and the wrong one in windows. Reading the documentation, I finally found something that would work both in windows and linux:
use Digest::MD5;
open ($fh, '<myfile.txt');
binmode ($fh);
print Digest::MD5->new->addfile($fh)->hexdigest;
I hope this helps other people having difficulty in windows, I find it so weird that I didn't find any mentions to problems on windows...
This also works:
use Digest::MD5 qw(md5_base64);
...
open(HANDLE, "<", $dirItemPath);
my $cksum = md5_base64(<HANDLE>);
print "\nFile checksum = ".$cksum;
context: I'm a beginner in Perl and struggling, please be patient, thanks.
the question: there is a one-liner that seems to do the job I want (in a cygwin console it does fine on my test file). So now I would need to turn it into a script, but I can't manage that unfortunately.
The one-liner in question is provided in the answer by Aki here Delete lines in perl
perl -ne 'print unless /HELLO/../GOODBYE/' <file_name>
Namely I would like to have a script that opens my file "test.dat" and removes the lines between some strings HELLO and GOODBYE. Here is what I tried and which fails (the path is fine for cygwin):
#!/bin/perl
use strict;
use warnings;
open (THEFILE, "+<test.dat") || die "error opening";
my $line;
while ($line =<THEFILE>){
next if /hello/../goodbye/;
print THEFILE $line;
}
close (THEFILE);
Many thanks in advance!
Your one-liner is equivalent to the following
while (<>) {
print unless /HELLO/../GOODBYE/;
}
Your code does something quite different. You should not attempt to read and write to the same file handle, that usually does not do what you think. When you want to quickly edit a file, you can use the -i "in-place edit" switch:
perl -ni -e 'print unless /HELLO/../GOODBYE/' file
Do note that changes to the file are irreversible, so you should make backups. You can use the backup option for that switch, e.g. -i.bak, but be aware that it is not flawless, as running the same command twice will still overwrite your backup (by saving to the same file name twice).
The simplest and safest way to do it, IMO, is to simply use shell redirection
perl script.pl file.txt > newfile.txt
While using the script file I showed at the top.
I'm trying to get color (colour) output using prove / TAP::Harness with Active state Perl on Windows 7.
The actual tests run fine, its just that there is no colour output.
I get a similar problem using Strawberry Perl and WinXP.
I am unable to use a *nix and cygwin or other thirdparty xterm both of which do
colour the output.
I know its a little picky thing but I think I've become addicted to the "green" :-)
Is there a simple fix? - couldn't see anything on the Activate state site - I was thinking of raising a bug.
Any guidance on debugging or what to check?
Is it worth writing my own formatter?
Thanks in advance for your help.
More detail on installed modules and approaches tried...
These are installed and to the best of my knowledge working
Win32::Console::ANSI;
Term::ANSIColor;
This test script worked:
#!/usr/bin/perl
use strict;
use warnings;
use Win32::Console::ANSI;
use Term::ANSIColor;
print "One fish\n";
print "Two fish\n";
print color("red"), "Red Fish\n", color("reset");
print color("blue"), "Blue Fish\n", color("reset");
I have tried:
prove
prove -c
and using the following test harness programs with and without formatter but
I was under the assumption colour was on by default.
#!/usr/bin/perl
use strict;
use warnings;
use TAP::Harness;
my #tests = glob( 't/*.t' );
my $harness = TAP::Harness->new();
$harness->runtests( #tests );
I have also install the HTML formatter and that appears to be working.
prove --formatter=TAP::Formatter::HTML
Running:
prove --formatter=TAP::Formatter::Color
Gives
Can't locate object method "verbosity" via package "TAP::Formatter::Color" at x:/Perl/site/lib/TAP/Harness.pm line 679.
Thanks
Mike
It appears to be a bug1 in TAP::Formatter::Color. It's attaching to the console's STDOUT handle but the messages that should be colored are on STDERR.
This:
my $console = Win32::Console->new( STD_OUTPUT_HANDLE() );
Should be this instead:
my $console = Win32::Console->new( STD_ERROR_HANDLE() );
Also, despite what the documentation says, --color is not the default on Windows. App::Prove (which is what's behind the "prove" executable) explicitly sets the default to false for Windows:
sub _color_default {
my $self = shift;
return -t STDOUT && !$ENV{HARNESS_NOTTY} && !IS_WIN32;
}
1. The bug was fixed in Test::Harness v3.41
Trying to run this little perl program from parsCit:
parsCit-client.pl e1.txt
Too late for -CSD option at [filename] line 1
e1.txt is here: http://dl.dropbox.com/u/10557283/parserProj/e1.txt
I'm running the program from win7 cmd, not Cygwin.
filename is parsCit-client.pl - entire program is here:
#!/usr/bin/perl -CSD
#
# Simple SOAP client for the ParsCit web service.
#
# Isaac Councill, 07/24/07
#
use strict;
use encoding 'utf8';
use utf8;
use SOAP::Lite +trace=>'debug';
use MIME::Base64;
use FindBin;
my $textFile = $ARGV[0];
my $repositoryID = $ARGV[1];
if (!defined $textFile || !defined $repositoryID) {
print "Usage: $0 textFile repositoryID\n".
"Specify \"LOCAL\" as repository if using local file system.\n";
exit;
}
my $wsdl = "$FindBin::Bin/../wsdl/ParsCit.wsdl";
my $parsCitService = SOAP::Lite
->service("file:$wsdl")
->on_fault(
sub {
my($soap, $res) = #_;
die ref $res ? $res->faultstring :
$soap->transport->status;
});
my ($citations, $citeFile, $bodyFile) =
$parsCitService->extractCitations($textFile, $repositoryID);
#print "$citations\n";
#print "CITEFILE: $citeFile\n";
#print "BODYFILE: $bodyFile\n";
From perldoc perlrun, about the -C switch:
Note: Since perl 5.10.1, if the -C option is used on the "#!" line, it
must be specified on the command line as well, since the standard
streams are already set up at this point in the execution of the perl
interpreter. You can also use binmode() to set the encoding of an I/O
stream.
Which is presumably what the compiler means by it being "too late".
In other words:
perl -CSD parsCit-client.pl
Because command-line options in a #! "shebang" are not passed consistently across all operating systems (see this answer), and Perl has already opened streams before parsing the script shebang, and so cannot compensate for this in some older OSs, it was decided in bug 34087 to forbid -C in the shebang. Of course, not everyone was happy with this "fix", particularly if it would have otherwise worked on their OS and they don't want to think about anything other than UTF-8.
If you think binmode() is ugly and unnecessary (and doesn't cover command-line arguments), you might like to consider the utf8::all package which has a similar effect to perl -CSDL.
Or were you using *nix, I would suggest export PERL_UNICODE="SDA" in the enclosing script to get Perl to realise it's in a UTF-8 environment.