Perl Inotify not responding to append (ie, echo 'test' >> file) - perl

When running the below test code to watch a file, events are only detected if I do a 'vim' on the file and write it out (or write-quit). Doing an 'echo' to the file, or adding text via perl is not detected.
test_inotify.pl:
#!/usr/bin/perl
use strict;
use warnings;
use diagnostics;
use Carp;
use IO::File;
use Linux::Inotify2;
$|++;
my $readfile = shift;
#my $action = "IN_CLOSE_WAIT";
#my $action = "IN_MODIFY";
#my $action = "IN_OPEN";
my $action = "IN_ALL_EVENTS";
unless ($readfile) { $readfile = "test.txt" };
my $inotify = Linux::Inotify2->new();
$inotify->watch($readfile, $action)
or die "Inotify watch on " . $readfile . "failed: $!\n";
while () {
my #events = $inotify->read();
unless (#events > 0) {
print "Inotify Read Error: $!\n";
exit;
};
foreach my $event (#events) {
print "Detected Event: " . $event->fullname . "\n";
};
};
test_fh_write.pl:
#!/usr/bin/perl -w
use strict;
use warnings;
use diagnostics;
use Carp;
use IO::File;
$|++;
my $readfile = shift;
unless ($readfile) { $readfile = "test.txt" };
my $readfh = IO::File->new( $readfile, ">>" ) or
#my $readfh = IO::File->new( $readfile, ">" ) or
die "Cannot open $readfile: $!";
$readfh->autoflush(1);
if ($readfh) {
print $readfh "test\n\n";
};
undef($readfh);
I've tried with test_fh_write.pl as well as echo commands like this: 'echo a >> test.txt', 'echo "test" >> test.txt', etc.
I have also tried with the "$|" character and without (even with $fh->autoflush(1)), but to no avail. Each of the $action variables defined in test_inotify.pl I've tried, but they all result the same.

I think the second argument to Linux::Inotify2::watch is a number/bitmask, not a string. You should be calling
$inotify->watch($readfile, IN_ALL_EVENTS)
instead of
$inotify->watch($readfile, "IN_ALL_EVENTS")
The bareword IN_ALL_EVENTS is resolved to the (presumably constant) function &Linux::Inotify2::IN_ALL_EVENTS.

Related

How do I execute a unix command containing a perl variable in perl

In the following perl code, I am tring to copy a perl variable $file from one directory to another directory with:
"system("cp $file $Output_Dir);
This command writes down the file name alright but then says:
cp: cannot stat 'tasmax_AFR-44_CNRM-CERFACS-CNRM-CM5_historical_r1i1p1_CLMcom-CCLM4-8-17_v1_day_19910101-19951231.nc': No such file or directory
The command
system("#sixfiles = ls $Vars[$kk]}*");
gives me the error:
sh: 1: =: not found
I wonder what is wrong with this code. Assistance will be appreciated.
#!/usr/bin/perl -w
use strict;
use warnings;
use File::Path;
use File::Copy;
my $debug = 1;
my #Vars = ("pr","tasmin","tasmax");
my $Vars;
my #sixfiles;
my $sixfiles;
my $Input_Dir = "/home/zmumba/DATA/Input_Dir";
my $Output_Dir = "/home/zmumba/DATA/Output_Dir";
for (my $kk=0; $kk < #Vars; ++$kk) {
opendir my $in_dir, $Input_Dir or die "opendir failed on $Input_Dir: $! ($^E)";
while (my $file=readdir $in_dir) {
next unless $file =~ /^$Vars[$kk]/;
next if -d $file;
print "$file\n";
print "Copying $file\n" if $debug;
my $cmd01 = "cp $file $Output_Dir";
print "Doing system ($cmd01)\n" if $debug;
system ($cmd01);
system("#sixfiles = ls $Vars[$kk]}*");
}
}
Try this:
use feature qw(say);
use strict;
use warnings;
use File::Spec;
my #Vars = ("pr","tasmin","tasmax");
my $Input_Dir = "/home/zmumba/DATA/Input_Dir";
my $Output_Dir = "/home/zmumba/DATA/Output_Dir";
opendir my $in_dir, $Input_Dir or die "opendir failed on $Input_Dir: $! ($^E)";
while (my $file=readdir $in_dir) {
next if ($file eq '.') || ($file eq '..');
next if -d $file;
next if !grep { $file =~ /^$_/ } #Vars;
say "Copying $file";
$file = File::Spec->catfile( $Input_Dir, $file );
system "cp", $file, $Output_Dir;
}
system ($cmd01);
Gives:
cp: cannot stat '<long-but-correct-file-name>': No such file or directory
This is almost certainly because you are not running the code from $Input_Dir, so that file doesn't exist in your current directory. You need to either chdir to the correct directory or add the directory path to the front of the file name variable.
system("#sixfiles = ls $Vars[$kk]}*");
This code makes no sense. The code passed to system() needs to be Unix shell code. That's the ls $Vars[$kk]}* bit (but I'm not sure where that } comes from). You can't populate a Perl array inside a shell command. You would need to capture the value returned from the ls command and then parse it somehow to separate it into a list.
You can give a try with the following code:
#!/usr/bin/env perl
use strict;
use warnings;
my $debug = 1;
my #Vars = ("pr", "tasmin", "tasmax");
my $Vars;
my $Input_Dir = "/home/zmumba/DATA/Input_Dir";
my $Output_Dir = "/home/zmumba/DATA/Output_Dir";
my $cpsrc, $cpdest = '';
print "No Write Permission: $!" unless(-w $Output_Dir);
for my $findex (0 .. $#Vars) {
$cpsrc = qq($Input_Dir/$Vars[$findex]);
print "$Vars[$findex]\n";
print "Copying $Vars[$findex]\n" if $debug;
my $cmd01 = "cp $cpsrc $Output_Dir";
print "Doing system ($cmd01)\n" if $debug;
system($cmd01);
}
You don't have to go through each file in source dir. You already know the files to copy from source.

perl- extract duplicate sequences from a multi-fasta file

I have a big fasta file input.fasta which consists many duplicate sequences. I want to enter a header name and extract out all the sequences with the matching header. I know this could be done easily done with awk/sed/grep but I need a Perl code.
input.fasta
>OGH38127_some_organism
PAAALGFSHLARQEDSALTPKHYTWTAPGEGDVRAPCPVLNTLANHEFLPHNGKNITVDK
AITALGDAMNISPALATTFFTGGLKTNPTPNATWFDLDMLHKHNVLEHDGSLSRRDMHFD
TSNKFDAATFANFLSYFDANATVLGVNETADARARHAYDMSKMNPEFTITSSMLPIMVGE
SVMMMLVWGSVEEPGAQRDYFEYFFRNERLPVELGWTPGETEIGVPVVTAMITAMVAASP
TDVP
>ABC14110_some_different_org_name
WWVAPGPGDSRGPCPGLNTLANHGYLPHDGKGITLSILADAMLDGFNIARSDALLLFTQ
AIRTSPQYPATNSFNLHDLGRDQLNRHNVLEHDASLSRADDFFGSNHIFNETVFDESRAY
AMLANSKIARQINSKAFNPQYKFTSKTEQFSLGEIAAPIIAFGNSTSGEVNRTLVEYFFM
NERLPIELGWKKSEDGIALDDILRVTQMISKAASLITPSALSWTAETLTP
>OGH38127_some_organism
LPWSRPGPGAVRAPCPMLNTLANHGFLPHDGKNISEARTVQALGRALNIEKELSQFLFEK
ALTTNPHTNATTFSLNDLSRHNLLEHDASLSRQDAYFGDNHDFNQTIFDETRSYWPHPVI
DIQAAALSRQARVNTSIAKNPTYNMSELGLDFSYGETAAYILILGDKDFGKVNRSWVEYL
FENERLPVELGWTRHNETITSDDLNTMLEKVVN
.
.
.
I have tried with the following script but it is not giving any output.
script.pl
#!/perl/bin/perl -w
use strict;
use warnings;
print "Enter a fasta header to search for:\n";
my $head = <>;
my $file = "input.fasta";
open (READ, "$file") || die "Cannot open $file: $!.\n";
my %seqs;
my $header;
while (my $line = <READ>){
chomp $line;
$line =~ s/^>(.*)\n//;
if ($line =~ m/$head/){
$header = $1;
}
}
close (READ);
open( my $out , ">", "out.fasta" ) or die $!;
my #count_seq = keys %seqs;
foreach (#count_seq){
print $out $header, "\n";
print $out $seqs{$header}, "\n";
}
exit;
Please help me correct this script.
Thanks!
If you use the Bioperl module Bio::SeqIO to handle the parsing of the fasta files, it becomes really simple:
#!/usr/bin/perl
use warnings;
use strict;
use Bio::SeqIO;
my ($file, $name) = #ARGV;
my $in = Bio::SeqIO->new(-file => $file, -format => "fasta");
my $out = Bio::SeqIO->new(-fh => \*STDOUT, -format => "fasta");
while (my $s = $in->next_seq) {
$out->write_seq($s) if $s->display_id eq $name;
}
run with perl grep_fasta.pl input.fasta OGH38127_some_organism
There's no need to store the sequences in memory, you can print them directly when reading the file. Use a flag variable ($inside in the example) that tells you whether you're reading the desired sequence or not.
#! /usr/bin/perl
use warnings;
use strict;
my ($file, $header) = #ARGV;
my $inside;
open my $in, '<', $file or die $!;
while (<$in>) {
$inside = $1 eq $header if /^>(.*)/;
print if $inside;
}
Run as
perl script.pl file.fasta OGH38127_some_organism > output.fasta

What produces the white space in my perl programm?

As the title says, I have a program or better two functions to read and write a file either in an array or to one. But now to the mean reason why I write this: when running my test several times my test program that tests my functions produces more and more white space. Is there somebody that could explain my fail and correct me?
my code
Helper.pm:
#!/usr/bin/env perl
package KconfCtl::Helper;
sub file_to_array($) {
my $file = shift();
my ( $filestream, $string );
my #rray;
open( $filestream, $file ) or die("cant open $file: $!");
#rray = <$filestream>;
close($filestream);
return #rray;
}
sub array_to_file($$;$) {
my #rray = #{ shift() };
my $file = shift();
my $mode = shift();
$mode='>' if not $mode;
my $filestream;
if ( not defined $file ) {
$filestream = STDOUT;
}
else {
open( $filestream, $mode, $file ) or die("cant open $file: $!");
}
my $l = #rray; print $l,"\n";
foreach my $line (#rray) {
print $filestream "$line\n";
}
close($filestream);
}
1;
test_helper.pl:
use KconfCtl::Helper;
use strict;
my #t;
#t= KconfCtl::Helper::file_to_array("kconf.test");
#print #t;
my $t_index=#t;
#t[$t_index]="n";
KconfCtl::Helper::array_to_file(\#t, "kconf.test", ">");
the result after the first:
n
and the 2nd run:
n
n
When you read from a file, the data includes the newline characters at the end of each line. You're not stripping those off, but you are adding an additional newline when you output your data again. That means your file is gaining additional blank lines each time you read and write it
Also, you must always use strict and use warnings 'all' at the top of every Perl script; you should avoid using subroutine prototypes; and you should declare all of your variables as late as possible
Here's a more idiomatic version of your module code which removes the newlines on input using chomp. Note that you don't need the #! line on the module file as it won't be run from the command line, but you my want it on the program file. It's also more normal to export symbols from a module using the Exporter module so that you don't have to qualify the subroutine names by prefixing them with the full package name
use strict;
use warnings 'all';
package KconfCtl::Helper;
sub file_to_array {
my ($file) = #_;
open my $fh, '<', $file or die qq{Can't open "$file" for input: $!}; #'
chomp(my #array = <$fh>);
return #array;
}
sub array_to_file {
my ($array, $file, $mode) = #_;
$mode //= '>';
my $fh;
if ( $file ) {
open $fh, $mode, $file or die qq{Can't open "$file" for output: $!}; #'
}
else {
$fh = \*STDOUT;
}
print $fh $_, "\n" for #$array;
}
1;
and your test program would be like this
#!/usr/bin/env perl
use strict;
use warnings 'all';
use KconfCtl::Helper;
use constant FILE => 'kconf.test';
my #t = KconfCtl::Helper::file_to_array(FILE);
push #t, 'n';
KconfCtl::Helper::array_to_file(\#t, FILE);
When you read in from your file, you need to chomp() the lines, or else the \n at the end of the line is included.
Try this and you'll see what's happening:
use Data::Dumper; ## add this line
sub file_to_array($) {
my $file = shift();
my ( $filestream, $string );
my #rray;
open( $filestream, '<', $file ) or die("cant open $file: $!");
#rray = <$filestream>;
close($filestream);
print Dumper( \#rray ); ### add this line
return #rray;
}
you can add
foreach(#rray){
chomp();
}
into your module to stop this happening.

Read multiple files from folder in perl

I'm pretty new on perl and in need for some help, basically what I want is a program that reads all .txt files from a folder, doing the script and throw the output in a new folder with a new name. Everything works when I'm working with one file at the time, specifying the name of the file.. But I can't get it to work with all of the files in the folder. This is how far I've gotten.
#!/usr/bin/perl
use warnings;
use strict;
use Path::Class;
use autodie;
use File::Find;
my #now = localtime();
my $timeStamp = sprintf(
"%04d%02d%02d-%02d:%02d:%02d",
$now[5] + 1900,
$now[4] + 1,
$now[3], $now[2], $now[1], $now[0]); #A function that translates time
my %wordcount;
my $dir = "/home/smenk/.filfolder";
opendir(DIR, $dir) || die "Kan inte öppna $dir: $!";
my #files = grep { /txt/ } readdir(DIR);
closedir DIR;
my $new_dir = dir("/home/smenk/.result"); # Reads in the folder for save
my $new_file = $new_dir->file("$timeStamp.log"); # Reads in the new file timestamp variable
open my $fh, '<', $dir or die "Kunde inte öppna '$dir' $!";
open my $fhn, '>', $new_file or die "test '$new_file'";
foreach my $file (#files) {
open(FH, "/home/smenk/.filfolder/$file") || die "Unable to open $file - $!\n";
while (<FH>) {
}
close(FH);
}
while (my $line = <$fh>) {
foreach my $str (split /\s+/, $line) {
$wordcount{$str}++;
}
}
my #listing = (sort { $wordcount{$b} <=> $wordcount{$a} } keys %wordcount)[0 .. 9];
foreach my $str (#listing) {
my $output = $wordcount{$str} . " $str\n";
print $fhn $output;
}
Here is the simplest skeleton for the reading part using Path::Class (see also dir and file:
#!/usr/bin/perl
use warnings;
use strict;
use Path::Class;
my $src = dir("/home/smenk/.filfolder");
my #txt_files = grep /[.] txt\z/x, $src->children;
for my $txt_file ( #txt_files ) {
my $in = $txt_file->openr;
while (my $line = <$in>) {
print "OUT: $line";
}
}
You can also use another great module Path::Tiny, for dir/file operations and the Time::Piece for the date/time functions - like:
#!/usr/bin/env perl
use strict;
use warnings;
use Path::Tiny;
use Time::Piece;
my #txtfiles = path("/home/smenk/.filfolder")->children(qr/\.txt\z/);
my $outdir = path("home/smenk/.result");
$outdir->mkpath; #create the dir...
my $t = localtime;
my $outfile = $outdir->child($t->strftime("%Y%m%d-%H%M%S.txt"));
$outfile->touch;
my #outdata;
for my $infile (#txtfiles) {
my #lines = $infile->lines({chomp => 1});
#do something with lines and create the output #data
push #outdata, scalar #lines;
}
$outfile->append({truncate => 1}, map { "$_\n" } #outdata); #or spew;

#INC hook unknown fatal error

Hey I'm writing a program that uses an #INC hook to decrypt the real perl source from blowfish. I'm having a quite annoying problem that doesn't show up using warnings or any of my standard tricks... Basically when I get to creating the new cipher object the loop skips to the next object in #INC without an error or anything.... I dont know what to do!
#!/usr/bin/perl -w
use strict;
use Crypt::CBC;
use File::Spec;
sub load_crypt {
my ($self, $filename) = #_;
print "Key?\n: ";
chomp(my $key = <STDIN>);
for my $prefix (#INC) {
my $buffer = undef;
my $cipher = Crypt::CBC->new( -key => $key, -cipher => 'Blowfish');
my $derp = undef;
$cipher ->start('decrypting');
open my $fh, '<', File::Spec->($prefix, "$filename.nc") or next;
while (read($fh,$buffer,1024)) {
$derp .= $cipher->crypt($buffer);
}
$derp .= $cipher->finish;
return ($fh, $derp);
}
}
BEGIN {
unshift #INC, \&load_crypt;
}
require 'gold.pl';
Also if I put the actual key in the initializing method it still fails
You've got a bunch of problems here. First of all, you're using File::Spec wrong. Second, you're returning a filehandle that's already at end of file, and a string that isn't a valid return value. (Also, I'd put the key prompt outside of the hook.)
#!/usr/bin/perl -w
use strict;
use Crypt::CBC;
use File::Spec;
# Only read the key once:
print "Key?\n: ";
chomp(my $key = <STDIN>);
sub load_crypt {
my ($self, $filename) = #_;
return unless $filename =~ /\.pl$/;
for my $prefix (#INC) {
next if ref $prefix;
#no autodie 'open'; # VERY IMPORTANT if you use autodie!
open(my $fh, '<:raw', File::Spec->catfile($prefix, "$filename.nc"))
or next;
my $buffer;
my $cipher = Crypt::CBC->new( -key => $key, -cipher => 'Blowfish');
my $derp;
$cipher->start('decrypting');
while (read($fh,$buffer,1024)) {
$derp .= $cipher->crypt($buffer);
}
$derp .= $cipher->finish;
# Subroutine writes 1 line of code into $_ and returns 1 (false at EOF):
return sub { $derp =~ /\G(.*\n?)/g and ($_ = $1, 1) };
}
return; # Didn't find the file; try next #INC entry
} # end load_crypt
# This doesn't need a BEGIN block, because we're only using the hook
# with require, and that's a runtime operation:
unshift #INC, \&load_crypt;
require 'gold.pl';