Im trying to make this script put the output files in results/ but i only get them in results/spool. How can I make them end up in the right place?
#!/usr/bin/perl -w
use POSIX;
my $date = strftime "%Y%m%d_%H%M.txt", localtime;
my #files = <spool/*.*>;
foreach $file (#files) {
my $date_file = $file;
system("$^X wfreq.pl $file >> results/${date_file}_$date");
unlink("$file");
}
Have you tested this ?
#!/usr/bin/perl -w
use POSIX;
my $date = strftime "%Y%m%d_%H%M.txt", localtime;
my #files = <spool/*.*>;
foreach $file (#files) {
(my $date_file = $file) =~ s!.*/!!;
system("$^X wfreq.pl $file >> results/${date_file}_$date");
unlink("$file");
}
The functions system(), qx() & exec() can handle variables without any problems.
Is this what your looking for?
This will give you a separate file for each date
#files = <spool/*.*>;
foreach $file (#files) {
system($^X, "wfreq.pl", $file);
$DATE = strftime "%Y%m%d";
open OUTPUT, '>results/$DATE.txt' or die;
print OUTPUT $DATE;
}
Or for a file with results all in one text file
#files = <spool/*.*>;
open OUTPUT, '>>results/results.txt' or die;
foreach $file (#files) {
system($^X, "wfreq.pl", $file);
$DATE = strftime "%Y%m%d";
print OUTPUT $DATE;
}
Related
i'm writing a basic Perl script to check if there're files in certain directories their names contain today's date, the script works fine when there are files in the directory and returns TRUE, but doesn't return FALSE when no files withe the given criteria exist
the script is as below:
#!/usr/bin/perl -w
use POSIX qw(strftime);
my $datestring = strftime "%Y%m%d", localtime;
opendir(DIR, 'C:\Perl');
#files = grep (/$datestring/,readdir(DIR));
closedir(DIR);
foreach $file (#files) {
my $count = () = $file =~ /$datestring/;
if ($count > 0) {
print ("TRUE");
}
else
{
print ("FALSE");
}
}
the files names are like
export_opportunities_20181111-040005_20181124-040010.csv
export_opportunities_20181111-040005_20181122-040010.csv
You can use #files in scalar context to tell if grep returned any matches from readdir. In scalar context, #files is the number of elements in the array.
#files = grep (/$datestring/,readdir(DIR));
# this is more commonly written
# #files = grep {/$datestring/} readdir(DIR);
print #files > 0 ? "TRUE" : "FALSE";
# or
print scalar(#files)." files matched $datestring";
See also List::Util.
use v5.10;
use List::Util qw<all any none>;
###
#files = readdir(DIR);
say 'all match' if all {/$datestring/} #files;
say 'at least one match' if any {/$datestring/} #files;
say 'no matches' if none {/$datestring/} #files;
As commented by toolic, when you are already removing non-matching files here :
#files = grep (/$datestring/,readdir(DIR));
Hence your for loop will never see non-matching files.
Here are a few other comments on your code :
always use strict and use warnings (there are several variables in your code snippet which are not properly declared)
always check the return code of system calls such as opendir
you can use the smartmatch operator in boolean context instead of assigning to the $count variable
you don't need to use parentheses around argument to the built-in print function
Here is a cleaner version of your code :
#!/usr/bin/perl -w
use strict;
use warnings;
use POSIX qw(strftime);
my $datestring = strftime "%Y%m%d", localtime;
opendir(DIR, 'C:\Perl') or die "cannot open dir : $!";
my #files = readdir(DIR);
closedir(DIR);
foreach my $file (#files) {
if ($file =~ /$datestring/) {
print "TRUE\n";
} else {
print "FALSE\n";
}
}
I have a big fasta file input.fasta which consists many duplicate sequences. I want to enter a header name and extract out all the sequences with the matching header. I know this could be done easily done with awk/sed/grep but I need a Perl code.
input.fasta
>OGH38127_some_organism
PAAALGFSHLARQEDSALTPKHYTWTAPGEGDVRAPCPVLNTLANHEFLPHNGKNITVDK
AITALGDAMNISPALATTFFTGGLKTNPTPNATWFDLDMLHKHNVLEHDGSLSRRDMHFD
TSNKFDAATFANFLSYFDANATVLGVNETADARARHAYDMSKMNPEFTITSSMLPIMVGE
SVMMMLVWGSVEEPGAQRDYFEYFFRNERLPVELGWTPGETEIGVPVVTAMITAMVAASP
TDVP
>ABC14110_some_different_org_name
WWVAPGPGDSRGPCPGLNTLANHGYLPHDGKGITLSILADAMLDGFNIARSDALLLFTQ
AIRTSPQYPATNSFNLHDLGRDQLNRHNVLEHDASLSRADDFFGSNHIFNETVFDESRAY
AMLANSKIARQINSKAFNPQYKFTSKTEQFSLGEIAAPIIAFGNSTSGEVNRTLVEYFFM
NERLPIELGWKKSEDGIALDDILRVTQMISKAASLITPSALSWTAETLTP
>OGH38127_some_organism
LPWSRPGPGAVRAPCPMLNTLANHGFLPHDGKNISEARTVQALGRALNIEKELSQFLFEK
ALTTNPHTNATTFSLNDLSRHNLLEHDASLSRQDAYFGDNHDFNQTIFDETRSYWPHPVI
DIQAAALSRQARVNTSIAKNPTYNMSELGLDFSYGETAAYILILGDKDFGKVNRSWVEYL
FENERLPVELGWTRHNETITSDDLNTMLEKVVN
.
.
.
I have tried with the following script but it is not giving any output.
script.pl
#!/perl/bin/perl -w
use strict;
use warnings;
print "Enter a fasta header to search for:\n";
my $head = <>;
my $file = "input.fasta";
open (READ, "$file") || die "Cannot open $file: $!.\n";
my %seqs;
my $header;
while (my $line = <READ>){
chomp $line;
$line =~ s/^>(.*)\n//;
if ($line =~ m/$head/){
$header = $1;
}
}
close (READ);
open( my $out , ">", "out.fasta" ) or die $!;
my #count_seq = keys %seqs;
foreach (#count_seq){
print $out $header, "\n";
print $out $seqs{$header}, "\n";
}
exit;
Please help me correct this script.
Thanks!
If you use the Bioperl module Bio::SeqIO to handle the parsing of the fasta files, it becomes really simple:
#!/usr/bin/perl
use warnings;
use strict;
use Bio::SeqIO;
my ($file, $name) = #ARGV;
my $in = Bio::SeqIO->new(-file => $file, -format => "fasta");
my $out = Bio::SeqIO->new(-fh => \*STDOUT, -format => "fasta");
while (my $s = $in->next_seq) {
$out->write_seq($s) if $s->display_id eq $name;
}
run with perl grep_fasta.pl input.fasta OGH38127_some_organism
There's no need to store the sequences in memory, you can print them directly when reading the file. Use a flag variable ($inside in the example) that tells you whether you're reading the desired sequence or not.
#! /usr/bin/perl
use warnings;
use strict;
my ($file, $header) = #ARGV;
my $inside;
open my $in, '<', $file or die $!;
while (<$in>) {
$inside = $1 eq $header if /^>(.*)/;
print if $inside;
}
Run as
perl script.pl file.fasta OGH38127_some_organism > output.fasta
could anyone please share with me a snippet where i can do a grep search in perl file. Example,
i need this grep: grep 1115852311 /opt/files/treated/postpaid/*
to be done in perl script and print all the matches
tried the below, but did not work :
my $start_dir= "\opt\files\treated\postpaid\";
my $file_name = "*";
my #filematches;
opendir(DIR, "$start_dir");
#xml_files = grep(1115852311,readdir(DIR));
print #xml_files;
A good start would be to read the documentation for grep(). If you do, you'll see that Perl's grep() works rather differently to the Unix grep command. The Unix command just looks for text in a list of files. The Perl version works on any list of data and returns any elements in that list for which an arbitrary Boolean expression is true.
A Perl version of the Unix command would look something like this:
while (<$some_open_filehandle>) {
print if /$some_string/;
}
That's not quite what you want, but we can use it as a start. First, let's write something that takes a filename and string and checks whether the string appears in the file:
sub is_string_in_file {
my ($filename, $string) = #_;
open my $fh, '<', $filename or die "Cannot open file '$filename': $!\n";
return grep { /$string/ } <$fh>;
}
We can now use that in a loop which uses readdir() to get a list of files.
my #files;
my $dir = '/opt/files/treated/postpaid/';
opendir my $dh, $dir or die $!;
while (my $file = readdir($dh)) {
if (is_string_in_file("$dir$file", 1115852311) {
push #files, "$dir$file";
}
}
After running that code, the list of files that contain your string will be in #files.
You might want to look at glob() instead of opendir() and readdir().
used the below snippet to achieve what i wanted
#!/usr/bin/perl
use strict;
use warnings;
sub is_string_in_file {
my ($filename, $string) = #_;
open my $fh, '<', $filename
or die "Cannot open file \n";
while(my $line = <$fh>){
if($line =~ /$string/){
print $string;
print $filename."\n";
}
}
#return grep { $_ eq $string } <$fh>;
}
my #files;
my $dir = '/opt/files/treated/postpaid/';
opendir my $dh, $dir or die $!;
while (my $file = readdir($dh)) {
is_string_in_file("$dir$file", 1115852311);
}
I'm pretty new on perl and in need for some help, basically what I want is a program that reads all .txt files from a folder, doing the script and throw the output in a new folder with a new name. Everything works when I'm working with one file at the time, specifying the name of the file.. But I can't get it to work with all of the files in the folder. This is how far I've gotten.
#!/usr/bin/perl
use warnings;
use strict;
use Path::Class;
use autodie;
use File::Find;
my #now = localtime();
my $timeStamp = sprintf(
"%04d%02d%02d-%02d:%02d:%02d",
$now[5] + 1900,
$now[4] + 1,
$now[3], $now[2], $now[1], $now[0]); #A function that translates time
my %wordcount;
my $dir = "/home/smenk/.filfolder";
opendir(DIR, $dir) || die "Kan inte öppna $dir: $!";
my #files = grep { /txt/ } readdir(DIR);
closedir DIR;
my $new_dir = dir("/home/smenk/.result"); # Reads in the folder for save
my $new_file = $new_dir->file("$timeStamp.log"); # Reads in the new file timestamp variable
open my $fh, '<', $dir or die "Kunde inte öppna '$dir' $!";
open my $fhn, '>', $new_file or die "test '$new_file'";
foreach my $file (#files) {
open(FH, "/home/smenk/.filfolder/$file") || die "Unable to open $file - $!\n";
while (<FH>) {
}
close(FH);
}
while (my $line = <$fh>) {
foreach my $str (split /\s+/, $line) {
$wordcount{$str}++;
}
}
my #listing = (sort { $wordcount{$b} <=> $wordcount{$a} } keys %wordcount)[0 .. 9];
foreach my $str (#listing) {
my $output = $wordcount{$str} . " $str\n";
print $fhn $output;
}
Here is the simplest skeleton for the reading part using Path::Class (see also dir and file:
#!/usr/bin/perl
use warnings;
use strict;
use Path::Class;
my $src = dir("/home/smenk/.filfolder");
my #txt_files = grep /[.] txt\z/x, $src->children;
for my $txt_file ( #txt_files ) {
my $in = $txt_file->openr;
while (my $line = <$in>) {
print "OUT: $line";
}
}
You can also use another great module Path::Tiny, for dir/file operations and the Time::Piece for the date/time functions - like:
#!/usr/bin/env perl
use strict;
use warnings;
use Path::Tiny;
use Time::Piece;
my #txtfiles = path("/home/smenk/.filfolder")->children(qr/\.txt\z/);
my $outdir = path("home/smenk/.result");
$outdir->mkpath; #create the dir...
my $t = localtime;
my $outfile = $outdir->child($t->strftime("%Y%m%d-%H%M%S.txt"));
$outfile->touch;
my #outdata;
for my $infile (#txtfiles) {
my #lines = $infile->lines({chomp => 1});
#do something with lines and create the output #data
push #outdata, scalar #lines;
}
$outfile->append({truncate => 1}, map { "$_\n" } #outdata); #or spew;
I'm trying to clean up a directory that contains a lot of sub directories that actually belong in some of the sub directories, not the main directory.
For example, there is
Main directory
sub1
sub2
sub3
HHH
And HHH belongs in sub3. HHH has multiple text files inside of it (as well as some ..txt and ...txt files that I would like to ignore), and each of these text files has a string
some_pattern [sub3].
So, I attempted to write a script that looks into the file and then moves it into its corresponding directory
use File::Find;
use strict;
use warnings;
use File::Copy;
my $DATA = "D:/DATA/DATA_x/*";
my #dirs = grep { -d } glob $DATA;
foreach (#dirs) {
if ($_ =~ m/HHH/) {
print "$_\n";
my $file = "$_/*";
my #files = grep { -f } glob $file;
foreach (#files) {
print "file $_\n";
}
foreach (#files) {
print "\t$_\n";
my #folders = split('/', $_);
if ($folders[4] eq '..txt' or $folders[4] eq '...txt') {
print "$folders[4] ..txt\n";
}
foreach (#folders) {
print "$_\n";
}
open(FH, '<', $_);
my $value;
while (my $line = <FH>) {
if ($line =~ m/some_pattern/) {
($value) = $line =~ /\[(.+?)\]/;
($value) =~ s/\s*$//;
print "ident'$value'\n";
my $new_dir = "$folders[0]/$folders[1]/$folders[2]/$value/$folders[3]/$folders[4]";
print "making $folders[0]/$folders[1]/$folders[2]/$value/$folders[3]\n";
print "file is $folders[4]\n";
my $new_over_dir = "$folders[0]/$folders[1]/$value/$folders[2]/$folders[3]";
mkdir $new_over_dir or die "Can't make it $!";
print "going to swap\n '$_'\n for\n '$new_dir'\n";
move($_, $new_dir) or die "Can't $!";
}
}
}
}
}
It's saying
Can't make it No such file or directory at foo.pl line 57, <FH> line 82.
Why is it saying that it won't make a file that doesn't exist?
A while later: here is my final script:
use File::Find;
use strict;
use warnings;
use File::Copy;
my $DATA = "D:/DATA/DATA_x/*";
my #dirs = grep { -d } glob $DATA;
foreach (#dirs) {
if ($_ =~ m/HHH/) {
my $value;
my #folders;
print "$_\n";
my $file = "$_/*";
my #files = grep { -f } glob $file;
foreach (#files) {
print "file $_\n";
}
foreach (#files) {
print "\t$_\n";
#folders = split('/', $_);
if ($folders[4] eq '..txt' or $folders[4] eq '...txt') {
print "$folders[4] ..txt\n";
}
foreach (#folders) {
print "$_\n";
}
open(FH, '<', $_);
while (my $line = <FH>) {
if ($line =~ m/some_pattern/) {
($value) = $line =~ /\[(.+?)\]/;
($value) =~ s/\s*$//;
print "ident'$value'\n";
}
}
}
if($value){
print "value $value\n";
my $dir1 = "/$folders[1]/$folders[2]/$folders[3]/$folders[4]/$folders[5]";
my $dir2 = "/$folders[1]/$folders[2]/$folders[3]/$folders[4]/$value";
system("cp -r $dir1 $dir2");
}
}
}
}
This works. It looks like part of my problem from before was that I was trying to run this on a directory in my D: drive--when I moved it to the C: drive, it worked fine without any permissions errors or anything. I did try to implement something with Path::Tiny, but this script was so close to being functional (and it was functional in a Unix environment), that I decided to just complete it.
You really should read the Path::Tiny doccu. It probably contains everything you need.
Some starting points, without error handling and so on...
use strict;
use warnings;
use Path::Tiny;
my $start=path('D:/DATA/DATA_x');
my $iter = path($start)->iterator({recurse => 1});
while ( $curr = $iter->() ) {
#select here the needed files - add more conditions if need
next if $curr->is_dir; #skip directories
next if $curr =~ m/HHH.*\.{2,3}txt$/; #skip ...?txt
#say "$curr";
my $content = $curr->slurp;
if( $content =~ m/some_pattern/ ) {
#do something wih the file
say "doing something with $curr";
my $newfilename = path("insert what you need here"); #create the needed new path for the file ..
path($newfilename->dirname)->mkpath; #make directories
$curr->move($newfilename); #move the file
}
}
Are you sure of the directory path you are trying to create. The mkdir call might be failing if some of the intermediate directories doesn't exist. If your code is robust to ensure that
the variable $new_over_dir contains the directory path you have to create, you can use method make_path from perl module File::Path to create the new directory, instead of 'mkdir'.
From the documentation of make_path:
The make_path function creates the given directories if they don't
exists before, much like the Unix command mkdir -p.