I wrote a Perl script in order to automate a process for sending a lot of jobs in an lsf based cluster.
The problem is that for a reason that I don't get it cuts the line with the job in two lines, and the job cannot run.
here is my script:
my $controler = $ARGV[0];
open my $cont, $controler or die "Could not open $controler: $!";
my $str = qx(wc -l $controler | awk '{print $1}');
#my $str2 = system($str);
#my #count = split /\n/,$str;
#print $str;
for (my $f = 1; $f <= $str; $f++) {
#print $count[0];
`mkdir ~/Multiple_alignments/ELEMENTS-AUTO-$f`;
#`mkdir ../SCORES-AUTO-$f`;
}
while ( $cont = <>){
chomp;
my #lines = split /\t/, $cont;
my $count2 = $lines[0];
my $cover = $lines[1];
my $length = $lines[2];
my $rho = $lines[3];
my #files = <*maf>;
foreach my $file (#files) {
#print $file . "\n";
my $base = basename($file, ".maf");
#print "$base\n";
#print "$cover\n";
#print "$length\n";
#print "$rho\n";
print "`bsub -q q_cf_htc_work -R \"select[type==X86_64 && mem>6000]\" rusage[mem=6000] -M6000 -o /home/vasilis.lenis/Multiple_alignments/out-files/phastCons_$base.o -e /home/vasilis.lenis/Multiple_alignments/out-files/phastCons_$base.e -J phastCons$base phastCons --target-coverage $cover --expected-length $length --rho $rho --most-conserved ../ELEMENTS-AUTO-$count2/most_conserved_$base.bed --msa-format MAF $file mytrees_no3.noncons.mod --no-post-probs`\n";
}
}
I just cannot understand why its happening.
(Also, the awk command that I have at the third line doesn't work)
Thank you in advance,
Vasilis.
Thank you very much for your guidance.
I believe that I solved the problem with the new line.
I was using wrong the chomp function.
chomp($cont);
Related
open my $directory, '<', abc.txt
chomp(my #values = <$directory>);
There is a file named abc.txt with the following contents:
abcde
abc
bckl
drfg
efgt
eghui
webnmferg
With the above lines, I am sending contents of file abc.txt into an array
Intention is to create a loop to run a command on all the lines of file abc.txt
Any suggestions for creating the loop?
open my $directory_fh, '<', abc.txt or die "Error $! opening abc.txt";
while (<$directory_fh>) {
chomp; # Remove final \n if any
print $_; # Do whatevery you want here
}
close $directory_fh;
I prefer to suffix all filehandles with _fh to make them more obvious.
while (<fh>) loops though all lines of the file.
You might need/want to remove a final \r if the file might have Windows/MS-DOS format.
create a loop to run a command on all the lines of file abc.txt
foreach my $line (#lines){
#assugming $cmd contains the command you want to execute
my $output = `$cmd $line`;
print "Executed $cmd on $line, output: $output\n";
}
Edit: As per Sebastian's feedback
my $i = 0;
while ($i <= $#lines){
my $output = `$cmd $lines[$i]`;
print "Executed $cmd on $lines[$i], output: $output\n";
}
OR if you are ok with destroying array then:
while (#lines){
my $line = shift #lines;
my $output = `$cmd $line`;
print "Executed $cmd on $line, output: $output\n";
}
If you wanted safe code that didn't refer to the array twice, you could use splice in a list assignment.
while (my ($line) = splice(#array, 0, 1)) {
my $output = `$cmd $line`;
print "Executed $cmd on $line, output: $output\n";
}
I have a file with several blocks of text separated by blank line. Ex.:
block1
block1
block2
block3
block3
I need a solution with sed, awk or Perl to locate the first blank line and redirect the previous block to another file and so on until the end of the file.
I have this command in sed that locates the first block, but not the rest:
sed -e '/./!Q'
Can someone help me?
give this line a try:
awk -v RS="" '{print > "file"++c".txt"}' input
it will generate file1...n.txt
Here's an awk:
$ awk 'BEGIN{file="file"++cont}/^$/{file="file"++cont;next}{print>file}' infile
Results
$ cat file1
block1
block1
$ cat file2
block2
$ cat file3
block3
block3
taking into account several empty string between block
awk '/./{if(!L)++C;print>"Out"C".txt"}{L=$0!~/^$/}' YourFile
Sed will not allow different external files (unspecified number of in fact) as output
Here's the solution in Perl
open( my $fh, '<', '/tmp/a.txt' ) or die $!;
{
## record delimiter
local $/ = "\n\n";
my $count = 1;
while ( chomp( my $block = <$fh> ) ) {
open( my $ofh, '>', sprintf( '/tmp/file%d', $count++ ) ) or die $!;
print {$ofh} $block;
close($ofh);
}
}
close($fh);
Here's my solution in Perl:
#!/usr/bin/perl
use strict;
use warnings;
my $n = 0;
my $block = '';
while (<DATA>) { # line gets stored in $_
if (/^\s*$/) { # blank line
write_to_file( 'file' . ++$n, $block );
$block = '';
} else {
$block .= $_;
}
}
# Write any remaining lines
write_to_file( 'file' . ++$n, $block );
sub write_to_file {
my $file = shift;
my $data = shift;
open my $fh, '>', $file or die $!;
print $fh $data;
close $fh;
}
__DATA__
block1
block1
block2
block3
block3
Output:
$ grep . file*
file1:block1
file1:block1
file2:block2
file3:block3
file3:block3
Another way to do it in Perl:
#!/usr/bin/perl
use strict;
use warnings;
# store all lines in $data
my $data = do { local $/; <DATA> };
my #blocks = split /\n\n/, $data;
my $n = 0;
write_to_file( 'file' . ++$n, $_ ) for #blocks;
sub write_to_file {
my $file = shift;
my $data = shift;
open my $fh, '>', $file or die $!;
print $fh $data;
close $fh;
}
__DATA__
block1
block1
block2
block3
block3
This might work for you (GNU csplit & sed):
csplit -qf uniqueFileName file '/^$/' '{*}' && sed -i '/^$/d' uniqueFileName*
or if you want to go with the defaults:
csplit -q file '/^$/' '{*}' && sed -i '/^$/d' xx*
Use:
tail -n+1 xx* # to check the results
I have a growing number of files to process using a simple Perl script I wrote. The script takes two files as input and prints an output. I want to use a bash script (or anything really) to automate the following usage:
perl Program.pl GeneLevels_A GeneLevels_B > GeneLevels_A_B
with every paired, non-directional combination of files in a particular directory.
Here is the Perl script:
#!/usr/bin/perl
use strict;
use warnings;
die "Usage: $0 <File_1> <File_2>\n" unless #ARGV == 2;
my $file1 = shift #ARGV;
my $file2 = shift #ARGV;
my %hash1;
my %hash2;
my $counter = 0;
my $amt = 25;
my $start = 244 - $amt;
open (REF, $file1);
while (<REF>) {
my $line = $_;
chomp $line;
if ($counter < $start) {
$counter++;
next;
}
my #cells = split('\t', $line);
my $ID = $cells[2];
my $row = $cells[0];
$hash1{$ID} = $row;
$counter++;
}
close REF;
$counter = 0;
open (FILE, $file2);
while (<FILE>) {
my $line = $_;
chomp $line;
if ($counter < $start) {
$counter++;
next;
}
my #cells = split('\t', $line);
my $ID = $cells[2];
my $row = $cells[0];
$hash2{$ID} = $row;
$counter++;
}
close FILE;
while ( my ($key, $value) = each(%hash1) ) {
if ( exists $hash2{$key} ) {
print "$key\t$value\t$hash2{$key}\n";
}
}
A good solution would allow me to run the Perl script on every file with an appropriate suffix.
An even better solution would assess the suffixes of existing files to determine which pairs of files have already been processed this way and omit those. For example if File_A, File_B, File_C, and File_B_C exist then only File_A_B and File_A_C would be produced. Note that File_A_B and File_B_A are equivalent.
This should work. Better checks for bad arguments would be a good thing to add:
#!/bin/bash
if [ $# != 2 ]; then
echo "usage: pair <suffix1> <suffix2>"
exit
fi
suffix1=$1
suffix2=$2
for file1 in *_${suffix1}; do
fileCheck=$(echo $file1 | sed -e "s#_$suffix2##")
if [ "$fileCheck" = "$file1" ]; then
file2=${file1/_$suffix1/_$suffix2}
if [[ ( ! -f ${file1}_${suffix2} ) && ( ! -f ${file2}_${suffix1} ) ]]; then
echo processing ${file1}_${suffix2}
perl Program.pl $file1 $file2 > ${file1}_${suffix2}
fi
fi
done
Dear all I am trying to get a script working and have no clue where to start with the error being produced
sh: 1: Syntax error: redirection unexpected
My script is below and if anyone can help I would be very grateful
Tony
ebay.pl
#!/usr/bin/perl -w
system("/usr/local/bin/ebaycurl.sh");
open (ITEMS, "/usr/local/data/eBaystuff") or die "stuff $!";
while (<ITEMS>) {
chomp;
next unless /ViewItem\&/;
s/Item not relisted/Item_not_relisted/g;
s/Item relisted/Item_relisted/g;
#words = split;
$relist = "";
foreach $word (#words) {
if ($word =~ /ViewItem\&/) {
print "\n";
$print_it = 1;
$link = $word;
($junk, $link) = split /f=/, $word;
$link =~ s/&/&/g;
#system("/usr/local/bin/ebaycurlitem.sh $link >/dev/null 2>/dev/null");
system("/usr/local/bin/ebaycurlitem.sh $link");
open (ITEM, "/usr/local/data/eBayitem") or die "item $!";
$relist = "";
while (<ITEM>) {
next unless /Relist/;
$relist = 'relist';
}
#($junk, $itemid) = split /item=/, $link;
#$itemid =~ s/\"//;
print "$relist\t";
next;
}
if (defined $print_it) {
if ($word =~ /\>/) {
$print_it = undef;
($rem, $junk) = split />/, $word;
print "$rem";
} else {
$word =~ s/title=//;
print "$word ";
}
}
if ($word =~ /Item_not_relisted/ and $relist =~ /relist/) {print "\t\t\t\tNOT RELISTED";}
}
print "\n";
}
ebaycurl.sh
#!/bin/bash
$(COOKIE_DIR)="cat /usr/local/etc/ebay_cookie_dir)
(/usr/bin/curl --cookie "COOKIE_DIR"/cookies.txt 'http://k2b-bulk.ebay.co.uk/ws/eBayISAPI.dll?SalesRecordConsole¤tpage=SCSold&ssPageName=STRK:ME:LNLK; -o /usr/local/data/eBaystuff)"
There's a lot wrong with the bash script you posted. I recommend reading up on bash syntax cause it looks like you just threw parentheses and quotes in at random. Rather than explain each correction I'm just gonna post this and hope it's self-explanatory...
#!/bin/bash
COOKIE_DIR=$(cat /usr/local/etc/ebay_cookie_dir)
curl --cookie "$COOKIE_DIR"/cookies.txt -o /usr/local/data/eBaystuff \
'http://k2b-bulk.ebay.co.uk/ws/eBayISAPI.dll?SalesRecordConsole¤tpage=SCSold&ssPageName=STRK:ME:LNLK'
I want to get the unique elements (lines) from a file which I will further send through email.
I have tried 2 methods but both are not working:
1st way:
my #array = "/tmp/myfile.$device";
my %seen = ();
my $file = grep { ! $seen{ $_ }++ } #array;
2nd way :
my $filename = "/tmp/myfile.$device";
cat $filename |sort | uniq > $file
How can I do it?
You seem to have forgotten to read the file!
open(my $fh, '<', $file_name)
or die("Can't open \"$file_name\": $!\n");
my %seen;
my #unique = grep !$seen{$_}++, <$fh>;
You need to open the file and read it.
"cat" is a shell command not perl
Try something like this
my $F;
die $! if(!open($F,"/tmp/myfile.$device"));
my #array = <$F>;
my %seen = (); my $file = grep { ! $seen{ $_ }++ } #array;
The die $! will stop the program with an error if the file doesn't open correctly;
#array=<$F> reads all the data from the file $F opened above into the array.
If you rig the argument list, you can make Perl open the file automatically, using:
perl -n -e 'BEGIN{#ARGV=("/tmp/myfile.device");} print if $count{$_}++ == 0;'