I have a file input as below;
#
volume stats
start_time 1
length 2
--------
ID
0x00a,1,2,3,4
0x00b,11,12,13,14
0x00c,21,22,23,24
volume stats
start_time 2
length 2
--------
ID
0x00a,31,32,33,34
0x00b,41,42,43,44
0x00c,51,52,53,54
volume stats
start_time 3
length 2
--------
ID
0x00a,61,62,63,64
0x00b,71,72,73,74
0x00c,81,82,83,84
#
I need output in below format;
1 33 36 39 42
2 123 126 129 132
3 213 216 219 222
#
Below is my code;
#!/usr/bin/perl
use strict;
use warnings;
#use File::Find;
# Define file names and its location
my $input = $ARGV[0];
# Grab the vols stats for different intervals
open (INFILE,"$input") or die "Could not open sample.txt: $!";
my $date_time;
my $length;
my $col_1;
my $col_2;
my $col_3;
my $col_4;
foreach my $line (<INFILE>)
{
if ($line =~ m/start/)
{
my #date_fields = split(/ /,$line);
$date_time = $date_fields[1];
}
if ($line =~ m/length/i)
{
my #length_fields = split(/ /,$line);
$length = $length_fields[1];
}
if ($line =~ m/0[xX][0-9a-fA-F]+/)
{
my #volume_fields = split(/,/,$line);
$col_1 += $volume_fields[1];
$col_2 += $volume_fields[2];
$col_3 += $volume_fields[3];
$col_4 += $volume_fields[4];
#print "$col_1\n";
}
if ($line =~ /^$/)
{
print "$date_time $col_1 $col_2 $col_3 $col_4\n";
$col_1=0;$col_2=0;$col_3=0;$col_4=0;
}
}
close (INFILE);
#
my code result is;
1
33 36 39 42
2
123 126 129 132
#
BAsically, for each time interval, it just sums up the columns for all the lines and displays all the columns against each time interval.
$/ is your friend here. Try setting it to '' to enable paragraph mode (separating your data by blank lines).
#!/usr/bin/env perl
use strict;
use warnings;
local $/ = '';
while ( <> ) {
my ( $start ) = m/start_time\s+(\d+)/;
my ( $length ) = m/length\s+(\d+)/;
my #row_sum;
for ( m/(0x.*)/g ) {
my ( $key, #values ) = split /,/;
for my $index ( 0..$#values ) {
$row_sum[$index] += $values[$index];
}
}
print join ( "\t", $start, #row_sum ), "\n";
}
Output:
1 33 36 39 42
2 123 126 129 132
3 213 216 219 222
NB - using tab stops for output. Can use sprintf if you need more flexible options.
I would also suggest that instead of:
my $input = $ARGV[0];
open (my $input_fh, '<', $input) or die "Could not open $input: $!";
You would be better off with:
while ( <> ) {
Because <> is the magic filehandle in perl, that - opens files specified on command line, and reads them one at a time, and if there isn't one, reads STDIN. This is just like how grep/sed/awk do it.
So you can still run this with scriptname.pl sample.txt or you can do curl http://somewebserver/sample.txt | scriptname.pl or scriptname.pl sample.txt anothersample.txt moresample.txt
Also - if you want to open the file yourself, you're better off using lexical vars and 3 arg open:
open ( my $input_fh, '<', $ARGV[0] ) or die $!;
And you really shouldn't ever be using 'numbered' variables like $col_1 etc. If there's numbers, then an array is almost always better.
Basically, a block begins with start_time and ends with a line of of whitespace. If instead end of block is always assured to be an empty line, you can change the test below.
It helps to use arrays instead of variables with integer suffixes.
When you hit the start of a new block, record the start_time value. When you hit a stat line, update column sums, and when you hit a line of whitespace, print the column sums, and clear them.
This way, you keep your program's memory footprint proportional to the longest line of input as apposed to the largest block of input. In this case, there isn't a huge difference, but, in real life, there can be. Your original program was reading the entire file into memory as a list of lines which would really cause your program's memory footprint to balloon when used with large input sizes.
#!/usr/bin/env perl
use strict;
use warnings;
my $start_time;
my #cols;
while (my $line = <DATA>) {
if ( $line =~ /^start_time \s+ ([0-9]+)/x) {
$start_time = $1;
}
elsif ( $line =~ /^0x/ ) {
my ($id, #vals) = split /,/, $line;
for my $i (0 .. $#vals) {
$cols[ $i ] += $vals[ $i ];
}
}
elsif ( !($line =~ /\S/) ) {
# guard against the possibility of
# multiple blank/whitespace lines between records
if ( #cols ) {
print join("\t", $start_time, #cols), "\n";
#cols = ();
}
}
}
# in case there is no blank/whitespace line after last record
if ( #cols ) {
print join("\t", $start_time, #cols), "\n";
}
__DATA__
volume stats
start_time 1
length 2
--------
ID
0x00a,1,2,3,4
0x00b,11,12,13,14
0x00c,21,22,23,24
volume stats
start_time 2
length 2
--------
ID
0x00a,31,32,33,34
0x00b,41,42,43,44
0x00c,51,52,53,54
volume stats
start_time 3
length 2
--------
ID
0x00a,61,62,63,64
0x00b,71,72,73,74
0x00c,81,82,83,84
Output:
1 33 36 39 42
2 123 126 129 132
3 213 216 219 222
When I run your code, I get warnings:
Use of uninitialized value $date_time in concatenation (.) or string
I fixed it by using \s+ instead of / /.
I also added a print after your loop in case the file does not end with a blank line.
Here is minimally-changed code to produce your desired output:
use strict;
use warnings;
# Define file names and its location
my $input = $ARGV[0];
# Grab the vols stats for different intervals
open (INFILE,"$input") or die "Could not open sample.txt: $!";
my $date_time;
my $length;
my $col_1;
my $col_2;
my $col_3;
my $col_4;
foreach my $line (<INFILE>)
{
if ($line =~ m/start/)
{
my #date_fields = split(/\s+/,$line);
$date_time = $date_fields[1];
}
if ($line =~ m/length/i)
{
my #length_fields = split(/\s+/,$line);
$length = $length_fields[1];
}
if ($line =~ m/0[xX][0-9a-fA-F]+/)
{
my #volume_fields = split(/,/,$line);
$col_1 += $volume_fields[1];
$col_2 += $volume_fields[2];
$col_3 += $volume_fields[3];
$col_4 += $volume_fields[4];
}
if ($line =~ /^$/)
{
print "$date_time $col_1 $col_2 $col_3 $col_4\n";
$col_1=0;$col_2=0;$col_3=0;$col_4=0;
}
}
print "$date_time $col_1 $col_2 $col_3 $col_4\n";
close (INFILE);
__END__
1 33 36 39 42
2 123 126 129 132
3 213 216 219 222
Suppressing system output from Perl code.
This code works fine functionally until I run into a hostname that can't resolved and want to suppress the output of a unresolvable domain.
If in the lists.hosts file there is a domain that can't be resolved, the screen output will contain: "ping: cannot resolve XXX.com: Unknown host"
my $ip;
open(HOSTLIST, "lists.hosts"); # Load domains
#hosts = <HOSTLIST>;
chomp($host);
foreach $host (#hosts) {
$results = `ping -c 1 $host`;
$record++;
my $pos = index($results, $find);
if (($results =~ /ttl=/) || ($results =~ /data bytes/)) {
#$count++;
chomp($host);
if (($results =~ /(?<=bytes from)(.*)(?=:)/) != 0) {
($ip) = ($results =~ /(?<=bytes from)(.*)(?=:)/);
}
elsif (($results =~ /(?<=\()(.*)(?=\))/) != 0) {
($ip) = ($results =~ /(?<=\()(.*)(?=\))/);
}
print "Record: $record Host: $host IP:$ip Status: Passed";
print "\n";
#print ("*** Record# $record: Ping Test Succeeded for Server: $host ***\n");
#print ("$results\n");
}
else {
$count++;
chomp($host);
#print ("*** Record# $record: Ping Test Failed for Server: $host ***\n");
print "Record: $record Host: $host Status: Failed\n";
#print ("$results\n");
}
}
close(HOSTLIST);
exit($errorcode);
Your invocation of ping needs to capture stderr:
ping -c 1 $host 2>&1
Also, you're not checking the return of your open, which you should do always. Finally, you should be using use warnings; and use strict; at the top.
I wrote a Perl script in order to automate a process for sending a lot of jobs in an lsf based cluster.
The problem is that for a reason that I don't get it cuts the line with the job in two lines, and the job cannot run.
here is my script:
my $controler = $ARGV[0];
open my $cont, $controler or die "Could not open $controler: $!";
my $str = qx(wc -l $controler | awk '{print $1}');
#my $str2 = system($str);
#my #count = split /\n/,$str;
#print $str;
for (my $f = 1; $f <= $str; $f++) {
#print $count[0];
`mkdir ~/Multiple_alignments/ELEMENTS-AUTO-$f`;
#`mkdir ../SCORES-AUTO-$f`;
}
while ( $cont = <>){
chomp;
my #lines = split /\t/, $cont;
my $count2 = $lines[0];
my $cover = $lines[1];
my $length = $lines[2];
my $rho = $lines[3];
my #files = <*maf>;
foreach my $file (#files) {
#print $file . "\n";
my $base = basename($file, ".maf");
#print "$base\n";
#print "$cover\n";
#print "$length\n";
#print "$rho\n";
print "`bsub -q q_cf_htc_work -R \"select[type==X86_64 && mem>6000]\" rusage[mem=6000] -M6000 -o /home/vasilis.lenis/Multiple_alignments/out-files/phastCons_$base.o -e /home/vasilis.lenis/Multiple_alignments/out-files/phastCons_$base.e -J phastCons$base phastCons --target-coverage $cover --expected-length $length --rho $rho --most-conserved ../ELEMENTS-AUTO-$count2/most_conserved_$base.bed --msa-format MAF $file mytrees_no3.noncons.mod --no-post-probs`\n";
}
}
I just cannot understand why its happening.
(Also, the awk command that I have at the third line doesn't work)
Thank you in advance,
Vasilis.
Thank you very much for your guidance.
I believe that I solved the problem with the new line.
I was using wrong the chomp function.
chomp($cont);
I am trying to filter a file (trying to find the best blast hits) by using a second file.
The file that I want to filter looks like this:
conserved1 chr22 100.00 92 0 0 1 92 19679676 19679767 2e-44 182
.....................
The second file (which is the first input in my script) that I’m using is like this:
conserved1 92
conserved2 76
.....................
(the first column is the name of my ‘item’ which is exactly with the first column of the previous file, and the second column is the size).
I stored the second file in a hash in order to connect the first file with the sizes of the conserved elements and to filter only the lines which the size (4th column) be the 70% of the size (from the 2nd file).
I wrote this script for that purpose, it works but it prints each chosen line more than once.
How can I fix this?
my $size_file = $ARGV[0];
my $alignment_file = $ARGV[1];
open my $con_info, $size_file or die "Could not open $size_file: $!";
my %hash;
while (<$con_info>)
{
chomp;
my ($key, $val) = split /\t/;
$hash{$key} .= exists $hash{$key} ? "$val" : $val;
}
#print "# %hash\n", Dump \%hash;
#print %hash;
#print "#{[%hash]}";
close $con_info;
open my $al_info, $alignment_file or die "Could not open $alignment_file: $!";
while (my $line = <$al_info>) {
chomp;
my#data = split('\t', $line);
my $con_name = $data[0];
my $evalue = $data[10];
my $percent = $data[2];
my $length = $data[3];
# print $con_name. "\n";
foreach my $key (keys %hash) {
if ($key == $con_name) {
#print "key: $key, value: $hash{$key}\n";
if ($evalue <= 1e-4 && $length >= 0.70 * $hash{$key}) {
print $line;
}
}
}
}
The output should be the first file (the file which is at the first code box) but with less lines, the lines that passes through the last if condition.
Thank you very- very much for your help!!!
if ($key == $con_name)
should be
if ($key eq $con_name)
as this should be string comparison.
And you don't really need foreach loop, just to pick one particular key:
while (my $line = <$al_info>) {
chomp($line);
my #data = split('\t', $line);
# my $con_name = $data[0];
# my $percent = $data[2];
# my $length = $data[3];
# my $evalue = $data[10];
my ($con_name, $percent, $length, $evalue) = #data[0,2,3,10];
# print $con_name. "\n";
if ($evalue <= 1e-4 && $length >= 0.70 * $hash{$con_name}) {
print $line;
}
}
Error:
Syntax error: end of file unexpected
Below is the Code
I changed.
"#!/usr/local/bin/perl"
The actual program is
#!/local/perl5/bin/perl5.003
use Socket;
$sockaddr = 'S n a4 x8';
$host = $ARGV[0];
$them = $host;
$port = 79;
print "Finger $host: \n";
$hostname = ``;
`nslookup $host |grep Name: >> $test`;
print $test;
#($name, $aliases, $proto) = getprotobyname('tcp');
($name, $aliases, $port) = getservbyname($port, 'tcp') unless $port =~ /^\d+$/;
($name, $aliases, $type, $len, $thisaddr) = gethostbyname($hostname);
$n1 = $name;
($name, $aliases, $type, $len, $thataddr) = gethostbyname($them);
$this = pack($sockaddr, &AF_INET, 0, $thisaddr);
$that = pack($sockaddr, &AF_INET, $port, $thataddr);
socket(S, &PF_INET, &SOCK_STREAM, $proto) || die "socket: $!";
bind(S, $this) || die "bind: $!";
connect(S, $that);
select(S); $| = 1; select(stdout);
print S "\n\n";
while (<S>) {print $_;};
Here:
`nslookup $host |grep Name: >> $test`;
$test is undefined at that point, so you're asking the shell to execute nslookup whatever.com |grep Name: >>. Where is the shell supposed to redirect the output to?
If you set $test to be something, like a filename.. or even $test = "$host.txt"; it will get you further.
Nothing to do with your Perl version, although being able to use strict;use warnings does help, as it would've caught the above error.