Comparing FILE1 value to FILE2 range and printing matches - perl

Question

I'm very new to Perl and am working on a Bioinformatics project at University. I have FILE1 containing a list of positions, in the format:
99269
550
100
126477
1700
And FILE2 in the format:
517 1878 forward
700 2500 forward
2156 3289 forward
99000 100000 forward
22000 23000 backward
I want to compare every position in FILE1 to every range in values on FILE2, and if a position falls into one of the ranges then I want to print the position, range and direction.
So my expected output would be:
99269 99000 100000 forward
550 517 1878 forward
1700 517 1878 forward
Currently it will run with no errors, however it doesn't output any information so I am unsure where I am going wrong! When I split the final 'if' rule it runs but will only work if the position is on exactly the same line as the range.
My code is as follows:
#!/usr/bin/perl
use strict;
use warnings;
my $outputfile = "/Users/edwardtickle/Documents/CC22CDS.txt";
open FILE1, "/Users/edwardtickle/Documents/CC22positions.txt"
or die "cannot open > CC22: $!";
open FILE2, "/Users/edwardtickle/Documents/CDSpositions.txt"
or die "cannot open > CDS: $!";
open( OUTPUTFILE, ">$outputfile" ) or die "Could not open output file: $! \n";
while (<FILE1>) {
if (/^(\d+)/) {
my $CC22 = $1;
while (<FILE2>) {
if (/^(\d+)\s+(\d+)\s+(\S+)/) {
my $CDS1 = $1;
my $CDS2 = $2;
my $CDS3 = $3;
if ( $CC22 > $CDS1 && $CC22 < $CDS2 ) {
print OUTPUTFILE "$CC22 $CDS1 $CDS2 $CDS3\n";
}
}
}
}
}
close(FILE1);
close(FILE2);
I have posted the same question on Perlmonks.

Answer 1

Because you are only reading FILE2 once it is only compared with the first line of FILE1
Subsequent lines are compared with the closed file
Stash the lines from FILE1 in an array and then compare each line in FILE2 with each array entry, as shown below
#!/usr/bin/perl
use strict;
use warnings;
my $outputfile = "out.txt";
open FILE1, "file1.txt"
or die "cannot open > CC22: $!";
open FILE2, "file2.txt"
or die "cannot open > CDS: $!";
open( OUTPUTFILE, ">$outputfile" ) or die "Could not open output file: $! \n";
my #file1list = ();
while (<FILE1>) {
if (/^(\d+)/) {
push #file1list, $1;
}
}
while (<FILE2>) {
if (/^(\d+)\s+(\d+)\s+(\S+)/) {
my $CDS1 = $1;
my $CDS2 = $2;
my $CDS3 = $3;
for my $CC22 (#file1list) {
if ( $CC22 > $CDS1 && $CC22 < $CDS2 ) {
print OUTPUTFILE "$CC22 $CDS1 $CDS2 $CDS3\n";
}
}
}
}
( there are also stylistic issues with the program (like capital letters for variables) but I've ignored these, it's quite a nice program for a beginner)

Answer 2

I thought I could simplify some of that by using split instead of regex, but I think my code is actually longer and more difficult to read! In any event, remember that split works great for problems like this:
# User config area
my $positions_file = 'input_positions.txt';
my $ranges_file = 'input_ranges.txt';
my $output_file = 'output_data.txt';
# Reading data
open my $positions_fh, "<", $positions_file;
open my $ranges_fh, "<", $ranges_file;
chomp( my #positions = <$positions_fh> );
# Store the range data in an array containing hash tables
my #range_data;
# to be used like $range_data[0] = {start => $start, end => $end, dir => $dir}
while (<$ranges_fh>) {
chomp;
my ( $start, $end, $dir ) = split; #splits $_ according to whitespace
push #range_data, { start => $start, end => $end, dir => $dir };
#print "start: $start, end: $end, direction: $dir\n";
} #/while
close $positions_fh;
close $ranges_fh;
# Data processing:
open my $output_fh, ">", $output_file;
#It feels like it should be more efficient to process one range at a time for all data points
foreach my $range (#range_data) { #start one range at a time
#each $range = $range_data[#] = { hash table }
foreach my $position (#positions) { #check all positions
if ( ( $range->{start} <= $position ) and ( $position <= $range->{end} ) ) {
my $output_string = "$position " . $range->{start} . " " . $range->{end} . " " . $range->{dir} . "\n";
print $output_fh $output_string;
} #/if
} #/foreach position
} #/foreach range
close $output_fh;
This code would probably run faster if the data processing was done during the while loop that's reading the range data.

Answer 3

Your bug was because you were embedding file processing, so your inner loop only went through the file's contents a single time and then was stuck at eof.
The easiest solution is just to load the inner loop file entirely into memory first.
The following demonstrates using more Modern Perl techniques:
#!/usr/bin/perl
use strict;
use warnings;
use autodie;
my $cc22file = "/Users/edwardtickle/Documents/CC22positions.txt";
my $cdsfile = "/Users/edwardtickle/Documents/CDSpositions.txt";
my $outfile = "/Users/edwardtickle/Documents/CC22CDS.txt";
my #ranges = do {
# open my $fh, '<', $cdsfile; # Using Fake Data instead below
open my $fh, '<', \ "517 1878 forward\n700 2500 forward\n2156 3289 forward\n99000 100000 forward\n22000 23000 backward\n";
map {[split]} <$fh>;
};
# open my $infh, '<', $cc22file; # Using Fake Data instead below
open my $infh, '<', \ "99269\n550\n100\n126477\n1700\n";
# open my $outfh, '>', $outfile; # Using STDOUT instead below
my $outfh = \*STDOUT;
CC22:
while (my $cc22 = <$infh>) {
chomp $cc22;
for my $cds (#ranges) {
if ($cc22 > $cds->[0] && $cc22 < $cds->[1]) {
print $outfh "$cc22 #$cds\n";
next CC22;
}
}
# warn "$cc22 No match found\n";
}
Outputs:
99269 99000 100000 forward
550 517 1878 forward
1700 517 1878 forward
Live Demo