Comparing two files, where one piece of information can be flexible - perl

Question

Comparing two files. So easy, but comparing two files where one piece of information can be flexible is proving to be very challenging for me.
fileA
4 "dup" 37036335 37044984
3 "dup" 100146708 100147504
7 "del" 100 203
2 "dup" 34 89
fileB
4 "dup" 37036335 37036735
3 "dup" 100146708 100147504
4 "dup" 68 109
Anticipated output:
output_file1 (matching hits)
fileA: 4 "dup" 37036335 37044984
fileB: 4 "dup" 37036335 37036735
fileA: 3 "dup" 100146708 100147504
fileB: 3 "dup" 100146708 100147504
output_file2 (found in fileA, but not in FileB including non-overlap)
7 "del" 100 203
2 "dup" 34 89
output_file3 (found in fileB, but not in FileA including non-overlap)
4 "dup" 68 109
The credentials are...
I need field 1 and field 2 in the first file to exactly match the second file and the coordinates in field 3 to be exact or overlap.
This would mean these are the same.
fileA :4 "dup" 37036335 37044984
fileB :4 "dup" 37036335 37036735
I also need to find differences between the two files. (no-overlap, 1 row isn't present in one file, but not in the other, etc)
Here's the gist of what I've tried. I've written this code probably 4 different ways, alas, still no success. I've put both files into arrays (I've tried a hash too...idk)
## if no hits in original, but hits in calculated
if((! #ori) && (#calc)){}
## if CNV calls in original, but none in calculated
if((#ori) && (! #calc)){}
## if CNV calls in both
if((#ori) && (#calc)){
## compare calls with double 'for' loop
foreach my $l (#ori){
my #l = split(/\s/,$l);
my $Ochromosome = $l[0];
my $Ostart = $l[2];
my $Oend = $l[3];
my $Otype = $l[1];
foreach my $l (#calc){
my #l = split(/\s/,$l);
my $Cchromosome = $l[0];
my $Cstart = $l[2];
my $Cend = $l[3];
my $Ctype = $l[1];
## check chromosome and type here
if(($Ochromosome eq $Cchromosome) && ($Otype eq $Ctype)){ ## what if there are two duplications on the same chromosome?
## check coordinates
if(($Ostart <= $Cend) && ($Cstart <= $Oend)){
## overlap
}else{
## noOverlap
}
}else{
## what if there is something found in one, but not in the other and they both have calls?
## ahhhh
}
}
}

Answer 1

Here is a simple solution which is also fairly efficient.
Iterate over lines of one file, checking each against all lines of the other (until a match is found). This is the very least we must do complexity wise, given all information that needs to be gathered.
If a line from A is not found in B, it is added to #not_in_B. To determine which lines in B are not in A, we prepare a hash where each element of B is a key with a value 0. Once/if an element of B is found, the value of its key in the hash is set to 1. Those that are not 1 at the end have never been found by elements of A, and so are the extra ones. They go in #not_in_A.
Both files are first read into arrays for simplicity (but this is needed for the inner loop).
use warnings;
use strict;
use feature 'say';
my $f1 = 'f1.txt';
my $f2 = 'f2.txt';
open my $fh, '<', $f1;
my #a1 = <$fh>; chomp(#a1);
open $fh, '<', $f2;
my #a2 = <$fh>; chomp(#a2);
close $fh;
my (#not_in_A, #not_in_B);
my %Bs_in_A = map { $_ => 0 } #a2;
foreach my $e1 (#a1)
{
my $match = 0;
foreach my $e2 (#a2)
{
if ( lines_match($e1, $e2) ) {
$match = 1;
say "Match:\n\tf1: $e1\n\tf2: $e2";
$Bs_in_A{$e2} = 1;
last;
}
}
push #not_in_B, $e1 if not $match;
}
#not_in_A = grep { $Bs_in_A{$_} == 0 } keys %Bs_in_A;
say '---';
say "Elements of A that are not in B:";
say "\t$_" for #not_in_B;
say "Elements of B that are not in A:";
say "\t$_" for #not_in_A;
sub lines_match
{
my ($l1, $l2) = #_;
my #t1 = split ' ', $l1;
my #t2 = split ' ', $l2;
# First two fields must be the same
return if $t1[0] ne $t2[0] or $t1[1] ne $t2[1];
# Third-to-fourth-field ranges must overlap
return
if ($t1[2] < $t2[2] and $t1[3] < $t2[2])
or ($t1[2] > $t2[3] and $t1[3] > $t2[3]);
return 1; # match
}
Output
Match:
f1: 4 "dup" 37036335 37044984
f2: 4 "dup" 37036335 37036735
Match:
f1: 3 "dup" 100146708 100147504
f2: 3 "dup" 100146708 100147504
---
Elements of A that are not in B:
7 "del" 100 203
2 "dup" 34 89
Elements of B that are not in A:
4 "dup" 68 109
Note that I've used 1 in place of A and 2 in place of B.