Parsing CSV files and matching - perl

Question

I have 10 folders and in each folder, i have two files (CSV,comma delimeted) in the following formats.
File 1:
Ensembl Gene ID,Ensembl Transcript ID,Exon Chr Start (bp),Exon Chr End (bp),Exon Rank in Transcript, Transcript count,Gene End (bp) ,Gene Start (bp),Strand
ENSG00000271782,ENST00000607815,50902700,50902978,1,1,50902978,50902700,-1
ENSG00000232753,ENST00000424955,103817769,103817825,1,1,103828355,103817769,1
ENSG00000232753,ENST00000424955,103827995,103828355,2,1,103828355,103817769,1
ENSG00000225767,ENST00000424664,50927141 50927168,1,1,50936822,50927141,1
File 2:
number,Start pos,End Pos
1,41035,41048
3,36738,36751
3,38169,38182
3,40264,40277
I am trying to match the second file to firstfile
The number in colum1 of second file is the key record number in first file.
Extract the last 3 colums from first file
where the output needed is :
1,ENSG00000271782,41035,41048,50902978,50902700,-1
3,ENSG00000225767,36738,36751,50936822,50927141,1
3,ENSG00000225767,38169,38182,50936822,50927141,1
3,ENSG00000225767,40264,40277,50936822,50927141,1
I have started reading from second using TexT::CSV, but need help.
use strict;
use warnings;
use lib 'C:/Perl/lib';
use Text::CSV;
my $file1 = "infile1";
open my $fh, "<", $file1 or die "$file1: $!";
my $file2 = "infile2"
open my $fh2, "<", $file2 or die "$file2: $!";
my $csv = Text::CSV->new ({
binary => 1,
auto_diag => 1,
});
while (my $row = $csv->getline ($fh2)) {
print "#$row\n"; # I am stuck in extraction ? do I need to put another while loop for fh1
}
close $fh1;
close $fh2;

Answer 1

Since there are no commas enclosed within double-quotes, you can just split on the commas instead of using Text::CSV (which is an excellent module). Given this, the following produces the output you want:
use strict;
use warnings;
use autodie;
my ( $num, %hash ) = 0;
my ( $file1, $file2 ) = qw/inFile1 inFile2/;
open my $fh1, '<', $file1;
while (<$fh1>) {
next if $. == 1;
chomp;
my #fields = split /,/;
$num++ if !$hash{ $fields[0] }++;
push #{ $hash{$num} }, [ #fields[ 0, 6 .. 8 ] ];
}
close $fh1;
open my $fh2, '<', $file2;
while (<$fh2>) {
next if $. == 1;
chomp;
my #fields = split /,/;
if ( my #arr = #{ $hash{ $fields[0] }->[0] } ) {
splice #arr, 1, 0, #fields[ 1, 2 ];
print join( ',', $fields[0], #arr ), "\n";
}
}
close $fh2;
This uses a hash to: 1) keep track of seen Gene IDs, and 2) build a hash of array of arrays (HoAoA). The count--your "key record"--is incremented on unique Gene IDs, so #1 keeps track of these IDs to insure that $num is incremented only if the Gene ID hasn't yet appeared. Number 2 (HoAoA) is used because there are multiple instances of the same Gene ID, but only the values from the first instance are used in the printing. (I did note, however, that the second skips #2, which is the multiple-instance Gene ID.) Perhaps you only needed a hash of arrays (HoA), but it works well the way it is--or you can just modify it, as needed. That is, if you aren't going to use the multiple Gene ID info, the code could be simplified.
Output on your datasets:
1,ENSG00000271782,41035,41048,50902978,50902700,-1
3,ENSG00000225767,36738,36751,50936822,50927141,1
3,ENSG00000225767,38169,38182,50936822,50927141,1
3,ENSG00000225767,40264,40277,50936822,50927141,1
Hope this helps!

Answer 2

The interesting part of this problem is that you need logic that read file 1 until it's ahead of file 2 and logic that reads file 2 until it's ahead of file 1 and logic to know how to act when one is behind the other, and when they are in balance.
You'll need to track unique gene ensemble ids, and their ordinal position in the list. So that when you read the second line of file2, you'll know how to skip the second and third line of file1, but also know not to skip and more in file1 when you've read the third and fourth line in file 1.
Or you can read file1 into memory, and create an array of arrays of lines, so that e.g.
file1arr[1] = [ $line1 ]
file1arr[2] = [ $line2, $line3 ]
file1arr[3] = [ $line4 ]
so when you loop over file 2, all the lines from file1 are in a neat little arrayref at the array index corresponding to the number column of file2.
Then it's just an exercise in iterating of the array of file1 lines, splitting them and building your output lines.