Generating 2 files based on two columns in a third file - perl

Question

I am trying to prepare two input files based on information in a third file. File 1 is for sample1 and File 2 is for sample2. Both these files have lines with tab delimited columns. The first column contains unique identifier and the second column contains information.
File 1
>ENT01 xxxxxxxxxxxxxx
>ENT02 xyxyxyxyxyxy
>ENT03 ththththththt
..so on. Similarly, File 2 contains
>ENG012 ggggggggggggg
>ENG098 ksksksksksks
>ENG234 wewewewewew
I have a File 3 that contains two columns each corresponding to the identifier from File 1 and File 2
>ENT01 >ENG78
>ENT02 >ENG098
>ENT02 >ENG012
>ENT02 >ENG234
>ENT03 >ENG012
and so on. I want to prepare input files for File 1 and File 2 by following the order in file 3. If an entry is repeated in file 3 (ex ENT02) I want to repeat the information for that entry. The expected output is
For File 1:
>ENT01 xxxxxxxxxx
>ENT02 xyxyxyxyxyxy
>ENT02 xyxyxyxyxyx
>ENT02 xyxyxyxyxyx
>ENT03 ththththththth
And for file 2
>ENG78 some info
>ENG098 ksksksksks
>ENG012 gggggggg
>ENG234 wewewewewew
>ENG012 gggggggg
All the the entries in file 1 and file 2 are unique but not in file 3. Also, there are some entries in file3 in either column that is not present in either file 1 or file 2. The current logic I am using is that finding an intersection of identifiers from column 1 in both files1&2 with respective columns in file 3, storing this as a list and using this list to compare with File1 and File 2 separately to generate output for File 1 & 2. I am working with the following lines
awk 'FNR==NR{a[$1]=$0;next};{print a[$1]}' file1 intersectlist
grep -v -x -f idsnotfoundinfile1 file3
I am not able to get the right output as I think at some point it is getting sorted and only uniq values are printed out. Can someone please help me clear work this out.

Answer 1

You need to read and remember the first 2 files into some data structure, and then for the third file, output to 2 new files:
$ awk -F'\t' -v OFS='\t' '
FNR == 1 {file_num++}
file_num == 1 || file_num == 2 {data[file_num,$1] = $2; next}
function value(str) {
return str ? str : "some info"
}
{
for (i=1; i<=2; i++) {
print $i, value(data[i,$i]) > ARGV[i] ".new"
}
}
' file1 file2 file3
$ cat file1.new
>ENT01 xxxxxxxxxxxxxx
>ENT02 xyxyxyxyxyxy
>ENT02 xyxyxyxyxyxy
>ENT02 xyxyxyxyxyxy
>ENT03 ththththththt
$ cat file2.new
>ENG78 some info
>ENG098 ksksksksksks
>ENG012 ggggggggggggg
>ENG234 wewewewewew
>ENG012 ggggggggggggg

Answer 2

The files 1 and 2 first need be read so that you can find their lines with identifiers from file 3. Since the identifiers in these files are unique you can build a hash for each file, with identifiers as keys.
Then process file 3 line by line, where for each identifier on the line retrieve its value from the hash for the appropriate file and write the corresponding lines to new files 1 and 2.
use warnings;
use strict;
use feature 'say';
use Path::Tiny;
my ($file1, $file2, $file3) = qw(File1.txt File2.txt File3.txt);
my ($fileout1, $fileout2) = map { $_ . 'new' } ($file1, $file2);
my %file1 = map { split } path($file1)->lines;
my %file2 = map { split } path($file2)->lines;
my ($ofh1, $ofh2) = map { path($_)->openw } ($fileout1, $fileout2);
open my $fh, '<', $file3 or die "Can't open $file3: $!";
while (<$fh>) {
my ($f1, $f2) = split;
say $ofh1 "$f1\t", $file1{$f1} // 'some info'; #/ see text
say $ofh2 "$f2\t", $file2{$f2} // 'some info';
}
close $_ for $ofh1, $ofh2, $fh;
This produces the correct output based on fragments of input files that are provided.
I use Path::Tiny here for its conciseness. Its lines method returns all lines, and in map's block each is split by default space. The list of such pairs returned by map is assigned to a hash, whereby each pair of successive strings forms a key-value pair.
Multiple files can be opened in one statement, and Path::Tiny again makes it clean with openw. Its methods throw the exception (die) on errors, so we get error checking as well.
If an identifier in File 3 is not found in File 1/2 I bluntly use 'some info' as stated in the question,† but I expect that there is a more rounded solution for such a case. Then the laconic // should be changed to accommodate extra processing (or call a sub in place of 'some info' string).
It is assumed that files 1 and 2 always have two entries on a line.
Some shortcuts are taken, like reading each file into a hash in one line. Please expand the code as appropriate, with whatever checks may be needed.
† In such a case $file1{$f1} is undef so // (defined-or) operator returns its right-hand-side argument. A "proper" way is to test if (exist $file1{$f1}) but // works as well.