How to sort CSV file by header? - perl

Question

There are two CSV files I want to compare. However, they have different order of headers and rows/values.
Here's a simple example:
INPUT FILE1:
NAME,AGE,BDAY
ABC,1,090214
DEF,1,122514
INPUT FILE2:
BDAY,NAME,AGE
122514,DEF,1
090214,ABC,1
INPUT FILE3:
BDAY,NAME,AGE
122514,DEFG,1
090214,ABC,1
Diff FILE1 and FILE2
No diffs.
Diff FILE1 and FILE3
Found diffs in FILE and FILE3.
<Any format of diffs is okay.>
I can easily create a perl script for this but before I do, does anyone know if there's an existing script/tool that already does this?
I have tried copying the files from UNIX to Windows, and sorting them using Excel. It works well but I encounter problems saving it.
I also have googled but can't find a reference for this.
Thanks for any inputs.

Answer 1

I think you need some kind of advanced comparation (with requires deeper analysis), so the use of a relational db approach maybe interesting.
In this respect, the module DBD::CSV is helpful. It allows writting SELECT statements, including join between tables.

Answer 2

Normalize your data
Use Text::CSV to reorder the columns of your CSV file.
Then you can use Perl’s sort or some other utility to reorder the rows of your files.
This also uses Text::Wrap to display the normalized files in a pleasing format:
use strict;
use warnings;
use autodie;
# Setup fake data
my #files;
{
local $/ = ''; # Paragraph mode
while (<DATA>) {
chomp;
my ( $file, $data ) = split "\n", $_, 2;
open my $fh, '>', $file;
print $fh $data, "\n";
push #files, $file;
}
}
# Normalize Files by Column Order
use Text::CSV;
my $csv = Text::CSV->new( { binary => 1, eol => $/ } )
or die "Cannot use CSV: " . Text::CSV->error_diag();
for my $file (#files) {
local #ARGV = $file;
local $^I = '.bak';
my #old_order;
my #new_order;
while (<>) {
if ( !$csv->parse($_) ) {
die "Bad parse $file, line $.: " . $csv->error_diag();
}
my #columns = $csv->fields();
if ( $. == 1 ) {
#old_order = #columns;
#new_order = sort #columns;
}
my %hash;
#hash{#old_order} = #columns;
if ( !$csv->combine( #hash{#new_order} ) ) {
die "Bad combine $file, line $.: " . $csv->error_diag();
}
print $csv->string();
}
unlink "$file$^I"; # Optionally delete backup
}
# Normalize Files by Row Order
for my $file (#files) {
my ( $header, #data ) = do { local #ARGV = $file; <> };
open my $fh, '>', $file;
print $fh $header, sort #data;
}
# View Normalized Files
use Text::Wrap;
for my $file (#files) {
open my $fh, '<', $file;
print wrap( sprintf( "%-12s", $file ), ' ' x 12, <$fh>, ), "\n";
}
__DATA__
file1.csv
NAME,AGE,BDAY
ABC,1,090214
DEF,1,122514
file2.csv
BDAY,NAME,AGE
122514,DEF,1
090214,ABC,1
file3.csv
BDAY,NAME,AGE
122514,DEFG,1
090214,ABC,1
Outputs:
file1.csv AGE,BDAY,NAME
1,090214,ABC
1,122514,DEF
file2.csv AGE,BDAY,NAME
1,090214,ABC
1,122514,DEF
file3.csv AGE,BDAY,NAME
1,090214,ABC
1,122514,DEFG