CSV import to MySQL - perl

Question

Hi I keep getting an error when trying to run the following perl script to import a csv file into an existing mysql database table. Every time I run it I get the message "Died at /home/perl/dep_import_2.pl line 10.
Any help would be appreciated
Thanks
#!/usr/bin/perl
use DBI;
use DBD::mysql;
use warnings "all";
if ($#ARGV != 0) {
print "Usage: dep_import_2.pl filename\n";
die;
}
$filename = $ARGV[0];
# MySQL CONFIG VARIABLES
$host = "localhost";
$user = "standard";
$pw = "standard";
$database = "data_track";
$dsn = "DBI:mysql:database=" . $database . ";host=" . $host;
$dbh = DBI->connect($dsn, $user, $pw)
or die "Can't connect to the DB: $DBI::errstr\n";
print "Connected to DB!\n";
open FILE, "/home/dep/new_study_forms_2.csv", $filename or die $!;
$_ = <FILE>;
$_ = <FILE>;
while (<FILE>) {
#f = split(/,/, $_);
$sql = "INSERT INTO dep (date, subject, weight, size, time, hi_pre, hi_post, hi_afternoon, hi_test, actical_on, actical_off, saggital_1, saggital_2, crown_heel1, crown_heel2, crown_rump1, crown_rump2, scan, record_number, tap, sample, dye, left_chip, right_chip) VALUES('$f[0]', '$f[1]', '$f[2]', '$f[3]' '$f[4]', '$f[5]', '$f[6]', '$f[7]', '$f[8]', '$f[9]', '$f[10]', '$f[11]', '$f[12]', '$f[13]', '$f[14]', '$f[15]', '$f[16]', '$f[17]', '$f[18]', '$f[19]', '$f[20]', '$f[21]', '$f[22]', '$f[23]')";
print "$sql\n";
my $query = $dbh->do($sql);
}

Answer 1

There are a few issues with your code. First, and most importantly, you are not using
use strict;
use warnings;
This is bad because you will not get information about errors in your code without them.
As others have pointed out, the reason the script dies is because $#ARGV is not zero. Meaning that you have either passed too few or too many arguments to the script. The arguments to the script must be exactly one, like the usage statement says.
However, that would not solve your problem, because your open statement below is screwed up. My guess is that you tried to add your file name directly. This line:
open FILE, "/home/dep/new_study_forms_2.csv", $filename or die $!;
It will probably give you the error unknown open() mode .... It should probably be
open FILE, "<", $filename or die $!;
And then you pass /home/dep/new_study_forms_2.csv to the script on the command line, assuming that is the correct file to use.
Also, in your query string, you should not interpolate variables, you should use placeholders, as is described in the documentation for DBI. The placeholders will take care of the quoting for you and avoid any data corruption. To make your query line a bit simpler, you can do something like:
my $sth = $dbh->prepare(
"INSERT INTO dep (date, subject, weight, size, time, hi_pre, hi_post,
hi_afternoon, hi_test, actical_on, actical_off, saggital_1, saggital_2,
crown_heel1, crown_heel2, crown_rump1, crown_rump2, scan, record_number,
tap, sample, dye, left_chip, right_chip)
VALUES(" . join(",", ("?") x #f) . ")");
$sth->execute(#f);

Answer 2

Here's a script which uses Text::CSV to properly parse CSV. It assumes that the first row contains column names, and then loads the CSV in batches, commiting after every 100 inserts. Every parameter (user, password, database) is configurable via command-line options. Usage is an in-line POD document.
#!/usr/bin/env perl
use strict;
use warnings qw(all);
use DBI;
use Getopt::Long;
use Pod::Usage;
use Text::CSV_XS;
=pod
=head1 SYNOPSIS
dep_import_2.pl --filename=file.csv --host=localhost --user=standard --pw=standard --database=data_track
=head1 DESCRIPTION
Loads a CSV file into the specified MySQL database.
=cut
my $host = 'localhost';
my $user = 'standard';
my $pw = 'standard';
my $database = 'data_track';
my $commit = 100;
GetOptions(
'help' => \my $help,
'filename=s' => \my $filename,
'host=s' => \$host,
'user=s' => \$user,
'pw=s' => \$pw,
'database=s' => \$database,
'commit=i' => \$commit,
) or pod2usage(q(-verbose) => 1);
pod2usage(q(-verbose) => 2) if $help;
my $dbh = DBI->connect("DBI:mysql:database=$database;host=$host", $user => $pw)
or die "Can't connect to the DB: $DBI::errstr";
my $csv = Text::CSV_XS->new
or die "Text::CSV error: " . Text::CSV->error_diag;
open(my $fh, '<:utf8', $filename)
or die "Can't open $filename: $!";
my #cols = #{$csv->getline($fh)};
$csv->column_names(\#cols);
my $query = "INSERT INTO dep (#{[ join ',', #cols ]}) VALUES (#{[ join ',', ('?') x (scalar #cols) ]})";
my $sth = $dbh->prepare($query);
my $i = 0;
while (my $row = $csv->getline_hr($fh)) {
$sth->execute(#{$row}{#cols});
$dbh->commit if ((++$i % $commit) == 0);
}
$dbh->commit;
$dbh->disconnect;
$csv->eof or $csv->error_diag;
close $fh;