Perl Parallel::ForkManager with DBI database handlers - perl

Question

im interested in performing multiple database actions in parallel. I have played with Perl Parallel::ForkManager but not used it with any databases yet. I have read that database connectivity is not supported very well with this. Does anyone have experience with this?
As an example i would probably be spawning a system call(which does the DBI work) NOT raw code, i.e.
#!/opt/local/bin/perl -w
use strict;
use Parallel::ForkManager;
$| = 1;
my $max_procs = 10;
my $pm = new Parallel::ForkManager($max_procs);
for (my $var = 0; $var <= 10; $var++) {
my $pid = $pm->start('proc'.$var) and next;
sleep ( 2 );
system( "./DBworker.pl $var" );
$pm->finish(0);
}
print "Waiting for child procs\n";
$pm->wait_all_children;
print "complete!\n";

Answer 1

If the work is being done by other programs, there is no danger to forking. The danger comes when you open a connection to the database and then fork. The child can't reuse the parents connection; however, take a look at DBIx::Connector, it handles the things you need to do after forking for you and running multiple programs is generally not the right answer.
#!/usr/bin/perl
use strict;
use warnings;
use DBIx::Connector;
use Parallel::ForkManager;
my $dsn = "dbi:SQLite:dbname=foo.db";
my $user = "";
my $pass = "";
my $conn = DBIx::Connector->new($dsn, $user, $pass,
{
AutoCommit => 0,
PrintError => 0,
RaiseError => 1,
ChopBlanks => 1,
FetchHashKeyName => 'NAME_lc',
}
);
END { unlink "foo.db" }
#setup table
$conn->run(fixup => sub {
my $dbh = $_;
$dbh->do("create table foo ( id integer, name char(35) )");
my $sth = $dbh->prepare("insert into foo (id, name) values (?, ?)");
while (<DATA>) {
chomp;
$sth->execute(split /,/);
}
});
my $pm = Parallel::ForkManager->new(3);
my $sth = $conn->dbh->prepare("select * from foo where id = ?");
for my $id (1 .. 3) {
next if $pm->start;
$sth->execute($id);
while (my $row = $sth->fetchrow_hashref) {
print "$id saw $row->{id} => $row->{name}\n";
}
$pm->finish;
}
$pm->wait_all_children;
print "done\n";
__DATA__
1,foo
2,bar
3,baz