Say I have a resource (e.g. a filehandle or network socket) which has to be freed:
open my $fh, "<", "filename" or die "Couldn't open filename: $!";
process($fh);
close $fh or die "Couldn't close filename: $!";
Suppose that process might die. Then the code block exits early, and $fh doesn't get closed.
I could explicitly check for errors:
open my $fh, "<", "filename" or die "Couldn't open filename: $!";
eval {process($fh)};
my $saved_error = $#;
close $fh or die "Couldn't close filename: $!";
die $saved_error if $saved_error;
but this kind of code is notoriously difficult to get right, and only gets more complicated when you add more resources.
In C++ I would use RAII to create an object which owns the resource, and whose destructor would free it. That way, I don't have to remember to free the resource, and resource cleanup happens correctly as soon as the RAII object goes out of scope - even if an exception is thrown. Unfortunately in Perl a DESTROY method is unsuitable for this purpose as there are no guarantees for when it will be called.
Is there a Perlish way to ensure resources are automatically freed like this even in the presence of exceptions? Or is explicit error checking the only option?
I think that's what Scope::Guard was designed to help with.
#!/usr/bin/perl
use strict; use warnings;
use Scope::Guard;
my $filename = 'file.test';
open my $fh, '>', $filename
or die "Couldn't open '$filename': $!";
{
my $sg = Scope::Guard->new(
sub {
close $fh or die "Could not close";
warn "file closed properly\n";
}
);
process($fh);
}
sub process { die "cannot process\n" }
However, as #Philip notes in the comments, Scope::Guard utilizes the DESTROY method which creates some uncertainty as to when the scope exit code will be run. Modules such as Hook::Scope and Sub::ScopeFinalizer look fine as well although I have never used them.
I do like Try::Tiny for its clean interface and sheer simplicity and it will help you handle exceptions the correct way:
#!/usr/bin/perl
use strict; use warnings;
use Try::Tiny;
my $filename = 'file.test';
open my $fh, '>', $filename
or die "Couldn't open '$filename': $!";
try {
process($fh);
}
catch {
warn $_;
}
finally {
close $fh
and warn "file closed properly\n";
};
sub process { die "cannot process\n" }
My module Scope::OnExit is intended for exactly that.
The nice thing about lexical filehandles is that they'll get closed (and freed) when they go out of scope. So you can just do something like this:
{
# bare block creates new scope
open my $fh, "<", "filename" or die "Couldn't open filename: $!";
eval { process($fh) };
# handle exceptions here
close $fh or die "Couldn't close filename: $!";
}
# $fh is now out of scope and goes away automagically.
Related
I've looked around here a bit and found similar questions but not exactly. If there is one, I apologize and please point me to it.
I have the following code. I'm trying to create a csv file of simply an ID pulled from a filename and the filename itself. This is the ENTIRE script.
use strict;
use warnings;
use File::Find;
find( \&findAllFiles, '.');
exit;
sub findAllFiles {
my #fp1;
my #fp2;
my $patId;
my $filename;
my $testvar = "hello again";
$filename = $File::Find::name;
if ($filename =~ /\.pdf$/) {
open (my $fh, '>', 'filenames.csv') or die "Failed to open - $!\n";
print $fh "starting...$testvar\n" or die "Failed to print to file - $!\n";
#fp1 = split('/', $filename);
#fp2 = split('_', $fp1[-1]);
$patId = $fp2[-1];
$patId =~ s/\.pdf$//;
print "Adding $patId, file = $filename\n";
print $fh "$patId,$filename\n" or die "File print error: $!";
close $fh or warn "close failed! - $!";
}
return;
}
The line that prints to the screen, prints perfectly.
If I take the file open/close and the first print statement out of the if block, it prints that line into the file, but not the data inside the block.
I've tried every combo I can think of and it doesn't work. I've alternated between '>' and '>>' since it clearly needs the append since it's looping over filenames, but neither works inside the if block.
Even this code above doesn't throw the die errors! It just ignores those lines! I'm figuring there's something obvious I'm missing.
Quoting File::Find::find's documentation:
Additionally, for each directory found, it will chdir() into that directory
It means that when you open inside findAllFiles, you are potentially opening a file filenames.csv inside a subdirectory of your initial directory. You can run something like find . -name filenames.csv from your terminal, and you'll see plenty of filenames.csv. You can change this behavior by passing no_chdir option to find:
find( { wanted => \&findAllFiles, no_chdir => 1}, '.');
(and additionally changing > for >> in your open)
However, personally, I'd avoid repeatedly opening and closing filenames.csv when you could open it just once before calling find. If you don't want to have your filehandle globally defined, you can always pass it as an argument to findAllFiles:
{
open my $fh, '>', 'filenames.csv' or die "Failed to open 'filenames.csv': $!";
find(sub { findAllFiles($fh) }, '.')
}
sub findAllFiles {
my ($fh) = #_;
...
filenames.csv will be created in the directory where the pdf is found, since find() changes directories as it searches. If that's not what you want, use an absolute path to open it (or open it before calling find, which seems like a better idea).
How exactly would I check to see if a file is locked exclusively? I have this function but it is returning 1 no matter what I do:
sub is_file_locked
{
my $theFile;
my $theRC;
($theFile) = #_;
$theRC = open(HANDLE, $theFile);
$theRC = flock(HANDLE, LOCK_EX|LOCK_NB);
close(HANDLE);
return !$theRC;
}
You have opened $theFile in read mode and LOCK_EX isn't meant to be used that way.
Note that the fcntl(2) emulation of flock(3) requires that
FILEHANDLE be open with read intent to use LOCK_SH and requires
that it be open with write intent to use LOCK_EX.
First off, you should check if open succeeded.
Also, you should check if you can get a shared lock. flock with LOCK_EX would (I think) fail, if there is a shared lock on the file.
However, the file can become locked between the check and the return, creating a race condition, so such a function is of dubious value.
#!/usr/bin/perl
use strict; use warnings;
use Fcntl qw( :flock );
print is_locked_ex($0)
? "$0 : locked exclusively\n"
: "$0 : not locked exclusively\n";
my $test_file = 'test.txt';
open my $fh, '>', $test_file
or die "Cannot open '$test_file' for writing: $!";
if ( flock $fh, LOCK_EX|LOCK_NB ) {
print is_locked_ex($test_file)
? "$test_file : locked exclusively\n"
: "$test_file : not locked exclusively\n";
}
close $fh or die "Cannot close '$test_file': $!";
sub is_locked_ex {
my ($path) = #_;
die "Not a plain file: '$path'" unless -f $path;
return 1 unless open my $fh, '<', $path;
my $ret = not flock $fh, LOCK_SH | LOCK_NB;
close $fh
or die "Cannot close '$path': $!";
return $ret;
}
The final solution:
flock($fh, LOCK_EX) or die "Cannot lock file - $!\n";
if ( is_file_locked($gTestQueuePath) ){ print "locked";} else { print "not locked";}
#1 = locked 0 = not locked
sub is_file_locked
{
my $theFile;
my $theRC;
($theFile) = #_;
$theRC = open(my $HANDLE, ">>", $theFile);
$theRC = flock($HANDLE, LOCK_EX|LOCK_NB);
close($HANDLE);
return !$theRC;
}
close $fh or die "Cannot close";
For some reason, I can't get filehandles working with Expect.pm's log_file method. I originally got help on How can I pass a filehandle to Perl Expect's log_file function?, where it was suggested that I use an IO::Handle filehandle to pass to the method. This seems to be a different issue, so I thought I'd start a new question.
This is the offending section of Expect.pm:
if (ref($file) ne 'CODE') {
croak "Given logfile doesn't have a 'print' method"
if not $fh->can("print");
$fh->autoflush(1); # so logfile is up to date
}
So, then, I tried this sample code:
use IO::Handle;
open $fh, ">>", "file.out" or die "Can't open file";
$fh->print("Hello, world");
if ($fh->can("print"))
{
print "Yes\n";
}
else
{
print "No\n";
}
When I run this, I get two (to my mind) conflicting items. A file with a single line that says 'Hello, world', and output of 'No'. To my mind, the $fh->can line should return true. Am I wrong here?
Odd, it looks like you need to create a real IO::File object to get the can method to work. Try
use IO::File;
my $fh = IO::File->new("file.out", ">>")
or die "Couldn't open file: $!";
IO::Handle doesn't overload the open() function, so you're not actually getting an IO::Handle object in $fh. I don't know why the $fh->print("Hello, world") line works (probably because you're calling the print() function, and when you do things like $foo->function it's equivalent to function $foo, so you're essentially printing to the filehandle like you'd normally expect).
If you change your code to something like:
use strict;
use IO::Handle;
open my $fh, ">>", "file.out" or die "Can't open file";
my $iofh = new IO::Handle;
$iofh->fdopen( $fh, "w" );
$iofh->print("Hello, world");
if ($iofh->can("print"))
{
print "Yes\n";
}
else
{
print "No\n";
}
...then your code will do as you expect. At least, it does for me!
I have a problem when I use Apache::DBI in child processes. The problem is that Apache::DBI provides a single handle for all processes which use it, so I get
DBD::mysql::db selectall_arrayref
failed: Commands out of sync; you
can't run this command now at
/usr/local/www/apache22/data/test-fork.cgi
line 20.
Reconnection doesn't help, since Apache::DBI reconnects in all processes, as I understood the following error
The server encountered an internal
error and was unable to complete your
request.
Error message: DBD driver has not
implemented the AutoCommit attribute
at
/usr/local/lib/perl5/site_perl/5.8.9/Apache/DBI.pm
line 283. ,
Here's the origin code:
use Data::Dumper 'Dumper';
use DBI ();
my $dbh = DBI->connect($dsn, $username, $password, {
RaiseError => 1,
PrintError => 0,
});
my $file = "/tmp/test-fork.tmp";
my $pid = fork;
defined $pid or die "fork: $!";
if ($pid) {
my $rows = eval { $dbh->selectall_arrayref('SELECT SLEEP(1)') };
print "Content-Type: text/plain\n\n";
print $rows ? "parent: " . Dumper($rows) : $#;
}
else {
my $rows = eval { $dbh->selectall_arrayref('SELECT SLEEP(1)') };
open FH, '>', $file or die "$file: $!";
print FH $rows ? "child: " . Dumper($rows) : $#;
close FH;
}
The code I used for reconnection:
...
else {
$dbh->disconnect;
$dbh = DBI->connect($dsn, $username, $password, $attrs);
my $rows = eval { $dbh->selectall_arrayref('SELECT SLEEP(1)') };
open FH, '>', $file or die "$file: $!";
print FH $rows ? "child: " . Dumper($rows) : $#;
close FH;
}
Is there a safe way to use Apache::DBI with forking? Is there a way to make it create a new connection perhaps?
I see a few options:
Explicitly close your DB handles when you fork, and reopen them as needed.
e.g.:
my $dbh = DBI->connect(...);
my $pid = fork;
defined $pid or die "fork: $!";
if ($pid) {
# parent...
}
else {
# child...
undef $dbh;
This could be made easier by storing the $dbh in an object, and passing around that object as needed to parts of your system. The object would be responsible for reopening the $dbh as needed, so the rest of the application doesn't have to concern itself with the details. Keep code encapsulated and well-separated from other parts of the system.
Don't use Apache::DBI. I can highly recommend DBIx::Connector, which opens a new connection as needed and doesn't preserve the bad behaviour of either plain DBI or Apache::DBI: see http://search.cpan.org/~dwheeler/DBIx-Connector-0.32/lib/DBIx/Connector.pm#Description for a detailed description of how it differs.
I use DBIx::Connector in my system inside a Moose object, which uses a method delegation to provide the dbh. The application simply does:
my $dbh = $db_dbj->dbh;
my $sth = $dbh->prepare(...);
# more boring DBI code here
...And the dbh is reconnected/regenerated as needed, invisibly.
As an aside, you should be really careful of using bare filehandles in a multiprocess environment. You could be very easily clobbering your data. open (my $fh, $file) or die "Cannot open $file: $!" is much safer.
I'm also a little nervous by seeing you using eval {} blocks without checking the contents of $#. You're just masking errors, rather than dealing with them, so there may be more things going on than you are aware of. Check your result values (or better, use an explicit exception-handling module, such as Try::Tiny. use use strict; use warnings;.
PS. I just noticed that you are explicitly including DBI in your code. Don't do that. If you use Apache::DBI in your startup_modperl.pl (or whatever you call your bootstrap file), you should never have to include DBI itself. I can't say for sure but I wouldn't be confident the right package is getting called (it's been a while since I looked at Apache::DBI's guts; it might take care of this for you though).
Don't fork under mod_perl2. Use Apache2::Subprocess. See also Is it a bad idea to fork under mod_perl2?
I have an app running under Catalyst+FastCGI. And I want it to fork() to do some work in background.
I used this code for plain CGI long ago (and it worked):
defined(my $pid = fork) or die "Can't fork: $!";
if ($pid) {
# produce some response
exit 0;
}
die "Can't start a new session: $!" if setsid == -1;
close STDIN or die $!;
close STDOUT or die $!;
close STDERR or die $!;
# do some work in background
I tried some variations on this under FastCGI but with no success. How should forking be done under FastCGI?
Update: This is what I have now:
defined(my $pid = fork) or die "Can't fork: $!";
if ($pid) {
$c->stash->{message} = 'ok';
$c->detach($c->view('JSON'));
}
die "Can't start a new session: $!" if setsid == -1;
close STDIN or die $!;
close STDOUT or die $!;
close STDERR or die $!;
# do some work, then exit()
I send the request with AJAX call, and have the "502 Bad Gateway" error in the firebug console.
I think this FAQ has the right answer:
https://fastcgi-archives.github.io/FastCGI_FAQ.html#Perlfork
You should do $request->Detach(); before the fork, and $request->Attach(); after the forking piece of code is done, where $request is the current FCGI object. At least, it worked for me.
In case of Catalyst::Engine::FastCGI you may need to patch the Catalyst::Engine::FastCGI to get access to the $request variable, since it is local to the run() method there (in the version that is currently on CPAN).
This part isn't going to work well with FastCGI:
if ($pid) {
# print response
exit 0;
}
You would exit in the parent process, thus it will stop responding to FastCGI requests.
The setsid()s and close()s are to daemonize your background process. This may or may not be necessary in your case.