Perl convert from script to module design ideas - perl

Question

I have a script that I would like to convert to a module and call its functions from a Perl script as I do with CPAN modules now. My question is how would others design the module, I'd like to get some idea how to proceed as I have never written a module before. The script, as written now does this:
1) Sets up logging to a db using am in house module
2) Establishes a connection to the db using DBI
3) Fetch a file(s) from a remote server using Net::SFTP:Foreign
4) Process the users in each file and add the data to the DB
The script currently takes command line options to override defaults using Getopt::Long.
Each file is a pipe delimited collection of user data, ~4000 lines, which goes in to the db, provided the user has an entry in our LDAP directory.
So more to the point:
How should I design my module? Should everything my script currently does be moved into the module, or are there some things that are best left in the script. For example, I was thinking of designing my module, so it would be called like this:
use MyModule;
$obj = MyModule->new; // possibly pass some arguments
$obj->setupLogging;
$obj->connectToDB;
$obj->fetchFiles;
$obj->processUsers;
That would keep the script nice and clean, but is this the best idea for the module? I was thinking of having the script retrieve the files and then just pass the paths to the module for processing.
Thanks

Answer 1

I think the most useful question is "What of this code is going to be usable by multiple scripts?" Surely the whole thing won't. I rarely create a perl module until I find myself writing the same subroutine more than once.
At that stage I generally dump it into something called "Utility.pm" until that gathers enough code that patterns (not in the gang of four sense really) start to suggest what belongs in a module with well-defined responsibilities.
The other thing you'll want to think about is whether your module is going to represent an object class or if it's going to be a collection of library subroutines.
In your example I could see Logging and Database connection management (though I'd likely use DBI) belonging in external modules.
But putting everything in there just leaves you with a five line script that calls "MyModule::DoStuff" and that's less than useful.
Most of the time ;)

Answer 2

"is this the best idea for the module? I was thinking of having the
script retrieve the files and then just pass the paths to the module
for processing."
It looks like a decent design to me. I also agree that you should not hardcode paths (or URLS) in the module, but I'm a little confused by what "having the script retrieve the files and then just pass the paths to the module for processing" means: do you mean, the script would write them to disk? Why do that? Instead, if it makes sense, have fetchFile(URL) retrieve a single file and return a reference/handle/object suitable for submission to processUsers, so the logic is like:
my #FileUrls = ( ... );
foreach (#FileUrls) {
my $file = $obj->fetchFile($_);
$obj->processUsers($file);
}
Although, if the purpose of fetchFile is just to get some raw text, you might want to make that a function independent of the database class -- either something that is part of the script, or another module.
If your reason for wanting to write to disk is that you don't want an entire file loaded into memory at once, you might want to adjust everything to process chunks of a file, such that you could have one kind of object for fetching files via a socket and outputting chunks as they are read to another kind of object (the process that adds to database one). In that case:
my $db = DBmodule->new(...);
my $netfiles = NetworkFileReader->new(...)
foreach (#FileUrls) {
$netfiles->connect($_);
$db->newfile(); # initialize/reset
while (my $chunk = $netfiles->more()) {
$db->process($chunk);
}
$db->complete();
$netfiles->close();
}
Or incorporate the chunk reading into the db class if you think it is more appropriate and unlikely to serve a generic purpose.