I have a config file with bunch of data structures (arrays, hashes) and I load them into my perl script using
require '<config>';
I can use the variables from config that I know of but is there a way that I can list all the variables loaded by the require function? Ideally I would want them to load into a hash variable and refer them to avoid variable name conflicts
Not easily, and this is why relying on global named variables is problematic. Instead, have your config file return a single data structure (like a hashref, so you can name parts of it) and load it with do into a lexical variable:
use strict;
use warnings;
my $file = '/path/to/foo.conf';
my $data = do $file;
die "Failed to parse $file: $#" if !defined $data and $#;
die "Failed to read $file: $!" if !defined $data;
Make sure either to pass an absolute path to the file (recommended, to avoid depending on what your current working directory happens to be) or prepend a relative path with ./, otherwise do (and require) will search #INC for the file, which since Perl 5.26 does not contain the current working directory. See Path::This for a way to get an absolute path relative to the current file.
Related
I have a read-only perl file with a huge hash defined in it. Is there anyway for me to read this perl file and dump out the hash contents?
this is basic structure of the hash within the file.
%hash_name = {
-files => [
'<some_path>',
],
-dirs => [
'<some_path>',
'<some_path>',
'<some_path>',
'<some_path>',
'<some_path>',
],
};
Ideally you'd copy the file so that you can edit it, then turn it into a module so to use it nicely.
But if for some reason this isn't feasible here are your options.
If that hash is the only thing in the file, "load" it using do† and assign to a hash
use warnings;
use strict;
my $file = './read_this.pl'; # the file has *only* that one hash
my %hash = do $file;
This form of do executes the file (runs it as a script), returning the last expression that is evaluated. With only the hash in the file that last expression is the hash definition, precisely what you need.
If the hash is undeclared, so a global variable (or declared with our), then declare as our a hash with the same name in your program and again load the file with do
our %hash_name; # same name as in the file
do $file; # file has "%hash" or "our %hash" (not "my %hash")
Here we "pick up" the hash that is evaluated as do runs the file by virtues of our
If the hash is "lexical", declared as my %hash (as it should be!) ... well, this is bad. Then you need to parse the text of the file so to extract lines with the hash. This is in general very hard to do, as it amounts to parsing Perl. (A hash can be built using map, returned from a sub as a reference or a flat list ...) Once that is done you eval the variable which contains the text defining that hash.
However, if you know how the hash is built, as you imply, with no () anywhere inside
use warnings;
use strict;
my $file = './read_this.pl';
my $content = do { # "slurp" the file -- read it into a variable
local $/;
open my $fh, '<', $file or die "Can't open $file: $!";
<$fh>;
};
my ($hash_text) = $content =~ /\%hash_name\s*=\s*(\(.*?\)/s;
my %hash = eval $hash_text;
This simple shot leaves out a lot, assuming squarely that the hash is as shown. Also note that this form of eval carries real and serious security risks.
†
Files are also loaded using require. Apart from it doing a lot more than do, the important thing here is that even if it runs multiple times require still loads that file only once. This matters for modules in the first place, which shouldn't be loaded multiple times, and use indeed uses require.
On the other hand, do does it every time, what makes it suitable for loading files to be used as data, which presumably should be read every time. This is the recommended method. Note that require itself uses do to actually load the file.
Thanks to Schwern for a comment.
I'm a little confused on how to save a file that is an IO::Handle.
Here is what I have
use IO::File;
my $iof = IO::File->new;
# open file
$iof->open($path, "w") || die "$! : $path";
# ensure binary
$iof->binmode;
# output file to disk
print $iof $self->File_Upload;
$iof->close;
File_Upload is the IO::Handle given to me via the CGI module for a file upload, but the output in the file is...
IO::Handle=GLOB(0x20dabec)
Not the binary data of the uploaded PDF.
If I have a file in a file handle how do I save it?
Do I need IO::File if I have an IO::Handle?
Your input is appreciated.
1DMF
Read from the CGI provided file handle using readline:
print $iof readline($self->File_Upload);
The fact that your output contents were 'IO::Handle=GLOB(0x20dabec)' implied that the $self->File_Upload is of type IO::Handle and should be treated as such.
Using readline in a list context pulls all the lines as demonstrated above. Alternatively, you could use the object method ->getlines():
print $iof $self->File_Upload->getlines();
How does one slurp a file?
my $fh = $self->File_Upload();
my $file = do { local $/; <$fh> };
Yes, this works for IO::Handle objects in addition to the usual globs (STDIN), references to globs (from open my $fh, ...) and IO scalars (*STDIN{IO}).
Then to print it,
print($iof $file);
In this particular case, you could simply use
print($iof $self->File_Upload()->getlines());
I am a perl novice, but have read the "Learning Perl" by Schwartz, foy and Phoenix and have a weak understanding of the language. I am still struggling, even after using the book and the web.
My goal is to be able to do the following:
Search a specific folder (current folder) and grab filenames with full path. Save filenames with complete path and current foldername.
Open a template file and insert the filenames with full path at a specific location (e.g. using substitution) as well as current foldername (in another location in the same text file, I have not gotten this far yet).
Save the new modified file to a new file in a specific location (current folder).
I have many files/folders that I want to process and plan to copy the perl program to each of these folders so the perl program can make new .
I have gotten so far ...:
use strict;
use warnings;
use Cwd;
use File::Spec;
use File::Basename;
my $current_dir = getcwd;
open SECONTROL_TEMPLATE, '<secontrol_template.txt' or die "Can't open SECONTROL_TEMPLATE: $!\n";
my #secontrol_template = <SECONTROL_TEMPLATE>;
close SECONTROL_TEMPLATE;
opendir(DIR, $current_dir) or die $!;
my #seq_files = grep {
/gz/
} readdir (DIR);
open FASTQFILENAMES, '> fastqfilenames.txt' or die "Can't open fastqfilenames.txt: $!\n";
my #fastqfiles;
foreach (#seq_files) {
$_ = File::Spec->catfile($current_dir, $_);
push(#fastqfiles,$_);
}
print FASTQFILENAMES #fastqfiles;
open (my ($fastqfilenames), "<", "fastqfilenames.txt") or die "Can't open fastqfilenames.txt: $!\n";
my #secontrol;
foreach (#secontrol_template) {
$_ =~ s/#/$fastqfilenames/eg;
push(#secontrol,$_);
}
open SECONTROL, '> secontrol.txt' or die "Can't open SECONTROL: $!\n";
print SECONTROL #secontrol;
close SECONTROL;
close FASTQFILENAMES;
My problem is that I cannot figure out how to use my list of files to replace the "#" in my template text file:
my #secontrol;
foreach (#secontrol_template) {
$_ =~ s/#/$fastqfilenames/eg;
push(#secontrol,$_);
}
The substitute function will not replace the "#" with the list of files listed in $fastqfilenames. I get the "#" replaced with GLOB(0x8ab1dc).
Am I doing this the wrong way? Should I not use substitute as this can not be done, and then rather insert the list of files ($fastqfilenames) in the template.txt file? Instead of the $fastqfilenames, can I substitute with content of file (e.g. s/A/{r file.txt ...). Any suggestions?
Cheers,
JamesT
EDIT:
This made it all better.
foreach (#secontrol_template) {
s/#/$fastqfilenames/g;
push #secontrol, $_;
}
And as both suggestions, the $fastqfiles is a filehandle.
replaced this: open (my ($fastqfilenames), "<", "fastqfilenames.txt") or die "Can't open fastqfilenames.txt: $!\n";
with this:
my $fastqfilenames = join "\n", #fastqfiles;
made it all good. Thanks both of you.
$fastqfilenames is a filehandle. You have to read the information out of the filehandle before you can use it.
However, you have other problems.
You are printing all of the filenames to a file, then reading them back out of the file. This is not only a questionable design (why read from the file again, since you already have what you need in an array?), it also won't even work:
Perl buffers file I/O for performance reasons. The lines you have written to the file may not actually be there yet, because Perl is waiting until it has a large chunk of data saved up, to write it all at once.
You can override this buffering behavior in a few different ways (closing the file handle being the simplest if you are done writing to it), but as I said, there is no reason to reopen the file again and read from it anyway.
Also note, the /e option in a regex replacement evaluates the replacement as Perl code. This is not necessary in your case, so you should remove it.
Solution: Instead of reopening the file and reading it, just use the #fastqfiles variable you previously created when replacing in the template. It is not clear exactly what you mean by replacing # with the filenames.
Do you want to to replace each # with a list of all filenames together? If so, you should probably need to join the filenames together in some way before doing the replacement.
Do you want to create a separate version of the template file for each filename? If so, you need an inner for loop that goes over each filename for each template. And you will need something other than a simple replacement, because the replacement will change the original string on the first time through. If you are on Perl 5.16, you could use the /r option to replace non-destructively: push(#secontrol,s/#/$file_name/gr); Otherwise, you should copy to another variable before doing the replacement.
$_ =~ s/#/$fastqfilenames/eg;
$fastqfilenames is a file handle, not the file contents.
In any case, I recommend the use of Text::Template module in order to do this kind of work (file text substitution).
I have just begun working with Perl, I am only at the introductory level, and I have been having trouble with opening filehandles.
Here is the code:
#!/usr/bin/perl -w
$proteinfilename = 'peptide';
open(PROTEINFILE, $proteinfilename) or die "Can't write to file '$proteinfilename' [$!]\n";
$protein = <PROTEINFILE>;
close PROTEINFILE;
print $protein;
exit;
Every time I tried to run the program, it gave me an error
readline() on closed filehandle PROTEINFILE at C:\BIN\protein.pl
or
Can't write to file 'peptide' [No such file or directory]
Can you please help me figure this out. I have the file peptide saved as a .txt and its in the same folder as the protein.pl. What else can I do to make this work?
You're telling perl to open file peptide in the current directory, but it doesn't find such a file there ("No such file or directory").
Perhaps the current directory isn't C:\BIN, the directory in which you claim the file is located. You can address that by moving the file, using an absolute path, or changing the
current directory to be the one where teh script is located.
use Cwd qw( realpath );
use Path::File qw( file );
chdir(file(realpath($0))->dir);
Perhaps the file isn't named peptide. It might actually be named peptide.txt, for example. Windows hides extensions it recognises by default, a feature I HATE. You can address this by renaming the file or by using the correct file name.
Are you looking to open the file for reading or writing? Your open statement opens it for reading; your error message says 'writing'. You use it for reading — so your error message is confusing, I believe.
If you get 'No such file or directory' errors, it means that despite what you thought, the name 'peptide' is not the name of a file in the current directory. Perl does not add extensions to file names for you; if your file is actually peptide.txt (since you mention that it is a 'txt file'), then that's what you need to specify to open. If you run perl protein.pl and peptide (or peptide.txt) is in the current directory, then it is not clear what your problem is. If your script is in C:\BIN directory and your current directory is not C:\BIN but peptide (or peptide.txt) is also in C:\BIN, then you need to arrange to open C:/bin/peptide or c:/bin/peptide.txt. Note the switch from backslashes to slashes. Backslashes have meanings specific to Perl as an escape character, and Windows is happy with slashes in place of backslashes. If you must use backslashes, then use single quotes around the name:
my $proteinfilename = 'C:\BIN\peptide.txt';
It may be simplest to take the protein file name from a command line argument; this gives you the flexibility of having the script anywhere on your PATH and the file anywhere you choose.
Two suggestions to help your Perl:
Use the 3-argument form of open and lexical file handles, as in:
open my $PROTEINFILE, '<', $proteinfilename or
die "Can't open file '$proteinfilename' for reading [$!]\n";
my $protein = <$PROTEINFILE>;
close $PROTEINFILE;
Note that this reads a single line from the file. If you need to slurp the whole file into $protein, then you have to do a little more work. There are modules to handle slurping for you, but you can also simply use:
my $protein;
{ local $/; $protein = <$PROTEINFILE>; }
This sets the line delimiter to undef which means the entire file is slurped in one read operation. The $/ variable is global, but this adjusts its value in a minimal scope. Note that $protein was declared outside the block containing the slurp operation!
Use use strict; as well as -w or use warnings;. It will save you grief over time.
I've only been using Perl for 20 years; I don't write a serious script without both use strict; and use warnings; because I don't trust my ability to spot silly mistakes (and Perl will do it for me). I don't make all that many mistakes, but Perl has saved me on many occasions because I use them.
Here how your program will go
#!/usr/bin/perl
use strict;
use warnings;
my $proteinfilename = 'peptide.txt';
open(PROTEINFILE, $proteinfilename) or die "Can't write to file '$proteinfilename' [$!]\n";
my $protein = <PROTEINFILE>;
close PROTEINFILE;
print $protein;
You need to add the file extension(for example .txt) at the end like below.
my $proteinfilename = 'peptide.txt';
Your program say peptide_test.pl and input text file peptide.txt should be in the same directory.
If they are not in the same directory, use absolute path like below.
my $proteinfilename = 'C:\somedirectory\peptide.txt';
Note: Use single quotes in case of absolute path.This will ignore the backslash\ in path.
Now about errors, If you don't use die statement, you will get error
readline<> on closed filehandle PROTEINFILE at C:\BIN\protein.pl
After using die,
or die $! ;
you will get error No such file or directory.
Also always
use strict;
use warnings;
-w is deprecated after perl 5.6. These two lines/statements will help you finding typos,syntax errors
And one more,I don't think you need exit;, at the end.
Refer exit function.
I am trying to create a temp file using the following code:
use File::Temp ;
$tmp = File::Temp->new( TEMPLATE => 'tempXXXXX',
DIR => 'mydir',
SUFFIX => '.dat');
This is create the temp file. Because of my permission issue, the other program is not able to write into file.
So I just want to generate the file name without creating the file. Is there any where to do that?
If you don't create the file at the same time you create the name then it is possible for the a file with the same name to be created before you create the file manually. If you need to have a different process open the file, simply close it first:
#!/usr/bin/perl
use strict;
use warnings;
use File::Temp;
sub get_temp_filename {
my $fh = File::Temp->new(
TEMPLATE => 'tempXXXXX',
DIR => 'mydir',
SUFFIX => '.dat',
);
return $fh->filename;
}
my $filename = get_temp_filename();
open my $fh, ">", $filename
or die "could not open $filename: $!";
The best way to handle the permissions problem is to make sure the users that run the two programs are both in the same group. You can then use chmod to change the permissions inside the first program to allow the second program (or any user in that group) to modify the file:
my $filename = get_temp_filename();
chmod 0660, $filename;
Just to obtain the name of the tempfile you can do:
#!/usr/bin/perl
use strict;
use warnings;
use 5.10.1;
use File::Temp qw/tempfile/;
my $file;
(undef, $file) = tempfile('tmpXXXXXX', OPEN=>0);
say $file;
But as Chas. Owens said, be careful the same name could be created before you use it.
The get_temp_filename function proposed by Chas. Owens uses a local filehandle object ($fh), which is destroyed upon function return, leading to the created tempfile destruction.
To avoid this, and therefore keep the file (less risk) add:
UNLINK => 0
to the new method arguments, forbidding file unlink at object deletion time.
Actually, I agree with Chas.Owens - the design is fatally flawed.
It really feels like you need to fix the design, so:
If you have control of the 2nd program, have that program create the filename and the file, and pass the filename to the 1st program.
But, if the 2nd program isn't something you wrote and so you cannot modify it then I'd recommend one of the following:
1 - Use the first processes PID as part of the file name in an attempt to minimize the risks of duplicate filenames.
2 - Have the 2nd program pipe its output to the 1st program, don't bother with a file at all. Personally, this is a much better solution than 1.
3 - Wrap the 2nd program in a script (shell, perl, whatever) which creates the name and the file and passes that to both programs.