Variable not getting substituted
Even after defining the 2 variable explicitly it's not getting substituted
sub updatekey{
my $key_url = File::Spec->catfile($dir. "/keys/cert.key.$label.$type")
$eol = '';
open(FILE, $key_url) or die "$!";
my $key_file;
while (read(FILE, $buf, 60*57)) {
$keyfile = $key_file . encode_base64($buf,$eol);
}
}
Open FILE getting failed because $type is not getting substituted. if I modify the line as below
my $key_url = File::Spec->catfile($dir. "/keys/cert.key.$label.pem")
it's working fine.
This is pretty basic debugging.
Add this to your code as the first lines in the subroutine:
if (defined $type) {
print "\$type is undefined\n";
} else {
print "\$type is '$type'\n";
}
That way, you'll see exactly what value $type has just before you try to use it. I expect you'll see either "$type is undefined" or "$type is ''".
Your problem then becomes finding where $type is supposed to be set and working out why that isn't happening.
Two other pieces of advice:
When writing a Perl program, it's always a good idea to have use strict and use warnings in your code (near the top of the file) as they will find a large number of basic programming errors.
When programming in any language, you should ensure that any subroutine only uses variables that are passed in at parameters or variables that are declared within the subroutine. Using global variables (as you seem to be doing here) makes your code less portable and harder to debug.
Related
I am trying to store my log messages in a hash depending upon message type as shown below:
#!/usr/bin/perl
use strict;
use warnings;
my %log;
opendir (DIR, '.') or die $!;
while (my $file = readdir(DIR)) {
# some code to select TAR filename in $1
if (-e $1.'.tar') {
push(#{$log->{$1}}, $file); /* line 12 */
}
else {
$log{$1} = [];
push(#{$log->{$1}}, $file); /* line 16 */
}
Now this code gives compilation error saying:
Global symbol "$log" requires explicit package name at at lines 12 & 16
where I am actually trying to use the hash "%log". What can be a possible way to get rid of this error ? Why exactly is this happening ?
I did see some explanation on context where people replied saying the variables were created in one context and were being referred in another but I feel this variable should be available inside while loop in this piece of code. This happens only when I have "use strict" and works fine otherwise.
I have started with Perl so I do not fully understand the basics! Please help me understand why this variable is not accessible.
my %log;
defines hash %log, but lines 12 and 16 don't use it. Instead, you're accessing the anonymous hash referenced by the scalar $log which you've never declared. You have two options.
You could continue to use an anonymous hash.
my $log = {}; # The creation of the hash ("{}") is currently being done
# implicitly by "->". This is called autovivification.
... $log->{...} ...
This adds a bit of a extra complexity and an unnoticeable reduction in speed.
You could use use a hash directly.
my %log;
... $log{...} ...
I'm not sure what are you trying to do with $1, but the hash access is not a reference, so change:
$log->{$1}
to
$log{$1}
The error message you got, says: Global symbol "$log" requires explicit package, because $log variable was not defined. Remember that %log and $log are two different variables (hash vs scalar).
I have a CGI script pulling bibliography data from a BibTeX file, building HTML from it. It uses CGI::Ajax to call the subroutine below with one or two arguments. Most of the time, it will be a search term that is passed as $s, but if I pass a string through my HTML form, the subroutine will not be entirely happy with it. There is a foreach loop checking the entries and jumping over the entries that do not match. Now I can print the argument outside this loop alright, but the loop itself won’t print anything for $s, nor will it find any entries matching it. If within the loop $s were simply empty, the subroutine would print the entire bibliography, but this is not the case.
Basically it is as if $s passed as an argument breaks the loop, whereas an explicit definition in the subroutine works fine.
Here is a simplified version of my code. Please excuse sloppy or ignorant coding, I’m just dabbling in Perl.
sub outputBib {
my ( $self,$s,$kw ) = #_;
my #k;
#k = ('foo','bar'); # this is fine
#k = keys (%{$self->{_bib}}); # this causes problems
foreach my $k (#k) {
$output .= "Key = $k<br/>";
$output .= "Search Term = $s<br/>";
}
return $output;
}
The problem seems to be the array built from the keys of the $self->{_bib} hash. It is odd that
the loop is fine when $s is not passed through CGI::Ajax. All elements are processed.
as soon as the subroutine is called with $s, the loop does not return anything.
if #k is defined as a straightforward array, the loop works and $s can be printed within the loop;
I build $self->{_bib} like so:
sub parseBib {
my ( $self ) = #_;
while (my $e = new Text::BibTeX::Entry $self->{_bibFileObject}) {
next unless $e->parse_ok;
my %entry_hash;
$entry_hash{'title'} = $e->get('title');
$entry_hash{'keywords'} = $e->get('keywords');
$self->{_bib}{$e->key} = \%entry_hash;
}
}
Any ideas? Thanks.
My first suggestion would be to use warn/print STDERR to verify on the live running copy that, when called via CGI::Ajax, all of your variables ($self, $s, $kw, $self->{_bib}) have the values that you're expecting. Although I'm a big fan of CGI::Ajax, it does a fair bit of magic behind your back and it may not be calling outputBib in quite the way you think it is.
Also keep in mind that CGI operates on a per-request model, not per-page. Are you perhaps populating $self->{_bib} when you send the initial page (and also doing all of your successful tests in that environment), then expecting it to still be there when the AJAX requests come in? If so, you're out of luck - you'll need to rebuild it in the AJAX handler, either within outputBib or earlier in your code, before you call ->build_html and hand it off to CGI::Ajax.
I've just been caught out when using someone else's API in conjunction with the default variable $_
foreach (#rps_server_details) {
#server_data = ();
#server_data = split(/,/);
#$esp_hosts = ();
$filters{server_name} = $server_data[0];
print "--->$_<--\n";
$esp_hosts = $esp->get_hosts(fields => $fields, %filters) || die "$#";
print "--->$_<--\n";
The output for this is:
--->igrid8873.someone.com,app_10<--
Use of uninitialized value in concatenation (.) or string at ./rps_inv_lookup.pl line 120.
---><--
Specifying my own loop variable instead of relying on $_ fixes the problem.
Am I just being naive by using $_ in conjunction with an API someone else has written? Or is this a bug in that API module?
It is a bug in the API. If you use $_ in a function it is important to add a
local($_);
inside the function to avoid clobbering the caller's $_, or otherwise avoid using $_in a library function to be called by others.
If you can limit yoursel to Perl versions > 5.9.1 then you can also make $_ lexical which makes it easier to understand than localwith
my $_;
But this will break on earlier versions of Perl.
From man perlvar:
As $_ is a global variable, this may lead in some cases to
unwanted side-effects. As of perl 5.9.1, you can now use a
lexical version of $_ by declaring it in a file or in a block
with "my". Moreover, declaring "our $_" restores the global $_
in the current scope.
I would say it's:
a violation of best practices on your part (always use as-local-as possible variable scope and avoid using $_ due to just the issue your encountered)
coupled with a bug in the API caused by the same violation of the best practices as well as not localizing the special variable with local $_ as proscribed by perldoc perlvar.
In addition to perldoc, the API violates Perl Best Practices (as in Conway's book's rules):
Section 5.6. Localizing Punctuation Variables
If you're forced to modify a punctuation variable, localize it.
The problems described earlier under "Localization can also crop up whenever you're forced to change the value in a punctuation variable (often in I/O operations). All punctuation variables are global in scope. They provide explicit control over what would be completely implicit behaviours in most other languages: output buffering, input line numbering, input and output line endings, array indexing, et cetera.
It's usually a grave error to change a punctuation variable without first localizing it. Unlocalized assignments can potentially change the behaviour of code in entirely unrelated parts of your system, even in modules you did not write yourself but are merely using.
Using local is the cleanest and most robust way to temporarily change the value of a global variable. It should always be applied in the smallest possible scope, so as to minimize the effects of any "ambient behaviour" the variable might control:
Here's full perldoc perlvar documentation as well - search for the word "nasty_break" in the web page (I couldn't find direct in-page link but it's close to the start of the page)
You should be very careful when
modifying the default values of most
special variables described in this
document. In most cases you want to
localize these variables before
changing them, since if you don't, the
change may affect other modules which
rely on the default values of the
special variables that you have
changed. This is one of the correct
ways to read the whole file at once:
open my $fh, "<", "foo" or die $!;
local $/; # enable localized slurp mode
my $content = ;
close $fh;
But the following code is quite bad:
open my $fh, "<", "foo" or die $!;
undef $/; # enable slurp mode
my $content = ;
close $fh;
since some other module, may want to
read data from some file in the
default "line mode", so if the code we
have just presented has been executed,
the global value of $/ is now changed
for any other code running inside the
same Perl interpreter.
Usually when a variable is localized
you want to make sure that this change
affects the shortest scope possible.
So unless you are already inside some
short {} block, you should create one
yourself. For example:
my $content = '';
open my $fh, "<", "foo" or die $!;
{
local $/;
$content = ;
}
close $fh;
Here is an example of how your own
code can go broken:
for (1..5){
nasty_break();
print "$_ ";
}
sub nasty_break {
$_ = 5;
# do something with $_
}
You probably expect this code to
print:
1 2 3 4 5
but instead you get:
5 5 5 5 5
Why? Because nasty_break() modifies $_
without localizing it first. The fix
is to add local():
local $_ = 5;
foreach (#rps_server_details) {
#server_data = ();
#server_data = split(/,/);
#$esp_hosts = ();
$filters{server_name} = $server_data[0];
print "--->$_<--\n";
{
local *_; # disconnects the remaining scope from the implicit
# variables so you can clean up after the dirty api.
# NOTE: Submit a bug report against the offending module.
# If you notice this across multiple api features
# consider finding a different module for this task.
$esp_hosts = $esp->get_hosts(fields => $fields, %filters) || die "$#";
}
print "--->$_<--\n";
Yes, the problem is with a library I'm using, and no, I cannot modify it. I need a workaround.
Basically, I'm dealing with a badly written Perl library, that exits with 'die' when a certain error condition is encountered reading a file. I call this routine from a program which is looping through thousands of files, a handful of which are bad. Bad files happen; I just want my routine to log an error and move on.
IF I COULD modify the library, I would simply change the
die "error";
to a
print "error";return;
, but I cannot. Is there any way I can couch the routine so that the bad files won't crash the entire process?
FOLLOWUP QUESTION: Using an "eval" to couch the crash-prone call works nicely, but how do I set up handling for catch-able errors within that framework? To describe:
I have a subroutine that calls the library-which-crashes-sometimes many times. Rather than couch each call within this subroutine with an eval{}, I just allow it to die, and use an eval{} on the level that calls my subroutine:
my $status=eval{function($param);};
unless($status){print $#; next;}; # print error and go to next file if function() fails
However, there are error conditions that I can and do catch in function(). What is the most proper/elegant way to design the error-catching in the subroutine and the calling routine so that I get the correct behavior for both caught and uncaught errors?
You could wrap it in an eval. See:
perldoc -f eval
For instance, you could write:
# warn if routine calls die
eval { routine_might_die }; warn $# if $#;
This will turn the fatal error into a warning, which is more or less what you suggested. If die is called, $# contains the string passed to it.
Does it trap $SIG{__DIE__}? If it does, then it's more local than you are. But there are a couple strategies:
You can evoke its package and override die:
package Library::Dumb::Dyer;
use subs 'die';
sub die {
my ( $package, $file, $line ) = caller();
unless ( $decider->decide( $file, $package, $line ) eq 'DUMB' ) {
say "It's a good death.";
die #_;
}
}
If not, can trap it. (look for $SIG on the page, markdown is not handling the full link.)
my $old_die_handler = $SIG{__DIE__};
sub _death_handler {
my ( $package, $file, $line ) = caller();
unless ( $decider->decide( $file, $package, $line ) eq 'DUMB DIE' ) {
say "It's a good death.";
goto &$old_die_handler;
}
}
$SIG{__DIE__} = \&_death_handler;
You might have to scan the library, find a sub that it always calls, and use that to load your $SIG handler by overriding that.
my $dumb_package_do_something_dumb = \&Dumb::do_something_dumb;
*Dumb::do_something_dumb = sub {
$SIG{__DIE__} = ...
goto &$dumb_package_do_something_dumb;
};
Or override a builtin that it always calls...
package Dumb;
use subs 'chdir';
sub chdir {
$SIG{__DIE__} = ...
CORE::chdir #_;
};
If all else fails, you can whip the horse's eyes with this:
package CORE::GLOBAL;
use subs 'die';
sub die {
...
CORE::die #_;
}
This will override die globally, the only way you can get back die is to address it as CORE::die.
Some combination of this will work.
Although changing a die to not die has a specific solution as shown in the other answers, in general you can always override subroutines in other packages. You don't change the original source at all.
First, load the original package so you get all of the original definitions. Once the original is in place, you can redefine the troublesome subroutine:
BEGIN {
use Original::Lib;
no warnings 'redefine';
sub Original::Lib::some_sub { ... }
}
You can even cut and paste the original definition and tweak what you need. It's not a great solution, but if you can't change the original source (or want to try something before you change the original), it can work.
Besides that, you can copy the original source file into a separate directory for your application. Since you control that directory, you can edit the files in it. You modify that copy and load it by adding that directory to Perl's module search path:
use lib qw(/that/new/directory);
use Original::Lib; # should find the one in /that/new/directory
Your copy sticks around even if someone updates the original module (although you might have to merge changes).
I talk about this quite a bit in Mastering Perl, where I show some other techniques to do that sort of thing. The trick is to not break things even more. How you not break things depends on what you are doing.
I have recently started learning Perl and one of my latest assignments involves searching a bunch of files for a particular string. The user provides the directory name as an argument and the program searches all the files in that directory for the pattern. Using readdir() I have managed to build an array with all the searchable file names and now need to search each and every file for the pattern, my implementation looks something like this -
sub searchDir($) {
my $dirN = shift;
my #dirList = glob("$dirN/*");
for(#dirList) {
push #fileList, $_ if -f $_;
}
#ARGV = #fileList;
while(<>) {
## Search for pattern
}
}
My question is - is it alright to manually load the #ARGV array as has been done above and use the <> operator to scan in individual lines or should I open / scan / close each file individually? Will it make any difference if this processing exists in a subroutine and not in the main function?
On the topic of manipulating #ARGV - that's definitely working code, Perl certainly allows you to do that. I don't think it's a good coding habit though. Most of the code I've seen that uses the "while (<>)" idiom is using it to read from standard input, and that's what I initially expect your code to do. A more readable pattern might be to open/close each input file individually:
foreach my $file (#files) {
open FILE, "<$file" or die "Error opening file $file ($!)";
my #lines = <FILE>;
close FILE or die $!;
foreach my $line (#file) {
if ( $line =~ /$pattern/ ) {
# do something here!
}
}
}
That would read more easily to me, although it is a few more lines of code. Perl allows you a lot of flexibility, but I think that makes it that much more important to develop your own style in Perl that's readable and understandable to you (and your co-workers, if that's important for your code/career).
Putting subroutines in the main function or in a subroutine is also mostly a stylistic decision that you should play around with and think about. Modern computers are so fast at this stuff that style and readability is much more important for scripts like this, as you're not likely to encounter situations in which such a script over-taxes your hardware.
Good luck! Perl is fun. :)
Edit: It's of course true that if he had a very large file, he should do something smarter than slurping the entire file into an array. In that case, something like this would definitely be better:
while ( my $line = <FILE> ) {
if ( $line =~ /$pattern/ ) {
# do something here!
}
}
The point when I wrote "you're not likely to encounter situations in which such a script over-taxes your hardware" was meant to cover that, sorry for not being more specific. Besides, who even has 4GB hard drives, let alone 4GB files? :P
Another Edit: After perusing the Internet on the advice of commenters, I've realized that there are hard drives that are much larger than 4GB available for purchase. I thank the commenters for pointing this out, and promise in the future to never-ever-ever try to write a sarcastic comment on the internet.
I would prefer this more explicit and readable version:
#!/usr/bin/perl -w
foreach my $file (<$ARGV[0]/*>){
open(F, $file) or die "$!: $file";
while(<F>){
# search for pattern
}
close F;
}
But it is also okay to manipulate #ARGV:
#!/usr/bin/perl -w
#ARGV = <$ARGV[0]/*>;
while(<>){
# search for pattern
}
Yes, it is OK to adjust the argument list before you start the 'while (<>)' loop; it would be more nearly foolhardy to adjust it while inside the loop. If you process option arguments, for instance, you typically remove items from #ARGV; here, you are adding items, but it still changes the original value of #ARGV.
It makes no odds whether the code is in a subroutine or in the 'main function'.
The previous answers cover your main Perl-programming question rather well.
So let me comment on the underlying question: How to find a pattern in a bunch of files.
Depending on the OS it might make sense to call a specialised external program, say
grep -l <pattern> <path>
on unix.
Depending on what you need to do with the files containing the pattern, and how big the hit/miss ratio is, this might save quite a bit of time (and re-uses proven code).
The big issue with tweaking #ARGV is that it is a global variable. Also, you should be aware that while (<>) has special magic attributes. (reading each file in #ARGV or processing STDIN if #ARGV is empty, testing for definedness rather than truth). To reduce the magic that needs to be understood, I would avoid it, except for quickie-hack-jobs.
You can get the filename of the current file by checking $ARGV.
You may not realize it, but you are actually affecting two global variables, not just #ARGV. You are also hitting $_. It is a very, very good idea to localize $_ as well.
You can reduce the impact of munging globals by using local to localize the changes.
BTW, there is another important, subtle bit of magic with <>. Say you want to return the line number of the match in the file. You might think, ok, check perlvar and find $. gives the linenumber in the last handle accessed--great. But there is an issue lurking here--$. is not reset between #ARGV files. This is great if you want to know how many lines total you have processed, but not if you want a line number for the current file. Fortunately there is a simple trick with eof that will solve this problem.
use strict;
use warnings;
...
searchDir( 'foo' );
sub searchDir {
my $dirN = shift;
my $pattern = shift;
local $_;
my #fileList = grep { -f $_ } glob("$dirN/*");
return unless #fileList; # Don't want to process STDIN.
local #ARGV;
#ARGV = #fileList;
while(<>) {
my $found = 0;
## Search for pattern
if ( $found ) {
print "Match at $. in $ARGV\n";
}
}
continue {
# reset line numbering after each file.
close ARGV if eof; # don't use eof().
}
}
WARNING: I just modified your code in my browser. I have not run it so it, may have typos, and probably won't work without a bit of tweaking
Update: The reason to use local instead of my is that they do very different things. my creates a new lexical variable that is only visible in the contained block and cannot be accessed through the symbol table. local saves the existing package variable and aliases it to a new variable. The new localized version is visible in any subsequent code, until we leave the enclosing block. See perlsub: Temporary Values Via local().
In the general case of making new variables and using them, my is the correct choice. local is appropriate when you are working with globals, but you want to make sure you don't propagate your changes to the rest of the program.
This short script demonstrates local:
$foo = 'foo';
print_foo();
print_bar();
print_foo();
sub print_bar {
local $foo;
$foo = 'bar';
print_foo();
}
sub print_foo {
print "Foo: $foo\n";
}