In perl text substitutions are very simple and powerful.
I want to do a script with variable substitutions, like:
if ( $IgnoreCase ) {$opt = "gi"} else {$opt = "g"}
$string =~ s/$source/$replace/$opt;
Results in:
Scalar found where operator expected ...
Is there a posibility to do the option variable?
Since you're using /g in all cases you can,
my $opt = $IgnoreCase ? "(?i)" : "";
$string =~ s/$opt$source/$replace/g;
More on this subject in perldoc perlre
One or more embedded pattern-match modifiers, to be turned on (or turned off if preceded by "-" ) for the remainder of the pattern or the remainder of the enclosing pattern group (if any).
This is particularly useful for dynamically-generated patterns, such as those read in from a configuration file, taken from an argument, or specified in a table somewhere. Consider the case where some patterns want to be case-sensitive and some do not: The case-insensitive ones merely need to include (?i) at the front of the pattern.
To slightly clarify the excellent answer from Сухой27:
As you have seen, you can't use a variable in the options section of the s/// operator (well, you could if you used eval but that would be a very bad idea).
However, that's not the only way to get options into a regex match. You can also use the (?...) syntax inside the regex string. For example, m/(?i)foo/ is exactly the same as m/foo/i. Note that /g is a slightly different class of option, so you can't use that option like this.
But because this (?...) string is just part of the regex, you can use a variable to embed it within your regex.
my $opt = $IgnoreCase ? "(?i)" : "";
$string =~ s/$opt$source/$replace/g;
See "Extended Patterns" in perldoc perlre for more details.
Related
I'm new to Perl, though not to programming, and am working through Learning Perl. The book has exercises to match successive lines of a small text file.
I had the idea of supplying match strings from STDIN, and going through the file for each one:
while(<STDIN>) {
chomp;
$regex = $_;
seek JUNK, 0, 0;
while(<JUNK>) {
chomp();
if(/$regex/) {
say;
}
}
say '';
}
This works fine, but I can't find a way to interpolate an entire match string, e.g.
/fred/i
into the predicate. I tried
if($$matcher) # with $matcher = '/fred/'
but Perl complained.
I imagine this is my ignorance, and should welcome enlightenment.
Statement modifiers, such as /i, are a part of the code telling Perl how to perform the match, not a part of the pattern to be matched. This is why that doesn't work for you.
You have three ways to work around this (well, probably more, since this is Perl we're talking about, but three ways that I can think of straight off):
1) Use extended regex syntax and, when you want a case-insensitive match, enter (?i:fred), as suggested in comments on the question.
2) Use string eval to allow the use of the regular statement modifiers: if (eval "$_ =~ $regex") { say } Note that this method will require you to also type the surrounding slashes. e.g., You'd have to enter /fred/i; just typing in fred would not work. Note also that it's a huge security hole to do this without validating your input first, since the user's entered text is executed as Perl code, just as if it were part of the original program. (Imagine if the user entered //, system("rm -rf /") - it would test against an empty regex, then delete all the files on your computer.) So probably not a recommended approach unless you really know what you're doing and/or you're the only one who will ever run the program.
3) The most complex, but also most correct, solution is to write a parser which inspects the user's entered string to see whether any special flags are present and then responds accordingly. A very simple example which allows the user to append /i for a case-insensitive search:
#!/usr/bin/env perl
use strict;
use warnings;
use 5.010;
while(<STDIN>) {
chomp;
my #parts = split '/', $_;
# If the user input starts with a /, the first part will be empty, so throw
# it away.
shift #parts unless $parts[0];
my $re = shift #parts;
my %flags;
for (#parts) {
for (split '') {
$flags{i} = 1 if $_ eq 'i';
}
}
my $f = join '', keys %flags;
say "Matched" if eval qq('foo' =~ /$re/$f);
}
This also uses string eval, so it is potentially vulnerable to the same kind of security issues as #2, but $re cannot contain any / characters (the split '/' would have ended $re immediately prior to the first /), which prevents code from being inserted there and $f can contain only the letter i (or any other flags you might choose to recognize if you expand on this). So it should be safe. (But, if anyone can demonstrate an exploit I missed, please tell me about it in comments!)
Problem
What you are trying to do can be summarized by:
my $regex = '/fred/i';
my #lines = (
'A line containing some words and Fred said Hello.',
'Another line. Here is a regex embedded in the line: /fred/i',
);
for ( #lines ) {
say if /$regex/;
}
Output:
Another line. Here is a regex embedded in the line: /fred/i
We see that the second line matches $regex, whereas we wanted the first line containing Fred to match the string fred with the (case insensitive) i flag added to the regex. The problem is that the characters / and i in $regex are taken as characters to be matched literally, i.e., they are not interpreted as special characters surrounding a Regex (as part of a Perl expression).
Note:
The character / is special as part of a Perl expression for a regular expression, but it is not special inside the Regex pattern. There are however characters that are special inside the pattern, the so-called meta characters:
\ | ( ) [ { ^ $ * + ? .
see perldoc quotemeta for more information.
A solution using extended patterns
Simply change the first line to:
my $regex = '(?i)fred'; # or alternatively: (?i:fred)
Regex flags can be added to a regex pattern using "Extended patterns" described in the manual perldoc perlre :
Extended Patterns
The syntax for most of these is a pair of parentheses with a question
mark as the first thing within the parentheses. The character after
the question mark indicates the extension.
[...]
(?adlupimnsx-imnsx)
(?^alupimnsx)
One or more embedded pattern-match modifiers, to be turned on (or
turned off if preceded by "-" ) for the remainder of the pattern or
the remainder of the enclosing pattern group (if any). This is
particularly useful for dynamically-generated patterns, such as those
read in from a configuration file, taken from an argument, or
specified in a table somewhere.
[...]
These modifiers are restored at the end of the enclosing group.
Alternatively the non-capturing form can be used:
(?:pattern)
(?adluimnsx-imnsx:pattern)
(?^aluimnsx:pattern)
This is for clustering, not capturing; it groups subexpressions like
"()" , but doesn't make backreferences as "()" does.
The question has been answered in the following comment:
Try (?i:fred), see Extended
patterns in
perldoc perlre for more information
– Håkon Hægland 7 hours ago.
I am trying to understand a program. Correct my if I'm wrong, but backticks are used to execute commands in a shell, so I'm not sure what it is its purpose in the following code:
my $end = $` if $dir =~ m/\/foldername/;
foreach my $folder (#dirs_) {
my $start_from = "$dir" . "\/" . "$folder";
my $move_to = "$end" . "\/" . "$folder";
rmtree $move_to;
dircopy($start_from, $move_to);
}
It's not very pretty is it.
The $` variable is one of the trinity $`, $& and $' which represent the pre-match, match, and post-match parts of the last string that was subjected to a successful regex comparison
For instance, if I have
my $s = 'abcdef';
then after
$s =~ /c./;
you will find that $` is ab, $& is cd, and $' is ef
It's important to remember that, just like the capture variables $1, $2 etc., these three are unaffected by failed regex matches. (They are not set to undef.) So it's vital to check whether a regex pattern matched before using any of them
This is archaic Perl, maintained primarily for backward compatability. It was a good idea at the time because Perl was keeping close to shell syntax (as were awk and sed, which still do). Nowadays it is best to use regex captures, or perhaps substr in conjunction with the newer #- and #+ arrays
All of the special built-in variables are documented in perldoc perlvar
The variable $` is a Perl special variable whose "English" name is $PREMATCH. From the perldoc website:
The string preceding whatever was matched by the last successful pattern match, not counting any matches hidden within a BLOCK or eval enclosed by the current BLOCK.
The $` is a regex-related special variable, containing the string preceding the last succesful match. From perlvar
$`
The string preceding whatever was matched by the last successful pattern match, not counting any matches hidden within a BLOCK or eval enclosed by the current BLOCK.
See Performance issues above for the serious performance implications of using this variable (even once) in your code.
This variable is read-only and dynamically-scoped.
Mnemonic: ` often precedes a quoted string.
In this case it contains the part of the string in $dir that precedes the matched |/foldername|, if the match happened. Note that this line of code, with the conditional declaration, results in undefined behavior if there is no match.
The code in foreach is then meant to copy folders "$dir/$folder" one level up. However, if the match failed this code runs after the program got into an invalid state. So I would urge you to rewrite it, along the lines of: declare $end in a separate statement, then conditionally assign the match and enter the loop, or skip the loop (if the match fails and $end is undef).
Following the link in documentation quote above, to Performance issues
In Perl 5.20.0 a new copy-on-write system was enabled by default, which finally fixes all performance issues with these three variables, and makes them safe to use anywhere.
The "three variables" refers to $\, $&, and $'. Thanks to stevieb for this remark.
However, I suggest following the recommendation by Borodin to use modern tools and techniques.
I have one perl script which parse the log and search for perticular pattern like Error, Fatal and decide the Pass and Fail status.
Coming to my question I need to search uniq pattern like "Error (E302/FEP0512SRA)" if such pattern is presnet in any line of Log file it should increase error_cnt by 1.
I tried "\" and Below approch but in both case i got fail to capture above mention patter.
my $str = "Error (E302/FEP0512SRA)";
if($line =~ /$str/) {
$error_cnt++;
}
Please let me know what else i can do so i can catch this string in my log.
Your string contains regex metacharacters (the parentheses). You should either escape them by hand, or use quotemeta:
my $str = quotemeta "Error (E302/FEP0512SRA)";
Regexes contain many operators, so-called metacharacters. Parens () are such metacharacters and have to be escaped. Perl provides the quotemeta function for that. Inside a regex, we can use the equivalent \Q...\E environment, which takes care of that for us. Then:
$error_cnt++ if $line =~ /\Q$str\E/;
What is the meaning of below statement in perl?
($script = $0) =~ s#^.*/##g;
I am trying to understand the operator =~ along with the statement on the right side s#^.*/##g.
Thanks
=~ applies the thing on the right (a pattern match or search and replace) to the thing on the left. There's lots of documentation about =~ out there, so I'm just going to point you at a pretty good one.
There's a couple of idioms going on there which are not obvious nor well documented which might be tripping you up. Let's cover them.
First is this...
($copy = $original) =~ s/foo/bar/;
This is a way of copying a variable and performing a search and replace on it in a single step. It is equivalent to:
$copy = $original;
$copy =~ s/foo/bar/;
The =~ operates on whatever is on the left after the left hand code has been run. ($copy = $original) evaluates to $copy so the =~ acts on the copy.
s#^.*/##g is the same as s/^.*\///g but using alternative delimiters to avoid Leaning Toothpick Syndrome. You can use just about anything as a regex delimiter. # is common, though I think its ugly and hard to read. I prefer {} because they balance. s{^.*/}{}g is equivalent code.
Unrolling the idioms, you have this:
$script = $0;
$script =~ s{^.*/}{}g;
$0 is the name of the script. So this is code to copy the name of the script and strip everything up to the last slash (.* is greedy and will match as much as possible) off it. It is getting just the filename of the script.
The /g indicates to perform the match on the string as many times as possible. Since this can only ever match once (the ^ anchors it to the beginning of the string) it serves no purpose.
There's a better and safer way to do this.
use File::Basename;
$script = basename($0);
It's very, very simple:
Perl quote-like expressions can take many different characters as part separators. The separator right after the command (in this case, the s) is the separator for the rest of the operation. For example:
# Out with the "Old" and "In" with the new
$string =~ s/old/new/;
$string =~ s#old#new#;
$string =~ s(old)(new);
$string =~ s#old#new#;
All four of those expressions are the same thing. They replace the string old with new in my $string. Whatever comes after the s is the separator. Note that parentheses, curly braces, and square brackets use parings. This works out rather nicely for the q and qq which can be used instead of single quotes and double quotes:
print "The value of \$foo is \"foo\"\n"; # A bit hard to read
print qq/The value of \$foo is "$foo"\n/; # Maybe slashes weren't a great choice...
print qq(The value of \$foo is "$foo"\n); # Very nice and clean!
print qq(The value of \$foo is (believe it or not) "$foo"\n); #Still works!
The last still works because the quote like operators count opening and closing parentheses. Of course, with regular expressions, parentheses and square brackets are part of the regular expression syntax, so you won't see them so much in substitutions.
Most of the time, it is highly recommended that you stick with the s/.../.../ form just for readability. It's what people are use to and it's easy to digest. However, what if you have this?
$bin_dir =~ s/\/home\/([^\/]+)\/bin/\/Users\/$1\bin/;
Those backslashes can make it hard to read, so the tradition has been to replace the backslash separators to avoid the hills and valleys effect.
$bin_dir =~ s#/home/([^/]+)/bin#/Users/$1/bin#;
This is a bit hard to read, but at least I don't have to quote each forward slash and backslash, so it's easier to see what I'm substituting. Regular expressions are hard because good quote characters are hard to find. Various special symbols such as the ^, *, |, and + are magical regular expression characters, and could probably be in a regular expression, the # is a common one to use. It's not common in strings, and it doesn't have any special meaning in a regular expression, so it won't be used.
Getting back to your original question:
($script = $0) =~ s#^.*/##g;
is the equivalent of:
($script = $0) =~ s/^.*\///g;
But because the original programmer didn't want to backquote that slash, they changed the separator character.
As for the:
($script = $0) =~ s#^.*/##g;`
It's the same as saying:
$script = $0;
$script =~ s#^.*/##g;
You're assigning the $script variable and doing the substitution in a single step. It's very common in Perl, but it is a bit hard to understand at first.
By the way, if I understand that basic expression (Removing all characters to the last forward slash. This would have been way cleaner:
use File::Basename;
...
$script = basename($0);
Much easier to read and understand -- even for an old Perl hand.
In perl, you can use many kinds of characters as quoting characters (string, regular expression, list). lets break it down:
Assign the $script variable the contents of $0 (the string that contains the name of the calling script.)
The =~ character is the binding operator. It invokes a regular expression match or a regex search and replace. In this case, it matches against the new variable, $script.
the s character indicates a search and replace regex.
The # character is being used as the delimiter for the regex. The regex pattern quote character is usually the / character, but you can use others, including # in this case.
The regex, ^.*/. It means, "at the start of string, search for zero or more characters until a slash. This will keep capturing on each line except for newline characters (which . does not match by default.)
The # indicating the start of the 'replace' value. Usually you have a pattern here that uses any captured part of the first line.
The # again. This ends the replace pattern. Since there was nothing between the start and end of the replace pattern, everything that was found in the first is replaced with nothing.
g, or global match. The search and replace will keep happening as many times as it matches in the value.
Effectively, searches for and empties every value before the / in the value , but keeps all the newlines, in the name of the script. It's a really lazy way of getting the script name when invoked in a long script that only works with a unix-like path.
If you have a chance, consider replacing with File::Basename, a core module in Perl:
use File::Basename;
# later ...
my $script = fileparse($0);
$string = 'a=1;b=2';
use Data::Dumper;
#array = split("; ?", $string);
print Dumper(\#array);
output:
$VAR1 = [
'a=1',
'b=2'
];
Anyone knows how "; ?" work here?It's not regex, but works quite like regex,so I don't understand.
I think it means "semicolon followed by optional space (just one or zero)".
It's not regex, but works quite like regex,so I don't understand.
The pattern parameter to split is always treated as a regular expression (would be better to not use a string, though). The only exception is the "single space", which is taken to mean "split on whitespace"
The first parameter of split is a regex. So I'd rather write split /; ?/, $string;.
When you use a string for the first parameter, it just means the regex can vary and has to be compiled anew each time the split is run. See perldoc -f split for details.
The regex could be read; the character ";" optionally followed by a space. See perlretut and perlreref for details.
A semicolon (the ;) followed by an optional (the ?) space (the ).