(Perl) Is it possible to have interpolated variables when a string is read from a file? - perl

I am working on a script that has some variables which are passed on to a string and then they a printed out. The initial string was only 6 lines I didn't need an external file for it but I now have a new string which can fill over 1000 lines. The new string also has some fields that are to be replaced by variables declared in the script.
The text file has something like:
Hello $name
The code is supposed to have several parts to it.
Declaration of variable
my $name = 'Foo';
Open file and read it into a string.
my $content;
open(my $fh, '<', $filename) or die "cannot open file $filename";
{
local $/;
$content = <$fh>;
}
close($fh);
Print string
print $content
Expected outcome:
Hello Foo
I am wondering if it's possible to read "Hello $name" from a file but print it as "Hello Foo" since the variable name is declared as Foo.

So you want your file to be a template. Why not use a proper template language like this one?
use Template qw( );
my %vars = (
name => "Foo",
);
my $tt = Template->new();
$tt->process($qfn, \%vars)
or die($tt->error());
Template:
Hello [% name %]
The output can be captured instead of printed by using ->process's third arg.

Simplest way:
my $foo = 'Fred';
my $bar = 'Barney';
my $string = 'Say hello to $foo and $bar';
say eval qq{"$string"}

The correct answer to the question (as you've already seen) is to use a proper templating system instead.
But it's worth noting that this is answered in the Perl FAQ.
How can I expand variables in text strings?
If you can avoid it, don't, or if you can use a templating system, such as Text::Template or Template Toolkit, do that instead. You might even be able to get the job done with sprintf or printf:
my $string = sprintf 'Say hello to %s and %s', $foo, $bar;
However, for the one-off simple case where I don't want to pull out a full templating system, I'll use a string that has two Perl scalar variables in it. In this example, I want to expand $foo and $bar to their variable's values:
my $foo = 'Fred';
my $bar = 'Barney';
$string = 'Say hello to $foo and $bar';
One way I can do this involves the substitution operator and a double /e flag. The first /e evaluates $1 on the replacement side and turns it into $foo. The second /e starts with $foo and replaces it with its value. $foo, then, turns into 'Fred', and that's finally what's left in the string:
$string =~ s/(\$\w+)/$1/eeg; # 'Say hello to Fred and Barney'
The /e will also silently ignore violations of strict, replacing undefined variable names with the empty string. Since I'm using the /e flag (twice even!), I have all of the same security problems I have with eval in its string form. If there's something odd in $foo, perhaps something like #{[ system "rm -rf /" ]}, then I could get myself in trouble.
To get around the security problem, I could also pull the values from a hash instead of evaluating variable names. Using a single /e, I can check the hash to ensure the value exists, and if it doesn't, I can replace the missing value with a marker, in this case ??? to signal that I missed something:
my $string = 'This has $foo and $bar';
my %Replacements = (
foo => 'Fred',
);
# $string =~ s/\$(\w+)/$Replacements{$1}/g;
$string =~ s/\$(\w+)/
exists $Replacements{$1} ? $Replacements{$1} : '???'
/eg;
print $string;
If you're going to be using Perl, then it's really worth your while to spend an afternoon getting to know the FAQ.

Related

Seeing value of Perl variable created at runtime

My code is below:
use strict;
my $store = 'Media Markt';
my $sentence = "I visited [store]";
# Replace characters "[" and "]"
$sentence =~ s/\[/\$/g;
$sentence =~ s/\]//g;
print $sentence;
I see following at screen:
I visited $store
Is it possible to see following? I want to see value of $store:
I visited Media Markt
You seem to be thinking of using a string, 'store', in order to build a variable name, $store. This gets to the subject of symbolic references, and you do not want to go there.
One way to do what you want is to build a hash that relates such strings to corresponding variables. Then capture the bracketed strings in the sentence and replace them by their hash values
use warnings;
use strict;
my $store = 'Media Markt';
my $time = 'morning';
my %repl = ( store => $store, time => $time );
my $sentence = "I visited [store] in the [time]";
$sentence =~ s/\[ ([^]]+) \]/$repl{$1}/gex;
print "$sentence\n";
This prints the line I visited Media Markt in the morning
The regex captures anything between [ ], by using the negated character class [^]] (any char other than ]), matched one-or-more times (+). Then it replaces that with its value in the hash, using /e to evaluate the replacement side as an expression. Since brackets are matched as well they end up being removed. The /x allows spaces inside, for readibilty.
For each string found in brackets there must be a key-value pair in the hash or you'll get a warning. To account for this, we can provide an alternative
$sentence =~ s{\[ ([^]+) \]}{$repl{$1}//"[$1]"}gex;
The defined-or operator (//) puts back "[$1]" if $repl{$1} returns undef (no key $1 in the hash, or it has undef value). Thus strings which have no hash pairs are unchanged. I changed the delimiters to s{}{} so that // can be used inside.
This does not allow nesting (like [store [name]]), does not handle multiline strings, and has other limitations. But it should work for reasonable cases.
As I told you on the Perl Programmers Facebook group, this is very similar to one of the answers in the Perl FAQ.
How can I expand variables in text strings?
If you can avoid it, don't, or if you can use a templating system, such as Text::Template or Template Toolkit, do that instead. You might even be able to get the job done with sprintf or printf:
my $string = sprintf 'Say hello to %s and %s', $foo, $bar;
However, for the one-off simple case where I don't want to pull out a full templating system, I'll use a string that has two Perl scalar variables in it. In this example, I want to expand $foo and $bar to their variable's values:
my $foo = 'Fred';
my $bar = 'Barney';
$string = 'Say hello to $foo and $bar';
One way I can do this involves the substitution operator and a double /e flag. The first /e evaluates $1 on the replacement side and turns it into $foo. The second /e starts with $foo and replaces it with its value. $foo, then, turns into 'Fred', and that's finally what's left in the string:
$string =~ s/(\$\w+)/$1/eeg; # 'Say hello to Fred and Barney'
The /e will also silently ignore violations of strict, replacing undefined variable names with the empty string. Since I'm using the /e flag (twice even!), I have all of the same security problems I have with eval in its string form. If there's something odd in $foo, perhaps something like #{[ system "rm -rf /" ]}, then I could get myself in trouble.
To get around the security problem, I could also pull the values from a hash instead of evaluating variable names. Using a single /e, I can check the hash to ensure the value exists, and if it doesn't, I can replace the missing value with a marker, in this case ??? to signal that I missed something:
my $string = 'This has $foo and $bar';
my %Replacements = (
foo => 'Fred',
);
# $string =~ s/\$(\w+)/$Replacements{$1}/g;
$string =~ s/\$(\w+)/
exists $Replacements{$1} ? $Replacements{$1} : '???'
/eg;
print $string;
And the actual (but really not recommended - for the reasons explained in the FAQ above) answer to your question is:
$sentence =~ s/\[(\w+)]/'$' . $1/ee;

Reading a scalar from one file then writing the string to another file?

I've got a file named 'input.file' with the following line:
Foo is $COLOR
$COLOR is assigned 'red' and I'm trying to create a second file named 'output.file' with the following line:
Foo is red
Here's my failed attempt:
#!/usr/bin/perl
use strict;
use warnings;
my $COLOR = "red";
open(FILE_IN, "< input.file");
open(FILE_OUT, "> output.file");
while (<FILE_IN>){
# Prints 'Foo is $COLOR' to 'output.file'
print FILE_OUT;
}
close FILE_IN;
close FILE_OUT;
# Prints 'Foo is red' to STDOUT
print "Foo is $COLOR\n";
So how do I print 'red' rather than '$COLOR' when when printing to 'output.file'?
Thanks
A general solution
Lets assume a string contains placeholders that match the regex /\$\w+/. We also have a hash mapping names to values, e.g.:
my %replacements = (
COLOUR => 'red',
x => 42,
perl_snippet => '$x++ == 3',
);
So the input
my $input = <<'END';
My favourite colour is $COLOUR.
The meaning of life is $x. Here is some Perl: `$perl_snippet`
$undefined
END
should transform to
My favourite colour is red.
The meaning of life is 42. Here is some Perl: `$x++ == 3`
$undefined
and not
My favourite colour is red.
The meaning of life is 42. Here is some Perl: `42++ == 3`
This can be achieved by matching for placeholders, using the name as a hash key, and only substituting if an appropriate entry exists in the replacement hash:
(my $output = $input) =~
s/\$(\w+)/exists $replacements{$1} ? $replacements{$1} : '$'.$1/eg;
or
(my $output = $input) =~
s/\$(\w+)(?(?{exists $replacements{$1}})|(*F))/$replacements{$1}/g;
This strategy of using a single substitution with a hash also guarantees that each part of the string is only evaluated once, and no double interpolation occurs.
A specific solution
If only one placeholder has to be interpolated, we can simplify by not using a hash:
s/\$COLOR/red/g;
This has the following disadvantages:
You can only have one such replacement. Adding another s/// makes double escaping possible, which is usually a bug.
You cannot easily query or modify the substitution values or placeholders at runtime.

Perl - How to create commands that users can input in console?

I'm just starting in Perl and I'm quite enjoying it. I'm writing some basic functions, but what I really want to be able to do is to use those functions intelligently using console commands. For example, say I have a function adding two numbers. I'd want to be able to type in console "add 2, 4" and read the first word, then pass the two numbers as parameters in an "add" function. Essentially, I'm asking for help in creating some basic scripting using Perl ^^'.
I have some vague ideas about how I might do this in VB, but Perl, I have no idea where I'd start, or what functions would be useful to me. Is there something like VB.net's "Split" function where you can break down the contents of a scalar into an array? Is there a simple way to analyse one word at a time in a scalar, or iterate through a scalar until you hit a separator, for example?
I hope you can help, any suggestions are appreciated! Bear in mind, I'm no expert, I started Perl all of a few weeks ago, and I've only been doing VB.net half a year.
Thank you!
Edit: If you're not sure what to suggest and you know any simple/intuitive resources that might be of help, that would also be appreciated.
Its rather easy to make a script which dispatches to a command by name. Here is a simple example:
#!/usr/bin/env perl
use strict;
use warnings;
# take the command name off the #ARGV stack
my $command_name = shift;
# get a reference to the subroutine by name
my $command = __PACKAGE__->can($command_name) || die "Unknown command: $command_name\n";
# execute the command, using the rest of #ARGV as arguments
# and print the return with a trailing newline
print $command->(#ARGV);
print "\n";
sub add {
my ($x, $y) = #_;
return $x + $y;
}
sub subtract {
my ($x, $y) = #_;
return $x - $y;
}
This script (say its named myscript.pl) can be called like
$ ./myscript.pl add 2 3
or
$ ./myscript.pl subtract 2 3
Once you have played with that for a while, you might want to take it further and use a framework for this kind of thing. There are several available, like App::Cmd or you can take the logic shown above and modularize as you see fit.
You want to parse command line arguments. A space serves as the delimiter, so just do a ./add.pl 2 3 Something like this:
$num1=$ARGV[0];
$num2=$ARGV[1];
print $num1 + $num2;
will print 5
Here is a short implementation of a simple scripting language.
Each statement is exactly one line long, and has the following structure:
Statement = [<Var> =] <Command> [<Arg> ...]
# This is a regular grammar, so we don't need a complicated parser.
Tokens are seperated by whitespace. A command may take any number of arguments. These can either be the contents of variables $var, a string "foo", or a number (int or float).
As these are Perl scalars, there is no visible difference between strings and numbers.
Here is the preamble of the script:
#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
strict and warnings are essential when learning Perl, else too much weird stuff would be possible. The use 5.010 is a minimum version, it also defines the say builtin (like a print but appends a newline).
Now we declare two global variables: The %env hash (table or dict) associates variable names with their values. %functions holds our builtin functions. The values are anonymous functions.
my %env;
my %functions = (
add => sub { $_[0] + $_[1] },
mul => sub { $_[0] * $_[1] },
say => sub { say $_[0] },
bye => sub { exit 0 },
);
Now comes our read-eval-loop (we don't print by default). The readline operator <> will read from the file specified as the first command line argument, or from STDIN if no filename is provided.
while (<>) {
next if /^\s*\#/; # jump comment lines
# parse the line. We get a destination $var, a $command, and any number of #args
my ($var, $command, #args) = parse($_);
# Execute the anonymous sub specified by $command with the #args
my $value = $functions{ $command }->(#args);
# Store the return value if a destination $var was specified
$env{ $var } = $value if defined $var;
}
That was fairly trivial. Now comes some parsing code. Perl “binds” regexes to strings with the =~ operator. Regexes may look like /foo/ or m/foo/. The /x flags allows us to include whitespace in our regex that doesn't match actual whitespace. The /g flag matches globally. This also enables the \G assertion. This is where the last successful match ended. The /c flag is important for this m//gc style parsing to consume one match at a time, and to prevent the position of the regex engine in out string to being reset.
sub parse {
my ($line) = #_; # get the $line, which is a argument
my ($var, $command, #args); # declare variables to be filled
# Test if this statement has a variable declaration
if ($line =~ m/\G\s* \$(\w+) \s*=\s* /xgc) {
$var = $1; # assign first capture if successful
}
# Parse the function of this statement.
if ($line =~ m/\G\s* (\w+) \s*/xgc) {
$command = $1;
# Test if the specified function exists in our %functions
if (not exists $functions{$command}) {
die "The command $command is not known\n";
}
} else {
die "Command required\n"; # Throw fatal exception on parse error.
}
# As long as our matches haven't consumed the whole string...
while (pos($line) < length($line)) {
# Try to match variables
if ($line =~ m/\G \$(\w+) \s*/xgc) {
die "The variable $1 does not exist\n" if not exists $env{$1};
push #args, $env{$1};
}
# Try to match strings
elsif ($line =~ m/\G "([^"]+)" \s*/xgc) {
push #args, $1;
}
# Try to match ints or floats
elsif ($line =~ m/\G (\d+ (?:\.\d+)? ) \s*/xgc) {
push #args, 0+$1;
}
# Throw error if nothing matched
else {
die "Didn't understand that line\n";
}
}
# return our -- now filled -- vars.
return $var, $command, #args;
}
Perl arrays can be handled like linked list: shift removes and returns the first element (pop does the same to the last element). push adds an element to the end, unshift to the beginning.
Out little programming language can execute simple programs like:
#!my_little_language
$a = mul 2 20
$b = add 0 2
$answer = add $a $b
say $answer
bye
If (1) our perl script is saved in my_little_language, set to be executable, and is in the system PATH, and (2) the above file in our little language saved as meaning_of_life.mll, and also set to be executable, then
$ ./meaning_of_life
should be able to run it.
Output is obviously 42. Note that our language doesn't yet have string manipulation or simple assignment to variables. Also, it would be nice to be able to call functions with the return value of other functions directly. This requires some sort of parens, or precedence mechanism. Also, the language requires better error reporting for batch processing (which it already supports).

How to convert single quoted string to double quoted string so it gets interpreted?

For example I have,
my $str = '\t';
print "My String is ".$str;
I want the output to interpret the tab character and return something like:
"My String is \t"
I am actually getting the value of the string from the database, and it returns it as a single quoted string.
String::Interpolate does exactly that
$ perl -MString::Interpolate=interpolate -E 'say "My String is [". interpolate(shift) . "]"' '\t'
My String is [ ]
'\t' and "\t" are string literals, pieces of Perl code that produces strings ("\","t" and the tab character respectively). The database doesn't return Perl code, so describing the problem in terms of single-quoted literals and double-quoted literals makes no sense. You have a string, period.
The string is formed of the characters "\" and "t". You want to convert that sequence of characters into the tab character. That's a simple substitution.
s/\\t/\t/g
I presume you don't want to deal with just \t. You can create a table of the sequences.
my %escapes = (
"t" => "\t",
"n" => "\n",
"\" => "\\",
);
my $escapes_pat = join('', map quotemeta, keys(%escapes));
$escapes_pat = qr/[$escapes_pat]/;
s/\\($escapes_pat)/$escapes{$1}/g;
You can follow the technique in perlfaq4's answer to How can I expand variables in text strings?:
If you can avoid it, don't, or if you can use a templating system, such as Text::Template or Template Toolkit, do that instead. You might even be able to get the job done with sprintf or printf:
my $string = sprintf 'Say hello to %s and %s', $foo, $bar;
However, for the one-off simple case where I don't want to pull out a full templating system, I'll use a string that has two Perl scalar variables in it. In this example, I want to expand $foo and $bar to their variable's values:
my $foo = 'Fred';
my $bar = 'Barney';
$string = 'Say hello to $foo and $bar';
One way I can do this involves the substitution operator and a double /e flag. The first /e evaluates $1 on the replacement side and turns it into $foo. The second /e starts with $foo and replaces it with its value. $foo, then, turns into 'Fred', and that's finally what's left in the string:
$string =~ s/(\$\w+)/$1/eeg; # 'Say hello to Fred and Barney'
The /e will also silently ignore violations of strict, replacing undefined variable names with the empty string. Since I'm using the /e flag (twice even!), I have all of the same security problems I have with eval in its string form. If there's something odd in $foo, perhaps something like #{[ system "rm -rf /" ]}, then I could get myself in trouble.
To get around the security problem, I could also pull the values from a hash instead of evaluating variable names. Using a single /e, I can check the hash to ensure the value exists, and if it doesn't, I can replace the missing value with a marker, in this case ??? to signal that I missed something:
my $string = 'This has $foo and $bar';
my %Replacements = (
foo => 'Fred',
);
# $string =~ s/\$(\w+)/$Replacements{$1}/g;
$string =~ s/\$(\w+)/
exists $Replacements{$1} ? $Replacements{$1} : '???'
/eg;
print $string;
Well, I just tried below workaround it worked. Please have a look
my $str1 = "1234\n\t5678";
print $str1;
#it prints
#1234
# 5678
$str1 =~ s/\t/\\t/g;
$str1 =~ s/\n/\\n/g;
print $str1;
#it prints exactly the same
#1234\n\t5678

What's an easy way to print a multi-line string without variable substitution in Perl?

I have a Perl program that reads in a bunch of data, munges it, and then outputs several different file formats. I'd like to make Perl be one of those formats (in the form of a .pm package) and allow people to use the munged data within their own Perl scripts.
Printing out the data is easy using Data::Dump::pp.
I'd also like to print some helper functions to the resulting package.
What's an easy way to print a multi-line string without variable substitution?
I'd like to be able to do:
print <<EOL;
sub xyz {
my $var = shift;
}
EOL
But then I'd have to escape all of the $'s.
Is there a simple way to do this? Perhaps I can create an actual sub and have some magic pretty-printer print the contents? The printed code doesn't have to match the input or even be legible.
Enclose the name of the delimiter in single quotes and interpolation will not occur.
print <<'EOL';
sub xyz {
my $var = shift;
}
EOL
You could use a templating package like Template::Toolkit or Text::Template.
Or, you could roll your own primitive templating system that looks something like this:
my %vars = qw( foo 1 bar 2 );
Write_Code(\$vars);
sub Write_Code {
my $vars = shift;
my $code = <<'END';
sub baz {
my $foo = <%foo%>;
my $bar = <%bar%>;
return $foo + $bar;
}
END
while ( my ($key, $value) = each %$vars ) {
$code =~ s/<%$key%>/$value/g;
}
return $code;
}
This looks nice and simple, but there are various traps and tricks waiting for you if you DIY. Did you notice that I failed to use quotemeta on my key names in the substituion?
I recommend that you use a time-tested templating library, like the ones I mentioned above.
You can actually continue a string literal on the next line, like this:
my $mail = "Hello!
Blah blah.";
Personally, I find that more readable than heredocs (the <<<EOL thing mentioned elsewhere).
Double quote " interpolates variables, but you can use '. Note you'll need to escape any ' in your string for this to work.
Perl is actually quite rich in convenient things to make things more readable, e.g. other quote-operations. qq and q correspond to " and ' and you can use whatever delimiter makes sense:
my $greeting = qq/Hello there $name!
Nice to meet you/; # Interpolation
my $url = q|http://perlmonks.org/|; # No need to escape /
(note how the syntax coloring here didn't quite keep up)
Read perldoc perlop (find in page: "Quote and Quote-like Operators") for more information.
Use a data section to store the Perl code:
#!/usr/bin/perl
use strict;
use warnings;
print <DATA>;
#print munged data
__DATA__
package MungedData;
use strict;
use warnings;
sub foo {
print "foo\n";
}
Try writing your code as an actual perl subroutine, then using B::Deparse to get the source code at runtime.