By default, perl prints \r\n in a win32 environment. How can I override this? I'm using perl to make some changes to some source code in a repository, and I don't want to change all the newline characters.
I tried changing the output record separator but with no luck.
Thanks!
Edit: Wanted to include a code sample - I'm doing a search and replace over some files that follow a relatively straightforward pattern like this:
#!/usr/bin/perl
# test.pl
use strict;
use warnings;
$/ = undef;
$\ = "\n";
$^I=".old~";
while (<>) {
while (s/hello/world/) {
}
print;
}
This should replace any instances of "hello" with "world" for any files passed on the cmd line.
Edit 2: I tried the binmode as suggested without any luck initially. I delved a bit more and found that $^I (the inplace edit special variable) was overriding binmode. Any work around to still be able to use the inplace edit?
Edit 3: As Sinan points out below, I needed to use binmode ARGVOUT with $^I instead of binmode STDOUT in my example. Thanks.
Printing "\n" to a filehandle on Windows emits, by default, a CARRIAGE RETURN ("\015") followed by a LINE FEED ("\012") character because that the standard newline sequence on Windows.
This happens transparently, so you need to override it for the special filehandle ARGVOUT (see perldoc perlvar):
#!/usr/bin/perl -i.bak
use strict; use warnings;
local ($\, $/);
while (<>) {
binmode ARGVOUT;
print;
}
Output:
C:\Temp> xxd test.txt
0000000: 7465 7374 0d0a 0d0a test....
C:\Temp> h test.txt
C:\Temp> xxd test.txt
0000000: 7465 7374 0a0a test..
See also perldoc open, perldoc binmode and perldoc perliol (thanks daotoad).
Does binmode( STDOUT ) work?
Re: your question about the binmode being lost when $^I opens a new output handle, you could solve this with the open pragma:
use open OUT => ':raw';
which will force all filehandles opened for writing to have the ':raw' PerlIO layer (equivalent to binmode with no argument) to apply to them. Just take care if you're opening anything else for output that you apply :crlf or any other layer as needed.
The data you are reading in contains line endings, so you're getting them back out again. You can strip them off yourself with chomp, then add your own ending back, provided you have set binmode as Sinan describes::
while (<>) {
binmode;
chomp; # strip off \r\n
while (s/search/replace/) {
# ...
}
print;
print "\n"; # add your own line ending back
}
By default, perl prints \r\n in a win32 environment. How can I override this?
I ended up creating my own file and setting binmode(fh) specifically. I could not get STDOUT (or ARGVOUT) to work reliably under both Windows 10 using perl 5.8.8 and Windows 7 with perl 5.14.4.
perl -e 'open(fh, ">x"); binmode(fh); print fh "\n";' ; od -c x
0000000 \n
Sometimes the binmode(fh) was needed here and sometimes it seemed to be the default.
I could not get binmode(STDOUT) to be work reliably. Some of the following did output just \n under Windows:
perl -e 'binmode(ARGVOUT); print "\n";' | od -c
perl -e 'binmode(STDOUT); print "\n";' | od -c
perl -e 'binmode(STDOUT); syswrite(STDOUT, "\n");' | od -c
... but then not when the output was going to a file. The following still spat out \r \n.
perl -e 'binmode(STDOUT); print "\n";' > x ; od -c x
perl -e 'binmode(ARGVOUT); print "\n";' > x ; od -c x
Interestingly, the following worked when piping to cat which then writes to a file. Perl must be seeing if STDOUT is a terminal, file, or pipe and enabling the cr-lf layer or not. Why a pipe is binary but a file is not is an interesting decision. There are also differences between running perl interactively from the command-line and running it from a script with the same args and redirects.
perl -e 'binmode(STDOUT); print "\n";' | cat > x ; od -c x
Noticed that I tried print and syswrite. I was surprised that syswrite didn't give me a direct layer to the file-handle. I also tried to copy the STDOUT file-handle and set binmode on that new file-handle but that didn't work either. PERLIO environmental variable didn't help either. The use out => ":raw"; worked under Windows 10 perl 5.8.8 but not Windows 7 perl 5.14.4 when redirected to an output file.
Btw, I wasn't doing a print "\n"; in my code when I stumbled over this problem. I was doing a print of pack("c", $num); where $num happened to be 10. Imagine my surprise when my binary file was corrupted by \rs.
Porting sucks!
A unix newline is a LINEFEED character, which is ASCII code 10.
print "\012";
Related
I want to replace a line with the $var.
runnew has
input= input/old/should/change;
replace= input/old/replace;
other = input/old/other;
replace_other= input/old/replace_other;
My output file should look like,
input= input/old/should/New;
replace= input/old/New_replace;
other = input/old/New_other;
replace_other= input/old/New_replace_other;
I want to replace "input =" by input = input/old/should/New;
I have used like,
if ($#ARGV != 0) {
die "\n********USAGE <cellname> <tech>********\n";
}
$newinput=$ARGV[0];
open(my $fh, "$runnew") or die "Could not open rerun.txt: $!";
while (<$fh>) {
system ( sed -i "/input=.*/c\input= $newinput" $runnew );
}
But there is error popping up "Scalar found where operator expected at run.pl" and its displaying sed line and asking " (Missing operator before $runnew?)."
When I used same sed on terminal its replacing the line .
Please can anyone point out where the error is ?
Yes Using Sed is simple but I have file with different lines and each line should be replaced .
Please let me knoe if you have better idea than this.
Thanks in advance.
system() takes a list of strings as its argument. You you need to put quotes around the command you pass it.
system ( "sed -i '/input=.*/c\input= $newinput' $runnew" );
But your code still looks very strange. You're running exactly the same sed command for every line in the input file. Is that what you meant to do?
It's not really clear what you're trying to do here. But I'm confident that the best approach would involve not using sed and using Perl to make your transformations.
Why do you want to call sed at all? Your requirement can be much easier handled in Perl directly:
add -i.bak to enable in-place replacement mode
use the first command line parameter as replacement string
remove it from the #ARGV array so it will not be interpreted as file
loop over all files on the command line
read line by line
apply substitution
print result
Perl automatically takes care of opening the files, writing to the correct file and renaming old files to .bak.
#!/usr/bin/perl -i.bak
use warnings;
use strict;
my($replacement) = shift(#ARGV);
while (<>) {
s/input=.*/input= $replacement/;
print;
}
exit 0;
Test run (taking an educated guess on your input data):
$ cat dummy1.txt.bak
input= test1
input= test2
$ cat dummy2.txt.bak
input= test3
input= test4
$ perl dummy.pl REPLACEMENT dummy1.txt dummy2.txt
$ cat dummy1.txt
input= REPLACEMENT
input= REPLACEMENT
$ cat dummy2.txt
input= REPLACEMENT
input= REPLACEMENT
or to use the contents of the file "rerun.txt":
$ perl dummy.pl REPLACEMENT $(cat rerun.txt)
I'm trying to run a tail command from within a perl script using the usual backticks.
The section in my perl script is as follows:
$nexusTime += nexusUploadTime(`tail $log -n 5`);
So I'm trying to get the last 5 lines of this file but I'm getting the following error when the perl script finishes:
sh: line 1: -n: command not found
Even though when I run the command on the command line it is indeed successful and I can see the 5 lines from that particular.
Not sure what is going on here. Why it works from command line but through perl it won't recognize the -n option.
Anybody have any suggestions?
$log has an extraneous trailing newline, so you are executing
tail file.log
-n 5 # Tries to execute a program named "-n"
Fix:
chomp($log);
Note that you will run into problems if log $log contains shell meta characters (such as spaces). Fix:
use String::ShellQuote qw( shell_quote );
my $tail_cmd = shell_quote('tail', '-n', '5', '--', $log);
$nexusTime += nexusUploadTime(`$tail_cmd`);
ikegami pointed out your error, but I would recommend avoiding external commands whenever possible. They aren't portable and debugging them can be a pain, among other things. You can simulate tail with pure Perl code like this:
use strict;
use warnings;
use File::ReadBackwards;
sub tail {
my ($file, $num_lines) = #_;
my $bw = File::ReadBackwards->new($file) or die "Can't read '$file': $!";
my ($lines, $count);
while (defined(my $line = $bw->readline) && $num_lines > $count++) {
$lines .= $line;
}
$bw->close;
return $lines;
}
print tail('/usr/share/dict/words', 5);
Output
ZZZ
zZt
Zz
ZZ
zyzzyvas
Note that if you pass a file name containing a newline, this will fail with
Can't read 'foo
': No such file or directory at tail.pl line 10.
instead of the more cryptic
sh: line 1: -n: command not found
that you got from running the tail utility in backticks.
The answer to this question is to place the option -n 5 before the target file
I am trying to execute below awk command inside a perl script, but it is failing.
#!/usr/bin/perl
print `awk -F, '{print $NF}' f1.txt > f2.txt`
This is the error:
syntax error at ./MO.pl line 3, near "print"
Execution of ./MO.pl aborted due to compilation errors.
Can anyone please help what I am doing wrong here?
This is a Perl error and has nothing to do with your awk script itself. The error is usually seen when the previous statement doesn't have a semicolon at the end.
Here's a very simple program (which should include use strict; and use warnings;, but I wanted to emulate what you have).
#! /usr/bin/env perl
#
print "Hello, World\n" # Line 4
print "Hello, Back\n"; # Line 5
And the error message is:
syntax error at test.pl line 5, near "print"
Execution of test.pl aborted due to compilation errors.
Note the error is near the print in Line #5, but the error is actually at the end of Line #4 where I'm missing a semicolon.
Running your exact program works on my system (although doesn't quite produce the results you want). I am assuming this isn't your exact program, but instead a simplification of your program. Is there a statement before that print?
Several other things:
You're redirecting your awk output, so there's nothing to print.
Use strict and warnings.
Better to use qx(....) than backticks (grave accent). It's more readable and allows you to do quoted executable in quoted executable.
Watch for Perlisms in your code. The $NF is interpreted by Perl, and without the use strict;, doesn't give you an error. Instead, the print in your Awk statement is a null print which prints the entire line.
Why do you use print if nothing is printing out? You're better off in this position to use system which allows you to put single quotes around your entire statement:
system q(awk -F, '{print $NF}' f1.txt > f2.txt);
This way, $NF doesn't have to be quoted.
Why are you doing Awk in a Perl program? Perl will do anything Awk will do and do it better:
Here's a version of your program using plain ol' Perl:
#! /usr/bin/env perl
use strict;
use warnings;
use autodie;
while ( my $line = <> ) {
my #array = split /\s*,\s*/, $line;
print $array[-1];
}
To run this program:
$ test.pl f1.txt > f2.txt
Yes, it's longer, but half of the program is taken up by pragmas that all Perl programs should use.
I'm sure people smarter than me can turn this into a Perl one-liner.
Since you're redirecting the awk output, there's nothing for perl to print. You might as well use system and the quoting operator q():
system q(awk -F, '{print $NF}' f1.txt > f2.txt)
Or, of course, do it in perl, which saves you from having to spawn a shell and then spawn awk:
open my $in, '<', 'f1.txt';
open my $out, '>', 'f1.txt';
while (<$in>) {
print $out (split " ")[-1], "\n";
}
close $in;
close $out;
If there are more lines in the script, you need a semi-colon at the end of the print statement.
When i am using "sed" in command line it is working but not when included in perl script.
An example is sed 's/\s+//g' aaa > bbb
but say when i am trying to call the same command through perl script
$gf = `sed 's/\s\+//g' aaa > bbb` ;
the output file remains same as the input file!!!! Please suggest.
In Perl, backticks have the same escape and interpolation rules as double quoted strings: A backslash forming an unknown escape code forgets the backslash, e.g. "\." eq ".".
Therefore, the Perl code
print `echo \"1\"`;
print `echo \\"1\\"`;
outputs
1
"1"
If you want to embed that sed command into Perl, you have to escape the backslashes so that they even reach the shell:
$gf = `sed 's/\\s\\+//g' aaa > bbb`;
Actually, you won't get any output into $gf as you redirect the output to a file. We could just do
use autodie;
system "sed 's/\\s\\+//g' aaa > bbb";
or with single quotes:
use autodie;
system q{ sed 's/\s\+//g' aaa > bbb };
which keeps the backslashes.
Still, this is quite unneccessary as Perl could apply the substitution itself.
use autodie; # automatic error handling
open my $out, ">", "bbb";
open my $in, "<", "aaa";
while (<$in>) {
s/\s\+//g; # remove all spaces followed by a plus
print {$out} $_;
}
In these weird situations, I ensure that I'm running the right command. I'll construct it, store it, and output the command so I can see exactly what I created:
my $command = '....';
print "Command is [$command]\n";
my $output = `$command`;
If you're running sed from Perl, you might be doing it wrong since Perl can already do all that.
have you got
use strict;
use warnings;
at the top of your file?
you could need backticks to execute the command
$gf = `sed 's/\s\+//g' aaa > bbb`;
I've scripted up a simple ksh that calls a Perl program to find and replace in files.
The passed-in arg is the home directory:
perl -pi -e 's/find/replace/g' $1/*.html
It works great. However, I'd like to output all the changes to a log file. I've tried piping and redirecting and haven't been able to get it work. Any ideas?
Thanks,
Glenn
Something like this to send all changes to STDERR:
perl -pi -e '$old = $_; s/find/replace/g and warn "${ARGV}[$.]: $old $_"; close ARGV if eof' $1/*.html
Updated: Fixed $. on multiple files.
You can print to STDERR and redirect just the STDERR output to a file as below:
perl -pi -e 'chomp($prev=$_);s/find/replace/g and print STDERR "$ARGV - $.: $prev -> $_"; close ARGV if eof' $1/*.html 2> logfile.txt
edit: added the filename, and fixed line number display when multiple input files are used