how should I use system command in perl - perl

I want to replace a line with the $var.
runnew has
input= input/old/should/change;
replace= input/old/replace;
other = input/old/other;
replace_other= input/old/replace_other;
My output file should look like,
input= input/old/should/New;
replace= input/old/New_replace;
other = input/old/New_other;
replace_other= input/old/New_replace_other;
I want to replace "input =" by input = input/old/should/New;
I have used like,
if ($#ARGV != 0) {
die "\n********USAGE <cellname> <tech>********\n";
}
$newinput=$ARGV[0];
open(my $fh, "$runnew") or die "Could not open rerun.txt: $!";
while (<$fh>) {
system ( sed -i "/input=.*/c\input= $newinput" $runnew );
}
But there is error popping up "Scalar found where operator expected at run.pl" and its displaying sed line and asking " (Missing operator before $runnew?)."
When I used same sed on terminal its replacing the line .
Please can anyone point out where the error is ?
Yes Using Sed is simple but I have file with different lines and each line should be replaced .
Please let me knoe if you have better idea than this.
Thanks in advance.

system() takes a list of strings as its argument. You you need to put quotes around the command you pass it.
system ( "sed -i '/input=.*/c\input= $newinput' $runnew" );
But your code still looks very strange. You're running exactly the same sed command for every line in the input file. Is that what you meant to do?
It's not really clear what you're trying to do here. But I'm confident that the best approach would involve not using sed and using Perl to make your transformations.

Why do you want to call sed at all? Your requirement can be much easier handled in Perl directly:
add -i.bak to enable in-place replacement mode
use the first command line parameter as replacement string
remove it from the #ARGV array so it will not be interpreted as file
loop over all files on the command line
read line by line
apply substitution
print result
Perl automatically takes care of opening the files, writing to the correct file and renaming old files to .bak.
#!/usr/bin/perl -i.bak
use warnings;
use strict;
my($replacement) = shift(#ARGV);
while (<>) {
s/input=.*/input= $replacement/;
print;
}
exit 0;
Test run (taking an educated guess on your input data):
$ cat dummy1.txt.bak
input= test1
input= test2
$ cat dummy2.txt.bak
input= test3
input= test4
$ perl dummy.pl REPLACEMENT dummy1.txt dummy2.txt
$ cat dummy1.txt
input= REPLACEMENT
input= REPLACEMENT
$ cat dummy2.txt
input= REPLACEMENT
input= REPLACEMENT
or to use the contents of the file "rerun.txt":
$ perl dummy.pl REPLACEMENT $(cat rerun.txt)

Related

Linux shell: change Perl code to linux shell, grep line by line

The follwoing code is Perl script, grep lines with 'Stage' from hostlog. and then line by line match the content with regex, if find add the count by 1:
$command = 'grep \'Stage \' '. $hostlog;
#stage_info = qx($command);
foreach (#stage_info) {
if ( /Stage\s(\d+)\s(.*)/ ) {
$stage_number = $stage_number+1;
}
}
so how to do this in linux shell? Based on my test, the we can not loop line by line, since there is space inside.
That is a horrible piece of Perl code you've got there. Here's why:
It looks like you are not using use strict; use warnings;. That is a huge mistake, and will not prevent errors, it will just hide them.
Using qx() to grep lines from a file is a completely redundant thing to do, as this is what Perl does best itself. "Shelling out" a process like that most often slows your program down.
Use some whitespace to make your code readable. This is hard to read, and looks more complicated than it is.
You capture strings by using parentheses in your regex, but you never use these strings.
Re: $stage_number=$stage_number+1, see point 3. And also, this can be written $stage_number++. Using the ++ operator will make your code clearer, will prevent the uninitialized warnings, and save you some typing.
Here is what your code should look like:
use strict;
use warnings;
open my $fh, "<", $hostlog or die "Cannot open $hostlog for reading: $!";
while (<$fh>) {
if (/Stage\s\d+/) {
$stage_number++;
}
}
You're not doing anything with the internal captures, so why bother? You could do everything with a grep:
$ stage_number=$(grep -E 'Stage\s\d+\s' | wc -l)
This is using extended regular expressions. I believe the GNU version takes these without a -E parameter, and in Solaris, even the egrep command might not quite allow for this regular expression.
If there's something more you have to do, you've got to explain it in your question.
If I understand the issue correctly, you should be able to do this just fine in the shell:
while read; do
if echo ${REPLY} | grep -q -P "'Stage' "; then
# Do what you need to do
fi
done < test.log
Note that if your grep command supports the -P option you may be able to use the Perl regular expression as-is for the second test.
this is almost it. bash has no expression for multiple digits.
#!/bin/bash
command=( grep 'Stage ' "$hostlog" )
while read line
do
[ "$line" != "${line/Stage [0-9]/}" ] && (( ++stage_number ))
done < <( "${command[#]}" )
On the other hand taking the function of the perl script into account rather than the operations it performs the whole thing could be rewritten as
(( stage_number += ` grep -c 'Stage \d\+\s' "$hostlog" ` ))
or this
stage_number=` grep -c 'Stage \d\+\s' "$hostlog" `
if, in the original perl, stage_number is uninitialised, or is initalised to 0.

SED command not working in perl script

When i am using "sed" in command line it is working but not when included in perl script.
An example is sed 's/\s+//g' aaa > bbb
but say when i am trying to call the same command through perl script
$gf = `sed 's/\s\+//g' aaa > bbb` ;
the output file remains same as the input file!!!! Please suggest.
In Perl, backticks have the same escape and interpolation rules as double quoted strings: A backslash forming an unknown escape code forgets the backslash, e.g. "\." eq ".".
Therefore, the Perl code
print `echo \"1\"`;
print `echo \\"1\\"`;
outputs
1
"1"
If you want to embed that sed command into Perl, you have to escape the backslashes so that they even reach the shell:
$gf = `sed 's/\\s\\+//g' aaa > bbb`;
Actually, you won't get any output into $gf as you redirect the output to a file. We could just do
use autodie;
system "sed 's/\\s\\+//g' aaa > bbb";
or with single quotes:
use autodie;
system q{ sed 's/\s\+//g' aaa > bbb };
which keeps the backslashes.
Still, this is quite unneccessary as Perl could apply the substitution itself.
use autodie; # automatic error handling
open my $out, ">", "bbb";
open my $in, "<", "aaa";
while (<$in>) {
s/\s\+//g; # remove all spaces followed by a plus
print {$out} $_;
}
In these weird situations, I ensure that I'm running the right command. I'll construct it, store it, and output the command so I can see exactly what I created:
my $command = '....';
print "Command is [$command]\n";
my $output = `$command`;
If you're running sed from Perl, you might be doing it wrong since Perl can already do all that.
have you got
use strict;
use warnings;
at the top of your file?
you could need backticks to execute the command
$gf = `sed 's/\s\+//g' aaa > bbb`;

How do insert lines of text into file after a particular line in unix [duplicate]

This question already has answers here:
How do I add a line of text to the middle of a file using bash?
(6 answers)
Closed 10 years ago.
How do insert lines of text into file after a particular line in unix ?
Background: The file is an autogenerated textfile but I manually have to edit it every time it is regenerated to add in 4 additional lines after a particular line. I can gurantee that this line will always be in the file but I cannot guarantee excalty what line it will be on so I want the additional lines to be added on the basis of the position of this line rather than adding to a fixed rownumber. I want to automate this process as it is part of my build process.
I'm using Mac OSX so I can make use of unix comand line tools, but Im not very familiar with such tools and cannot work out how to do this.
EDIT
Thanks for the solution, although I havent managed to get them working yet:
I tried Sed solution
sed -i '/<string>1.0</string>/ a <key>CFBundleHelpBookFolder</key>\
<string>SongKongHelp</string>\
<key>CFBundleHelpBookName</key>\
<string>com.jthink.songkong.Help</string>
' /Applications/SongKong.app/Contents/Info.plist
but get error
sed: 1: "/Applications/SongKong. ...": invalid command code S
and I tried the bash solution
#!/bin/bash
while read line; do
echo "$line"
if [[ "$line" = "<string>1.0</string>"]]; then
cat mergefile.txt # or echo or printf your extra lines
fi
done < /Applications/SongKong.app/Contents/Info.plist
but got error
./buildosx4.sh: line 5: syntax error in conditional expression: unexpected token `;'
./buildosx4.sh: line 5: syntax error near `;'
./buildosx4.sh: line 5: ` if [[ "$line" = "<string>1.0</string>"]]; then'
EDIT 2
Now working, i was missing a space
#!/bin/bash
while read line; do
echo "$line"
if [[ "$line" = "<string>1.0</string>" ]]; then
cat mergefile.txt # or echo or printf your extra lines
fi
done < /Applications/SongKong.app/Contents/Info.plist
Assuming the marker line contains fnord and nothing else;
awk '1;/^fnord$/{print "foo"; print "bar";
print "baz"; print "quux"}' input >output
Another way to look at this is that you want to merge two files at some point in one of the files. If your extra four lines were in a separate file, you could make a more generic tool like this:
#!/usr/bin/awk
BEGIN {
SEARCH=ARGV[1]; # Get the search string from the command line
delete ARGV[1]; # Delete the argument, so subsequent arguments are files
}
# Collect the contents of the first file into a variable
NR==FNR {
ins=ins $0 "\n";
next;
}
1 # print every line in the second (or rather the non-first) file
# Once we're in the second file, if we see the match, print stuff...
$0==SEARCH {
printf("%s", ins); # Using printf instead of print to avoid the extra newline
next;
}
I've spelled this out for ease of documentation; you could obviously shorten it to something that looked more like triplee's answer. You'd invoke this like:
$ scriptname "Text to match" mergefile.txt origfile.txt > outputfile.txt
Done this way, you'd have a tool that could be used to achieve this kind of merge on different files and with different text.
Alternately, you could of course do this in pure bash.
#!/bin/bash
while read line; do
echo "$line"
if [[ "$line" = "matchtext" ]]; then
cat mergefile.txt # or echo or printf your extra lines
fi
done < origfile.txt
The problem can be solved efficiently for any filesize by this algorithm:
Read each line from the original file and print it to a tempfile.
If the last line was the marker line, print your insertion lines to the tempfile
Print the remaining lines
Rename the tempfile to the original filename.
As a Perl script:
#!perl
use strict; use warnings;
$^I = ".bak"; # create a backup file
while (<>) {
print;
last if /regex to determine if this is the line/;
}
print <<'END';
Yourstuff
END
print while <>; # print remaining lines
# renaming automatically done.
Testfile:
foo
bar
baz
qux
quux
Regex is /^ba/.
Usage: $ perl this_script.pl source-file
The testfile after processing:
foo
bar
Yourstuff
baz
qux
quux
use the sed 'a command with a regex for the line you need to match
sed -i '/the target line looks like this/ a this is line 1\
this is line 2\
this is line 3\
this is line 4
' FILENAME

How do I use perl like sed?

I have a file that has some entries like
--ERROR--- Failed to execute the command with employee Name="shayam" Age="34"
--Successfully executed the command with employee Name="ram" Age="55"
--ERROR--- Failed to execute the command with employee Name="sam" Age="23"
--ERROR--- Failed to execute the command with employee Name="yam" Age="3"
I have to extract only the Name and Age of those for whom the command execution was failed.
in this case i need to extract shayam 34 sam 23 yam 3. I need to do this in perl. thanks a lot..
perl -p -e 's/../../g' file
Or to inline replace:
perl -pi -e 's/../../g' file
As a one-liner:
perl -lne '/^--ERROR---.*Name="(.*?)" Age="(.*?)"/ && print "$1 $2"' file
Your title makes it not clear. Anyway...
while(<>) {
next if !/^--ERROR/;
/Name="([^"]+)"\s+Age="([^"]+)"/;
print $1, " ", $2, "\n";
}
can do it reading from stdin; of course, you can change the reading loop to anything else and the print with something to populate an hash or whatever according to your needs.
As a one liner, try:
perl -ne 'print "$1 $2\n" if /^--ERROR/ && /Name="(.*?)"\s+Age="(.*?)"/;'
This is a lot like using sed, but with Perl syntax.
The immediate question of "how do I use perl like sed?" is best answered with s2p, the sed to perl converter. Given the command line, "sed $script", simply invoke "s2p $script" to generate a (typically unreadable) perl script that emulates sed for the given set of commands.
Refer to comments :
my #a = <>; # Reading entire file into an array
chomp #a; # Removing extra spaces
#a = grep {/ERROR/} #a; # Removing lines that do not contain ERROR
# mapping with sed-like regexp to keep only names and ages :
#a = map {s/^.*Name=\"([a-z]+)\" Age=\"([0-9]+)\".*$/$1 $2/; $_} #a;
print join " ",#a; # print of array content

How can I print just a unix newline in Perl on Win32?

By default, perl prints \r\n in a win32 environment. How can I override this? I'm using perl to make some changes to some source code in a repository, and I don't want to change all the newline characters.
I tried changing the output record separator but with no luck.
Thanks!
Edit: Wanted to include a code sample - I'm doing a search and replace over some files that follow a relatively straightforward pattern like this:
#!/usr/bin/perl
# test.pl
use strict;
use warnings;
$/ = undef;
$\ = "\n";
$^I=".old~";
while (<>) {
while (s/hello/world/) {
}
print;
}
This should replace any instances of "hello" with "world" for any files passed on the cmd line.
Edit 2: I tried the binmode as suggested without any luck initially. I delved a bit more and found that $^I (the inplace edit special variable) was overriding binmode. Any work around to still be able to use the inplace edit?
Edit 3: As Sinan points out below, I needed to use binmode ARGVOUT with $^I instead of binmode STDOUT in my example. Thanks.
Printing "\n" to a filehandle on Windows emits, by default, a CARRIAGE RETURN ("\015") followed by a LINE FEED ("\012") character because that the standard newline sequence on Windows.
This happens transparently, so you need to override it for the special filehandle ARGVOUT (see perldoc perlvar):
#!/usr/bin/perl -i.bak
use strict; use warnings;
local ($\, $/);
while (<>) {
binmode ARGVOUT;
print;
}
Output:
C:\Temp> xxd test.txt
0000000: 7465 7374 0d0a 0d0a test....
C:\Temp> h test.txt
C:\Temp> xxd test.txt
0000000: 7465 7374 0a0a test..
See also perldoc open, perldoc binmode and perldoc perliol (thanks daotoad).
Does binmode( STDOUT ) work?
Re: your question about the binmode being lost when $^I opens a new output handle, you could solve this with the open pragma:
use open OUT => ':raw';
which will force all filehandles opened for writing to have the ':raw' PerlIO layer (equivalent to binmode with no argument) to apply to them. Just take care if you're opening anything else for output that you apply :crlf or any other layer as needed.
The data you are reading in contains line endings, so you're getting them back out again. You can strip them off yourself with chomp, then add your own ending back, provided you have set binmode as Sinan describes::
while (<>) {
binmode;
chomp; # strip off \r\n
while (s/search/replace/) {
# ...
}
print;
print "\n"; # add your own line ending back
}
By default, perl prints \r\n in a win32 environment. How can I override this?
I ended up creating my own file and setting binmode(fh) specifically. I could not get STDOUT (or ARGVOUT) to work reliably under both Windows 10 using perl 5.8.8 and Windows 7 with perl 5.14.4.
perl -e 'open(fh, ">x"); binmode(fh); print fh "\n";' ; od -c x
0000000 \n
Sometimes the binmode(fh) was needed here and sometimes it seemed to be the default.
I could not get binmode(STDOUT) to be work reliably. Some of the following did output just \n under Windows:
perl -e 'binmode(ARGVOUT); print "\n";' | od -c
perl -e 'binmode(STDOUT); print "\n";' | od -c
perl -e 'binmode(STDOUT); syswrite(STDOUT, "\n");' | od -c
... but then not when the output was going to a file. The following still spat out \r \n.
perl -e 'binmode(STDOUT); print "\n";' > x ; od -c x
perl -e 'binmode(ARGVOUT); print "\n";' > x ; od -c x
Interestingly, the following worked when piping to cat which then writes to a file. Perl must be seeing if STDOUT is a terminal, file, or pipe and enabling the cr-lf layer or not. Why a pipe is binary but a file is not is an interesting decision. There are also differences between running perl interactively from the command-line and running it from a script with the same args and redirects.
perl -e 'binmode(STDOUT); print "\n";' | cat > x ; od -c x
Noticed that I tried print and syswrite. I was surprised that syswrite didn't give me a direct layer to the file-handle. I also tried to copy the STDOUT file-handle and set binmode on that new file-handle but that didn't work either. PERLIO environmental variable didn't help either. The use out => ":raw"; worked under Windows 10 perl 5.8.8 but not Windows 7 perl 5.14.4 when redirected to an output file.
Btw, I wasn't doing a print "\n"; in my code when I stumbled over this problem. I was doing a print of pack("c", $num); where $num happened to be 10. Imagine my surprise when my binary file was corrupted by \rs.
Porting sucks!
A unix newline is a LINEFEED character, which is ASCII code 10.
print "\012";