I'm trying to edit a text file that looks like this:
TYPE=Ethernet
HWADDR=00:....
IPV6INIT=no
MTU=1500
IPADDR=192.168.2.247
...
(Its actually the /etc/sysconfig/network-scripts/ifcfg- file on red hat Linux)
Instead of reading and rewriting the file each time I want to modify it, I figured I could use grep, sed, awk or the native text parsing functionality provided in Perl.
For instance, if I wanted to change the IPADDR field of the file, is there a way I can just retrieve and modify the line directly? Maybe something like
grep 'IPADDR=' <filename>
but add some additional arguments to modify that line? I'm a little new to UNIX based text processing languages so bear with me...
Thanks!
Here's a Perl oneliner to replace the IPADDR value with the IP address 127.0.01. It's short enough that you should be able to see what you need to modify to alter other fields*:
perl -p -i.orig -e 's/^IPADDR=.*$/IPADDR=127.0.0.1/' filename
It will rename "filename" to "filename.orig", and write out the new version of the file into "filename".
Perl command-line options are explained at perldoc perlrun (thanks for the reminder toolic!), and the syntax of perl regular expressions is at perldoc perlre.
*The regular expression ^IPADDR=.*$, split into components, means:
^ # bind to the beginning of the line
IPADDR= # plain text: match "IPADDR="
.* # followed by any number of any character (`.` means "any one character"; `*` means "any number of them")
$ # bind to the end of the line
since you are on redhat, you can try using the shell
#!/bin/bash
file="file"
read -p "Enter field to change: " field
read -p "Enter new value: " newvalue
shopt -s nocasematch
while IFS="=" read -r f v
do
case "$f" in
$field)
v=$newvalue;;
esac
echo "$f=$v"
done <$file > temp
mv temp file
UPDATE:
file="file"
read -p "Enter field to change: " field
read -p "Enter new value: " newvalue
shopt -s nocasematch
EOL=false
IFS="="
until $EOL
do
read -r f v || EOL=true
case "$f" in
$field)
v=$newvalue;;
esac
echo "$f=$v"
done <$file #> temp
#mv temp file
OR , using just awk
awk 'BEGIN{
printf "Enter field to change: "
getline field < "-"
printf "Enter new value: "
getline newvalue <"-"
IGNORECASE=1
OFS=FS="="
}
field == $1{
$2=newvalue
}
{
print $0 > "temp"
}END{
cmd="mv temp "FILENAME
system(cmd)
}' file
Or with Perl
printf "Enter field: ";
chomp($field=<STDIN>);
printf "Enter new value: ";
chomp($newvalue=<STDIN>);
while (<>){
my ( $f , $v ) = split /=/;
if ( $field =~ /^$f/i){
$v=$newvalue;
}
print join("=",$f,$v);
}
That would be the 'ed' command line editor, like sed but will put the file back where it came from.
Related
I have a file which contains each users userid and password. I need to fetch userid and password from that file by passing userid as an search element using awk command.
user101,smith,smith#123
user102,jones,passj#007
user103,albert,albpass#01
I am using a awk command inside my perl script like this:
...
...
my $userid = ARGV[0];
my $user_report_file = "report_file.txt";
my $data = `awk -F, '$1 ~ /$userid/ {print $2, $3}' $user_report_file`;
my ($user,$pw) = split(" ",$data);
...
...
Here I am getting the error:
awk: ~ /user101/ {print , }
awk: ^ syntax error
But if I run same command in terminal window its able to give result like below:
$] awk -F, '$1 ~ /user101/ {print $2, $3}' report_file.txt
smith smith#123
What could be the issue here?
The backticks are a double-quoted context, so you need to escape any literal $ that you want awk to interpret.
my $data = `awk -F, '\$1 ~ /$userid/ {print \$2, \$3}' $user_report_file`;
If you don't do that, you're interpolating the capture variables from the last successful Perl match.
When I have these sorts of problems, I try the command as a string first to see if it is what I expect:
my $data = "awk -F, '\$1 ~ /$userid/ {print \$2, \$3}' $user_report_file";
say $data;
Here's the Perl equivalent of that command:
$ perl -aF, -e '$F[0]=~/101/ && print "#F[1,2]"' report_file
But, this is something you probably want to do in Perl instead of creating another process:
Interpolating data into external commands can go wrong, such as a filename that is foo.txt; rm -rf /.
The awk you run is the first one in the path, so someone can make that a completely different program (so use the full path, like /usr/bin/awk).
Taint checking can tell you when you are passing unsanitized data to the shell.
Inside a program you don't get all the shortcuts, but if this is the part of your program that is slow, you probably want to rethink how you are accessing this data because scanning the entire file with any tool isn't going to be that fast:
open my $fh, '<', $user_report_file or die;
while( <$fh> ) {
chomp;
my #F = split /,/;
next unless $F[0] =~ /\Q$userid/;
print "#F[1,2]";
last; # if you only want the first one
}
Basically right now I have a for loop running that runs a series of tests. Once the tests pass I input the results into a csv file:
for (( some statement ))
do
if[[ something ]]
input this value into a specific row and column
fi
done
What I can't figure out right now is how to input a specific value into a specific cell in the csv file. I know in awk you can read a cell with this command:
awk -v "row=2" -F'#' 'NR == row { print $2 }' some.csv and this will print the cell in the 2nd row and 2nd column. I need something similar to this except it can input a value into a specific cell instead of read it. Is there a function that does this?
You can use the following:
awk -v value=$value -v row=$row -v col=$col 'BEGIN{FS=OFS="#"} NR==row {$col=value}1' file
And set the bash values $value, $row and $col. Then you can redirect and move to the original:
awk ... file > new_file && mv new_file file
This && means that just if the first command (awk...) is executed successfully, then the second one will be performed.
Explanation
-v value=$value -v row=$row -v col=$col pass the bash variables to awk. Note value, row and col could be other names, I just used the same as bash to make it easier to understand.
BEGIN{FS=OFS="#"} set the Field Separator and Output Field Separator to be #. The OFS="#" is not necessary here, but can be useful in case you do some print.
NR==row {$col=value} when the number of record (number of line here) is equal to row, then set the col column with value value.
1 perform the default awk action: {print $0}.
Example
$ cat a
hello#how#are#you
i#am#fine#thanks
hoho#haha#hehe
$ row=2
$ col=3
$ value="XXX"
$ awk -v value=$value -v row=$row -v col=$col 'BEGIN{FS=OFS="#"} NR==row {$col=value}1' a
hello#how#are#you
i#am#XXX#thanks
hoho#haha#hehe
Your question has a 'perl' tag so here is a way to do it using Tie::Array::CSV which allows you to treat the CSV file as an array of arrays and use standard array operations:
use strict;
use warnings;
use Tie::Array::CSV;
my $row = 2;
my $col = 3;
my $value = 'value';
my $filename = '/path/to/file.csv';
tie my #file, 'Tie::Array::CSV', $filename, sep_char => '#';
$file[$row][$col] = $value;
untie #file;
using sed
row=2 # define the row number
col=3 # define the column number
value="value" # define the value you need change.
sed "$row s/[^#]\{1,\}/$value/$col" file.csv # use shell variable in sed to find row number first, then replace any word between #, and only replace the nominate column.
# So above sed command is converted to sed "2 s/[^#]\{1,\}/value/3" file.csv
If the above command is fine, and your sed command support the option -i, then run the command to change the content directly in file.csv
sed -i "$row s/[^#]\{1,\}/$value/$col" file.csv
Otherwise, you need export to temp file, and change the name back.
sed "$row s/[^#]\{1,\}/$value/$col" file.csv > temp.csv
mv temp.csv file.csv
I need to search for a specific word in a file starting from specific line and return the line numbers only for the matched lines.
Let's say I want to search a file called myfile for the word my_word and then store the returned line numbers.
By using shell script the command :
sed -n '10,$ { /$my_word /= }' $myfile
works fine but how to write that command on tcl shell?
% exec sed -n '10,$ { /$my_word/= }' $file
extra characters after close-brace.
I want to add that the following command works fine on tcl shell but it starts from the beginning of the file
% exec sed -n "/$my_word/=" $file
447431
447445
448434
448696
448711
448759
450979
451006
451119
451209
451245
452936
454408
I have solved the problem as follows
set lineno 10
if { ! [catch {exec sed -n "/$new_token/=" $file} lineFound] && [string length $lineFound] > 0 } {
set lineNumbers [split $lineFound "\n"]
foreach num $lineNumbers {
if {[expr {$num >= $lineno}] } {
lappend col $num
}
}
}
Still can't find a single line that solve the problem
Any suggestions ??
I don't understand a thing: is the text you are looking for stored inside the variable called my_word or is the literal value my_word?
In your line
% exec sed -n '10,$ { /$my_word/= }' $file
I'd say it's the first case. So you have before it something like
% set my_word wordtosearch
% set file filetosearchin
Your mistake is to use the single quote character ' to enclose the sed expression. That character is an enclosing operator in sh, but has no meaning in Tcl.
You use it in sh to group many words in a single argument that is passed to sed, so you have to do the same, but using Tcl syntax:
% set my_word wordtosearch
% set file filetosearchin
% exec sed -n "10,$ { /$my_word/= }" $file
Here, you use the "..." to group.
You don't escape the $ in $my_word because you want $my_word to be substitued with the string wordtosearch.
I hope this helps.
After a few trial-and-error I came up with:
set output [exec sed -n "10,\$ \{ /$myword/= \}" $myfile]
# Do something with the output
puts $output
The key is to escape characters that are special to TCL, such as the dollar sign, curly braces.
Update
Per Donal Fellows, we do not need to escape the dollar sign:
set output [exec sed -n "10,$ \{ /$myword/= \}" $myfile]
I have tried the new revision and found it works. Thank you, Donal.
Update 2
I finally gained access to a Windows 7 machine, installed Cygwin (which includes sed and tclsh). I tried out the above script and it works just fine. I don't know what your problem is. Interestingly, the same script failed on my Mac OS X system with the following error:
sed: 1: "10,$ { /ipsum/= }": extra characters at the end of = command
while executing
"exec sed -n "10,$ \{ /$myword/= \}" $myfile"
invoked from within
"set output [exec sed -n "10,$ \{ /$myword/= \}" $myfile]"
(file "sed.tcl" line 6)
I guess there is a difference between Linux and BSD systems.
Update 3
I have tried the same script under Linux/Tcl 8.4 and it works. That might mean Tcl 8.4 has nothing to do with it. Here is something else that might help: Tcl comes with a package called fileutil, which is part of the tcllib. The fileutil package contains a useful tool for this case: fileutil::grep. Here is a sample on how to use it in your case:
package require fileutil
proc grep_demo {myword myfile} {
foreach line [fileutil::grep $myword $myfile] {
# Each line is in the format:
# filename:linenumber:text
set lineNumber [lindex [split $line :] 1]
if {$lineNumber >= 10} { puts $lineNumber}
}
}
puts [grep_demo $myword $myfile]
Here is how to do it with awk
awk 'NR>10 && $0~f {print NR}' f="$my_word" "$myfile"
This search for all line larger than line number 10 that contains word in variable $my_word in file name stored in variable myfile
I want to feed input to a C program with a perl script like this
./cprogram $(perl -e 'print "\xab\xcd\xef";').
However, the string must be read from a file. So I get something like this:
./cprogram $(perl -e 'open FILE, "<myfile.txt"; $file_contents = do { local $/; <FILE> }; print $file_contents'. However, now perl interprets the string as the string "\xab\xcd\xef", and I want it to interpret it as the byte sequence as in the first example.
How can this be achieved? It has to be ran on a server without File::Slurp.
In the first case, you pass the three bytes AB CD EF (produced by the string literal "\xAB\xCD\xEF") to print.
In the second case, you must be passing something other than those three bytes to print. I suspect you are passing the twelve character string \xAB\xCD\xEF to print.
So your question becomes: How does one convert the twelve-character string \xAB\xCD\xEF into the three bytes AB CD EF. Well, you'd require some kind of parser such as
s/\\x([0-9a-fA-F][0-9a-fA-F])|\\([^x])|([^\\]+)/
$1 ? chr(hex($1)) : $2 ? $2 : $3
/eg
And here it is at work:
$ perl -e'print "\\xAB\\xCD\\xEF";' >file
$ echo -n "$( perl -0777pe'
s{\\x([0-9a-fA-F][0-9a-fA-F])|\\([^x])|([^\\]+)}{
$1 ? chr(hex($1)) : $2 // $3
}eg;
' file )" | od -t x1
0000000 ab cd ef
0000003
Is Perl's eval too evil? If not, end in print eval("\"$file_contents\"");
Or can you prepare the file in advance using Perl? EG print FILE "\xAB\xCD\xED"; then read the resulting file with your existing code.
using a bash trick:
perl -e "$(echo "print \"$(cat input)"\")"
which for your example becomes:
./cprogram "$(perl -e "$(echo "print \"$(cat myfile.txt)"\")")"
This works:
perl -pi -e 's/abc/cba/g' hellofile
But this does not:
perl -pie 's/cba/abc/g' hellofile
In other words -pi -e works but -pie does not. Why?
The -i flag takes an optional argument (which, if present, must be immediately after it, not in a separate command-line argument) that specifies the suffix to append to the name of the input file for the purposes of creating a backup. Writing perl -pie 's/cba/abc/g' hellofile causes the e to be taken as this suffix, and as the e isn't interpreted as the normal -e option, Perl tries to run the script located in s/cba/abc/g, which probably doesn't exist.
Because -i takes an optional extension for backup files, e.g. -i.bak, and therefore additional flags cannot follow directly after -i.
From perldoc perlrun
-i[extension]
specifies that files processed by the <> construct are to be edited
in-place. It does this by renaming the input file, opening the output
file by the original name, and selecting that output file as the
default for print() statements. The extension, if supplied, is used to
modify the name of the old file to make a backup copy, following these
rules:
If no extension is supplied, no backup is made and the current file is
overwritten.
If the extension doesn't contain a * , then it is appended to the end
of the current filename as a suffix. If the extension does contain one
or more * characters, then each * is replaced with the current
filename. In Perl terms, you could think of this as:
perl already tells you why :) Try-It-To-See
$ perl -pie " s/abc/cba/g " NUL
Can't open perl script " s/abc/cba/g ": No such file or directory
If you use B::Deparse you can see how perl compiles your code
$ perl -MO=Deparse -pi -e " s/abc/cba/g " NUL
BEGIN { $^I = ""; }
LINE: while (defined($_ = <ARGV>)) {
s/abc/cba/g;
}
continue {
die "-p destination: $!\n" unless print $_;
}
-e syntax OK
If you lookup $^I in perlvar you can learn about the -i switch :)
$ perldoc -v "$^I"
$INPLACE_EDIT
$^I The current value of the inplace-edit extension. Use "undef" to
disable inplace editing.
Mnemonic: value of -i switch.
Now if we revisit the first part, add an extra -e, then add Deparse, the -i switch is explained
$ perl -pie -e " s/abc/cba/g " NUL
Can't do inplace edit: NUL is not a regular file.
$ perl -MO=Deparse -pie -e " s/abc/cba/g " NUL
BEGIN { $^I = "e"; }
LINE: while (defined($_ = <ARGV>)) {
s/abc/cba/g;
}
continue {
die "-p destination: $!\n" unless print $_;
}
-e syntax OK
Could it really be that e in -pie is taken as extension? I guess so
$ perl -MO=Deparse -pilogicus -e " s/abc/cba/g " NUL
BEGIN { $^I = "logicus"; }
LINE: while (defined($_ = <ARGV>)) {
s/abc/cba/g;
}
continue {
die "-p destination: $!\n" unless print $_;
}
-e syntax OK
When in doubt, Deparse or Deparse,-p