Say I want to substitute test for TEST, but not after a "!" The substitution should be ignored after the "!" but not before.
And I also want to skip lines containing the word "skipme"
eg:
test
! test
test ! test
blahblah ! test
blah skipme test
test
should become:
TEST
! test
TEST ! test
blahblah ! test
blah skipme test
TEST
^ perhaps is all you need.
sed -e 's|^test|TEST|' your_file.txt
Outputs:
TEST
! test
TEST ! test
blahblah ! test
blah skipme test
TEST
As some notes in grep:
The caret ^ and the dollar sign $ are meta-characters that
respectively match the empty string at the beginning and end of a
line.
Here is another awk
awk '{f=0;for (i=1;i<=NF;i++) {if ($i~/!|skipme/) f=1; if (!f && $i=="test") $i=toupper($i)}}1' file
TEST
! test
TEST ! test
blahblah ! test
blah skipme test
TEST
How it works
awk '{
f=0 # set flag to 0
for (i=1;i<=NF;i++) { # run trough all fields one by one
if ($i~/!|skipme/) # if a field contains "!" or "skipme"
f=1 # set flag "f" to "1"
if (!f && $i=="test") # If field contains "test" and flag "f" is not set, do
$i=toupper($i) # Set field to upper characters
}
}
1 # Print the line
' file # Read the file
An updated version of Jidders version that should work i most cases:
(this will not work if there are more than on test to change, but
It will work even if there are no space between words)
awk '(/!/ && index($0,w)<index($0,"!")) || (/skipme/ && index($0,w)<index($0,"skipme")) || !/!|skipme/ {c=toupper(w);sub(w,c)}1' w="test" file
And the one through perl,
$ perl -pe 's/(?<!\! )test/TEST/ unless /skipme/' file
TEST
! test
TEST ! test
blahblah ! test
blah skipme test
TEST
It replaces test with TEST only when it is not preceeded by ! and skips the line which contains the string skipme.
Related
I have one command which is used to extract lines between two string patterns 'string1' and 'string2'. This is stored in variable called 'var1'.
var1=$(awk '/string1/{flag=1; next} /string2/{flag=0} flag' text.txt)
This command works well and the output is a set of lines.
Do you hear the people sing?
Singing a song of angry men?
It is the music of a people
Who will not be slaves again
I want the output of the above command to be inserted after a string pattern 'string3' in another file called stat.txt. I used sed as follows
sed '/string3/a'$var1'' stat.txt
I am having trouble getting the new output. Here, the $var1 seems to be working partially i.e. only one line -
string3
Do you hear the people sing?
Any other suggestions to solve this?
I would be tempted to use sed to extract the lines, and awk to insert them into the other text:
lines=$(sed -n '/string1/,/string2/ p' text.txt)
awk -v new="$lines" '{print} /string3/ {print new}' stat.txt
or perhaps both tasks in a single awk call
awk '
NR == FNR && /string1/ {flag = 1}
NR == FNR && /string2/ {flag = 0}
NR == FNR && flag {lines = lines $0 ORS}
NR == FNR {next}
{print}
/string3/ {printf "%s", lines} # it already ends with a newline
' text.txt stat.txt
It's a data format problem...
Appending a multi-line block of text with the sed append command requires that every line in the block to be appended ends with a \ -- except for the last line of that block. So if we take the two lines of code that didn't work in the question, and reformat the text as required by the append command, the original code should work as expected:
var1=$(awk '/string1/{flag=1; next} /string2/{flag=0} flag' text.txt)
var1="$(sed '$!s/$/\\/' <<< "$var1")"
sed '/string3/a'$var1'' stat.txt
Note that the 2nd line above contains a bashism. A more portable version would be:
var1="$(echo "$var1" | sed '$!s/$/\\/')"
Either variant would convert $var1 to:
Do you hear the people sing?\
Singing a song of angry men?\
It is the music of a people\
Who will not be slaves again
Want to rename the (known) 3th folder within a (unknown) file path from a string, when positioned on 3th level while separator is /
Need a one-liner explicitly for sed. Because I later want use it for tar --transform=EXPRESSION
string="/db/foo/db/bar/db/folder"
echo "$string" | sed 's,db,databases,'
sed replace "db" only on 3th level
expected result
/db/foo/databases/bar/db/folder
You could use a capturing group to capture /db/foo/ and then match db. Then use use the first caputring group in the replacement using \1:
string="/db/foo/db/bar/db/folder"
echo -e "$string" | sed 's,^\(/[^/]*/[^/]*/\)db,\1databases,'
About the pattern
^ Start of string
\( Start capture group
/[^/]*/[^/]*/ Match the first 2 parts using a negated character class
\) Close capture group
db Match literally
That will give you
/db/foo/databases/bar/db/folder
If awk is also an option for this task:
$ awk 'BEGIN{FS=OFS="/"} $4=="db"{$4="database"} 1' <<<'/db/foo/db/bar/db/folder'
/db/foo/database/bar/db/folder
FS = OFS = "/" assign / to both input and output field separators,
$4 == "db" { $4 = "database }" if fourth field is db, make it database,
1 print the record.
Here is a pure bash way to get this done by setting IFS=/ without calling any external utility:
string="/db/foo/db/bar/db/folder"
string=$(IFS=/; read -a arr <<< "$string"; arr[3]='databases'; echo "${arr[*]}")
echo "$string"
/db/foo/databases/bar/db/folder
I am trying replace a block of code between two patterns with blank lines
Tried using below command
sed '/PATTERN-1/,/PATTERN-2/d' input.pl
But it only removes the lines between the patterns
PATTERN-1 : "=head"
PATTERN-2 : "=cut"
input.pl contains below text
=head
hello
hello world
world
morning
gud
=cut
Required output :
=head
=cut
Can anyone help me on this?
$ awk '/=cut/{f=0} {print (f ? "" : $0)} /=head/{f=1}' file
=head
=cut
To modify the given sed command, try
$ sed '/=head/,/=cut/{//! s/.*//}' ip.txt
=head
=cut
//! to match other than start/end ranges, might depend on sed implementation whether it dynamically matches both the ranges or statically only one of them. Works on GNU sed
s/.*// to clear these lines
awk '/=cut/{found=0}found{print "";next}/=head/{found=1}1' infile
# OR
# ^ to take care of line starts with regexp
awk '/^=cut/{found=0}found{print "";next}/^=head/{found=1}1' infile
Explanation:
awk '/=cut/{ # if line contains regexp
found=0 # set variable found = 0
}
found{ # if variable found is nonzero value
print ""; # print ""
next # go to next line
}
/=head/{ # if line contains regexp
found=1 # set variable found = 1
}1 # 1 at the end does default operation
# print current line/row/record
' infile
Test Results:
$ cat infile
=head
hello
hello world
world
morning
gud
=cut
$ awk '/=cut/{found=0}found{print "";next}/=head/{found=1}1' infile
=head
=cut
This might work for you (GNU sed):
sed '/=head/,/=cut/{//!z}' file
Zap the lines between =head and =cut.
I have a file with a certain line, let's say...
AAA BBB CCC
I need to replace that entire line, after finding it, so I did:
q1=`grep -Hnm 1 "AAA" FILE | cut -d : -f 2`
That outputs me the line number of the first occurrence (in q1), because it has more than one occurrence, now, here comes my problem... In a previous step I was using this sed to replace a certain line in the file:
sed -e '3s/.*/WHATEVER/' FILE
To replace (in the example, line 3) the full line with WHATEVER, but now if I try to use $q1 instead of the "3" indicating the line number it doesn't work:
sed -e '$q1s/.*/WHATEVER/' FILE
It's probably a stupid syntax mistake, any help is welcome; thanks in advance
Try:
sed -e "${q1}s/.*/WHATEVER/" FILE
I'd use awk for this:
awk '/AAA/ && !r {print "WHATEVER"; r=1; next} {print}' <<END
a
b
AAA BBB CCC
d
e
AAA foo bar
f
END
a
b
WHATEVER
d
e
AAA foo bar
f
If you want to replace the first occurrence of a string in a file, you could use this awk script:
awk '/occurrence/ && !f++ {print "replacement"; next}1' file
The replacement will only be printed the first time, as !f++ will only evaluate to true once (on subsequent evaluations, f will be greater than zero so !f will be false. The 1 at the end is always true, so for each line other than the matched one, awk does the default action and prints the line.
Testing it out:
$ cat file
blah
blah
occurrence 1 and some other stuff
blah
blah
some more stuff and occurrence 2
blah
$ awk '/occurrence/ && !f++ {print "replacement"; next}1' file
blah
blah
replacement
blah
blah
some more stuff and occurrence 2
blah
The "replacement" string could easily be set to the value of a shell variable in the following way:
awk -v rep="$r" '/occurrence/ && !f++ {print rep; next}1' file
where $r is a shell variable.
Using the same file as above and the example variable in your comment:
$ q2="x=\"Second\""
$ awk -v rep="$q2" '/occurrence/ && !f++ {print rep; next}1' file
blah
blah
x="Second"
stuff
blah
blah
some more stuff and occurrence 2
blah
sed "${q1} c\
WHATEVER" YourFile
but you can directly use
sed '/YourPatternToFound/ {s/.*/WHATEVER/
:a
N;$!ba
}' YourFile
I need to search for a specific word in a file starting from specific line and return the line numbers only for the matched lines.
Let's say I want to search a file called myfile for the word my_word and then store the returned line numbers.
By using shell script the command :
sed -n '10,$ { /$my_word /= }' $myfile
works fine but how to write that command on tcl shell?
% exec sed -n '10,$ { /$my_word/= }' $file
extra characters after close-brace.
I want to add that the following command works fine on tcl shell but it starts from the beginning of the file
% exec sed -n "/$my_word/=" $file
447431
447445
448434
448696
448711
448759
450979
451006
451119
451209
451245
452936
454408
I have solved the problem as follows
set lineno 10
if { ! [catch {exec sed -n "/$new_token/=" $file} lineFound] && [string length $lineFound] > 0 } {
set lineNumbers [split $lineFound "\n"]
foreach num $lineNumbers {
if {[expr {$num >= $lineno}] } {
lappend col $num
}
}
}
Still can't find a single line that solve the problem
Any suggestions ??
I don't understand a thing: is the text you are looking for stored inside the variable called my_word or is the literal value my_word?
In your line
% exec sed -n '10,$ { /$my_word/= }' $file
I'd say it's the first case. So you have before it something like
% set my_word wordtosearch
% set file filetosearchin
Your mistake is to use the single quote character ' to enclose the sed expression. That character is an enclosing operator in sh, but has no meaning in Tcl.
You use it in sh to group many words in a single argument that is passed to sed, so you have to do the same, but using Tcl syntax:
% set my_word wordtosearch
% set file filetosearchin
% exec sed -n "10,$ { /$my_word/= }" $file
Here, you use the "..." to group.
You don't escape the $ in $my_word because you want $my_word to be substitued with the string wordtosearch.
I hope this helps.
After a few trial-and-error I came up with:
set output [exec sed -n "10,\$ \{ /$myword/= \}" $myfile]
# Do something with the output
puts $output
The key is to escape characters that are special to TCL, such as the dollar sign, curly braces.
Update
Per Donal Fellows, we do not need to escape the dollar sign:
set output [exec sed -n "10,$ \{ /$myword/= \}" $myfile]
I have tried the new revision and found it works. Thank you, Donal.
Update 2
I finally gained access to a Windows 7 machine, installed Cygwin (which includes sed and tclsh). I tried out the above script and it works just fine. I don't know what your problem is. Interestingly, the same script failed on my Mac OS X system with the following error:
sed: 1: "10,$ { /ipsum/= }": extra characters at the end of = command
while executing
"exec sed -n "10,$ \{ /$myword/= \}" $myfile"
invoked from within
"set output [exec sed -n "10,$ \{ /$myword/= \}" $myfile]"
(file "sed.tcl" line 6)
I guess there is a difference between Linux and BSD systems.
Update 3
I have tried the same script under Linux/Tcl 8.4 and it works. That might mean Tcl 8.4 has nothing to do with it. Here is something else that might help: Tcl comes with a package called fileutil, which is part of the tcllib. The fileutil package contains a useful tool for this case: fileutil::grep. Here is a sample on how to use it in your case:
package require fileutil
proc grep_demo {myword myfile} {
foreach line [fileutil::grep $myword $myfile] {
# Each line is in the format:
# filename:linenumber:text
set lineNumber [lindex [split $line :] 1]
if {$lineNumber >= 10} { puts $lineNumber}
}
}
puts [grep_demo $myword $myfile]
Here is how to do it with awk
awk 'NR>10 && $0~f {print NR}' f="$my_word" "$myfile"
This search for all line larger than line number 10 that contains word in variable $my_word in file name stored in variable myfile