I am trying to replace all escaped characters \" in a string with "" but not if \" is preceded by a \
So that input such as:
\"\"\"\" would return """"""""
\"\\"\"\" would return ""\\"""""
\" would return ""
\"\" would return """"
\\"\" would return \\"""
\"\\" would return ""\\"
\\\\\\\" would return \\\\\\\"
So far I have
$ echo sed -e 's/\([^\]\)\\"/\1""/;s/^\\"/""/'
but in the case of
$ echo '\"\"\"\"\"' | sed -e 's/\([^\]\)\\"/\1""/;s/^\\"/""/'`
I am getting incorrect results.
Any help would be appreciated.
This might work for you (GNU sed):
sed 's/\\\\"/\n/g;s/\\"/""/g;s/\n/\\\\"/g' file
Replace all occurances of the string you want untouched by something else (\n is a good choice), replace the string you want changed globally, reinstate the first set of strings.
How about this:
#!/bin/bash
function myreplace {
echo "$1" | sed -e "s/[\\]\"/MYDUMMY/g" \
-e 's/\\MYDUMMY/\\\\"/g' \
-e 's/MYDUMMY/""/g'
}
myreplace '\"\"\"\"'
myreplace '\"\\"\"\"'
myreplace '\"'
myreplace '\"\"'
myreplace '\\"\"'
myreplace '\"\\"'
myreplace '\\\\\\\"'
Executing the script above results in:
""""""""
""\\"""""
""
""""
\\"""
""\\"
\\\\\\\"
Using a sed loop will allow not having to pick a unique replacement string for an unknown dataset.
sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
$ echo '\"\"\"\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g;t inner'
""""""""
$ echo '\"\\"\"\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
""\\"""""
$ echo '\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
""
$ echo '\"\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
""""
$ echo '\\"\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
\\"""
$ echo '\"\\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
""\\"
$ echo '\\\\\\\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
\\\\\\\"
Related
I expected sed 's/\s+$//g' to strip trailing spaces
echo "'$(echo 'Magnetic ' | sed 's/\s+$//g')'"
outputs 'Magnetic ', as does
echo "'$(echo 'Magnetic ' | sed 's/[\n\s]+$//g')'"
How do I remove the trailing space with sed?
You have to escape the plus sign + because sed uses BRE, so:
echo "'$(echo 'Magnetic ' | sed 's/\s\+$//g')'"
if there's -r or -E flag, sed uses ERE instead so you don't have to escape it:
echo "'$(echo 'Magnetic ' | sed -r 's/\s+$//g')'"
The goal is to insert the following complex lines before a specific pattern in a file:
NDPI_VERSION_SHORT=$(cat Makefile | grep -P "^NDPI_VERSION_SHORT = " | sed -E 's|^NDPI_VERSION_SHORT = (.*)$|\1|g') \
NDPI_VERSION_SHORT=${NDPI_VERSION_SHORT//[[:space:]]/} \
NDPI_MAJOR=$(cat Makefile | grep -P "^NDPI_MAJOR = " | sed -E 's|^NDPI_MAJOR = (.*)$|\1|g') \
NDPI_MAJOR=${NDPI_MAJOR//[[:space:]]/}
I unsuccessfully tried the following:
sed -i '/pattern/i \
NDPI_VERSION_SHORT=$(cat Makefile | grep -P "^NDPI_VERSION_SHORT = " | sed -E \'s|^NDPI_VERSION_SHORT = (.*)$|\1|g\') \
NDPI_VERSION_SHORT=${NDPI_VERSION_SHORT\/\/[[:space:]]\/} \
NDPI_MAJOR=$(cat Makefile | grep -P "^NDPI_MAJOR = " | sed -E \'s|^NDPI_MAJOR = (.*)$|\1|g\') \
NDPI_MAJOR=${NDPI_MAJOR\/\/[[:space:]]\/}' file
bash: syntax error near unexpected token `('
I also tried to quote all inserted lines leading to the same result.
What am I doing wrong?
This should work:
sed "/pattern/i \
NDPI_VERSION_SHORT=\$\(cat Makefile | grep -P \"^NDPI_VERSION_SHORT = \" | sed -E 's|^NDPI_VERSION_SHORT = \(.*\)\$|\\\1|g'\) \\\ \n\
NDPI_VERSION_SHORT=\${NDPI_VERSION_SHORT//[[:space:]]/} \\\ \n\
NDPI_MAJOR=\$\(cat Makefile | grep -P \"^NDPI_MAJOR = \" | sed -E 's|^NDPI_MAJOR = \(.*\)\$|\\\1|g'\) \\\ \n\
NDPI_MAJOR=\${NDPI_MAJOR//[[:space:]]/}" file
The problem is the single quote within the inserted text, which will end the sed script and which cannot be escaped. You can use single quotes, though, if you use double quotes to enclose the script. This, however, means you'll need to escape quite a lot of things in your text: The $, ", (, ). Since the shell itself uses up a backslash for escaping, you need to write \\\ where you have a \. And the line break is achieved via a \n. Note that the / does not need to be escaped since sed does not use it as delimiter here.
I have the following script for extraction of logs.
Need to make work it with cpulimit, to not overload the server. Can anyone help?
nice -20 zcat /var/detail-20150418.gz | sed '/./{H;$!d;};x;/46.241.178.96/!d' > /tmp/new.txt
nice -20 sed -n '/Acct-Status-Type/,/Called-Station-Id/p' /tmp/new.txt > /tmp/new_2.txt
rm /tmp/new.txt
nice -20 sed '/Acct-Status-Type/{x;p;x;}' /tmp/new_2.txt > /tmp/new_3.txt
rm /tmp/new_2.txt
grep -v '\(Acct-Authentic\|Acct-Input-Octets\|Acct-Input-Gigawords\|Acct-Output-Octets\|Acct-Output-Gigawords\|Acct-Input-Packets\|Acct-Output-Packets\|Acct-Session-Time\)' /tmp/new_3.txt > /tmp/new_4.txt
rm /tmp/new_3.txt
sed 's/^[ \t]*//;s/[ \t]*$//' /tmp/new_4.txt | paste -s -d ',' | sed 's/,,/\n/g' | sed 's/^[,]*//;s/[,]*$//' > /tmp/new_5.txt
rm /tmp/new_4.txt
sed 's/Acct-Status-Type = //' /tmp/new_5.txt | sed 's/User-Name = //' |sed 's/Event-Timestamp = //' | sed 's/Acct-Terminate-Cause = //' | sed 's/Framed-IP-Address = //' | sed 's/Called-Station-Id = //' > /tmp/new_6.txt
rm /tmp/new_5.txt
awk -F"," '{ print $3 "," $1 "," $2 "," $6 "," $5 "," $4}' /tmp/new_6.txt | sed -e 's/"//g' | sed -e 's/,,/,/g' > /tmp/log.txt
rm /tmp/new_6.txt
Getting rid of all that disk IO will help a lot:
nice -20 sh <<'END_SCRIPT' > /tmp/log.txt
zcat /var/detail-${date}.gz |
sed '/./{H;$!d;};x;/46.241.178.96/!d' |
sed -n '/Acct-Status-Type/,/Called-Station-Id/p' |
sed '/Acct-Status-Type/{x;p;x;}' |
grep -Ev '(Acct-Authentic|Acct-Input-Octets|Acct-Input-Gigawords|Acct-Output-Octets|Acct-Output-Gigawords|Acct-Input-Packets|Acct-Output-Packets|Acct-Session-Time)' |
sed 's/^[ \t]*//;s/[ \t]*$//' |
paste -s -d ',' |
sed -e 's/,,/\n/g' \
-e 's/^[,]*//;s/[,]*$//' \
-e 's/Acct-Status-Type = //' \
-e 's/User-Name = //' \
-e 's/Event-Timestamp = //' \
-e 's/Acct-Terminate-Cause = //' \
-e 's/Framed-IP-Address = //' \
-e 's/Called-Station-Id = //' |
awk -F"," '{ print $3 "," $1 "," $2 "," $6 "," $5 "," $4}' |
sed -e 's/"//g' |
sed -e 's/,,/,/g'
END_SCRIPT
All the commands except zcat can be consolidated into a single awk or perl script: I don't have the time to help you with that right now.
I'm using this code to get all titles from urls with http://something.txt:
#!/usr/bin/perl -w
$output = `cat source.html | grep -o '<a .*href=.*>' | grep -E 'txt' | sed -e 's/<a /\n<a /g' | sed -e 's/<a .*title="//' | cut -f1 -d '"'`;
print("$output");
When i run this on perl i get the error:
sed: -e expression #1, char 6: unterminated `s' command
The error is related with this portion of code:
sed -e 's/<a /\n<a /g'
In backquotes, Perl uses the same rules as in double quotes. Therefore, \n corresponds to a newline; you have to backslash the backslash to pass literal \ to the shell:
`sed -e 's/<a /\\n<a /g'`
I have a string ABCD20110420.txt and I want to extract the date out of it. Expected 2011-04-20
I can use replace to remove the text part, but how do I insert the "-" ?
# echo "ABCD20110420.txt" | replace 'ABCD' '' | replace '.txt' ''
20110420
echo "ABCD20110420.txt" | sed -e 's/ABCD//' -e 's/.txt//' -e 's/\(....\)\(..\)\(..\)/\1-\2-\3/'
Read: sed FAQ
Just use the shell (bash)
$> file=ABCD20110420.txt
$> echo "${file//[^0-9]/}"
20110420
$> file="${file//[^0-9]/}"
$> echo $file
20110420
$> echo ${file:0:4}-${file:4:2}-${file:6:2}
2011-04-20
The above is applicable to files like your sample. If you have files like A1BCD20110420.txt, then will not work.
For that case,
$> file=A1BCD20110420.txt
$> echo ${file%.*} #get rid of .txt
A1BCD20110420
$> file=${file%.*}
$> echo "2011${file#*2011}"
20110420
Or you can use regular expression (Bash 3.2+)
$> file=ABCD20110420.txt
$> [[ $file =~ ^.*(2011)([0-9][0-9])([0-9][0-9])\.*$ ]]
$> echo ${BASH_REMATCH[1]}
2011
$> echo ${BASH_REMATCH[2]}
04
$> echo ${BASH_REMATCH[3]}
20
echo "ABCD20110420.txt" | sed -r 's/.+([0-9]{4})([0-9]{2})([0-9]{2}).+/\1-\2-\3/'
$ file=ABCD20110420.txt
$ echo "$file" | sed -e 's/^[A-Za-z]*\([0-9][0-9][0-9][0-9]\)\([0-9][0-9]\)\([0-9][0-9]\)\.txt$/\1-\2-\3/'
This only requires a single call to sed.
echo "ABCD20110420.txt" | sed -r 's/.{4}(.{4})(.{2})(.{2}).txt/\1-\2-\3/'