Parsing ENV with SED - sed

I have an env file
ENV=development
PORT=3000
SECRET_KEY=-----BEGIN PRIVATE KEY-----\nKEYKEYKEY\n-----END PRIVATE KEY-----\n
How can I parse this file using SED to properly format it?
Used this solution
for line in `cat ${FILE_NAME_ENV}`
do
KEY=$(echo $line | sed 's/=.*//')
VAL=$(echo $line | sed 's/[^=]*=//')
echo $KEY
echo $VAL
done
But SECRET_KEY is splitting over multiple lines due to spaces in the value
ENV
development
PORT
3000
SECRET_KEY
-----BEGIN
PRIVATE
PRIVATE
KEY-----\nKEYKEYKEY\n-----END
KEY-----\nKEYKEYKEY\n-----END
PRIVATE
PRIVATE
KEY-----\n
KEY-----\n
How can i make key value in one line?
SECRET_KEY
-----BEGIN PRIVATE KEY-----\nKEYKEYKEY\n-----END PRIVATE KEY-----\n

Using sed
$ sed -n 's/^\(SECRET_KEY\)=/\1\n/p' input_file
SECRET_KEY
-----BEGIN PRIVATE KEY-----\nKEYKEYKEY\n-----END PRIVATE KEY-----\n

Suggesting to enclose echo command arguments with " to contain each line as single argument to command.
Suggestion 1: fix missing "
for line in "$(cat ${FILE_NAME_ENV})"; do
KEY=$(echo "$line" | sed 's/=.*//')
VAL=$(echo "$line" | sed 's/[^=]*=//')
echo "$KEY"
echo "$VAL"
done
Output 1:
ENV
PORT
SECRET_KEY
development
3000
-----BEGIN PRIVATE KEY-----\nKEYKEYKEY\n-----END PRIVATE KEY-----\n
Suggestion 2: fix with echo -e to accept \n as newline
for line in "$(cat ${FILE_NAME_ENV})"; do
KEY=$(echo "$line" | sed 's/=.*//')
VAL=$(echo "$line" | sed 's/[^=]*=//')
echo "$KEY"
echo -e "$VAL"
done
Output 2:
ENV
PORT
SECRET_KEY
development
3000
-----BEGIN PRIVATE KEY-----
KEYKEYKEY
-----END PRIVATE KEY-----

Related

How to insert multiple complex lines containing spaces, pipes, grep and sed commands before pattern

The goal is to insert the following complex lines before a specific pattern in a file:
NDPI_VERSION_SHORT=$(cat Makefile | grep -P "^NDPI_VERSION_SHORT = " | sed -E 's|^NDPI_VERSION_SHORT = (.*)$|\1|g') \
NDPI_VERSION_SHORT=${NDPI_VERSION_SHORT//[[:space:]]/} \
NDPI_MAJOR=$(cat Makefile | grep -P "^NDPI_MAJOR = " | sed -E 's|^NDPI_MAJOR = (.*)$|\1|g') \
NDPI_MAJOR=${NDPI_MAJOR//[[:space:]]/}
I unsuccessfully tried the following:
sed -i '/pattern/i \
NDPI_VERSION_SHORT=$(cat Makefile | grep -P "^NDPI_VERSION_SHORT = " | sed -E \'s|^NDPI_VERSION_SHORT = (.*)$|\1|g\') \
NDPI_VERSION_SHORT=${NDPI_VERSION_SHORT\/\/[[:space:]]\/} \
NDPI_MAJOR=$(cat Makefile | grep -P "^NDPI_MAJOR = " | sed -E \'s|^NDPI_MAJOR = (.*)$|\1|g\') \
NDPI_MAJOR=${NDPI_MAJOR\/\/[[:space:]]\/}' file
bash: syntax error near unexpected token `('
I also tried to quote all inserted lines leading to the same result.
What am I doing wrong?
This should work:
sed "/pattern/i \
NDPI_VERSION_SHORT=\$\(cat Makefile | grep -P \"^NDPI_VERSION_SHORT = \" | sed -E 's|^NDPI_VERSION_SHORT = \(.*\)\$|\\\1|g'\) \\\ \n\
NDPI_VERSION_SHORT=\${NDPI_VERSION_SHORT//[[:space:]]/} \\\ \n\
NDPI_MAJOR=\$\(cat Makefile | grep -P \"^NDPI_MAJOR = \" | sed -E 's|^NDPI_MAJOR = \(.*\)\$|\\\1|g'\) \\\ \n\
NDPI_MAJOR=\${NDPI_MAJOR//[[:space:]]/}" file
The problem is the single quote within the inserted text, which will end the sed script and which cannot be escaped. You can use single quotes, though, if you use double quotes to enclose the script. This, however, means you'll need to escape quite a lot of things in your text: The $, ", (, ). Since the shell itself uses up a backslash for escaping, you need to write \\\ where you have a \. And the line break is achieved via a \n. Note that the / does not need to be escaped since sed does not use it as delimiter here.

sed search and replace \" but not \\"

I am trying to replace all escaped characters \" in a string with "" but not if \" is preceded by a \
So that input such as:
\"\"\"\" would return """"""""
\"\\"\"\" would return ""\\"""""
\" would return ""
\"\" would return """"
\\"\" would return \\"""
\"\\" would return ""\\"
\\\\\\\" would return \\\\\\\"
So far I have
$ echo sed -e 's/\([^\]\)\\"/\1""/;s/^\\"/""/'
but in the case of
$ echo '\"\"\"\"\"' | sed -e 's/\([^\]\)\\"/\1""/;s/^\\"/""/'`
I am getting incorrect results.
Any help would be appreciated.
This might work for you (GNU sed):
sed 's/\\\\"/\n/g;s/\\"/""/g;s/\n/\\\\"/g' file
Replace all occurances of the string you want untouched by something else (\n is a good choice), replace the string you want changed globally, reinstate the first set of strings.
How about this:
#!/bin/bash
function myreplace {
echo "$1" | sed -e "s/[\\]\"/MYDUMMY/g" \
-e 's/\\MYDUMMY/\\\\"/g' \
-e 's/MYDUMMY/""/g'
}
myreplace '\"\"\"\"'
myreplace '\"\\"\"\"'
myreplace '\"'
myreplace '\"\"'
myreplace '\\"\"'
myreplace '\"\\"'
myreplace '\\\\\\\"'
Executing the script above results in:
""""""""
""\\"""""
""
""""
\\"""
""\\"
\\\\\\\"
Using a sed loop will allow not having to pick a unique replacement string for an unknown dataset.
sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
$ echo '\"\"\"\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g;t inner'
""""""""
$ echo '\"\\"\"\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
""\\"""""
$ echo '\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
""
$ echo '\"\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
""""
$ echo '\\"\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
\\"""
$ echo '\"\\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
""\\"
$ echo '\\\\\\\"' | sed -e 's/^\\"/""/;:inner; s/\([^\]\)[\]"/\1""/g; t inner'
\\\\\\\"

Replace everyting after every time different string

Want to change everything after security.server.ip=* with the result ip from the second grep.
First Grep:
cat admin.conf|grep security.server.ip|grep -v ^#
Result:
security.server.ip=10.10.1.2
Second Grep:
cat /etc/hosts|grep -i admin-server|head -1|awk '{ print $1}
Result:
10.10.1.2
Sometimes security.server.ip will be different on admin.conf and I'm wondering how to replace it with one command which will catch IP address form second grep and replace it in the first one.
You can use a script:
#!/bin/sh
IP=$(exec grep -i admin-server /etc/hosts | awk '{ print $1; exit }')
sed -i "/^security\.server\.ip=/s|=.*|=$IP|" admin.conf
You could save it in a variable:
NEWIP=`grep -i admin-server /etc/hosts|head -1|awk '{ print $1}'` \
sed -i "s/^security\.server\.ip=[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/security\.server\.ip=$NEWIP/" admin.conf
With GNU awk for inplace editing, nextfile, and gensub():
gawk -i inplace '
NR==FNR{ if (tolower($0) ~ /admin-server/) { ip=$1; nextfile } next }
{ $0=gensub(/(security\.server\.ip=).*/,"\\1"ip,""); print }
' /etc/hosts admin.conf

extract number from string

I have a string ABCD20110420.txt and I want to extract the date out of it. Expected 2011-04-20
I can use replace to remove the text part, but how do I insert the "-" ?
# echo "ABCD20110420.txt" | replace 'ABCD' '' | replace '.txt' ''
20110420
echo "ABCD20110420.txt" | sed -e 's/ABCD//' -e 's/.txt//' -e 's/\(....\)\(..\)\(..\)/\1-\2-\3/'
Read: sed FAQ
Just use the shell (bash)
$> file=ABCD20110420.txt
$> echo "${file//[^0-9]/}"
20110420
$> file="${file//[^0-9]/}"
$> echo $file
20110420
$> echo ${file:0:4}-${file:4:2}-${file:6:2}
2011-04-20
The above is applicable to files like your sample. If you have files like A1BCD20110420.txt, then will not work.
For that case,
$> file=A1BCD20110420.txt
$> echo ${file%.*} #get rid of .txt
A1BCD20110420
$> file=${file%.*}
$> echo "2011${file#*2011}"
20110420
Or you can use regular expression (Bash 3.2+)
$> file=ABCD20110420.txt
$> [[ $file =~ ^.*(2011)([0-9][0-9])([0-9][0-9])\.*$ ]]
$> echo ${BASH_REMATCH[1]}
2011
$> echo ${BASH_REMATCH[2]}
04
$> echo ${BASH_REMATCH[3]}
20
echo "ABCD20110420.txt" | sed -r 's/.+([0-9]{4})([0-9]{2})([0-9]{2}).+/\1-\2-\3/'
$ file=ABCD20110420.txt
$ echo "$file" | sed -e 's/^[A-Za-z]*\([0-9][0-9][0-9][0-9]\)\([0-9][0-9]\)\([0-9][0-9]\)\.txt$/\1-\2-\3/'
This only requires a single call to sed.
echo "ABCD20110420.txt" | sed -r 's/.{4}(.{4})(.{2})(.{2}).txt/\1-\2-\3/'

get list of sections from ini-file using shell (sed/awk)

I want to create a var from the section names of an ini file like:
[foo]
; ...
[bar]
; ...
[baz:bar]
;...
now I need a var like
SECTIONS="foo bar baz"
thanks in advance
One line solution could be:
export SECTIONS=`grep "^\[" test.ini |sort -u | xargs | tr '\[' ' ' | tr '\]' ' ' `
SECTIONS=$(crudini --get your.ini | sed 's/:.*//')
I'm now using this construct, don't need to know if a section exists. just read it, if it's empty it does not exist.
INI_FILE=test.ini
function ini_get
{
eval `sed -e 's/[[:space:]]*\=[[:space:]]*/=/g' \
-e 's/;.*$//' \
-e 's/[[:space:]]*$//' \
-e 's/^[[:space:]]*//' \
-e "s/^\(.*\)=\([^\"']*\)$/\1=\"\2\"/" \
< $INI_FILE \
| sed -n -e "/^\[$1\]/,/^\s*\[/{/^[^;].*\=.*/p;}"
echo ${!2}
}
IP=$(ini_get 50001 ip)
PORT=$(ini_get 50001 port)
echo $IP:$PORT