I want to replace the value with a variable. The following does not work because of the single quote used in sed.
#!/bin/sh
set myvarM='pqr'
sed 's/:P03:M15/:P02:M1$myvarM/' mychange.txt > new_mychange.txt
Can I change the sed command or should I use something else?
#!/bin/sh
myvarM='pqr'
sed "s/:P03:M15/:P02:M1$myvarM/" mychange.txt > new_mychange.txt
Incidentally, to make the replacement in-place (ie not create a new file, but alter the original file), do this:
sed -i '' "s/:P03:M15/:P02:M1$myvarM/" mychange.txt
This says "use a blank as the increment suffix" - ie write out the same filename as the input
awk -v val="$myvarM" '{sub(/:P03:M15/, ":P02:M1" val); print}' filename
Note that in a bourne-type shell ("/bin/sh") the set command sets the positional parameters. Your first line sets $1 to the value myvarM='pqr' -- the myvarM variable continues to be unset.
Related
Playing with makefile and bash shell scripting where I ended-up having a variable containing:
--global-option=build_ext --global-option=--b2-args=libtorrent-python-pic=on --global-option=--b2-args=address-model=32
I need to convert it so double-quotes gets appended at the right place such as:
--global-option=build_ext --global-option=--b2-args="libtorrent-python-pic=on" --global-option=--b2-args="address-model=32"
I tried the following without success:
echo $myvar | sed -e 's/ /\n/' | sed -z '{s/=/="/2;t;s/$/"/}'
--global-option=build_ext
--global-option="--b2-args=libtorrent-python-pic=on
EDIT: Note that this time it's --b2-args= but this could be a conjunction of --anything=, and the reason why I was focussing on the second instance of = to change for =" and if true append = at the end of word.
Since your question doesn't discuss anything about prepending --global-option= if it's missing as in the final --b2-args... string on the provided input line, I think your input was really supposed to be:
$ cat file
--global-option=build_ext --global-option=--b2-args=libtorrent-python-pic=on --global-option=--b2-args=address-model=32
in which case using any sed in any shell on every Unix box:
$ sed 's/\([^ =]*=[^= ]*=\)\([^ ]*\)/\1"\2"/g' file
--global-option=build_ext --global-option=--b2-args="libtorrent-python-pic=on" --global-option=--b2-args="address-model=32"
I need to comment out a line in a crontab file through a script, so it contains directories, spaces and symbols. This specific line is stored in a variable and I am starting to get mixed up on how to escape the variable. Since the line changes on a regular basis I dont want any escaping in there. I don't want to simply add # in front of it, since I also need to switch it around and replace the line again with the original without the #.
So the goal is to replace $line with #$line (comment) with the possibility to do it the other way around (uncomment).
So I have a variable:
line="* * * hello/this/line & /still/this/line"
This is a line that occurs in a file, file.txt. Wich needs to get comment out.
First try:
sed -i "s/^${line}/#${line}/" file.txt
Second try:
sed -i 's|'${line}'|'"#${line}"'|g' file.txt
choroba's helpful answer shows an effective solution using perl.
sed solution
If you want to use sed, you must use a separate sed command just to escape the $line variable value, because sed has no built-in way to escape strings for use as literals in a regex context:
lineEscaped=$(sed 's/[^^]/[&]/g; s/\^/\\^/g' <<<"$line") # escape $line for use in regex
sed -i "s/^$lineEscaped\$/#&/" file.txt # Note the \$ to escape the end-of-line anchor $
With BSD/macOS sed, use -i '' instead of just -i for in-place updating without backup.
And the reverse (un-commenting):
sed -i "s/^#\($lineEscaped\)\$/\1/" file.txt
See this answer of mine for an explanation of the sed command used for escaping, which should work with any input string.
Also note how variable $lineEscaped is only referenced once, in the regex portion of the s command, whereas the substitution-string portion simply references what the regex matched (which avoids the need to escape the variable again, using different rules):
& in the substitution string represents the entire match, and \1 the first capture group (parenthesized subexpression, \(...\)).
For simplicity, the second sed command uses double quotes in order to embed the value of shell variable $lineEscaped in the sed script, but it is generally preferable to use single-quoted scripts so as to avoid confusion between what the shell interprets up front vs. what sed ends up seeing.
For instance, $ is special to both the shell and sed, and in the above script the end-of-line anchor $ in the sed regex must therefore be escaped as \$ to prevent the shell from interpreting it.
One way to avoid confusion is to selectively splice double-quoted shell-variable references into the otherwise single-quoted script:
sed -i 's/^'"$lineEscaped"'$/#&/' file.txt
awk solution
awk offers literal string matching, which obviates the need for escaping:
awk -v line="$line" '$0 == line { $0 = "#" $0 } 1' file.txt > $$.tmp && mv $$.tmp file.txt
If you have GNU Awk v4.1+, you can use -i inplace for in-place updating.
And the reverse (un-commenting):
awk -v line="#$line" '$0 == line { $0 = substr($0, 2) } 1' file.txt > $$.tmp &&
mv $$.tmp file.txt
Perl has ways to do the quoting/escaping for you:
line=$line perl -i~ -pe '$regex = quotemeta $ENV{line}; s/^$regex/#$ENV{line}/' -- input.txt
How to find structures matching a pattern
struct struct_name {
....
....
};
I'm using
sed -n -e '/struct{/,/}/p'
how to search for any struct_name
To extract all struct definitions (POSIX-compliant command):
sed -n '/struct [^ {]\{1,\} {/,/}/p' file
More robust with respect to whitespace variations (POSIX-compliant):
sed -n '/struct[[:blank:]]\{1,\}[^ {]\{1,\}[[:blank:]]*{/,/}/p' file
Alternative, using an extended regular expression (works with both GNU and BSD/macOS sed):
sed -E -n '/struct[[:blank:]]+[^ {]+[[:blank:]]*\{/,/\}/p' file
awk alternative (awk only uses extended regexes):
awk '/struct[[:blank:]]+[^ {]+[[:blank:]]*\{/,/\}/' file
The awk solution has the added advantage that a given struct definition will also be extracted correctly if it is all on a single line: awk looks for the end of a range on the same input line as the start of the range, whereas sed does not.
To extract a specific struct definition by name:
sed doesn't support variables, so your best bet is to splice in a shell variable that the shell expands up front.
name='struct_name' # define name to search for as shell var.
sed -n '/struct '"$name"' {/,/}/p' file # splice shell var. into sed script
Note that I've deliberately not used sed -n "/struct $name {/,/}/p" - a single, double-quoted string expanded by the shell as a whole - so as to make it clear which part of the sed script is expanded by the shell up front.
This works in this simple case, but is tricky business in general, because you must ensure that the expanded variable value contains no regex/sed metacharacters that break the command.
Here's an awk alternative that uses awk variables and literal substring matching to bypass the problem of potentially having to escape the variable value:
awk -v name='struct_name' 'index($0, "struct " name " {"),/}/' file
This solution has the added advantage that the struct definition will also be extracted correctly if it is all on a single line: awk looks for the end of a range on the same input line as the start of the range, whereas sed does not.
This will search a text file for struct_name. You can use the -E switch to use a regular expression.
grep -no struct_name test.txt
The -n switch causes the line number to be included, the -o means only the matching element of the line will be displayed.
My code is like below; and it is not working.
line_number=$1;
variable1=$2;
sed '\$line_number c\
\$variable1' file > tmp.out
If I write like below then its working. Can you please suggest how can I make above code running?
sed '1 c\
<replace>abc</replace>' file > tmp.out
Take the variables out of single quotes, and you don't need any backslash before the variables:
sed "$line_number"' c\
'"$variable1" file > tmp.out
You can try this,
#!/bin/bash
line_number=$1;
variable1=$2;
sed "$line_number c\
$variable1" file > tmp.out
If you want to use your variables values inside sed, then you have use " (double quotes).
Then, It will be interpreted by shell.
I'm on Linux command line and I have file with
127.0.0.1
128.0.0.0
121.121.33.111
I want
127.0.0.1:80
128.0.0.0:80
121.121.33.111:80
I remember my colleagues were using sed for that, but after reading sed manual still not clear how to do it on command line?
You could try using something like:
sed -n 's/$/:80/' ips.txt > new-ips.txt
Provided that your file format is just as you have described in your question.
The s/// substitution command matches (finds) the end of each line in your file (using the $ character) and then appends (replaces) the :80 to the end of each line. The ips.txt file is your input file... and new-ips.txt is your newly-created file (the final result of your changes.)
Also, if you have a list of IP numbers that happen to have port numbers attached already, (as noted by Vlad and as given by aragaer,) you could try using something like:
sed '/:[0-9]*$/ ! s/$/:80/' ips.txt > new-ips.txt
So, for example, if your input file looked something like this (note the :80):
127.0.0.1
128.0.0.0:80
121.121.33.111
The final result would look something like this:
127.0.0.1:80
128.0.0.0:80
121.121.33.111:80
Concise version of the sed command:
sed -i s/$/:80/ file.txt
Explanation:
sed stream editor
-i in-place (edit file in place)
s substitution command
/replacement_from_reg_exp/replacement_to_text/ statement
$ matches the end of line (replacement_from_reg_exp)
:80 text you want to add at the end of every line (replacement_to_text)
file.txt the file name
How can this be achieved without modifying the original file?
If you want to leave the original file unchanged and have the results in another file, then give up -i option and add the redirection (>) to another file:
sed s/$/:80/ file.txt > another_file.txt
sed 's/.*/&:80/' abcd.txt >abcde.txt
If you'd like to add text at the end of each line in-place (in the same file), you can use -i parameter, for example:
sed -i'.bak' 's/$/:80/' foo.txt
However -i option is non-standard Unix extension and may not be available on all operating systems.
So you can consider using ex (which is equivalent to vi -e/vim -e):
ex +"%s/$/:80/g" -cwq foo.txt
which will add :80 to each line, but sometimes it can append it to blank lines.
So better method is to check if the line actually contain any number, and then append it, for example:
ex +"g/[0-9]/s/$/:80/g" -cwq foo.txt
If the file has more complex format, consider using proper regex, instead of [0-9].
You can also achieve this using the backreference technique
sed -i.bak 's/\(.*\)/\1:80/' foo.txt
You can also use with awk like this
awk '{print $0":80"}' foo.txt > tmp && mv tmp foo.txt
Using a text editor, check for ^M (control-M, or carriage return) at the end of each line. You will need to remove them first, then append the additional text at the end of the line.
sed -i 's|^M||g' ips.txt
sed -i 's|$|:80|g' ips.txt
sed -i 's/$/,/g' foo.txt
I do this quite often to add a comma to the end of an output so I can just easily copy and paste it into a Python(or your fav lang) array