I am trying to replace lines containing the word timeout with a specific value using sed. See the example below. Can someone point me to what I am doing wrong here?
There are two types of timeouts, one that is declared as a duration (i.e. with s or m appended ) and one without.
Content in text ($DIR/config.txt) :
timeout="{{key_or_default "/config/timeout" "2s"}}" # declared as a duration
interval="{{key_or_default "/config/fetchInterval" "5m"}}"
another_timeout="{{key_or_default "/config/timeout" "2"}}" # declared not as a duration
yet_another_timeout_ms="{{key_or_default "/config/timeout" "2"}}"
Expected outcome
timeout="30s" # declared as a duration
interval="{{key_or_default "/config/fetchInterval" "5m"}}" # no change needed as it does not have the word timeout
another_timeout="3000" # declared not as a duration
yet_another_timeout_ms="3000"
sed commands which I tried are below,
# to convert duration-based timeouts to 30s
sed -i 's/"{{.*timeout.*[0-9]+s."/"30s"/' $DIR/config.txt
# to convert non-duration-based timeouts to 3000
sed -i 's/"{{.*timeout.*[0-9]+.}}"/"3000"/' $DIR/config.txt
With duration based:
sed -E -i 's/([a-z_]*timeout\w*=)[^0-9]*([0-9]+[a-z]+)[^0-9]+/\1"30s"/g' $DIR/config.txt
Without duration based:
sed -E -i 's/([a-z_]*timeout\w*=)[^0-9]*([0-9]+")[^0-9]+/\1"3000"/g' $DIR/config.txt
Output:
timeout="30s"
interval="{{key_or_default "/config/fetchInterval" "5m"}}"
another_timeout="3000"
yet_another_timeout_ms="3000"
Using sed
$ sed -Ei.bak 's/\{+[^}]*timeout" "[0-9]+([a-z]*)"}+/3000\1/' input_file
timeout="3000s"
interval="{{key_or_default "/config/fetchInterval" "5m"}}"
another_timeout="3000"
Related
I have a file where some entries look like:
EMIG_BAD_ID = syscall.Errno( -0x12f)
I want to use sed to replace that negative number to make it positive,
EMIG_BAD_ID = syscall.Errno( 0x12f)
I've tried some ideas from web searches but not succeded.
E.g. this one exits with an error:
egrep EMIG_* _error.grep | \
sed -e 's/syscall.Errno(\1)/syscall.Errno(-\1)/g' _error.grep
sed: -e expression #1, char 40: Invalid back reference
What is wrong here?
If the format is exactly like you posted, you can use this fairly easy replacement:
sed 's/syscall.Errno( -/syscall.Errno( /g' _error.grep
To make the space between ( and - optional:
sed 's/syscall.Errno( \?-/syscall.Errno( /g' _error.grep
To answer your question / if you insist on using back references (and optional space), here's how to use back references:
sed 's/syscall.Errno( \?-\(.*\))/syscall.Errno(\1)/g' _error.grep
Some additional notes:
You don't need grep - if EMIG_BAD_ID is on the same line it's very easy to include that in the sed matching pattern.
You pipe from egrep to sed and let sed read from a file. That doesn't make sense. You should prefer reading files directly with sed; but if you need grep, just read from stdin (without the file argument). Specify -i with sed to perform an in-place edit.
Using sed
$ sed -E '/^EMIG/s/-([0-9]+)/\1/' input_file
EMIG_BAD_ID = syscall.Errno( 0x12f)
Thank you for your answers. Unfortunately I don't have a working answer yet.
Code below: (part of a bigger file)
cat _error.out
E2BIG = 0x40000007
EMIG_ARRAY_TOO_LARGE = -0x133
cat test.sh
cat _error.out | sed 's/=\(.*\)/= syscall.Errno(\1)/'
Result:
E2BIG = syscall.Errno( 0x40000007)
EMIG_ARRAY_TOO_LARGE = syscall.Errno( -0x133)
The problem is to use bash/grep/sed to make the negative number positive.
Thanks!
I have a file with multiple paths in the same line:
cat modules.dep
kernel/mm/zsmalloc.ko:
kernel/crypto/lzo.ko:
kernel/drivers/char/tpm/tpm_vtpm_proxy.ko: kernel/drivers/char/tpm/tpm.ko
kernel/drivers/block/virtio_blk.ko:
kernel/drivers/block/zram/zram.ko: kernel/mm/zsmalloc.ko
kernel/drivers/nvdimm/virtio_pmem.ko: kernel/drivers/nvdimm/nd_virtio.ko
kernel/drivers/nvdimm/nd_virtio.ko:
kernel/drivers/net/virtio_net.ko: kernel/drivers/net/net_failover.ko kernel/net/core/failover.ko
kernel/drivers/net/net_failover.ko: kernel/net/core/failover.ko
extra/virtio_gpu/virtio-gpu.ko: kernel/drivers/virtio/virtio_dma_buf.ko
extra/wlan_simulation/virt_wifi_sim.ko: kernel/drivers/net/wireless/virt_wifi.ko
I would like to change it to:
/lib/modules/zsmalloc.ko:
/lib/modules/lzo.ko:
/lib/modules/tpm_vtpm_proxy.ko: /lib/modules/tpm.ko
/lib/modules/virtio_blk.ko:
/lib/modules/zram.ko: /lib/modules/zsmalloc.ko
/lib/modules/virtio_pmem.ko: /lib/modules/nd_virtio.ko
/lib/modules/nd_virtio.ko:
/lib/modules/virtio_net.ko: /lib/modules/net_failover.ko /lib/modules/failover.ko
/lib/modules/net_failover.ko: /lib/modules/failover.ko
/lib/modules/virtio-gpu.ko: /lib/modules/virtio_dma_buf.ko
/lib/modules/virt_wifi_sim.ko: /lib/modules/virt_wifi.ko
But my attempt:
sed -i 's/\(.*\)\//\/lib\/modules\//g' modules.load
works only, if there is just one path per line.
How can I achieve this, via sed, with multiple paths per line?
I am using sed from BusyBox in D(ASH) Standalone.
BusyBox v1.32.1-Magisk (2021-01-21 00:17:27 PST) multi-call binary.
Usage: sed [-i[SFX]] [-nrE] [-f FILE]... [-e CMD]... [FILE]...
or: sed [-i[SFX]] [-nrE] CMD [FILE]...
-e CMD Add CMD to sed commands to be executed
-f FILE Add FILE contents to sed commands to be executed
-i[SFX] Edit files in-place (otherwise sends to stdout)
Optionally back files up, appending SFX
-n Suppress automatic printing of pattern space
-r,-E Use extended regex syntax
If no -e or -f, the first non-option argument is the sed command string.
Remaining arguments are input files (stdin if none).
This sed should work:
sed -E 's~[^[:blank:]]+/~/lib/modules/~g' modules.dep
/lib/modules/zsmalloc.ko:
/lib/modules/lzo.ko:
/lib/modules/tpm_vtpm_proxy.ko: /lib/modules/tpm.ko
/lib/modules/virtio_blk.ko:
/lib/modules/zram.ko: /lib/modules/zsmalloc.ko
/lib/modules/virtio_pmem.ko: /lib/modules/nd_virtio.ko
/lib/modules/nd_virtio.ko:
/lib/modules/virtio_net.ko: /lib/modules/net_failover.ko /lib/modules/failover.ko
/lib/modules/net_failover.ko: /lib/modules/failover.ko
/lib/modules/virtio-gpu.ko: /lib/modules/virtio_dma_buf.ko
/lib/modules/virt_wifi_sim.ko: /lib/modules/virt_wifi.ko
[^[:blank:]]+/ finds 1+ non-whitespace characters followed by a / thus matching longest string until it gets a / in each of the multiple string per line.
I'm currently trying to find a line in a file
#define IMAX 8000
and replacing 8000 with another number.
Currently, stuck trying to pipe arguments from awk into sed.
grep '#define IMAX' 1d_Euler_mpi_test.c | awk '{print $3}' | sed
Not too sure how to proceed from here.
I would do something like:
sed -i '' '/^#define IMAX 8000$/s/8000/NEW_NUMBER/' 1d_Euler_mpi_test.c
Could you please try following. Place new number's value in place of new_number too.(tested this with GNU sed)
echo "#define IMAX 8000" | sed -E '/#define IMAX /s/[0-9]+$/new_number/'
In case you are reading input from an Input_file and want to save its output into Input_file itself use following then.
sed -E '/#define IMAX /s/[0-9]+$/new_number/' Input_file
Add -i flag in above code in case you want to save output into Input_file itself. Also my codes will catch any digits which are coming at the end of the line which has string #define IMAX so in case you only want to look for 8000 or any fixed number change [0-9]+$ to 8000 etc in above codes then.
You may use GNU sed.
sed -i -e 's/IMAX 8000/IMAX 9000/g' /tmp/file.txt
Which will invoke sed to do an in-place edit due to the -i option. This can be called from bash.
If you really really want to use just bash, then the following can work:
while read a ; do echo ${a//IMAX 8000/IMAX 9000} ; done < /tmp/file.txt > /tmp/file.txt.t ; mv /tmp/file.txt{.t,}
This loops over each line, doing a substitution, and writing to a temporary file (don't want to clobber the input). The move at the end just moves temporary to the original name.
Is there some way in sed to create multiple output lines from a single input line? I have a template file (there are more lines in the file, I'm just simplifying it):
http://hostname:#PORT#
I am currently using sed to replace #PORT# with a real port. However, I'd like to be able to pass in multiple ports, and have sed create a line for each. Is that possible?
I'm assuming you would want to duplicate the whole line for each port number. In that case it's easier to think of it as replacing the port numbers with the URL:
$ cat ports.in
1
2
3
4
5
$ sed 's#^\([0-9]*\$)#http://hostname:\1#' ports.in
http://hostname:1
http://hostname:2
http://hostname:3
http://hostname:4
http://hostname:5
To do it the other way around is easier with awk:
$ cat url.in
http://hostname:#PORT#
$ awk '/^[0-9]/ {ports[++i]=$0} /^http/ {sub(":#PORT#", ":%d\n"); for (p in ports) printf($0, ports[p])}' ports.in url.in
http://hostname:2
http://hostname:3
http://hostname:4
http://hostname:5
http://hostname:1
This reads both ports.in and url.in, and if a line starts with a number it is assumed that it's a port number from ports.in. Otherwise, if the line starts with http it's assumed to be an URL from url.in and will replace the port placeholder with a printf formatting string and then print the URL once for each port number read. It will fail to do the right thing if the files are fed in the wrong order.
A similar solution, but taking the URL from a shell variable:
$ myurl="http://hostname:#PORT#"
$ awk -v url="$myurl" 'BEGIN{sub(":#PORT#", ":%d\n",url)} /^[0-9]/ {ports[++i]=$0} END {for (p in ports) printf(url, ports[p])}' ports.in
http://hostname:2
http://hostname:3
http://hostname:4
http://hostname:5
http://hostname:1
It seems you have multiple templates and multiple ports to apply to them. Here's how to do it in a shell script (tested with bash), but you'll need to do it in two sed executions if you want to keep it simple because you have two multiply valued inputs. It is mathematically a cross product of the templates and the substitution values.
ports='80
8080
8081'
templates='http://domain1.net:%PORT/
http://domain2.org:%PORT/
http://domain3.com:%PORT/'
meta="s/(.*)/g; s|%PORT|\1|p; /p"
sed="`echo \"$ports\" |sed -rn \"$meta\" |tr '\n' ' '`"
echo "$templates" |sed -rn "h; $sed"
The shell variable meta is a meta sed script because it writes another sed script. The h saves the pattern buffer in the sed hold space. The sed commands generated from the meta sed recall, substitute, and print for each port. This is the result.
http://domain1.net:80/
http://domain1.net:8080/
http://domain1.net:8081/
http://domain2.org:80/
http://domain2.org:8080/
http://domain2.org:8081/
http://domain3.com:80/
http://domain3.com:8080/
http://domain3.com:8081/
Following is the content of a file:
xxx_component1-1.0-2-2acd314.xc-linux-x86-64-Release-devel.r
xxx_component2-3.0-1-fg3sdhd.xc-linux-x86-64-Release-devel.r
xxx_component3-1.0-2-3gsjcgd.xc-linux-x86-64-Release-devel.r
xxx_component4-0.0-2-2acd314.xc-linux-x86-64-Release-devel.r
I want to extract component names component1 component2 etc.
This is what I tried:
for line in `sed -n -e '/^xxx-/p' $file`
do
comp=`echo $line | sed -e '/xxx-/,/[0-9]/p'`
echo "comp - $comp"
done
I also tried sed -e 's/.*xxx-\(.*\)[^0-9].*/\1/'
This is based on some info on net. Please give me sed command and if possible also explain stepwise
Part 2. I also need to extract version number from the string.
version number starts with digit and ends with . followed by xc-linux.
As you can see to maintain the uniqueness its has random alphanumeric characters ( length is 7) as part of the version number.
For example :
xxx_component1-1.0-2-2acd314.xc-linux-x86-64-Release-devel.r
In this string the version number is : 1.0-2-2acd314
There are quite a few ways to extract the data. The simplest form would be grep.
GNU grep:
You can grab the required data using GNU grep with PCRE option -P:
$ cat file
xxx_component1-1.0-2-2acd314.xc-linux-x86-64-Release-devel.r
xxx_component2-3.0-1-fg3sdhd.xc-linux-x86-64-Release-devel.r
xxx_component3-1.0-2-3gsjcgd.xc-linux-x86-64-Release-devel.r
xxx_component4-0.0-2-2acd314.xc-linux-x86-64-Release-devel.r
$ grep -oP '(?<=_)[^-]*' file
component1
component2
component3
component4
Here we use negative look behind assertion tell to capture everything from _ to a - not incusive.
awk:
$ awk -F"[_-]" '{print $2}' file
component1
component2
component3
component4
Here we tell awk to use - and _ as delimiters and print the second column.
sed:
Having said that, you can also use sed to extract required data using group capture:
$ sed 's/.*_\([^-]*\)-.*/\1/' file
component1
component2
component3
component4
The regex states that match any character zero or more times up to an _. From that point onwards, capture everything until a - in a group. In the replacement part we just use the data captured in the group by calling it using back reference, that is \1.