So I have a project that uses sed and bash and I'm trying to understand why on a command line output it gives me one value whereas in actual project it gives me a different value.
In the command line I'm executing
sed 's/||/ || /g' filename
and in the sed file I'm inputting
s/||/ || /g
Now the first command in the command line gives me the proper output but I'm trying to understand why when I use the command in the sed file it's not giving me the same output.
In the bash file I have I'm calling the sed file as such
sed -rf sedFile $1
and then running the program as
./bashfile.bash fileName
sed 's/||/ || /g' filename
is totatlly different from
sed -rf sedFile filename
The latter is essentially
sed -r 's/||/ || /g' filename
and it will generate a totally different results because of -r will change the meaning of | from literal vertial bar to alternation. Thus, the || in the search pattern matches a zero length string, or a zero length string, or a zero length string, which means it will match any position (zero length) between the characters (including before the first character and after the last character).
echo ,,,, | sed -r 's/||/_||_/g' # underscores for visibility in the output
has the following output
_||_,_||_,_||_,_||_,_||_
By the way, s/|/ || /g would have the same effect, whereas s// || /g is invalid.
Related
I'm trying to run the command below to replace every char in DECEMBER by itself followed by $n question marks. I tried both escaping {$n} like so {$n} and leaving it as is. Yet my output just keeps being D?{$n}E?{$n}... Is it just not possible to do this with a sed?
How should i got about this.
echo 'DECEMBER' > a.txt
sed -i "s%\(.\)%\1\(?\){$n}%g" a.txt
cat a.txt
This might work for you (GNU sed):
n=5
sed -E ':a;s/[^\n]/&\n/g;x;s/^/x/;/x{'"$n"'}/{z;x;y/\n/?/;b};x;ba' file
Append a newline to each non-newline character in a line $n times then replace all newlines by the intended character ?.
N.B. The newline is chosen as the initial substitute character as it is not possible for it to be within a line (sed uses newlines to separate lines) and if the final substitution character already exists within the current line, the substitutions are correct.
Range (also, interval or limiting quantifiers), like {3} / {3,} / {3,6}, are part of regex, and not replacement patterns.
You can use
sed -i "s/./&$(for i in {1..7}; do echo -n '?'; done)/g" a.txt
See the online demo:
#!/bin/bash
sed "s/./&$(for i in {1..7}; do echo -n '?'; done)/g" <<< "DECEMBER"
# => D???????E???????C???????E???????M???????B???????E???????R???????
Here, . matches any char, and & in the replacement pattern puts it back and $(for i in {1..7}; do echo -n '?'; done) adds seven question marks right after it.
This one-liner should do the trick:
sed 's/./&'$(printf '%*s' "$n" '' | tr ' ' '?')'/g' a.txt
with the assumption that $n expands to a positive integer and the command is executed in a POSIX shell.
Efficiently using any awk in any shell on every Unix box after setting n=2:
$ awk -v n="$n" '
BEGIN {
new = sprintf("%*s",n,"")
gsub(/./,"?",new)
}
{
gsub(/./,"&"new)
print
}
' a.txt
D??E??C??E??M??B??E??R??
To make the changes "inplace" use GNU awk with -i inplace just like GNU sed has -i.
Caveat - if the character you want to use in the replacement text is & then you'd need to use gsub(/./,"\\\\\\&",new) in the BEGIN section to make it is treated as literal instead of a backreference metachar. You'd have that issue and more (e.g. handling \1 or /) with any sed solution and any solution that uses double quotes around the script would have more issues with handling $s and the solutions that have a shell script expanding unquoted would have even more issues with globbing chars.
I have a file with multiple paths in the same line:
cat modules.dep
kernel/mm/zsmalloc.ko:
kernel/crypto/lzo.ko:
kernel/drivers/char/tpm/tpm_vtpm_proxy.ko: kernel/drivers/char/tpm/tpm.ko
kernel/drivers/block/virtio_blk.ko:
kernel/drivers/block/zram/zram.ko: kernel/mm/zsmalloc.ko
kernel/drivers/nvdimm/virtio_pmem.ko: kernel/drivers/nvdimm/nd_virtio.ko
kernel/drivers/nvdimm/nd_virtio.ko:
kernel/drivers/net/virtio_net.ko: kernel/drivers/net/net_failover.ko kernel/net/core/failover.ko
kernel/drivers/net/net_failover.ko: kernel/net/core/failover.ko
extra/virtio_gpu/virtio-gpu.ko: kernel/drivers/virtio/virtio_dma_buf.ko
extra/wlan_simulation/virt_wifi_sim.ko: kernel/drivers/net/wireless/virt_wifi.ko
I would like to change it to:
/lib/modules/zsmalloc.ko:
/lib/modules/lzo.ko:
/lib/modules/tpm_vtpm_proxy.ko: /lib/modules/tpm.ko
/lib/modules/virtio_blk.ko:
/lib/modules/zram.ko: /lib/modules/zsmalloc.ko
/lib/modules/virtio_pmem.ko: /lib/modules/nd_virtio.ko
/lib/modules/nd_virtio.ko:
/lib/modules/virtio_net.ko: /lib/modules/net_failover.ko /lib/modules/failover.ko
/lib/modules/net_failover.ko: /lib/modules/failover.ko
/lib/modules/virtio-gpu.ko: /lib/modules/virtio_dma_buf.ko
/lib/modules/virt_wifi_sim.ko: /lib/modules/virt_wifi.ko
But my attempt:
sed -i 's/\(.*\)\//\/lib\/modules\//g' modules.load
works only, if there is just one path per line.
How can I achieve this, via sed, with multiple paths per line?
I am using sed from BusyBox in D(ASH) Standalone.
BusyBox v1.32.1-Magisk (2021-01-21 00:17:27 PST) multi-call binary.
Usage: sed [-i[SFX]] [-nrE] [-f FILE]... [-e CMD]... [FILE]...
or: sed [-i[SFX]] [-nrE] CMD [FILE]...
-e CMD Add CMD to sed commands to be executed
-f FILE Add FILE contents to sed commands to be executed
-i[SFX] Edit files in-place (otherwise sends to stdout)
Optionally back files up, appending SFX
-n Suppress automatic printing of pattern space
-r,-E Use extended regex syntax
If no -e or -f, the first non-option argument is the sed command string.
Remaining arguments are input files (stdin if none).
This sed should work:
sed -E 's~[^[:blank:]]+/~/lib/modules/~g' modules.dep
/lib/modules/zsmalloc.ko:
/lib/modules/lzo.ko:
/lib/modules/tpm_vtpm_proxy.ko: /lib/modules/tpm.ko
/lib/modules/virtio_blk.ko:
/lib/modules/zram.ko: /lib/modules/zsmalloc.ko
/lib/modules/virtio_pmem.ko: /lib/modules/nd_virtio.ko
/lib/modules/nd_virtio.ko:
/lib/modules/virtio_net.ko: /lib/modules/net_failover.ko /lib/modules/failover.ko
/lib/modules/net_failover.ko: /lib/modules/failover.ko
/lib/modules/virtio-gpu.ko: /lib/modules/virtio_dma_buf.ko
/lib/modules/virt_wifi_sim.ko: /lib/modules/virt_wifi.ko
[^[:blank:]]+/ finds 1+ non-whitespace characters followed by a / thus matching longest string until it gets a / in each of the multiple string per line.
I'm trying to add array brackets at the beginning and end of a file using sed (after first removing the trailing comma at the end of the file) to put all the content of the file in an array. I'm first using this sed command to remove the last comma from a file
sed '$ s/,$//' "$path"
After that, I'm using the middle command below to add array brackets at beginning and ending of file
sed '$ s/,$//' "$path" | sed 's/^.*$/[&]/' | tee $filename
This sed 's/^.*$/[&]/' was supposed to match everything (from beginning to end ^$) and then put brackets around the whole match [&] (i.e. as if to make it into an array), but it instead put array brackets around the beginning and end of each line.
Question, how to edit the beginning and ending of a file with sed?
whole script
for path in dirname/* do
name="${path##*/}"
sed '$ s/,$//' "$path" | sed 's/^.*$/[&]/' | tee "newdir/$name"
done
sed is an editor that works line by line, so the command sed 's/^.*$/[&]/' would add brackets to every line. If you want to edit just the beginning and end of a file you need to put line numbers in front of the substitutions ($ stands for the last line):
sed -e '1 s/^/[/' -e '$ s/$/]/'
Since you already have a command that removes trailing ,'s you could combine it with the aforementioned substitutions. Your command line would then look like this:
sed -e '1 s/^/[/' -e '$ s/,*$/]/' "$path" | tee $filename
I have a file in which some lines start by a >
For these lines, and only these ones, I want to keep the first eleven characters.
How can I do that using sed ?
Or maybe something else is better ?
Thanks !
Muriel
Let's start with this test file:
$ cat file
line one with something or other
>1234567890abc
other line in file
To keep only the first 11 characters of lines starting with > while keeping all other lines:
$ sed -r '/^>/ s/(.{11}).*/\1/' file
line one with something or other
>1234567890
other line in file
To keep only the first eleven characters of lines starting with > and deleting all other lines:
$ sed -rn '/^>/ s/(.{11}).*/\1/p' file
>1234567890
The above was tested with GNU sed. For BSD sed, replace the -r option with -E.
Explanation:
/^>/ is a condition. It means that the command which follows only applies to lines that start with >
s/(.{11}).*/\1/ is a substitution command. It replaces the whole line with just the first eleven characters.
-r turns on extended regular expression format, eliminating the need for some escape characters.
-n turns off automatic printing. With -n in effect, lines are only printed if we explicitly ask them to be printed. In the second case above, that is done by adding a p after the substitute command.
Other forms:
$ sed -r 's/(>.{10}).*/\1/' file
line one with something or other
>1234567890
other line in file
And:
$ sed -rn 's/(>.{10}).*/\1/p' file
>1234567890
This code removes newlines, then quotes and separates the characters.
Semicolon isn't working. Piping to a second sed does work, but the semicolon doesn't.
Script
# Piping works
echo "$1" | sed -r ':a;N;s/\n/\\n/;$!ba' | sed -r 's/(\\?.)/'"'\1',/g"
# Semicolon doesn't work on single lines
echo "$1" | sed -r ':a;N;s/\n/\\n/;$!ba;s/(\\?.)/'"'\1',/g"
# Skipping N command on single line works
echo "$1" | sed -r ':a;$bb;N;s/\n/\\n/;$!ba;:b;s/(\\?.)/'"'\1',/g"
Output:
$ wchar "test\n"
't','e','s','t','\n',
test\n
't','e','s','t','\n',
$ wchar "test
test"
't','e','s','t','\n','t','e','s','t',
't','e','s','t','\n','t','e','s','t',
't','e','s','t','\n','t','e','s','t',
The problem you run into is that the first line is also the last one (note that "test\n" does not contain a newline but a backslash), so the first N command is executed before any test whether the current line is the last one and ends up trying to fetch past the end.
Since all you're trying to do with the label and N loop is to assemble the file into the hold buffer (the replacement of the newlines can wait until afterwards), I suggest the following replacement:
echo -e "test\n" | sed -rn '1h;1!H;${x;s/\n/\\n/g; s/(\\?.)/'\''\1'\'', /g;p}'
This follows the basic pattern
sed -n '1 h; 1! H; $ { x; do stuff; p }'
...in other words, it reads the file into the hold buffer, swaps it back into the pattern space when the last line is being handled (i.e., when the whole file is in the hold buffer), does stuff with it and then prints the result. The s commands in place of do stuff are lifted from your code.