I have a textfile and would like to save all lines that are ending with .m2 or .M2. I tried several ways including this here
D:\filetype\core\sed.exe -n -e "s/^\(.*\)\.M2//" D:\filetype\listfile\test.txt > D:\filetype\listfile\test2.txt
But as result i only get a emtpy textfile, so i guess something is wrong with my code.
The other way was
D:\filetype\core\sed.exe -n -e "^/\(.*)\/.M2\/" D:\filetype\listfile\test.txt > D:\filetype\listfile\test2.txt
But in this case i wasn't able to locate the source of the error
unknown command: `^'
Thanks if someone can see my fault.
You can use below sed command:
sed -n -e '/\.[mM]2$/p' <file_name>
This will print all the lines which have .m2 or .M2 at the end
Now comming to issues with your commands. Your first command does:
sed -n -e "s/^\(.*\)\.M2//"
which is a search and replace command indiacated by s in the command. Syntax for this command is s/search_text/replace_text/. Now if you are look at your command carefully, you are searching for something and replacing it with nothing - as indicated by last // in your command - where replace_text is missing.
Your second command does
sed -n -e "^/\(.*)\/.M2\/"
which is incorrect syntax. General syntax of a sed command is :
sed -e 'line_range command'
where line range - which is optional - can be line numbers like1, 5 , 2-5, or a regular expression like /[gG]/, /[gG][iIuU]rl/.
If line_range is missing, the first letter in sed command should be a command. In your case: line_range is missing - which is fine syntax wise -, however the first letter is ^ - which is not a sed command - because of which you are getting syntax error.
The command that I suggested is
sed -n -e '/\.[mM]2$/p'
Here, line_range is the regular expression /\.[mM]2$/ - which says "any line which has .m2 or .M2 at the end", and command is p, which is the letter for print command.
Sed is mostly used to transform text. Why not use grep instead?
grep -i "\.m2$"
This will match case insensitively (-i) any line ending with .m2.
Related
I have a file with multiple paths in the same line:
cat modules.dep
kernel/mm/zsmalloc.ko:
kernel/crypto/lzo.ko:
kernel/drivers/char/tpm/tpm_vtpm_proxy.ko: kernel/drivers/char/tpm/tpm.ko
kernel/drivers/block/virtio_blk.ko:
kernel/drivers/block/zram/zram.ko: kernel/mm/zsmalloc.ko
kernel/drivers/nvdimm/virtio_pmem.ko: kernel/drivers/nvdimm/nd_virtio.ko
kernel/drivers/nvdimm/nd_virtio.ko:
kernel/drivers/net/virtio_net.ko: kernel/drivers/net/net_failover.ko kernel/net/core/failover.ko
kernel/drivers/net/net_failover.ko: kernel/net/core/failover.ko
extra/virtio_gpu/virtio-gpu.ko: kernel/drivers/virtio/virtio_dma_buf.ko
extra/wlan_simulation/virt_wifi_sim.ko: kernel/drivers/net/wireless/virt_wifi.ko
I would like to change it to:
/lib/modules/zsmalloc.ko:
/lib/modules/lzo.ko:
/lib/modules/tpm_vtpm_proxy.ko: /lib/modules/tpm.ko
/lib/modules/virtio_blk.ko:
/lib/modules/zram.ko: /lib/modules/zsmalloc.ko
/lib/modules/virtio_pmem.ko: /lib/modules/nd_virtio.ko
/lib/modules/nd_virtio.ko:
/lib/modules/virtio_net.ko: /lib/modules/net_failover.ko /lib/modules/failover.ko
/lib/modules/net_failover.ko: /lib/modules/failover.ko
/lib/modules/virtio-gpu.ko: /lib/modules/virtio_dma_buf.ko
/lib/modules/virt_wifi_sim.ko: /lib/modules/virt_wifi.ko
But my attempt:
sed -i 's/\(.*\)\//\/lib\/modules\//g' modules.load
works only, if there is just one path per line.
How can I achieve this, via sed, with multiple paths per line?
I am using sed from BusyBox in D(ASH) Standalone.
BusyBox v1.32.1-Magisk (2021-01-21 00:17:27 PST) multi-call binary.
Usage: sed [-i[SFX]] [-nrE] [-f FILE]... [-e CMD]... [FILE]...
or: sed [-i[SFX]] [-nrE] CMD [FILE]...
-e CMD Add CMD to sed commands to be executed
-f FILE Add FILE contents to sed commands to be executed
-i[SFX] Edit files in-place (otherwise sends to stdout)
Optionally back files up, appending SFX
-n Suppress automatic printing of pattern space
-r,-E Use extended regex syntax
If no -e or -f, the first non-option argument is the sed command string.
Remaining arguments are input files (stdin if none).
This sed should work:
sed -E 's~[^[:blank:]]+/~/lib/modules/~g' modules.dep
/lib/modules/zsmalloc.ko:
/lib/modules/lzo.ko:
/lib/modules/tpm_vtpm_proxy.ko: /lib/modules/tpm.ko
/lib/modules/virtio_blk.ko:
/lib/modules/zram.ko: /lib/modules/zsmalloc.ko
/lib/modules/virtio_pmem.ko: /lib/modules/nd_virtio.ko
/lib/modules/nd_virtio.ko:
/lib/modules/virtio_net.ko: /lib/modules/net_failover.ko /lib/modules/failover.ko
/lib/modules/net_failover.ko: /lib/modules/failover.ko
/lib/modules/virtio-gpu.ko: /lib/modules/virtio_dma_buf.ko
/lib/modules/virt_wifi_sim.ko: /lib/modules/virt_wifi.ko
[^[:blank:]]+/ finds 1+ non-whitespace characters followed by a / thus matching longest string until it gets a / in each of the multiple string per line.
having trouble with a sed command.
I'm looking to find a line in a file and replace it.
In my script I've used this command without issue; (I use it to set variables)
sed -i '/job=empty/c\job='$job'' $sd/pingcheck-mon-$job.sh
The line I want to replace looks like this,
bash home/user/pingcheck/pingcheck-jobs/job1/pingcheck-mon-job1.sh
This is the command I can't get to run:
sed -i '/bash '$sd'/pingcheck-mon-'$job'.sh/c\jobslot=empty' $wd/pingcheck-worker.sh
Error I get:
sed: -e expression #1, char 9: extra characters after command
Could someone please tell me where I'm going wrong?
Thanks in advance!
I am trying to add a line in a file using sed after first match. I looked my threads online, addressing the same issue and tried below versions of the command:
sudo sed -e '/cfg_file/cfg_file='$NEW_FILE -e '/cfg_file/cfg_file=/q' MAIN_CONF_FILE
sudo sed -i '0,/cfg_file/ a\cfg_file='$NEW_FILE $MAIN_CONF_FILE
sudo sed -i '1s/cfg_file/cfg_file='$NEW_FILE $MAIN_CONF_FILE
sudo sed -i '1/cfg_file/s/cfg_file='$NEW_FILE $MAIN_CONF_FILE
Unfortunately, nothing worked for me. Either they show error, in case of point 3, or show similar behavior of adding lines after each match.
SAMPLE FILE
cfg_file=some_line1
cfg_file=some_line2
Now I want to add a line after first match of cg_file, like below.
EXPECTED RESULT
cfg_file=some_line1
cfg_file=my_added_line_after_first_match_only.
cfg_file=some_line2
Help me in adding line after first match and correcting my command.
Since you're on Ubuntu, you are using GNU sed. GNU sed has some weird features and some useful ones; you should ignore the weird ones and use the useful ones.
In context, the useful one is ranges starting at line 0. Weird ones are the way it messes with a, i and c commands.
MAIN_CONF_FILE=/tmp/copy.of.main.config.file
NEWFILE="my_added_line_after_first_match_only."
sed -e '0,/^cfg_file=/ { /^cfg_file/ a\' \
-e "cfg_file=$NEWFILE" \
-e '}' \
"$MAIN_CONF_FILE"
In classic sed, the a command is followed by backslash-newline, and each subsequent line of the script is appended up to and including the first line without a backslash at the end (and the backslash is removed). Each -e argument functions as a line in the script. Distinguish between the shell lines escaped with backslash at the end and the sed script lines with backslash at the end.
Example at work
$ cat /tmp/copy.of.main.config.file | so
cfg_file=some_line1
cfg_file=some_line2
$ cat script.sh
MAIN_CONF_FILE=/tmp/copy.of.main.config.file
NEWFILE="my_added_line_after_first_match_only."
SED=/opt/gnu/bin/sed
${SED} \
-e '0,/^cfg_file=/ { /^cfg_file/ a\' \
-e "cfg_file=$NEWFILE" \
-e '}' \
"$MAIN_CONF_FILE"
$ bash script.sh
cfg_file=some_line1
cfg_file=my_added_line_after_first_match_only.
cfg_file=some_line2
$
This is based on your attempt 2, but avoids some of the weird stuff.
Basic sanity
As I noted, it is not sensible to experiment with sudo and the -i option to sed. You don't use those until you know that the script will do the job correctly. It is dangerous to do anything as root via sudo. It is doubly dangerous when you don't know whether what you're trying to use will work. Don't risk wrecking your system.
I have tried to scan through the other posts in stack overflow for this, but couldn't get my code work, hence I am posting a new question.
Below is the content of file temp.
<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/<env:Body><dp:response xmlns:dp="http://www.datapower.com/schemas/management"><dp:timestamp>2015-01-
22T13:38:04Z</dp:timestamp><dp:file name="temporary://test.txt">XJzLXJlc3VsdHMtYWN0aW9uX18i</dp:file><dp:file name="temporary://test1.txt">lc3VsdHMtYWN0aW9uX18i</dp:file></dp:response></env:Body></env:Envelope>
This file contains the base64 encoded contents of two files names test.txt and test1.txt. I want to extract the base64 encoded content of each file to seperate files test.txt and text1.txt respectively.
To achieve this, I have to remove the xml tags around the base64 contents. I am trying below commands to achieve this. However, it is not working as expected.
sed -n '/test.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test.txt">##g'|perl -p -e 's#</dp:file>##g' > test.txt
sed -n '/test1.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test1.txt">##g'|perl -p -e 's#</dp:file></dp:response></env:Body></env:Envelope>##g' > test1.txt
Below command:
sed -n '/test.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test.txt">##g'|perl -p -e 's#</dp:file>##g'
produces output:
XJzLXJlc3VsdHMtYWN0aW9uX18i
<dp:file name="temporary://test1.txt">lc3VsdHMtYWN0aW9uX18i</dp:response> </env:Body></env:Envelope>`
Howeveer, in the output I am expecting only first line XJzLXJlc3VsdHMtYWN0aW9uX18i. Where I am commiting mistake?
When i run below command, I am getting expected output:
sed -n '/test1.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test1.txt">##g'|perl -p -e 's#</dp:file></dp:response></env:Body></env:Envelope>##g'
It produces below string
lc3VsdHMtYWN0aW9uX18i
I can then easily route this to test1.txt file.
UPDATE
I have edited the question by updating the source file content. The source file doesn't contain any newline character. The current solution will not work in that case, I have tried it and failed. wc -l temp must output to 1.
OS: solaris 10
Shell: bash
sed -n 's_<dp:file name="\([^"]*\)">\([^<]*\).*_\1 -> \2_p' temp
I add \1 -> to show link from file name to content but for content only, just remove this part
posix version so on GNU sed use --posix
assuming that base64 encoded contents is on the same line as the tag around (and not spread on several lines, that need some modification in this case)
Thanks to JID for full explaination below
How it works
sed -n
The -n means no printing so unless explicitly told to print, then there will be no output from sed
's_
This is to substitute the following regex using _ to separate regex from the replacement.
<dp:file name=
Regular text
"\([^"]*\)"
The brackets are a capture group and must be escaped unless the -r option is used( -r is not available on posix). Everything inside the brackets is captured. [^"]* means 0 or more occurrences of any character that is not a quote. So really this just captures anything between the two quotes.
>\([^<]*\)<
Again uses the capture group this time to capture everything between the > and <
.*
Everything else on the line
_\1 -> \2
This is the replacement, so replace everything in the regex before with the first capture group then a -> and then the second capture group.
_p
Means print the line
Resources
http://unixhelp.ed.ac.uk/CGI/man-cgi?sed
http://www.grymoire.com/Unix/Sed.html
/usr/xpg4/bin/sed works well here.
/usr/bin/sed is not working as expected in case if the file contains just 1 line.
below command works for a file containing only single line.
/usr/xpg4/bin/sed -n 's_<env:Envelope\(.*\)<dp:file name="temporary://BackUpDir/backupmanifest.xml">\([^>]*\)</dp:file>\(.*\)_\2_p' securebackup.xml 2>/dev/null
Without 2>/dev/null this sed command outputs the warning sed: Missing newline at end of file.
This because of the below reason:
Solaris default sed ignores the last line not to break existing scripts because a line was required to be terminated by a new line in the original Unix implementation.
GNU sed has a more relaxed behavior and the POSIX implementation accept the fact but outputs a warning.
So, I have looked around for an answer to this, and indeed I have found some, but none seem to work...
I have a folder full of bash scripts that I need to modify. specifically, I need to replace the line
INPUT=/data/scratch02/mpgussert/HAWC-30/${RUN}_reco
with
INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/${RUN}
I have tried this
perl -w -i -p -e "s'INPUT=/data/scratch02/mpgussert/HAWC-30/${RUN}_reco'INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/${RUN}'g" *.sh
which executes without error, but does not find and replace the desired text. From my understanding, using ' to deliminate the regex should search without special character replacement. Is that correct? If so, any ideas why it fails?
I have also tried
perl -w -i -p -e "s/INPUT=\/data\/scratch02\/mpgussert\/HAWC-30\/\$\{RUN\}_reco/INPUT=\/data\/hawc01\/hawcroot\/data\/hawc\/reconstructed\/quesadilla\/\$\{RUN\}/g" *.sh
the backslash should ignore special character replacement, but this returns the following error.
Backslash found where operator expected at -e line 1, near "RUN\"
syntax error at -e line 1, near "RUN\"
Execution of -e aborted due to compilation errors.
So it's searching for RUN\, which is not what I want... Any thoughts? I would appreciate any help you can give.
Thanks!
You want the pattern to be ...\$\{RUN\}..., but that's not what you're passing:
$ echo "...\$\{RUN\}..."
...$\{RUN\}...
You either need do more escaping, or switch to single quotes.
$ echo '...\$\{RUN\}...'
...\$\{RUN\}...
All together:
perl -i -wpe'
s{INPUT=/data/scratch02/mpgussert/HAWC-30/\$\{RUN\}_reco}
{INPUT=/data/hawc01/hawcroot/data/hawc/reconstructed/quesadilla/\${RUN}}g
' *.sh