I have scripts like this
#!/bin/sh
SHELL_CMD=busybox
and try to substitute some pattern to bash shell using makefile,
#!/bin/bash
#SHELL_CMD=busybox
Followings are the procedures in Makefile
release:
#rm -rf my_temp/
#mkdir my_temp/
#cp dir1/burn_*.sh dir1/dump_*.sh my_temp/
#cd my_temp/; \
for f in $(shell ls); do \
sed 's:#!/bin/sh\nSHELL_CMD=busybox:#!/bin/bash\n#SHELL_CMD=busybox:1' $${f} > temp; mv temp $${f}; \
done; \
cd ..;
#cd my_temp/; tar -jcv -f bash.tar.bz2 *.sh; cd ..;
My questions are:
1 in the for loop, it didn't get the correct script names in for loop.
How to patch it ?
Any better patterns in this sed substitution?
You are much better off doing the substitution without trying to match the newline in the source string. Unless you are ready to do some complex sed-fu (How can I replace a newline (\n) using sed?) you can just apply the substitution on each of the line with a
You can do this to apply the action on both 1st and 2nd lines. Also the $(shell ls) part is not needed. You can just run a shell glob expression to get you the files ending with .sh
#for f in *.sh; do \
sed -i -e '1 s:#!/bin/sh:#!/bin/bash:1' -e '2 s:^:#:1' $${f} ;\
done
If you don't want the -i in-place substitution, use the tmp file approach as you had originally shown.
Related
I have a folder of files that start with specific strings and would like to replace part of their strings using the corresponding column from textfile
Folder with files
ABC_S1_002.txt
ABC_S1_003.html
ABC_S1_007.png
NMC_D1_002.png
NMC_D2_003.html
And I have a text file that has the strings to be replaced as:
ABC ABC_newfiles
NMC NMC_extra
So the folder after renaming will be
ABC_newfiles_S1_002.txt
ABC_newfiles_S1_003.html
ABC_newfiles_S1_007.png
NMC_extra_D1_002.png
NMC_extra_D2_003.html
I tried file by file using mv
for f in ABC*; do mv "$f" "${f/ABC/ABC_newfiles}"; done
How can I read in the textfile that has the old strings in first column and replace that with new strings from second column? I tried
IFS=$'\n'; for i in $(cat file_rename);do oldName=$(echo $i | cut -d $'\t' -f1); newName=$(echo $i | cut -d $'\t' -f2); for f in oldName*; do mv "$f" "${f/oldName/newName}"; done ; done
Did not work though.
This might work for you (GNU parallel and rename):
parallel --colsep ' ' rename -n 's/{1}/{2}/' {1}* :::: textFile
This will list out the rename commands for each line in textFile.
Once the output has been checked, remove the -n option and run for real.
For a sed solution, try:
sed -E 's#(.*) (.*)#ls \1*| sed "h;s/\1/\2/;H;g;s/\\n/ /;s/^/echo mv /e"#e' testFile
Again, this will echo the mv commands out, once checked, remove echo and run for real.
Review the result of
sed -r 's#([^ ]*) (.*)#for f in \1*; do mv "$f" "${f/\1/\2}"; done#' textfile
When that looks well, you can copy paste the result or wrap it in source:
source <(sed -r 's#([^ ]*) (.*)#for f in \1*; do mv "$f" "${f/\1/\2}"; done#' textfile)
I have a working sed command that searches for rm a-file, comments it and adds another line (rm another-file) below it:
sed -e '/^rm a-file/s;^;# ;' \
-e '/# rm a-file/a rm another file' \
my.script
I am wondering if it is somehow possible to combine both edits, the commenting and the appending into one command so that I have to specify the matching pattern (/^rm a-file/) only once.
In case it matters, I am using gnu sed.
You can just regroup it like this:
SED COMMANDS:
$ cat commands.sed
/^rm a-file/{
s#^## #
a rm another file
}
INPUT:
$ cat myscript.sh
rm a-file
blabla
rm a-file
blabla2
OUTPUT:
$ sed -f commands.sed myscript.sh
# rm a-file
rm another file
blabla
# rm a-file
rm another file
blabla2
Explanations:
This will look for lines that start with rm a-file then replace ^ by # (commenting the initial rm command) then it will append the line rm another file only if the line respects the condition ^rm a-file .
This can be done in a one-liner as well, as kindly suggested by Sundeep: (https://www.gnu.org/software/sed/manual/sed.html#Commands-Requiring-a-newline)
sed -e '/^rm a-file/{s#^## #; a rm another file' -e '}'
With GNU sed, this particular case can be covered with simple substitution as well
$ sed 's/^rm a-file.*/# &\nrm another file/' my.script
# rm a-file
rm another file
blabla
# rm a-file
rm another file
blabla2
^rm a-file.* to match line starting with rm a-file, and then .* to capture rest of line for re-use
# &\nrm another file the & here would have entire matched text while \n would add the required newline
I would like to execute this make command to first replace the first line of all csv files inside the directory and then replace the # for commas through the other lines.
The second command is working fine and does what it is supposed to do, but the first one only replaces the line on the first file.
Could anyone give me a help on that?
csv:
$(DOCKER_RUN) npm run csv-generator
make format-csv
format-csv:
#sed -i '' '1 s/^.*$$/"bar","repository"/g' $(CURDIR)/foo/npm/*.csv
#sed -i '' 's/\(.*\)#/\1","/g' $(CURDIR)/foo/npm/*.csv
The reason that the first sed command "fails" is that sed doesn't reset the line counter between input files (on your system, and neither on my Mac OS X machine, see comments):
$ cat test1
a
b
g
$ cat test2
aa
bb
cc
$ sed -n '=' test1 test2 # the '=' sed command outputs line numbers
1
2
3
4
5
6
This is why the first sed command isn't doing what you want it to do, it only affects the first file's first line.
The solution is to loop over the files and call sed for each of them (untested in Makefile):
#for f in $(CURDIR)/foo/npm/*.csv; do \
sed -i '' '1 s/^.*$$/"bar","repository"/g' $f; \
done
Using find and xargs will also work, just make sure that find isn't picking up files further down in the folders.
EDIT: In light of the comments on this answer, I would recommend avoiding the use of sed -i on multiple files altogether, and convert both statements into for-loops (in this case, they may be collapsed into one loop with two statements):
#for f in $(CURDIR)/foo/npm/*.csv; do \
sed -i '' '1 s/^.*$$/"bar","repository"/g' $f; \
sed -i '' 's/\(.*\)#/\1","/g' $f; \
done
In my experience, using for-loops in Makefiles seems to be far more common compared to using find and xargs. This is probably due to incompatibility between find and xargs versions between Unices. It also makes the Makefile a lot easier to read if one uses explicit loops.
I managed to solve with:
#find $(CURDIR)/foo/npm -name "*.csv" -type f | xargs -L 1 sed -i '' '1 s/^.*$$/"bar"/g'
sed -i is creating a backup of all files in subdirectories before editing in place (as expected) but it's not actually editing files in subdirectories.
$ mkdir -p a/b
$ echo "A" > a/a.txt
$ echo "B" > a/b/b.txt
Now I have two text files, one in a one in a subdirectory of a
$ sed -i.bac "1s/^/PREPENDED /" a/**/*.txt
Backups are created for both:
$ find a
a
a/a.txt
a/a.txt.bac
a/b
a/b/b.txt
a/b/b.txt.bac
Only a.txt is edited:
$ cat a/a.txt
PREPENDED A
$ cat a/b/b.txt
B
I'm using ZSH (so I have globstar support) and I'm on Mac.
Why is this happening and how can I fix it?
It's happening because your sed invocation only has a single line 1, which happens to be in a.txt. If you want it to do it for each file then you need to invoke sed multiple times.
for f in a/**/*.txt
do
sed ... "$f"
done
Since you are needing to descend through several levels of directories, a single invocation of sed alone is not sufficient. However, using find you can accomplish what you want in a single line. If you are not familiar with find ... -exec '{}' \; it is worth taking a few minutes with startpage.com and do a quick search. In your case, the following invocation works well:
find a -type f -name "*.txt" -exec sed -i.bac 's/^/PREPENDED /' '{}' \;
Here find searches directory a and all below for any file (-type f) matching *.txt, then for each file (indicated by '{}') -exec executes sed -i.bac 's/^/PREPENDED /' and lastly an escaped \; is given to indicate the end of the -exec command.
results:
$ ls -1 a
b
a.txt
a.txt.bac
$ ls -1 a/b
b.txt
b.txt.bac
$ cat a/a.txt
PREPENDED A
$ cat a/b/b.txt
PREPENDED B
As was correctly pointed out, with globstar set shopt -s globstar it is unnecessary to use find as the following invocation of sed is sufficient:
sed -i.bac 's/^/PREPENDED /' a/**/*.txt
I want to replace
ExecStart=/sbin/runuser -l <USER> -c "/usr/bin/vncserver %i"
PIDFile=/home/<USER>/.vnc/%H %i.pid
with
ExecStart=/sbin/runuser -l root -c "/usr/bin/vncserver %i"
PIDFile=/root/.vnc/%H %i.pid
I have tried:
sed -i 's$ExecStart=/sbin/runuser -l <USER> -c "/usr/bin/vncserver %i" \n PIDFile=/home/<USER>/.vnc/%H %i.pid$ExecStart=/sbin/runuser -l root -c "/usr/bin/vncserver %i" \n PIDFile=/root/.vnc/%H %i.pid$g' file
I used $ instead of / for special characters. But that doesn't work, nothing is getting edited.
Please tell me what I am missing. Thx.
Got the Soution.Thx to the answerer :) :
sed -i 's$ExecStart=/sbin/runuser -l <USER> -c "/usr/bin/vncserver %i"$ExecStart=/sbin/runuser -l root -c "/usr/bin/vncserver %i"$g' uu && sed -i 's$PIDFile=/home/<USER>/.vnc/%H %i.pid$PIDFile=/root/.vnc/%H %i.pid$g' uu
sed primarily works on one line at a time, so you'll need to define two separate substitute operations, one for each line:
sed -e 's/-l <USER> -c/-l root -c/' \
-e 's%home/<USER>%root%' \
file
Note that the character after s is the delimiter for the regular expression. Rather than messing with backslashes, I changed to % in the second to avoid the collision with slashes.
If you're really worried, you can make your matches match more, but what I showed is unlikely to fail you. It is odd that you want /root to replace /home/<USER>. You might do better with a marker like <HOME> which can be set correctly.