Can someone please give me the exact syntax for removing ^# from thousands of html files in nested directories using sed? The ^# is a control character inserted by a windows program that generated these files. I cannot seem to get the syntax right.
I tried this (but it did not work) using a file since I could not enter the control-character at the command prompt:
find ./ *.html -type f -exec sed -i 's/^#//g' {} ;
POSIX sed doesn't handle NUL in input but GNU sed can with hex escape:
find . -name '*.html' -type f -exec sed -i 's/\x0//g' '{}' +
Kind of new to Perl, still navigating my way through.
Is there another way to write the bash command below in "Perl"?
find $INPUT_DIR -ctime -$DAYS_NUM -type f -exec grep -hs EDI_DC {} \; |
grep -i -v xml >> $OUTPUT_DIR/$OUTPUT_FILENAME
where INPUT_DIR, DAYS_NUM, OUTPUT_DIR and OUTPUT_FILENAME are arguments passed during runtime.
When you try to convert find command to perl, consider using find2perl script.
It generate the perl code.
find2perl 'INPUT_DIR' -ctime -'DAYS_NUM' -type f -exec grep -hs EDI_DC {} \;
I want to run something like:
find . -type files -exec echo "touch -cmd '"$(date --utc -r '{}' +"%Y-%m-%d %H:%M:%S.%N +0000")"' "$(ls --quoting-style=shell '{}')
It doesn't seem to work since "{}" doesn't seem to be expanded in the $(). How could I do to make it work right?
Simplify your problem.
Write a shell script, and then get the shell script working. Then execute your shell script using find.
find . -type files -exec myScript '{}'
Why is there a difference in output between using
find . -exec ls '{}' \+
and
find . -exec ls '{}' \;
I got:
$ find . -exec ls \{\} \+
./file1 ./file2
.:
file1 file2 testdir1
./testdir1:
testdir2
./testdir1/testdir2:
$ find . -exec ls \{\} \;
file1 file2 testdir1
testdir2
./file2
./file1
This might be best illustrated with an example. Let's say that find turns up these files:
file1
file2
file3
Using -exec with a semicolon (find . -exec ls '{}' \;), will execute
ls file1
ls file2
ls file3
But if you use a plus sign instead (find . -exec ls '{}' \+), as many filenames as possible are passed as arguments to a single command:
ls file1 file2 file3
The number of filenames is only limited by the system's maximum command line length. If the command exceeds this length, the command will be called multiple times.
All of the answers so far are correct. I offer this as a clearer (to me) demonstration of the behaviour that is described using echo rather than ls:
With a semicolon, the command echo is called once per file (or other filesystem object) found:
$ find . -name 'test*' -exec echo {} \;
./test.c
./test.cpp
./test.new
./test.php
./test.py
./test.sh
With a plus, the command echo is called once only. Every file found is passed in as an argument.
$ find . -name 'test*' -exec echo {} \+
./test.c ./test.cpp ./test.new ./test.php ./test.py ./test.sh
If find turns up large numbers of results, you may find that the command being called chokes on the number of arguments.
From man find:
-exec command ;
Execute command; true if 0 status is returned. All following
arguments to find are taken to be arguments to the command until
an argument consisting of ';' is encountered. The string '{}'
is replaced by the current file name being processed everywhere
it occurs in the arguments to the command, not just in arguments
where it is alone, as in some versions of find. Both of these
constructions might need to be escaped (with a '\') or quoted to
protect them from expansion by the shell. See the EXAMPLES sec
section for examples of the use of the '-exec' option. The
specified command is run once for each matched file.
The command is executed in the starting directory. There are
unavoidable security problems surrounding use of the -exec option;
you should use the -execdir option instead.
-exec command {} +
This variant of the -exec option runs the specified command on
the selected files, but the command line is built by appending
each selected file name at the end; the total number of
invocations of the command will be much less than the number of
matched files. The command line is built in much the same way
that xargs builds its command lines. Only one instance of '{}'
is allowed within the command. The command is executed in
the starting directory.
So, the way I understand it, \; executes a separate command for each file found by find, whereas \+ appends the files and executes a single command on all of them. The \ is an escape character, so it's:
ls testdir1; ls testdir2
vs
ls testdir1 testdir2
Doing the above in my shell mirrored the output in your question.
example of when you would want to use \+
Suppose two files, 1.tmp and 2.tmp:
1.tmp:
1
2
3
2.tmp:
0
2
3
With \;:
find *.tmp -exec diff {} \;
> diff: missing operand after `1.tmp'
> diff: Try `diff --help' for more information.
> diff: missing operand after `2.tmp'
> diff: Try `diff --help' for more information.
Whereas if you use \+ (to concatenate the results of find):
find *.tmp -exec diff {} \+
1c1,3
< 1
---
> 0
> 2
> 30
So in this case it's the difference between diff 1.tmp; diff 2.tmp and diff 1.tmp 2.tmp
There are cases where \; is appropriate and \+ will be necessary. Using \+ with rm is one such instance, where if you are removing a large number of files performance (speed) will be superior to \;.
find has special syntax. You use the {} as they are because they have meaning to find as the pathname of the found file and (most) shells don't interpret them otherwise. You need the backslash \; because the semicolon has meaning to the shell, which eats it up before find can get it. So what find wants to see AFTER the shell is done, in the argument list passed to the C program, is
"-exec", "rm", "{}", ";"
but you need \; on the command line to get a semicolon through the shell to the arguments.
You can get away with \{\} because the shell-quoted interpretation of \{\} is just {}. Similarly, you could use '{}'.
What you cannot do is use
-exec 'rm {} ;'
because the shell interprets that as one argument,
"-exec", "rm {} ;"
and rm {} ; isn't the name of a command. (At least unless someone is really screwing around.)
Update
the difference is between
$ ls file1
$ ls file2
and
$ ls file1 file2
The + is catenating the names onto a command line.
The difference between ; (semicolon) or + (plus sign) is how the arguments are passed into find's -exec/-execdir parameter. For example:
using ; will execute multiple commands (separately for each argument),
Example:
$ find /etc/rc* -exec echo Arg: {} ';'
Arg: /etc/rc.common
Arg: /etc/rc.common~previous
Arg: /etc/rc.local
Arg: /etc/rc.netboot
All following arguments to find are taken to be arguments to the command.
The string {} is replaced by the current file name being processed.
using + will execute the least possible commands (as the arguments are combined together). It's very similar to how xargs command works, so it will use as many arguments per command as possible to avoid exceeding the maximum limit of arguments per line.
Example:
$ find /etc/rc* -exec echo Arg: {} '+'
Arg: /etc/rc.common /etc/rc.common~previous /etc/rc.local /etc/rc.netboot
The command line is built by appending each selected file name at the end.
Only one instance of {} is allowed within the command.
See also:
man find
Using semicolon (;) vs plus (+) with exec in find at SO
Simple unix command, what is the {} and \; for at SO
What is meaning of {} + in find's -exec command? at Unix
we were trying to find file for housekeeping.
find . -exec echo {} \; command ran over night in the end no result.
find . -exec echo {} \ + have results and only took a few hours.
Hope this helps.
I'm looking for a sed command that, with find, I can take a directory tree of JSP files and write the name of the file in an HTML comment to the top of the file.
This will allow me to review a legacy application JSP call tree of in the HTML source.
I'm thinking it will be a one liner for a talented sed guru...
something like:
find . -name '.jsp' -exec sed ? ? ? {} \;
Maybe something using xargs is more appropriate, but I think sed is the tool that will do the work.
If you want to use sed, you can try
find -name "*.jsp" -exec sed -i '1i <!-- {} -->' {} \;
Works fine for me in the presence of /.
On Unix the filename will contain slashes (/) which are special characters for sed, so I would recommend this simpler approach that writes the filename at the bottom of the file:
find . -name '*.jsp' -exec sh -c "echo '<\!-- {} -->' >> '{}'" \;
To write the filename at the top of the file use this:
find . -name '*.jsp' -exec sh -c \
'echo "<!-- {} -->" > "{}.new" && cat "{}" >> "{}.new" && mv "{}.new" "{}"' \;
N.B. The filename might contain characters that might render your HTML invalid, e.g. &, although I doubt that a JSP could have such a strange name.