This is my tree
├── test
│ ├── dir1
│ └── dir2
│ ├── file
│ └── file2
└── test2
└── dir2
├── file
└── file2
I use diff: diff -r test/ test2/
Only in test: dir1
So the only difference is that there is an empty directory (dir1) in in test/ which does not exist in test2/.
I want to ignore empty directories as a difference. So I want in this case that diff tells me that the content of test/ is the same as the content of test2/
How can I achieve this?
I found an way of doing it but i'm not really happy with it.
diff can be told to exclude files matching a pattern. Sadly the pattern only works on filename, so my solution may exclude more directories (and files too) than expected.
Here it is, fwiw :
diff $(find test -empty -type d -exec sh -c 'echo -n "-x $(basename $1) "' _ {} \;) -r test/ test2/
I added the following command
find test -empty -type d -exec sh -c 'echo -n "-x $(basename $1) "' _ {} \;)
which outputs the basename of every empty dir preceding by diff's exclude option -x. With your example tree, it would output : -x dir1
Related
Does the compadd command for ZSH not support completion when some characters are entered？
I have a executable file called 'index_for_test.js',and i add a shell script to .zshrc.
$PATH:
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/hanqing/Development/compadd-test
the index_for_test.js at the root of /Users/hanqing/Development/compadd-test
./
├── dir1
│ ├── a.js
│ └── b.js
├── dir2
│ ├── a.ts
│ └── b.ts
└── index_for_test.js
total 24
drwxr-xr-x 6 hanqing staff 192 11 4 13:45 .
drwxr-xr-x 10 hanqing staff 320 11 4 13:42 ..
-rw-r--r--# 1 hanqing staff 6148 11 4 14:04 .DS_Store
drwxr-xr-x 4 hanqing staff 128 11 4 13:43 dir1
drwxr-xr-x 4 hanqing staff 128 11 4 13:43 dir2
-rwxr-xr-x 1 hanqing staff 155 11 4 13:50 index_for_test.js
// index_for_test.js
#! /usr/bin/env node
const fs =require('fs')
const path=require('path')
const files=fs.readdirSync(path.join(process.cwd()))
console.log(files.join('\n'))
the script be added to .zshrc :
_index_for_test_completion() {
local abc=(`index_for_test.js`)
echo '\nabc:\n'
echo $abc'\n'
compadd -- $abc
}
compdef _index_for_test_completion index_for_test.js
When input index_for_test.js followed by input a space then press tab, it work find.But when input index_for_test.js ../ then press tab,it does not show completion list, even if compadd accept the arguments.
Image:
normal: index_for_test.js
error: index_for_test.js ../
Expect
If this is my mistake,please let me know the reason,thanks.
In addition,if the behavior is right,I wonder that how to achieve completion like cd command;
behavior of cd
index_for_test.js only prints names of files in the current directory. A file name in the current directory cannot start with ../, so there is no completion starting with ../.
If you want to complete files in a directory, you need to pass this directory to your completion script, and have it work in that directory.
In addition, you should complete directory names inside zsh, if all directories may potentially contain interesting files. If you want to allow only certain directories, have your script complete directories.
In addition, your script is broken when file names contain whitespace. Use a null byte as the separator: file names can't contain null bytes.
Untested code. May need some tweaking.
#!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const files = fs.readdirSync(process.argv[2]);
console.log(files.join('\0'))
_index_for_test_completion() {
local dir=$words[CURRENT] files
if [[ ! -d $dir ]]; then dir=$dir:h; fi
files=(${(ps:\0:)$(index_for_test.js $dir))
print -lr '' abc: $files
if [[ $dir != $words[CURRENT] ]]; then dir+="/"; fi
compadd -- $^dir$files
}
compdef _index_for_test_completion index_for_test.js
POWER SHELL ERROR Picture of code and directories
I would like to create a batch file that moves all folders that contain more than one file to another directory.
I tried the code below
mkdir "OOOO3_MORE_THAN_ONE"
for dir in *; do
# if the file is a directory
if [ -d "$dir" ]; then
# count number of files
count=$(find "$dir" -type f | wc -l)
#i f count=2 then move
if [ "$count" -le 1 ]; then
# move dir
mv -v "$dir" /completepath/"OOOO3_MORE_THAN_ONE"
fi
fi
done
I just get a new folder without any folders inside. The folders with multiple files did not move to the new directory
I also tried the below code, it's a little different, but still resulted in an empty folder
#! /bin/bash -p
shopt -s nullglob # glob patterns that match nothing expand to nothing
shopt -s dotglob # glob patterns expand names that start with '.'
destdir='subset'
[[ -d $destdir ]] || mkdir -- "$destdir"
for dir in * ; do
[[ -L $dir ]] && continue # Skip symbolic links
[[ -d $dir ]] || continue # Skip non-directories
[[ $dir -ef $destdir ]] && continue # Skip the destination dir.
numfiles=$(find "./$dir//." -type f -print | grep -c //)
(( numfiles > 1 )) && mv -v -- "$dir" "$destdir"
done
Alright, you have two problems (as originally posted).
(1) you are attempting to move to /completepath/OOOO3_MORE_THAN_ONE after creating "OOOO3_MORE_THAN_ONE" in the current working directory. Unless you are executing the script in /completepath when the directory ./OOOO3_MORE_THAN_ONE is created, your calls to mv -v "$dir" /completepath/"OOOO3_MORE_THAN_ONE"will fail. (the double-quotes are superfluous here)
(2) you have a "chicken-and-the-egg" problem because you:
mkdir "OOOO3_MORE_THAN_ONE"
before you call:
for dir in *; do
(OOOO3_MORE_THAN_ONE is included in '*', but not excluded before your calls to find and mv)
So you will effectively try and move OOOO3_MORE_THAN_ONE below itself when its name is reached in your list.
So How To Correct the Problems?
Rearrange your code. Since your Question is tagged [sh] (POSIX shell), you do not have the benefit of arrays available to pre-store the count and dir names, but you can always use a temporary file created with mktemp. You will want to read through each directory entry identified with for dir in * and write all count and dir information out to your temporary file before you start changing the directory structure. Then you can simply loop over the entries in your temporary file, checking if $count -gt 1 and moving to your new $dir if so, e.g.
#!/bin/sh
## initialize newdir from 1st argument (or default: OOOO3_MORE_THAN_ONE)
newdir="${1:-OOOO3_MORE_THAN_ONE}"
## set your complete path from 2nd arg (or '.' by default)
cmpltpath="${2:-.}"
## now create a temporary file to hold count dirname pairs
tmpfile=$(mktemp)
for dir in *; do ## write count and dirname pairs to temporary file
[ -d "$dir" ] && echo "$(find "$dir" -type f | wc -l) $dir" >> "$tmpfile"
done
## now create the directory to move to using cmpltpath, exit on failure
mkdir -p "$cmpltpath/$newdir" || exit 1
## read count and dirname from tmpfile & move if count > 1
while read -r count dir || [ -n "$dir" ]; do
## if count=2 then move
if [ "$count" -gt 1 ]; then
## move to dir
mv -v "$dir" "$cmpltpath/$newdir"
fi
done < "$tmpfile"
rm "$tmpfile" ## tidy up and remove tmpfile (or set trap after mktemp)
(note: the script takes the directory name to create and move files below as the first argument (positional parameter) for the script, and the complete path (absolute or relative) to precede your new directory)
(also note: if you have bash (or another advanced shell that supports associative arrays), you can simply save the directory name and count within an associative array keyed on directory name and avoid using a temporary file altogether)
Original directory
Using a directory tree where each subdirectory d1, d2, d3 has 1, 2 or 3 files below them, e.g.:
$ tree
.
├── d1
│ └── file1
├── d2
│ ├── file1
│ └── file2
├── d3
│ ├── file1
│ ├── file2
│ └── file3
└── mvscript.sh
Example Use/Resulting Directory Structure
Now running the script will move all directories with greater than 1 file below into your new directory:
$ sh mvscript.sh
'd2' -> './OOOO3_MORE_THAN_ONE/d2'
'd3' -> './OOOO3_MORE_THAN_ONE/d3'
$ tree
.
├── OOOO3_MORE_THAN_ONE
│ ├── d2
│ │ ├── file1
│ │ └── file2
│ └── d3
│ ├── file1
│ ├── file2
│ └── file3
├── d1
│ └── file1
└── mvscript.sh
Your Second Approach
Your second approach is not bad, but unless you have some special requirements and need to match dot-files, you may want to adjust GLOBIGNORE instead of setting dotglob as specified in man bash under the Pathname Expansion section. Also note there is no space between the #! and /bin/bash on the first line.
A basic tweak of your second attempt could be:
#!/bin/bash
destdir='subset'
mkdir -p -- "$destdir" || exit 1
for dir in * ; do
[[ -L $dir ]] && continue # Skip symbolic links
[[ -d $dir ]] || continue # Skip non-directories
[[ $dir -ef $destdir ]] && continue # Skip the destination dir.
numfiles=$(find "$dir" -type f -printf ".\n" | wc -l)
(( numfiles > 1 )) && mv -v -- "$dir" "$destdir"
done
Example Use/Output
A similar test would result in:
$ bash mvscript2.sh
'd2' -> 'subset/d2'
'd3' -> 'subset/d3'
$ tree
.
├── d1
│ └── file1
├── mvscript2.sh
└── subset
├── d2
│ ├── file1
│ └── file2
└── d3
├── file1
├── file2
└── file3
I am using Debian Linux. I'm a newbie. I'll do my best to ask in the simplest way I know.
I have a pretty deep tree of directories on a drive that contain thousands of .tif files and .txt files. I'd like to recursively find (list) all .txt files that do not have a matching .tif file (basename). The .tif files and .txt files are also located in separate directories throughout the tree.
In simple form it could look like this...
directory1: hf-770.tif, hf-771.tif, hf-772.tif
directory2: hf-770.txt, hf-771.txt, hf-771.txt, hr-001.txt, tb-789.txt
I need to find (list) hr-001.txt and tb-789.txt as they do not have a matching .tif file. Again the directory tree is quite deep with multiple sub-directories throughout.
I researched and experimented with variations of the following commands but cannot seem to make it work. Thank you so much.
find -name "*.tif" -name "*.txt" | ls -1 | sed 's/\([^.]*\).*/\1/' | uniq
You can write a shell script for this:
#!/bin/bash
set -ue
while IFS= read -r -d '' txt
do
tif=$(basename "$txt" | sed s/\.txt$/.tif/)
found=$(find . -name "$tif")
if [ -z "$found" ]
then
echo "$txt has no tif"
fi
done < <(find . -name \*.txt -print0)
This has a loop over all .txt files it finds in the current directory or below. For each found file, it replaces the .txt extension with .tif, then tries to find that file. If it cannot find it (returned text is empty), it prints the .txt file name.
robert#saaz:$ tree
.
├── bar
│ └── a.txt
├── foo
│ ├── a.tif
│ ├── b.tif
│ ├── c.tif
│ └── d.txt
└── txt-without-tif
2 directories, 6 files
robert#saaz:$ bash txt-without-tif
./foo/d.txt has no tif
Currently I run prove like this:
prove -v -r .
Is there a way to exclude files using regex? I could not see that in perldoc prove or the output of prove -H.
I usually do something like this:
$ find t/ -name '*.t' ! -name '*timeout*' | xargs prove -l
(skips slow timeout tests)
I managed to do this by using a sub-directory :
$ tree t/
t/
├── 01_tests.t
├── 02_tests.t
├── 03_tests.t
└── ignored
└── 01_ignored.t
Then I do this to execute the normal tests:
$ prove -v t/*.t
t/01_tests.t ........ ok
t/02_tests.t ........ ok
t/03_tests.t ........ ok
Result: PASS
And this to execute the ignored tests in another context:
$ prove -v t/ignored/
t/ignored/01_ignored.t ........ ok
Result: PASS
Simple and efficient solution if your tests are not already stored in different sub-directories.
For making it easier to use, and document how to run the tests, I usually put those in a Makefile.
how do i create a standard patch using diff -u without using a different name for the "new" file?
when i submitted a patch for an Apache project, the committer advised that i don't need to rename the file when submitting patches. i can somewhat understand how this breaks patching since the name of the "new" file should somehow match the name of the patch target - however they can't be in the same directory with the same name.
is it okay (for ease of patching) to rename the "old" file, such that i should have used:
diff -u Source-old.java Source.java
instead of:
diff -u Source.java Source-new.java
?
Given an existing project 'a', copy whole project to 'b', make changes in 'b'. Generate diff between original directory and your copied directory.
E.g., checkout or download project to directory 'a', copy to 'b':
$ tree a
a
`-- dir
|-- Bar.java
`-- Foo.java
$ cp -r a b
$ tree b
b
`-- dir
|-- Bar.java
`-- Foo.java
Make changes to 'b' (and only 'b'):
$ diff -r -s a b
Files a/dir/Bar.java and b/dir/Bar.java are identical
Files a/dir/Foo.java and b/dir/Foo.java are identical
$ sed -i 's/Foo.*$/& \/* Change...*\//' b/dir/Foo.java
$ diff -ruN a b | tee a.patch
diff -ruN a/dir/Foo.java b/dir/Foo.java
--- a/dir/Foo.java 2012-08-02 18:41:39.444720785 -0700
+++ b/dir/Foo.java 2012-08-02 18:46:45.319932802 -0700
## -1,2 +1,2 ##
package dir;
-public class Foo {}
+public class Foo {} /* Change...*/
$ gzip a.patch
Another alternative is to store the original source in a temporary, local git repository, then use git's built-in diff to generate the patch. Or, better, if the original source is using git, then just clone the repo and work directly in the source tree itself, and (still) using git to generate the patch.