I have a need to regularly copy files from a specific set of source sub directories (100's of them) into a 'flat" directory structure, i.e. i want all the files from the multiple source directories in a single destination directory. I can't seem to find a way of copying that can look into the source sub directories & copy the files that doesn't re-create the sub-directory folder structure in the destination directory.
Any help appreciated.
sourcedir=/root/of/subdirectory/set
destdir=/where/the/files/go
find $sourcedir -type f -print | while read file; do cp $file $destdir; done
or (prevent overwrites)
find $sourcedir -type f -print | while read file; do base=$(basename $file); test -f $destdir/$base || cp $file $destdir; done
Note this will not work if any of the names of the files or subdirectories in $sourcedir contain spaces.
Related
I created a powershell command in order to compress mdb files from a certain folder. The structure looks like this:
C:\Projects\MySoftware\templates\A\a.mdb
C:\Projects\MySoftware\templates\B\b.mdb
C:\Projects\MySoftware\templates\C\c.mdb
Basically I want that my archive must contains the directories with "templates" as root node. So:
templates\A\a.mdb
templates\B\b.mdb
templates\C\c.mdb
And in PowerShell
powershell -Command "$pa = 'C:\Projects'; Set-Location C:\Projects\MySoftware\; gci -Path templates -Recurse -Include *.mdb | sort LastWriteTime | ForEach-Object { cd 'C:\Projects\MySoftware\'; $fn = $_.Fullname; tar rf BASetupUpdate.tgz $($fn.Replace('C:\Projects\MySoftware\','')) }"
The tar creates correctly the archive but without compression. I'm struggling with command line and help in order to set the compression level.
You need to add the z option to compress, which cannot be used with r, only c. You would need to make a list of all the things you want to put in the tar file, and use a single invocation of the tar command with all of those names. You cannot incrementally add files to a compressed tar.gz (or .tgz if you prefer) archive. So something like:
tar -czf archive.tgz file1 file2 ... filen
As for the compression level, you can add --option gzip:compression-level=9. However to start, you should leave it at the default level and see if that meets your needs. Higher compression levels can take much more time for a small reduction in size.
Im trying to use logrotate with very little experience, Currently working i have the files rotating, compressing and renaming into the same folder. Now i need instead of dropping the files in the same place, i need to have them dropped in another location. They also need to have the same folder structure and if it isn't there than it needs to create the new folder. All the compressed files need to be added and not override the existing files
I'm thinking that the olddir will drop them into a destination folder but not sure on how to have it drop it in the corresponding folder or create it if its not already there.
Example source
var/log/device1/*.log
var/log/device2/*.log
var/log/device3/*.log
Example Destination to drop .gz files into
opt/archive/device1/
opt/archvie/device2/
(needs to create opt/archive/device3 and put rotated file in here)
Didn't end up finding a way to move with logrotate but came up with script to do the same sort of thing. pretty simplistic and wont work for more than 1 level deep of subfolders.
#!/bin/bash
source="/opt/log/host"
destination="/opt/archive/"
for i in $(find $source -maxdepth 2 -type f -name "*.gz")
do
#removing /opt/log/host from string
dd="$( echo "$i" | sed -e 's#^/opt/log/host/##' )"
#removing everything after the first /
ff=$( echo "$dd" | cut -f1 -d"/" )
#setting the correct destination string
ee=$destination$dd
#create new folders if they do not exist
mkdir -p -- "$destination$ff"
#move files
mv $i $ee
done
Okay so I want to know how I would go about doing this, using grep to locate .txt files named "cocacola1", "cocacola2", "cocacola3" & then copying them to another directory. So searching for files named "cocacola" &/even if it contains other characters within the file name to then copy them to another directory/location.
You can just use unix find. Assuming the files you're searching for are in 'source' and you want to copy to 'destination':
find source -name '*cocacola*' -exec cp {} destination \;
I put the wildcard '*' before and after cocacola since you said other characters might exist in the file name.
I'm developing a simple launchdaemon that copies files from one directory to another. I've gotten the files to transfer over fine.
I just want the files in the directory to be .mp3's instead of .dat's
Some of the files look like this:
6546785.8786.dat
3678685.9834.dat
4658679.4375.dat
I want them to look like this:
6546785.8786.mp3
3678685.9834.mp3
4658679.4375.mp3
This is what I have at the end of the bash script to rename the file extensions.
cd $mp3_dir
mv *.dat *.mp3
exit 0
Problem is the file comes out as *.mp3 instead of 6546785.8786.mp3
and when another 6546785.8786.dat file is imported to $mp3_dir, the *.mp3 is overwritten with the new .mp3
I need to rename just the .dat file extensions to .mp3 and keep the filename.
Ideas? Suggestions?
Try:
for file in *.dat; do mv "$file" "${file%dat}mp3"; done
Or, if your shell has it:
rename .dat .mp3 *.dat
Now, why your command didn't work: first of all, it is more than certain that you only had one file in your directory when it was renamed to *.mp3, otherwise mv would have failed with *.mp3: not a directory.
And mv does NOT do any magic with file globs, it is the shell which expands globs. Which means, if you had this file in the directory:
t.dat
and you typed:
mv *.dat *.mp3
the shell would have expanded *.dat to t.dat. However, as nothing would match *.mp3, the shell would have left it as is, meaning the fully expanded command is:
mv t.dat *.mp3
Which will create a file named, literally, *.mp3.
If, on the other hand, you had several files named *.dat, as in:
t1.dat t2.dat
the command would have expanded to:
mv t1.dat t2.dat *.mp3
But this will fail: if there are more than two arguments to mv, it expects the last argument (ie, *.mp3) to be a directory.
For anyone on a mac, this is quite easy if you have BREW, if you don't have brew then my advice is get it. then when installed just simply do this
$ brew install rename
then once rename is installed just type (in the directory where the files are)
$ rename -s dat mp3 *
Is it possible using any sort of diff utility to diff based on filename only, ignoring all folders and subfolders?
So if I have
/folder_1/a/1243.txt
and
/folder_2/b/1243.txt
or
/folder_2/1234.txt
It would match the files when doing a diff between folder_1 and folder_2?
Are you trying to do a diff on lists of filenames, to see which filenames one folder contains that the other doesn't? If so, do find folder_1 -type f in Linux or dir /s /b /ad folder_1 in DOS and pipe the output into text files, then diff the contents of the text files.
In Vim I'd skip the temp files, and do :.!find folder_1 -type f in one window, :.!find folder_2 -type f in second window, then :windo diffthis to diff them.
diff -r directory another
compares the files in directories and outputs if they differs, or is missing from either of the directory.