I would need your help with ExifTool. I am trying to check if all of my .wav files have the same amount of channels through meta-data. How should I proceed? Should I print out the tags first and then write a script to check if they are all the same or is there a better way?
Thank you for your help.
You'd have to do it externally (via a file or your shell controls or something.
e.g., in bash:
if [ "$(exiftool -NUMCHANNELS a.wav)" == "$(exiftool -NUMCHANNELS b.wav)" ]
then
echo match
fi
To see if many files all match, you could do something like
exiftool -q -NUMCHANNELS *.wav | uniq | wc -l
And verify the output is 1
Related
I have a text file C:\folder\filelist.txt containing a list of numbers, for example:
345651
342679
344000
349080
I want to append the URL as shown below, download only the files that are >1000KB, and strip the parameters after "-a1" from the filename, for example:
URL
Size
Output File
https://some.thing.com/gab/abc-345651-def-a1?scl=1&fmt=jpeg
1024kb
C:\folder\abc-345651-def-a1.jpeg
https://some.thing.com/gab/abc-342679-def-a1?scl=1&fmt=jpeg
3201kb
C:\folder\abc-342679-def-a1.jpeg
https://some.thing.com/gab/abc-342679-def-a1?scl=1&fmt=jpeg
644kb
-
https://some.thing.com/gab/abc-349080-def-a1?scl=1&fmt=jpeg
2312kb
C:\folder\abc-349080-def-a1.jpeg
This is the code I currently have, which works for downloading the files and appending the .jpeg extension, given the full URL is in the text file. It does not filter out the smaller images or strip the parameters following "-a1".
cd C:\folder\
wget --adjust-extension --content-disposition -i C:\folder\filelist.txt
I'm running Windows and I'm a beginner at writing batch scripts. The most important thing 'm trying to accomplish is to avoid downloading images <1000kb: it would be acceptable if I had to manually append the URL in the text file and rename the files after the fact. Is it possible to do what I'm trying to do? I've tried modifying the script by referencing the posts below, but I can't seem to get it to work. Thanks in advance!
Wget images larger than x kb
Downloading pdf files with wget. (characters after file extension?)
Spider a Website and Return URLs Only
#change working directory
cd /c/folder/
#convert input file list to unix
dos2unix filelist.txt
for image in $(cat filelist.txt)
do
imageURL="https://some.thing.com/gab/abc-$image-def-a1?scl=1&fmt=jpeg"
size=`wget -d -qO- "$imageURL" 2>&1 | grep 'Content-Length' | awk {'print $2'}`
if [[ $size -gt 1024000 ]] ;then
imgname="/c/folder/abc-$image-def-a1.jpeg"
wget -O $imgname $imageURL
fi
done
I need to grep about 1 million files. If there's a better way to do this, let me know. I was thinking there may be a faster way to do it in perl.
What I'm trying to do is export every line that contains the text httpsfile in it.
Here's what I'm trying to run:
grep 'httpsfile' * >> grepped.txt
Here's the error I'm getting:
-bash: /bin/grep: Argument list too long
Any help would be appreciated.
You can do it in parallel if you want:
ls > /tmp/files
parallel -a /tmp/files --xargs -s 100 grep 'httpsfile'
Unless you have a lot of RAM and your on million files are already in the buffer cache, parallelizing won't be of any help given the fact the operation will be I/O bound so here is the fastest still portable (POSIX) way:
find . -exec grep httpsfile {} + > grepped.txt
Note that unlike the accepted answer solution, using find won't fail with oddly named files. Have a look to https://unix.stackexchange.com/questions/128985/why-not-parse-ls
Try ls | xargs grep httpsfile.
Just change * to ./ or, whatever is the root directory that contains the 1 million files. You might need to add -r as well to make grep recursive and look into nested directories.
* in the shell expands out into all the files.
I have been searching for a utility/tool that can provide the md5sum(or any unique checksum) of a data block inside ext3 inode structure.
The requirement is to verify whether certain data blocks get zeroed, after a particular operation.
I am new to file systems and do not know if any existing tool can do the job, or I need to write this test utility myself.
Thanks...
A colleague provided a very elegant solution. Here is the script.
It needs the name of file as a parameter, and assumes the file system blocksize to be 4K
A further extension of this idea:
If you know the data blocks associated with the file (stat ), you can use 'skip' option of 'dd' command and build small files, each of 1 block size length. Further, you can get the md5sum of these blocks. So, this way you can get md5sum directly from the block device. Not something you would want to do everyday, but a nice analytical trick.
==================================================================================
#!/bin/bash
absname=$1
testdir="/root/test/"
mdfile="md5"
statfile="stat"
blksize=4096
fname=$(basename $absname)
fsize=$( ls -al $absname | cut -d " " -f 5 )
numblk=$(( fsize/blksize ))
x=1
#Create the test directory, if it does not exist already
if [[ ! -d $testdir ]];
then
`mkdir -p $testdir`
fi
#Create multiple files from the test file, each 1 block sized
while [[ $x -le $numblk ]]
do
(( s=x-1 ))
`dd if=$absname of=$testdir$fname$x bs=4096 count=1 skip=$s`
`md5sum $testdir$fname$x >> $testdir$mdfile`
(( x=x+1 ))
done
Say I had a script that checked honeypot locations using md5sum.
#!/bin/bash
#cryptocheck.sh
#Designed to check md5 CRC's of honeypot files located throughout the filesystem.
#Must develop file with specific hashes and create crypto.chk using following command:
#/opt/bin/md5sum * > crypto.chk
#After creating file, copy honeypot folder out to specific folders
locations=("/share/ConfData" "/share/ConfData/Archive" "/share/ConfData/Application"
"/share/ConfData/Graphics")
for i in "${locations[#]}"
do
cd "$i/aaaCryptoAudit"
/opt/bin/md5sum -c /share/homes/admin/crypto.chk
done
And the output looked like this:
http://pastebin.com/b4AU4s6k
Where would you start to try and recognize the output and perhaps trigger some sort of response by the system if there is a 'FAILED'?
I've worked a bit with PERL trying to parse log files before but my attempts typically failed miserably for one reason or another.
This may not be the proper way to go about this, but I'd want to be putting this script into a cronjob that would run every minute. I had some guys telling me that an inotify job or script (I'm not familiar with this) would be better than doing it this way.
Any suggestions?
--- edit
I made another script to call the script above and send the output to a file. The new script then runs a grep -q on 'FAILED' and if it picks anything up, it sounds the alarm (tbd what the alarm will be).
#!/bin/bash
#cryptocheckinit.sh
#
#rm /share/homes/admin/cryptoalert.warn
/share/homes/admin/cryptocheck.sh > /share/homes/admin/cryptoalert.warn
grep -q "FAILED" /share/homes/admin/cryptoalert.warn && echo "LIGHT THE SIGNAL FIRES"
Use:
if ! /opt/bin/md5sum -c /share/homes/admin/crypto.chk
then
# Do something
fi
Or pipe the output of the loop:
for i in "${locations[#]}"
do
cd "$i/aaaCryptoAudit"
/opt/bin/md5sum -c /share/homes/admin/crypto.chk
done | grep -q FAILED && echo "LIGHT THE SIGNAL FIRES"
I am back with a second no-brainer question, but I would like to get this straight in my head.
I have an assignment in which I am charged with providing a command to find a file named test in my home directory (one command using find, and one using grep). I understand that using find is just 'find ~/test', but using grep, wouldn't I have to search out a pattern within the file 'test'? Or is there a way to search for the file (using grep), even if the file is empty?
ls ~ | grep test
I understand that using find is just 'find ~/test'
No. find ~/test will also have a match for every file or directory under the directory $HOME/test/. Rather use find ~ -type f -name test.
The assignment sounds unclear. But yes, if you give any filenames to grep, it will look at the contents of the files and ignore the names of the files. Perhaps you can grep the output of another command? Maybe ls as #Reese suggested, or maybe a different find command.
ls -R ~ | grep test
Explanation: ls -R ~ will recursively list all files and directories in your home folder. grep test will narrow down that list to files (and directories) that have "test" in their name.