i want to tail log file with grep and sent it via mail
like:
tail -f /var/log/foo.log | grep error | mail -s subject name#example.com
how can i do this?
You want to send an email when emailing errors occur? That might fail ;)
You can however try something like this:
tail -f $log |
grep --line-buffered error |
while read line
do
echo "$line" | mail -s subject "$email"
done
Which for every line in the grep output sends an email.
Run above shell script with
nohup ./monitor.sh &
so it will keep running in the background.
I'll have a go at this. Perhaps I'll learn something if my icky bash code gets scrutinised. There is a chance there are already a gazillion solutions to do this, but I am not going to find out, as I am sure you have trawled the depths and widths of the cyberocean. It sounds like what you want can be separated into two bits: 1) at regular intervals obtain the 'latest tail' of the file, 2) if the latest tail actually exists, send it by e-mail. For the regular intervals in 1), use cron. For obtaining the latest tail in 2), you'll have to keep track of the file size. The bash script below does that - it's a solution to 2) that can be invoked by cron. It uses the cached file size to compute the chunk of the file it needs to mail. Note that for a file myfile another file .offset.myfile is created. Also, the script does not allow path components in the file name. Rewrite, or fix it in the invocation [e.g. (cd /foo/bar && segtail.sh zut), assuming it is called segtail.sh ].
#!/usr/local/bin/bash
file=$1
size=0
offset=0
if [[ $file =~ / ]]; then
echo "$0 does not accept path components in the file name" 2>&1
exit 1
fi
if [[ -e .offset.$file ]]; then
offset=$(<".offset.$file")
fi
if [[ -e $file ]]; then
size=$(stat -c "%s" "$file") # this assumes GNU stat, possibly present as gstat. CHECK!
# (gstat can also be Ganglias Status tool - careful).
fi
if (( $size < $offset )); then # file might have been reduced in size
echo "reset offset to zero" 2>&1
offset=0
fi
echo $size > ".offset.$file"
if [[ -e $file && $size -gt $offset ]]; then
tail -c +$(($offset+1)) "$file" | head -c $(($size - $offset)) | mail -s "tail $file" foo#bar
fi
How about:
mail -s "catalina.out errors" blah#myaddress.com < grep ERROR catalina.out
Related
I'm working on a script to monitor a log of jobs executed and I want to receive a mail notification with the line where appears the job in the body of the mail. This is what I got so far but it keeps throwing error, I can make it work but just with an empty body. Could you please help?
Job="jobname"
tail -fn0 logfile.log | awk -v Jobs="$Job"'/jobname/
{
system("grep -i "Jobs" logfile.log | mail -s "Jobs Is Completed" mail#mail.com")
exit
}'
What's wrong with just:
Job="jobname"
tail -fn0 logfile.log |
grep --line-buffered -i "jobname.*$Job" |
mail -s "$Job Is Completed" mail#mail.com"
Your use of jobname as literal in 2 places and a shell variable named Job with an awk variable named Jobs populated from it and both containing jobname anyway was very confusing so hopefully you can tweak the above to do whatever you need to do if the variable usage above is not quite right.
watchdog.sh
#!/bin/bash
mailWorker(){
while read -r line; do
if [[ $line == *$match* ]]
then
# mailing
grep -i "Jobs" "$logfile" | mail -s "Jobs Is Completed" mail#mail.com
break
fi
done
}
logfile="/path/to/logfile.log"
match="jobname"
if [ ! -f "$logfile" ]; then
touch "$logfile"
fi
tail -f --lines=0 "$logfile" | mailWorker
Can't boot my Windows PC today and I am on 2nd OS Linux Mint. With my limited knowledge on Linux and shell scripts, I really don't have an idea how to do this.
I have a bunch of files in a directory generated from my system, need to remove the last 12 characters from the left of ".txt"
Sample filenames:
filename1--2c4wRK77Wk.txt
filename2-2ZUX3j6WLiQ.txt
filename3-8MJT42wEGqQ.txt
filename4-sQ5Q1-l3ozU.txt
filename5--Way7CDEyAI.txt
Desired result:
filename1.txt
filename2.txt
filename3.txt
filename4.txt
filename5.txt
Any help would be greatly appreciated.
Here is a programmatic way of doing this while still trying to account for pesky edge cases:
#!/bin/sh
set -e
find . -name "filename*" > /tmp/filenames.list
while read -r FILENAME; do
NEW_FILENAME="$(
echo "$FILENAME" | \
awk -F '.' '{$NF=""; gsub(/ /, "", $0); print}' | \
awk -F '/' '{print $NF}' | \
awk -F '-' '{print $1}'
)"
EXTENSION="$(echo "$FILENAME" | awk -F '.' '{print $NF}')"
if [[ "$EXTENSION" == "backup" ]]; then
continue
else
cp "$FILENAME" "${FILENAME}.backup"
fi
if [[ -z "$EXTENSION" ]]; then
mv "$FILENAME" "$NEW_FILENAME"
else
mv "$FILENAME" "${NEW_FILENAME}.${EXTENSION}"
fi
done < /tmp/filenames.list
Create a List of Files to Edit
First up create a list of files that you would like to edit (assuming that they all start with filename) and under the current working directory (.):
find . -name "filename*" > /tmp/filenames.list
If they don't start with filename fret not you could always use a find command like:
find . -type f > /tmp/filenames.list
Iterate over a list of files
To accomplish this we use a while read loop:
while read -r LINE; do
# perform action
done < file
If you had the ability to use bash you could always use a named pipe redirect:
while read -r LINE; do
# perform action
done < <(
find . -type f
)
Create a rename variable
Next, we create a variable NEW_FILENAME and using awk we strip off the file extension and any trailing spaces using:
awk -F '.' '{$NF=""; gsub(/ /, "", $0); print}'
We could just use the following though if you know for certain that there aren't multiple periods in the filename:
awk -F '.' '{print $1}'
The leading ./ is stripped off via
awk -F '/' '{print $NF}'
although this could have been easily done via basename
With the following command, we strip everything after the first -:
awk -F '-' '{print $1}'
Creating backups
Feel free to remove this if you deem unnecessary:
if [[ "$EXTENSION" == "backup" ]]; then
continue
else
cp "$FILENAME" "${FILENAME}.backup"
fi
One thing that we definitely don't want is to make backups of backups. The above logic accounts for this.
Renaming the files
One thing that we don't want to do is append a period to a filename that doesn't have an extension. This accounts for that.
if [[ -z "$EXTENSION" ]]; then
mv "$FILENAME" "$NEW_FILENAME"
else
mv "$FILENAME" "${NEW_FILENAME}.${EXTENSION}"
fi
Other things of note
Odds are that your Linux Mint installation has a bash shell so you could simplify some of these commands. For instance, you could use variable substitution: echo "$FILENAME" | awk -F '.' '{print $NF}' would become "${FILENAME##.*}"
[[ is not defined in POSIX sh so you will likely just need to replace [[ with [, but review this document first:
https://mywiki.wooledge.org/BashFAQ/031
From the pattern of filenames it looks like that the first token can be picked before "-" from filenames. Use following command to rename these files after changing directory to where files are located -
for srcFile in `ls -1`; do fileN=`echo $srcFile | cut -d"-" -f1`; targetFile="$fileN.txt"; mv $srcFile $targetFile; done
If above observation is wrong, following command can be used to remove exactly 12 characters before .txt (4 chars) -
for srcFile in `ls -1`; do fileN=`echo $srcFile | rev | cut -c17- | rev`; targetFile="$fileN.txt"; mv $srcFile $targetFile; done
In ls -1, a pattern can be added to filter files from current directory if that is required.
Problem: I can't find any way to reliably get the current playing file in an MPlayer playlist.
Here is how far I have gotten. This working ash script monitors a text file with the path to the current playlist. When I update the file, the script closes the old instance of MPlayer and opens a new one with the new playlist:
# POLL PLAYLIST FILE FOR CHANGES
CURRENTPLAYLISTPATH=/home/tc/currentplaylist
INFIFO=/tmp/mplayer-in
CURRENTPLAYLIST="NEVERMATCHAPLAYLIST"
FIRSTRUN=1
while [ 1 ];
do
# CHECK FOR NEW PLAYLIST
NEWPLAYLIST=$(head -n 1 $CURRENTPLAYLISTPATH)
if [[ "$NEWPLAYLIST" != "$CURRENTPLAYLIST" ]]; then
if [ "$FIRSTRUN" == 0 ]; then
echo "quit" > "$INFIFO"
fi
# CREATE NAMED PIPE, IF NEEDED
trap "rm -f $INFIFO" EXIT
if [ ! -p $INFIFO ]; then
mkfifo $INFIFO
fi
# START MPLAYER
mplayer -fixed-vo -nolirc -vc ffmpeg12vdpau,ffh264vdpau, -playlist $NEWPLAYLIST -loop 0 -geometry 1696x954 -slave -idle -input file=$INFIFO -quiet -msglevel all=0 -identify | tee -a /home/tc/mplayer.log &
CURRENTPLAYLIST=$NEWPLAYLIST
FIRSTRUN=0
fi
sleep 5;
done
My original plan was just to use the "-identify" flag and parse the log file. This actually works really well up until I need to truncate the log file to keep it from getting too large. As soon as my truncating script is run, MPlayer stops writing to the log file:
FILENAME=/home/tc/mplayer.log
MAXCOUNT=100
if [ -f "$FILENAME" ]; then
LINECOUNT=`wc -l "$FILENAME" | awk '{print $1}'`
if [ "$LINECOUNT" -gt "$MAXCOUNT" ]; then
REMOVECOUNT=`expr $LINECOUNT - $MAXCOUNT`
sed -i 1,"$REMOVECOUNT"d "$FILENAME"
fi
fi
I have searched and searched but have been unable to find any other way of getting the current playing file that works.
I have tried piping the output to another named pipe and then monitoring it, but only works for a few seconds, then MPlayer completely freezes.
I have also tried using bash (instead of ash) and piping the output to a function like the following, but get the same freezing problem:
function parseOutput()
{
while read LINE
do
echo "get_file_name" > /tmp/mplayer-in
if [[ "$LINE" == *ANS_FILENAME* ]]
then
echo ${LINE##ANS_FILENAME=} > "$CURRENTFILEPATH"
fi
sleep 1
done
}
# START MPLAYER
mplayer -fixed-vo -nolirc -vc ffmpeg12vdpau,ffh264vdpau, -playlist $NEWPLAYLIST -loop 0 -geometry 1696x954 -slave -idle -input file=/tmp/mplayer-in -quiet | parseOutput &
I suspect I am missing something very obvious here, so any help, ideas, points in the right direction would be greatly appreciated.
fodder
Alright then, so I'll post mine too.
Give this one a try (assuming there is only one instance running, like on fodder's machine):
basename "$(readlink /proc/$(pidof mplayer)/fd/* | grep -v '\(/dev/\|pipe:\|socket:\)')"
This is probably the safer way, since the file descriptors might not always be in the same order on all systems.
However, this can be shortened, with a little risk:
basename "$(readlink /proc/$(pidof mplayer)/fd/*)" | head -1
You might probably like to install this, too:
http://mplayer-tools.sourceforge.net/
Well, I gave up on getting the track from MPlayer itself.
My 'solution' is probably too hackish, but works for my needs since I know my machine will only ever have one instance of MPlayer running:
lsof -p $(pidof mplayer) | grep -o "/path/to/my/assets/.*"
If anyone has a better option I'm certainly still interested in doing this the right way, I just couldn't make any of the methods work.
fodder
You can use the run command.
Put this in ~/.mplayer/input.conf:
DEL run "echo ${filename} ${stream_pos} >> /home/knarf/out"
Now if you press the delete key while playing a file it will do what you expect i.e. append the current file playing and the position in the stream to the ~/out file. You can replace echo with your program.
See slave mod docs for more info (Ctrl-F somevar).
About getting properties from MPlayer
I have used a non-elegant solution, but it is working for me.
stdbuf -oL mplayer --slave --input=file=$FIFO awesome_awesome.mp3 |
{
while IFS= read -r line
do
if [[ "${line}" == ANS_* ]]; then
echo "${line#*=}" > ${line%=*} # echo property_value > property_name
fi
done
} &
mplayer_pid=&!
read filename < ./ANS_FILENAME
read timeLength < ./ANS_LENGTH
echo ($timeLength) $filename
and so on..
It is in another proccess, that's why I've used files to bring properties
'stdbuf' is for not to miss anything
I started putting together a bash library to handle tasks like this. Basically, you can accomplish this by dumping the mplayer output to a file. Then you grep that dump for "Playing " and take the last result with tail. This should give you the name of the file that's currently playing or that last finished playing.
Take a look at my bash code. You'll want to modify the playMediaFile function to your needs, but the getMediaFileName function should do exactly what you're asking. You'll find the code on my github.
I have the dSYM file for build created on client's machine. Client got a crash in build and now I am trying to de-symbol by use of the symbolicatecrash by the simple following command in terminal:
symbolicatecrash myapp_iPod-Touch.crash myapp.app.dSYM > test.txt
but it's not creating any meaningful de-symboled file. and it's giving the follwoing error in terminal:
Can't understand the output from otool
then I found a solution in following link:
iPhone SDK 3.0 and symbolicatecrash not getting along?
but still it's not de-symbolicating the exact memory location to exact code line responsible for crash.
Then I tried some other options too:
Following is the other option but didn't work:
symbolicatecrash.sh -A -v [crashlog-filename] MyApp.dSYM
For reference: http://apptech.next-munich.com/2010/01/symbolicatecrash.html
The best option that helped me is atos command to get the exact code line number of the crash but I want the systematic symbolicatecrash to create the dump.
NOTE: When I create build in my machine and desymbolicate (the my machine created) build's crash log in my machine it creates perfectly good dump file (show's exact memory location VS code line responsible for crash).
If you have the DSYM file for the crash then you can use this one:
#!/bin/bash
if [[ $# < 2 ]]
then
echo "Usage: $0 [-arch <arch> (defaults to whatever is specified in the crashlog- file] <dSYM-file> <crashlog-file>"
exit 1
fi
#Get the architecture either from the params or from the crashlog itself
ARCH_PARAMS=''
if [[ "${1}" == '-arch' ]]
then
ARCH_PARAMS="-arch ${2}"
shift 2
else
ARCHITECTURE=$(cat "${2}" | grep -A1 "Binary Images:" | grep 0x | sed -E -n 's/.*(armv[6-9]).*/\1/p')
if [ -n "${ARCHITECTURE}" ]
then
ARCH_PARAMS="-arch ${ARCHITECTURE}"
else
echo "Couldn't determine architecture based on the crashlog. Please specify it by calling $0 -arch <arch> <dSYM-file> <crashlog-file>"
exit
fi
fi
echo "Assuming architecture:" ${ARCHITECTURE}
#Store the other params
SYMBOL_FILE="${1}"
CRASHLOG="${2}"
#Get the identifier out of the crashlog
IDENTIFIER=$(cat "${CRASHLOG}" | egrep -o "^Identifier:[[:space:]]*.*$" | sed 's/^Identifier:[[:space:]]*\(.*\)$/\1/')
echo "Identifier:" $IDENTIFIER
echo
echo
#Iterate through the crashlog files, find the ones that belong to the $IDENTIFIER, sed the address out of those files, symbolicate them with atos and finally replace them back into those line again. Print all other lines untouched.
while read line
do
SYMBOL=$(echo $line | sed -E -n "s/.*(${IDENTIFIER}[[:space:]]*)(0x[[:alnum:]]*).*/\2/p" | atos -o "${SYMBOL_FILE}/Contents/Resources/DWARF/${IDENTIFIER}" ${ARCH_PARAMS})
if [ -n "$SYMBOL" ]
then
echo $line | sed -E "s/(${IDENTIFIER}[[:space:]]*)(0x[[:alnum:]]*)(.*)/\1\2 ${SYMBOL}/"
else
echo $line
fi
done < "${CRASHLOG}"
How it is possible to make a dry run with sed?
I have this command:
find ./ -type f | xargs sed -i 's/string1/string2/g'
But before I really substitute in all the files, i want to check what it WOULD substitute. Copying the whole directory structure to check is no option!
Remove the -i and pipe it to less to paginate though the results. Alternatively, you can redirect the whole thing to one large file by removing the -i and appending > dryrun.out
I should note that this script of yours will fail miserably with files that contain spaces in their name or other nefarious characters like newlines or whatnot. A better way to do it would be:
while IFS= read -r -d $'\0' file; do
sed -i 's/string1/string2/g' "$file"
done < <(find ./ -type f -print0)
I would prefer to use the p-option:
find ./ -type f | xargs sed 's/string1/string2/gp'
Could be combined with the --quiet parameter for less verbose output:
find ./ -type f | xargs sed --quiet 's/string1/string2/gp'
From man sed:
p:
Print the current pattern space.
--quiet:
suppress automatic printing of pattern space
I know this is a very old thread and the OP doesn't really need this answer, but I came here looking for a dry run mode myself, so thought of adding the below piece of advice for anyone coming here in future. What I wanted to do was to avoid stomping the backup file unless there is something really changing. If you blindly run sed using the -i option with backup suffix, existing backup file gets overwritten, even when there is nothing substituted.
The way I ended up doing is to pipe sed output to diff and see if anything changed and then rerun sed with in-place update option, something like this:
if ! sed -e 's/string1/string2/g' $fpath | diff -q $fpath - > /dev/null 2>&1; then
sed -i.bak -e 's/string1/string2/g' $fpath
fi
As per OP's question, if the requirement is to just see what would change, then instead of running the in-pace sed, you could do the diff again with some informative messages:
if ! sed -e 's/string1/string2/g' $fpath | diff -q $fpath - > /dev/null 2>&1; then
echo "File $fpath will change with the below diff:"
sed -e 's/string1/string2/g' $fpath | diff $fpath -
fi
You could also capture the output in a variable to avoid doing it twice:
diff=$(sed -e 's/string1/string2/g' $fpath | diff $fpath -)
if [[ $? -ne 0 ]]; then
echo "File $fpath will change with the below diff:"
echo "$diff"
fi