Currently my log file sits at 32 meg. Did I miss an option that would split the log file as it grows?
You can use logrotate to do this job for you.
Put this in /etc/logrotate.d/mongod (assuming you use Linux and have logrotate installed):
/var/log/mongo/*.log {
daily
rotate 30
compress
dateext
missingok
notifempty
sharedscripts
copytruncate
postrotate
/bin/kill -SIGUSR1 `cat /var/lib/mongo/mongod.lock 2> /dev/null` 2> /dev/null || true
endscript
}
If you think that 32 megs is too large for a log file, you may also want to look inside to what it contains.
If the logs seem mostly harmless ("open connection", "close connection"), then you may want to start mongod with the --quiet switch. This will reduce some of the more verbose logging.
Rotate the logs yourself
http://www.mongodb.org/display/DOCS/Logging
or use 'logrotate' with an appropriate configuration.
Using logrotate is a good option. while, it will generate 2 log files that fmchan commented, and you will have to follow Brett's suggestion to "add a line to your postrotate script to delete all mongod style rotated logs".
Also copytruncate is not the best option. There is always a window between copy and truncate. Some mongod logs may get lost. Could check logrotate man page or refer to this copytruncate discussion.
Just provide one more option. You could write a script that sends the rotate signal to mongod and remove the old log files. mongologrotate.sh is a simple reference script that I have written. You could write a simple cron job or script to call it periodically like every 30 minutes.
Related
My MongoDB Log file is over 16GB, the log file is not important to me. How can I disable the logging of MongoDB and if I do, will I encounter any problems?
It would not be a smart idea to turn off logging. Use Rotate Log Files to rotate them and keep them small.
logrotate is standard function on Linux.
Simplest way to rotate the log file is kill -USR1 $(/usr/sbin/pidof mongod)
My logrotate.conf file looks like this:
missingok
compress
delaycompress
notifempty
create
/var/log/mongodb/mongod.log{
size 10M
rotate 9
sharedscripts
postrotate
kill -USR1 $(/usr/sbin/pidof mongod)
endscript
}
When the logfile reaches 10MB then it is rotated. Up to 9 files are kept. logrotate is executed by a daily cron job.
Though you can disable logging, it is really not recommended.
I have a problem with logrotate. The application itself produces following logs:
xxx.log
and at 23:59 the application changes the log to:
xxx.log.2019-01-05
and so on. Right now I am getting following in the log directory:
xxx.log
xxx.log.2019-01-01
xxx.log.2019-01-02
etc.
What I need to to is want to rotate the logs that get created on 23:59 and not to touch the xxx.log file itself.
I have tried with following logrotate rule:
/var/log/xxx/xxx/xxx.log.* {
daily
missingok
rotate 30
compress
notifempty
copytruncate
nosharedscripts
prerotate
bash -c "[[ ! $1 =~ *.gz ]]"
endscript
}
But first of all logrotate does not compress the log that was created last and it also adds .1.gz extension to previously compressed files.
logrotate does not compress the log that was created last
Do you have "delaycompress" defined in /etc/logrotate.conf? Per logrotate man:
delaycompress
Postpone compression of the previous log file to the next rotation cycle.
it also adds .1.gz extension
While you're at the aforementioned man page, you should check out what the "extension" option does:
extension ext
Log files with ext extension can keep it after the rotation.
in last days i get daily mail from cron's logrotate task:
/etc/cron.daily/logrotate:
gzip: stdin: file size changed while zipping
How can I fix it?
Thanks,
Gian Marco.
Here's a blog post in French which gives a solution.
In English, you can read this bug report.
To summarise:
First you have to add the --verbose option in the script /etc/cron.daily/logrotate to have more information the next time it runs to identify which rotation log cause the problem.
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate --verbose /etc/logrotate.conf`
Next you have to add the delaycompress option in logrotate configuration.
Like exemple, I add the nginx's logrotate configiguration in /etc/logrotate.d/nginx:
/var/log/nginx/*.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 0640 www-data adm
sharedscripts
...
}
upstart will close (and reopen) its log file when it notices that the file is deleted. However, if you look at what gzip does, you see that it doesn't delete the file until after it's one writing the output file. That means that there always is a race condition where log lines might be lost for lines logs being written gzipping.
You can disable the warning using gzip --quiet, but really that doesn't hide the fact that you might still loose log lines.
This means that delaycompress is not a generic fix to this. It's a specific fix to a specific problem.
The real solution for this is probably a combination of delaycompress and being able to send a signal to the process. It will make the race condition go away in practise (unless you rotate multiple times per second :) ).
Hello people
It's my first time using logrotate and I don't know if I'm configuring it in the right way. I'm using it with loggerhead log file in Ubuntu 11.04
Log is under
/log/loggerhead/loggerheadd.log
My configuration file looks like this
/log/loggerhead/loggerheadd.log {
daily
rotate 7
compress
delaycompress
missingok
}
Then I run a force rotation
logrotate -f /etc/logrotate.d/loggerhead
and that change the name of the log file to
/log/loggerhead/loggerheadd.log.1
And didn't create the original file (loggerheadd.log) again, so I couldn't run a new force rotation, because "the file doesn't exist".
So, it's supposed that the application write entries in "loggerheadd.log" but when logrotate run the file will be renamed, so where will be written the log entries? Am I missing something?
Hope you can help me
By default logrotate will just rename your files, so your old file will be gone.
You can either use the create option to create a new file after the old one is used, or copytruncate to copy the original file to one with a new name, then truncate the original. Either option will do what you're asking for (more details on the man page here)
I looked at logrotate.conf examples and everything in my /etc/logrotate.d directory. Nowhere was I able to find documentation on variables in these files.
I am trying to create a config file for rotating the logs of an application we are writing. I want to set the directory where logs are stored once, and then use it as a variable, like so:
my_app_log_dir=/where/it/is/deployed/logs
${my_app_log_dir}/*.log ${my_app_log_dir}/some_sub_dir/*.log {
missingok
# and so on
# ...
}
Is that possible?
You can achieve what you are looking for by implementing this kludge:
my-logrotate.conf ( NOTE: double quotes " are mandatory, also note that file names don't have to appear on the same line )
"*.log"
"some_sub_dir/*.log"
{
missingok
# and so on
# ...
}
Then the actual logrotate script - my-logrotate.sh
#!/bin/sh
set -eu
cd "${my_app_log_dir}"
exec logrotate /path/to/my-logrotate.conf
Now you can add logrotate.sh to your crontab.
You can use a bash "here document" to create a suitable config file on the fly, either at installation time or before running logrotate.
A bash script might look like this:
cat >rt.conf <<.
"${my_app_log_dir}/*.log" {
rotate 5
size 15k
missingok
}
.
logrotate rt.conf
Directly in the config file no (as far as my knowledge in logrotate goes).
Other solution:
Use the include option to include parts of the configuration file from a directory. This can help you if you have a package for your application, the package can leave a file in that directory containing only the entries for your app.
With logrotate 3.8.7,a test reveals that you can set and use variables in the pre-rotate and post-rotate script sections.
I tried this for a particular service log file.
postrotate
pid_file="/run/some_service/some_serviced.pid"
test -e "${pid_file}" && kill -s HUP $(cat "${pid_file}") || true
touch "${pid_file}.WAS_USED"
endscript
After running logrotate in force mode to ensure the log file was rotated and the post-rotate script executed, on looking in /run/some_service, there was an additional file "some_serviced.pid.WAS_USED", thus proving that the use of the variable worked.