I have a custom application (myApp) which is writing logs to the file '/var/log/myApp'. I can see the logs being written and it works fine. Now I am trying to setup logrotate for this file and for this I have created a config file '/etc/logrotate.d/myApp', the contents of which are -
/var/log/myApp {
missingok
size +10k
start 0
nocompress
create 0600 root root
rotate 10
postrotate
/etc/init.d/rsyslog restart > /dev/null 2>&1 || true
endscript
}
Now if i do a logrotate -dv /etc/logrotate.d/myApp I don't see any errors as such and when logrotate -f /etc/logrotate.d/myApp is executed i.e., a forceful logrotate the log is rotated. But when the log file size exceeds 10k the log is not automatically rotated. Any help would be appreciated.
logrotate rotates the logs specified in /etc/logrotate.d/ according to the time interval configured in /etc/logrotate.conf. On some distros, the default value is a week. You can override that time interval in your specific config using e.g. 'daily' in your config file.
Log files will not be rotated until logrotate has known about the files for at least as long as the time specified. I assume that you waited this long and/or modified the conf file?
logrotate is working only once a day , So it's not checking your file size all the time.
if you want an effective way to keep the size lower , you should use a cron job like
for example , force the rotate every hour in your crontab
0 * * * * logrotate <my conf here>
and your logs will be , if needed, rotated every hour. I had such problem in my free-tier aws server , as the size exceed rapidly and i didn't have enough disk space to wait 24 hours for a rotate
Related
My MongoDB Log file is over 16GB, the log file is not important to me. How can I disable the logging of MongoDB and if I do, will I encounter any problems?
It would not be a smart idea to turn off logging. Use Rotate Log Files to rotate them and keep them small.
logrotate is standard function on Linux.
Simplest way to rotate the log file is kill -USR1 $(/usr/sbin/pidof mongod)
My logrotate.conf file looks like this:
missingok
compress
delaycompress
notifempty
create
/var/log/mongodb/mongod.log{
size 10M
rotate 9
sharedscripts
postrotate
kill -USR1 $(/usr/sbin/pidof mongod)
endscript
}
When the logfile reaches 10MB then it is rotated. Up to 9 files are kept. logrotate is executed by a daily cron job.
Though you can disable logging, it is really not recommended.
I have a problem with logrotate. The application itself produces following logs:
xxx.log
and at 23:59 the application changes the log to:
xxx.log.2019-01-05
and so on. Right now I am getting following in the log directory:
xxx.log
xxx.log.2019-01-01
xxx.log.2019-01-02
etc.
What I need to to is want to rotate the logs that get created on 23:59 and not to touch the xxx.log file itself.
I have tried with following logrotate rule:
/var/log/xxx/xxx/xxx.log.* {
daily
missingok
rotate 30
compress
notifempty
copytruncate
nosharedscripts
prerotate
bash -c "[[ ! $1 =~ *.gz ]]"
endscript
}
But first of all logrotate does not compress the log that was created last and it also adds .1.gz extension to previously compressed files.
logrotate does not compress the log that was created last
Do you have "delaycompress" defined in /etc/logrotate.conf? Per logrotate man:
delaycompress
Postpone compression of the previous log file to the next rotation cycle.
it also adds .1.gz extension
While you're at the aforementioned man page, you should check out what the "extension" option does:
extension ext
Log files with ext extension can keep it after the rotation.
in last days i get daily mail from cron's logrotate task:
/etc/cron.daily/logrotate:
gzip: stdin: file size changed while zipping
How can I fix it?
Thanks,
Gian Marco.
Here's a blog post in French which gives a solution.
In English, you can read this bug report.
To summarise:
First you have to add the --verbose option in the script /etc/cron.daily/logrotate to have more information the next time it runs to identify which rotation log cause the problem.
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate --verbose /etc/logrotate.conf`
Next you have to add the delaycompress option in logrotate configuration.
Like exemple, I add the nginx's logrotate configiguration in /etc/logrotate.d/nginx:
/var/log/nginx/*.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 0640 www-data adm
sharedscripts
...
}
upstart will close (and reopen) its log file when it notices that the file is deleted. However, if you look at what gzip does, you see that it doesn't delete the file until after it's one writing the output file. That means that there always is a race condition where log lines might be lost for lines logs being written gzipping.
You can disable the warning using gzip --quiet, but really that doesn't hide the fact that you might still loose log lines.
This means that delaycompress is not a generic fix to this. It's a specific fix to a specific problem.
The real solution for this is probably a combination of delaycompress and being able to send a signal to the process. It will make the race condition go away in practise (unless you rotate multiple times per second :) ).
I have a logrotate setting in logrotate.conf that doesn't want to run. What I'm trying to do is:
Rotate the log every day and truncate the log
Emailing me the rotated lines of the log
Adding the lines of the rotated log to a monthly archive
Create olddir rotated/ if it doesn't exist yet
What am i missing here?
(log file path){
daily
rotate 0
olddir rotated
copytruncate
nodateext
missingok
notifempty
compress
mailfirst
mail email (at) email . com
prerotate
original = $1
replacement = 'rotated'
olddir_path = "${original/php-error.log/$replacement}"
mkdir olddir_path
endscript
postrotate
original = $1
replacement = 'rotated'
olddir_path = "${original/php-error.log/$replacement}"
cat "${olddir_path}/php-error.log.1" >> "${olddir_path}/php-error-monthly.log"
endscript
}
You were great all the way up to "Create olddir rotate/ if it doesn't exist yet". Run logrotate -d /etc/logrotate.conf and I'll bet you get something like this back for that configuration:
error: /etc/logrotate.conf: error verifying olddir path rotated: No such file or directory
This is because that directory needs to exist at the time logrotate reads its configuration file; this is before any prerotate scripts are executed so naturally that directory does not exist and parsing the configuration fails. Make that directory first, however you'd like but first.
Currently my log file sits at 32 meg. Did I miss an option that would split the log file as it grows?
You can use logrotate to do this job for you.
Put this in /etc/logrotate.d/mongod (assuming you use Linux and have logrotate installed):
/var/log/mongo/*.log {
daily
rotate 30
compress
dateext
missingok
notifempty
sharedscripts
copytruncate
postrotate
/bin/kill -SIGUSR1 `cat /var/lib/mongo/mongod.lock 2> /dev/null` 2> /dev/null || true
endscript
}
If you think that 32 megs is too large for a log file, you may also want to look inside to what it contains.
If the logs seem mostly harmless ("open connection", "close connection"), then you may want to start mongod with the --quiet switch. This will reduce some of the more verbose logging.
Rotate the logs yourself
http://www.mongodb.org/display/DOCS/Logging
or use 'logrotate' with an appropriate configuration.
Using logrotate is a good option. while, it will generate 2 log files that fmchan commented, and you will have to follow Brett's suggestion to "add a line to your postrotate script to delete all mongod style rotated logs".
Also copytruncate is not the best option. There is always a window between copy and truncate. Some mongod logs may get lost. Could check logrotate man page or refer to this copytruncate discussion.
Just provide one more option. You could write a script that sends the rotate signal to mongod and remove the old log files. mongologrotate.sh is a simple reference script that I have written. You could write a simple cron job or script to call it periodically like every 30 minutes.