How do I roll sensu logs without restarting? - sensu

Question

Sensu logs can fill up with large amounts of data. You can setup an outside infrastructure with logrotate to restart sensu software on a periodic basis to eliminate open file handles but we would prefer not to restart.
Is there a way to roll the logs to a set number of backups with a set disk usage? I'm looking for configuration similar to how you can configure a Java application's logging with log4j and rolling file appenders/loggers. I cannot find anything on the sensu website.

Answer 1

Update: In my case, it turned out that the PID files from /var/run/sensu/sensu-.*.pid were missing, which seems to be due to the fact that we're managing the Sensu processes via /opt/sensu/embedded/bin/sensu-ctl. I ended up fixing it by applying this patch to logrotate.d/sensu:
diff --git a/sensu_configs/logrotate.d/sensu b/sensu_configs/logrotate.d/sensu
index 8457e29..42a80f9 100644
--- a/sensu_configs/logrotate.d/sensu
+++ b/sensu_configs/logrotate.d/sensu
## -6,7 +6,7 ##
sharedscripts
compress
postrotate
- kill -USR2 `cat /var/run/sensu/sensu-client.pid 2> /dev/null` 2> /dev/null || true
+ /opt/sensu/embedded/bin/sensu-ctl sensu-client 2
endscript
}
## -18,7 +18,7 ##
sharedscripts
compress
postrotate
- kill -USR2 `cat /var/run/sensu/sensu-server.pid 2> /dev/null` 2> /dev/null || true
+ /opt/sensu/embedded/bin/sensu-ctl sensu-server 2
endscript
}
## -30,6 +30,6 ##
sharedscripts
compress
postrotate
- kill -USR2 `cat /var/run/sensu/sensu-api.pid 2> /dev/null` 2> /dev/null || true
+ /opt/sensu/embedded/bin/sensu-ctl sensu-api 2
endscript
}
I am leaving the original answer below, in case somebody finds it useful.
I think logrotate.d/sensu should do what you need, by sending the -USR2 signal to Sensu when rotating logs. You might need to apply this patch to it, though:
diff --git a/sensu.logrotate b/sensu.logrotate
index 8457e29..a5178fa 100644
--- a/sensu.logrotate
+++ b/sensu.logrotate
## -1,4 +1,5 ##
/var/log/sensu/sensu-client.log {
+ su sensu sensu
rotate 7
daily
missingok
## -11,6 +12,7 ##
}
/var/log/sensu/sensu-server.log {
+ su sensu sensu
rotate 7
daily
missingok
## -23,6 +25,7 ##
}
/var/log/sensu/sensu-api.log {
+ su sensu sensu
rotate 7
daily
missingok
Please let me know if you ever get a chance to test it out.