logrotate fails to rotate syslog occassionally - logrotate

Can someone please tell me why occasionally syslog does not rotate and keeps logging in the same file?
After the following customization to the default setting the occasional syslog file rotation issue is observed:
Path for syslog is changed from default /var/log/syslog to /opt/vortex/log/syslog (changed in /etc/rsyslog.conf as given below)
$template ATMFormat,"%$YEAR% %timegenerated%::%syslogtag%:%msg:::$%\n"
auth,authpriv.* /var/log/auth.log
.;auth,authpriv.none -/opt/vortex/log/syslog;ATMFormat
syslog rotation rule is changed from default weekly to daily and set to retain last 30 files (changed in /etc/logrotate.d/rsyslog)
Please find the given below details of configurations and scripts for your reference.
logrotate is scheduled daily through cron by placing logrotate script file in /etc/cron.daily directory. crontab has the following definition
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user command
17 * * * * root cd / && run-parts --report /etc/cron.hourly
0 0 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#
/etc/cron.daily/logrotate script has the below rules
#!/bin/sh
test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf
The contents of logrotate.conf is given below:
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
missingok
monthly
create 0664 root utmp
rotate 1
}
/var/log/btmp {
missingok
monthly
create 0660 root utmp
rotate 1
}
# system-specific logs may be configured here
log rotate configuration for system log is specified in /etc/logrotate.d/rsyslog as given below :
/opt/vortex/log/syslog
{
rotate 30
daily
missingok
notifempty
delaycompress
compress
postrotate
invoke-rc.d rsyslog reload > /dev/null
endscript
}
/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/debug
/var/log/messages
{
rotate 4
weekly
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
invoke-rc.d rsyslog reload > /dev/null
endscript
}
Thanks for your help

Related

How to update modules.conf for SELINUX in BUILDROOT?

looking to disable some SELinux modules (set to off) and create others in modules.conf. I don't see an obvious way of updating modules.conf as I tried adding my changes as a modules.conf patch but it failed given that the modules.conf file gets built and is not just downloaded by BR so it is not available for patching like other things under the refpolicy directory:
Build window output:
refpolicy 2.20190609 PatchingApplying 0001-refpolicy-update-modules-conf.patch using patch:
can't find file to patch at input line 3
I did see in the log that there is a support/sedoctool.py that autogenerates the policy/modules.conf file so that the file is NOT patchable like most other things in the ref policy.
The relevant section of the buildroot/output/build/refpolicy-2.20190609/Makefile:
# policy building support tools
support := support
genxml := $(PYTHON) $(support)/segenxml.py
gendoc := $(PYTHON) $(support)/sedoctool.py
<...snip...>
########################################
#
# Create config files
#
conf: $(mod_conf) $(booleans) generate$(booleans) $(mod_conf): conf.intermediate.INTERMEDIATE: conf.intermediate
conf.intermediate: $(polxml)
#echo "Updating $(booleans) and $(mod_conf)"
$(verbose) $(gendoc) -b $(booleans) -m $(mod_conf) -x $(polxml)
Part of the hsmlinux build.log showing the sedoctool.py (gendoc) being run:
Updating policy/booleans.conf and policy/modules.conf
.../build-buildroot-sawshark/buildroot/output/host/usr/bin/python3 support/sedoctool.py -b policy/booleans.conf -m policy/modules.conf -x doc/policy.xml
I'm sure there is a standard way of doing this, just doesn't seem to be documented anywhere I can find.
Thanks.
Turns out that the sedoctool.py script is reading the doc/policy.xml. Looking at sedoctool.py:
#modules enabled and disabled values
MOD_BASE = "base"
MOD_ENABLED = "module"
MOD_DISABLED = "off"
<...snip...>
def gen_module_conf(doc, file_name, namevalue_list):
"""
Generates the module configuration file using the XML provided and the
previous module configuration.
"""
# If file exists, preserve settings and modify if needed.
# Otherwise, create it.
<...snip...>
mod_name = node.getAttribute("name")
mod_layer = node.parentNode.getAttribute("name")
<...snip...>
if mod_name and mod_layer:
file_name.write("# Layer: %s\n# Module: %s\n" % (mod_layer,mod_name))
if required:
file_name.write("# Required in base\n")
file_name.write("#\n")
if [mod_name, MOD_DISABLED] in namevalue_list:
file_name.write("%s = %s\n\n" % (mod_name, MOD_DISABLED))
# If the module is set as enabled.
elif [mod_name, MOD_ENABLED] in namevalue_list:
file_name.write("%s = %s\n\n" % (mod_name, MOD_ENABLED))
# If the module is set as base.
elif [mod_name, MOD_BASE] in namevalue_list:
file_name.write("%s = %s\n\n" % (mod_name, MOD_BASE))
So sedoctool.py has the nice feature of: "# If file exists, preserve settings and modify if needed." and modules.conf can just be added whole here via a complete file patch and the modules that are not desired set as "off" : refpolicy-2.20190609/policy/modules.conf and the script will update as needed based on desired policy.
One more detail is that in the next stage of the refpolicy Makefile (Building) the modules.conf with the updates is deleted in the beginning which kind of clashes with the ability of sedoctool to preserve the patched version of modules.conf...so patched the removal in the Building stage of the Makefile.
[7m>>> refpolicy 2.20190609 Building^[
<...snip...>
rm -f policy/modules.conf
The Makefile in refpolicy-2.20190609 has this line that I patched out because we are patching in our own modules.conf:
bare: clean
<...snip...>
$(verbose) rm -f $(mod_conf)
That patch looks like:
--- BUILDROOT/Makefile 2020-08-17 13:25:06.963804709 -0400
+++ FIX/Makefile 2020-08-17 19:25:29.540607763 -0400
## -636,7 +636,6 ##
$(verbose) rm -f $(modxml)
$(verbose) rm -f $(tunxml)
$(verbose) rm -f $(boolxml)
- $(verbose) rm -f $(mod_conf)
$(verbose) rm -f $(booleans)
$(verbose) rm -fR $(htmldir)
$(verbose) rm -f $(tags)
BTW,
Creating a patch with a complete new file in pp1:q!:
diff -crB --new-file pp0 pp1 > pp0.patch

u-boot bbappend causes file system corruption

I need to configure u-boot to boot immediately (disabling the press key to interrupt prompt) for a yocto build.
I've added the following bbappend to do so:
# u-boot_%.bbappend
FILESEXTRAPATHS_prepend := "${THISDIR}/u-boot:"
SRC_URI_append_edge = " \
file://disable-boot-delay.cfg \
"
where disable-boot-delay.cfg looks like this:
CONFIG_AUTOBOOT_KEYED=y
CONFIG_AUTOBOOT_KEYED_CTRLC=n
CONFIG_BOOTDELAY=-2
This successfully disables the boot prompt, but also causes the FAT partition that u-boot is installed on to become corrupt:
$ # without above bbappend
$ dosfsck /dev/mapper/loop0p1
fsck.fat 4.1 (2017-01-24)
/dev/mapper/loop0p1: 49 files, 6510/20431 clusters
$ # with above bbappend
$ dosfsck /dev/mapper/loop0p1
fsck.fat 4.1 (2017-01-24)
/OVERLAYS/.
Bad short file name (.).
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it
? 4
/OVERLAYS/..
Bad short file name (..).
1) Drop file
2) Rename file
3) Auto-rename
4) Keep it
? 4
/dev/mapper/loop0p1: 49 files, 6510/20431 clusters
I'm massively, massively confused as to how this change could be causing file system corruption.
Environment:
Yocto Warrior (2.7)
meta-raspberrypi (warrior branch)
RPI_USE_U_BOOT = "1"
Diffing workdir/build/.config with and without the above append:
229c229
< CONFIG_BOOTDELAY=2
---
> CONFIG_BOOTDELAY=-2
306c306,311
< # CONFIG_AUTOBOOT_KEYED is not set
---
> CONFIG_AUTOBOOT_KEYED=y
> CONFIG_AUTOBOOT_PROMPT="Autoboot in %d seconds\n"
> # CONFIG_AUTOBOOT_ENCRYPTION is not set
> CONFIG_AUTOBOOT_DELAY_STR=""
> CONFIG_AUTOBOOT_STOP_STR=""
> # CONFIG_AUTOBOOT_KEYED_CTRLC is not set

logrotate working but ignoring size

CentOS v.7
Logrotate v.3.8.6
I set logrotate to rotate when file reaches 5M but it ignores it, if I add daily it will rotate daily regardless of size, i tried with size, minsize and maxsize all the same the only difference is with "size" it doesnt even refer to it in the output, here is my config and output of logrotate -vdf /etc/logrotate.d/maillog
(the actual log file size when running the following tests was 45K)
(the conf file is the same for all tests only the size parameter changed)
/var/log/maillog {
size 5M
rotate 50
create 644 root root
dateext
dateformat -%Y-%m-%d_%H_%s
notifempty
postrotate systemctl restart rsyslog
systemctl restart postfix
endscript }
SIZE:
logrotate -vdf /etc/logrotate.d/maillog
reading config file /etc/logrotate.d/maillog
Allocating hash table for state file, size 15360 B
Handling 1 logs
rotating pattern: /var/log/maillog forced from command line (50 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/maillog
log needs rotating
rotating log /var/log/maillog, log->rotateCount is 50
Converted ' -%Y-%m-%d_%H_%s' -> '-%Y-%m-%d_%H_%s'
dateext suffix '-2017-12-19_13_1513689486'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]_[0-9][0-9]_[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
fscreate context set to unconfined_u:object_r:var_log_t:s0
renaming /var/log/maillog to /var/log/maillog-2017-12-19_13_1513689486
creating new /var/log/maillog mode = 0644 uid = 0 gid = 0
running postrotate script
running script with arg /var/log/maillog: "
systemctl restart rsyslog
systemctl restart postfix
"
No reason for "log needs rotating" is given.
MINSIZE:
logrotate -vdf /etc/logrotate.d/maillog
reading config file /etc/logrotate.d/maillog
Allocating hash table for state file, size 15360 B
Handling 1 logs
rotating pattern: /var/log/maillog forced from command line (50 rotations)
empty log files are not rotated, only log files >= 5242880 bytes are rotated, old logs are removed
considering log /var/log/maillog
log needs rotating
rotating log /var/log/maillog, log->rotateCount is 50
Converted ' -%Y-%m-%d_%H_%s' -> '-%Y-%m-%d_%H_%s'
dateext suffix '-2017-12-19_13_1513689869'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]_[0-9][0-9]_[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
fscreate context set to unconfined_u:object_r:var_log_t:s0
renaming /var/log/maillog to /var/log/maillog-2017-12-19_13_1513689869
creating new /var/log/maillog mode = 0644 uid = 0 gid = 0
running postrotate script
running script with arg /var/log/maillog: "
systemctl restart rsyslog
systemctl restart postfix
"
Here it shows, "only log files >= are rotated" but no reason for "log needs rotating" is given.
MAXSIZE:
reading config file /etc/logrotate.d/maillog
Allocating hash table for state file, size 15360 B
Handling 1 logs
rotating pattern: /var/log/maillog forced from command line (50 rotations)
empty log files are not rotated, log files >= 5242880 are rotated earlier, old logs are removed
considering log /var/log/maillog
log needs rotating
rotating log /var/log/maillog, log->rotateCount is 50
Converted ' -%Y-%m-%d_%H_%s' -> '-%Y-%m-%d_%H_%s'
dateext suffix '-2017-12-19_13_1513690859'
glob pattern '-[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]_[0-9][0-9]_[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
fscreate context set to unconfined_u:object_r:var_log_t:s0
renaming /var/log/maillog to /var/log/maillog-2017-12-19_13_1513690859
creating new /var/log/maillog mode = 0644 uid = 0 gid = 0
running postrotate script
running script with arg /var/log/maillog: "
systemctl restart rsyslog
systemctl restart postfix
"
Here it shows, "log files => are rotated" but no reason for "log needs rotating" is given.
Why is it ignoring file size when rotating?
You're running logrotate with -f, in that scenario it's always going to force a rotation with a complete lack of regard for your other options:
https://manpages.debian.org/jessie/logrotate/logrotate.8.en.html
-f, --force
Tells logrotate to force the rotation, even if it doesn't think this is necessary.
It gives you no reason because you in fact were the reason-giver.

How do I roll sensu logs without restarting?

Sensu logs can fill up with large amounts of data. You can setup an outside infrastructure with logrotate to restart sensu software on a periodic basis to eliminate open file handles but we would prefer not to restart.
Is there a way to roll the logs to a set number of backups with a set disk usage? I'm looking for configuration similar to how you can configure a Java application's logging with log4j and rolling file appenders/loggers. I cannot find anything on the sensu website.
Update: In my case, it turned out that the PID files from /var/run/sensu/sensu-.*.pid were missing, which seems to be due to the fact that we're managing the Sensu processes via /opt/sensu/embedded/bin/sensu-ctl. I ended up fixing it by applying this patch to logrotate.d/sensu:
diff --git a/sensu_configs/logrotate.d/sensu b/sensu_configs/logrotate.d/sensu
index 8457e29..42a80f9 100644
--- a/sensu_configs/logrotate.d/sensu
+++ b/sensu_configs/logrotate.d/sensu
## -6,7 +6,7 ##
sharedscripts
compress
postrotate
- kill -USR2 `cat /var/run/sensu/sensu-client.pid 2> /dev/null` 2> /dev/null || true
+ /opt/sensu/embedded/bin/sensu-ctl sensu-client 2
endscript
}
## -18,7 +18,7 ##
sharedscripts
compress
postrotate
- kill -USR2 `cat /var/run/sensu/sensu-server.pid 2> /dev/null` 2> /dev/null || true
+ /opt/sensu/embedded/bin/sensu-ctl sensu-server 2
endscript
}
## -30,6 +30,6 ##
sharedscripts
compress
postrotate
- kill -USR2 `cat /var/run/sensu/sensu-api.pid 2> /dev/null` 2> /dev/null || true
+ /opt/sensu/embedded/bin/sensu-ctl sensu-api 2
endscript
}
I am leaving the original answer below, in case somebody finds it useful.
I think logrotate.d/sensu should do what you need, by sending the -USR2 signal to Sensu when rotating logs. You might need to apply this patch to it, though:
diff --git a/sensu.logrotate b/sensu.logrotate
index 8457e29..a5178fa 100644
--- a/sensu.logrotate
+++ b/sensu.logrotate
## -1,4 +1,5 ##
/var/log/sensu/sensu-client.log {
+ su sensu sensu
rotate 7
daily
missingok
## -11,6 +12,7 ##
}
/var/log/sensu/sensu-server.log {
+ su sensu sensu
rotate 7
daily
missingok
## -23,6 +25,7 ##
}
/var/log/sensu/sensu-api.log {
+ su sensu sensu
rotate 7
daily
missingok
Please let me know if you ever get a chance to test it out.

Magento: upgrade pre 1.6 version to most recent one

I've seen a lot of questions about pre 1.6 Magento installations to the most recent version (at the current moment 1.7.0.2) but there are a lot of answers that don't work for everybody.
So below the answer to the question:
How to upgrade Magento from a pre 1.6 installation to the most recent one.
There are a lot of versions and not all of them are working. This one has worked for me for a lot of versions, as far as 1.3 to 1.7.
Please add comments with solutions to problems you're experiencing, I can update the answer so other people get help from this topic too!
What you need:
- SUDO rights/root account on your server.
- The linux package 'nohub'
- make sure NOBODY can trigger the index.php. If your version supports maintenance.flag, put an empty maintenance.flag file in your Magento root.
Walkthrough
1) Download the latest Magento. Overwrite: ./download/* ./lib/* ./mage
2) Run these steps from you Magento root als SUDOer (if you're not root, put 'sudo' for all the commands)
find . -type f -exec chmod 644 {} \;
find . -type d -exec chmod 755 {} \;
chmod -R 777 ./var
chmod 550 mage
3) Go to your Magento root folder and type:
./mage list-upgrades
./mage config-set preferred_state stable
./mage upgrade-all --force
./mage install http://connect20.magentocommerce.com/community Mage_All_Latest --force
4) Now there is the last step. Note: In some situations this process can take up to 8+ hours!
nohup php -f ./index.php
Known issues
1) it's possible that your update gets in a loop. To find this loop, enable debugging.Edit: /lib/Varien/Db/Adapter/Pdo/Mysql.php (+/- line 112 and 112)
protected $_debug = true;
protected $_debuglogeverything = true;
This will write a debug to: /var/debug/[debug_file]
2) Read the file by opening the dir:
cd /var/debug/[debug_file] <-- replace with the actual filename
tail -f [debug_file]
3) If you use debug, the file will get HUGE! Make sure you delete it once in a while.
Tip: as a root user, type:
crontab -e
*/5 * * * * rm /[my_magento_base_folder]/var/debug/[debug_file] <-- add this line
If you want to read the file, add a # to this line and use tail to read it.
These steps help you find common errors and loops (if the tail shows a repeating error message)