How to setup MongoDB log rotation on Windows? - mongodb

Question

I have a conf file for mongod where I am trying to set log rotation, the log file size, and the number of old instances to be kept.
I have added logRotate: rename to the conf file as follows:
systemLog:
timeStampFormat: ctime
destination: file
logAppend: true
logRotate: rename
path: C:\Program Files\ApplicationName\logs\db.log
But I cannot find any documentation or examples that show how to specify the max size of the log file before it rotates and how many old instances to keep.
Also I don't know how to test this.
Apperently you need setup some kind of external process to send SIGUSR1 signal from OS to trigger rotation.
Any ideas? Has anyone actually done this successfully?

Answer 1

There is nothing available in MongoDB itself. Either you write all logs into one file:
systemLog
logAppend: true
or you create a new log file every time you start MonogDB server:
systemLog
logAppend: true
logRotate: rename
In either case you have to delete/compress old log files manually with a 3rd party tool or by script.
In Linux you can trigger a log rotate by kill -USR1 but I am not aware this would be available in Windows. You need to initiate log rotate from Mongo shell:
use admin
db.runCommand({ logRotate: 1 })
or
db.getSiblingDB('admin').runCommand({ logRotate: 1 })
db.adminCommand({ logRotate: 1 })
You can run it from command line with
mongo --eval "db.adminCommand({ logRotate: 1 })"
Setup such task with Windows Task Scheduler. You may have a look at LogRotateWin which is an implementation of logrotate utility for Windows Platform. I never tried it, but it seems to work as supposed to be. For sample logrotate configuration see How can I disable the logging of MongoDB?
Note, for Linux in order not to lose any logs, you first have to rename the logfile and afterwards initiate the log rotate command. I don't know whether Windows behaves in the same way.

Answer 2

With the help of Wernfried-Domscheit answer, a lot of research, weeks of trial and error and frustration I figured out a working solution on windows.
Prerequisites
You have a MongoDB-Service running with a configuration file mongod.cfg
LogRotateWin is installed. This is a third party package to rotate the logfiles, based on the unix implementation. It provides many customizable functionalities like compression, when rotating is applied, old file deletion and more. You can find a full option list here LogRotateWin Configuration.
Basic knowledge of Windows Taskscheduler.
Very basic knowledge of bat files.
1. Adjust the configuration file of the MongoDB-Service
Stop the service
Open the mongod.cfg and find the code lines where the systemLog is configured:
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: E:\MongoDB\Server\4.4\log\mongod.log
logRotate: reopen
Be sure, that you overwrite the path to your mongod.log. Be also sure, that logAppend: true and logRotate: reopen are set.
Delete the current mongod.log file
Restart the service
2. Configure the logrotate configuration
This is my configuration. You can customize that to your own needs. But dont use copy, copytruncate and create and dont remove the postrotate commands!
E:\MongoDB\Server\4.4\log\mongod.log {
nocompress
daily
size 100m
missingok
rotate 50
postrotate
E:\logrotate\notify_mongodb_service.bat
endscript
}
This configuration rotates the logs uncompressed every day or if the size exceeded 100 megabyte. Maximum of 50 logs are stored, older files will be deleted. The postrotate script will be executed when the rotation succeeded.
3. Create the notify_mongodb_service.bat file
This file sends a command to the MongoDB-Service, that a new file will be used. If you enabled authorization you can add -u username -p password aswell.
E:\MongoDB\Server\4.4\bin\mongo.exe --eval "db.adminCommand({ logRotate: 1 })"
Change the path to mongo.exe to the location on your system.
Save the file and be sure that the path in the logrotate configuration is the same! (the line between postrotate and endscript
The command is stored in an extra file, because the LogRotateWin interprets the Brackets ({}) of the mongo command and throws an Exception.
4. Check that all is working
Open the folder of the logs. You should see a single file mongod.log.
Open a terminal and check that logrotation is working ( -f forces the logrotation even if no trigger was triggered):
logrotate logrotate.conf -f
A new log file should occur. (for me it is mongod.log.1)
The mongod.log should be empty.
Trigger something that will be logged. For example connect with monogdbCompass to your MongoDB.
Check mongod.log. The connection should be logged there.
5. Create a Taskscheduler Job to run logrotation periodically
I wont talk about taskscheduler creation but here is an example of my configuration. You can import that file and modify it to your needs.
Also change the path of the logrotate.exe to your location of your system.
This job runs every hour to check if one or multiple triggers of logrotate triggered:
<?xml version="1.0" encoding="UTF-16"?>
<Task version="1.4" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">
<RegistrationInfo>
<Date>2021-12-02T17:57:55.9541897</Date>
<Author>Grayknife</Author>
<Description>Execute Logrotate Hourly</Description>
<URI>\docker\LogRotate</URI>
</RegistrationInfo>
<Triggers>
<CalendarTrigger>
<Repetition>
<Interval>PT1H</Interval>
<StopAtDurationEnd>false</StopAtDurationEnd>
</Repetition>
<StartBoundary>2021-12-02T18:00:00</StartBoundary>
<ExecutionTimeLimit>PT30M</ExecutionTimeLimit>
<Enabled>true</Enabled>
<ScheduleByDay>
<DaysInterval>1</DaysInterval>
</ScheduleByDay>
</CalendarTrigger>
</Triggers>
<Principals>
<Principal id="Author">
<UserId>1234</UserId>
<LogonType>Password</LogonType>
<RunLevel>LeastPrivilege</RunLevel>
</Principal>
</Principals>
<Settings>
<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>
<DisallowStartIfOnBatteries>true</DisallowStartIfOnBatteries>
<StopIfGoingOnBatteries>true</StopIfGoingOnBatteries>
<AllowHardTerminate>true</AllowHardTerminate>
<StartWhenAvailable>false</StartWhenAvailable>
<RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>
<IdleSettings>
<StopOnIdleEnd>true</StopOnIdleEnd>
<RestartOnIdle>false</RestartOnIdle>
</IdleSettings>
<AllowStartOnDemand>true</AllowStartOnDemand>
<Enabled>true</Enabled>
<Hidden>false</Hidden>
<RunOnlyIfIdle>false</RunOnlyIfIdle>
<DisallowStartOnRemoteAppSession>false</DisallowStartOnRemoteAppSession>
<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>
<WakeToRun>false</WakeToRun>
<ExecutionTimeLimit>PT0S</ExecutionTimeLimit>
<Priority>7</Priority>
</Settings>
<Actions Context="Author">
<Exec>
<Command>E:\logrotate\logrotate.exe</Command>
<Arguments>logrotate.conf</Arguments>
<WorkingDirectory>E:\logrotate</WorkingDirectory>
</Exec>
</Actions>
</Task>
I hope i could help someone with that guide.
Edit 2022-05-01
I faced the issue that logrotate throws exception when there are more than 9 files:
E:\logrotate>E:\logrotate\logrotate.exe logrotate.conf -f
logrotate: Force option set to true
logrotate: Exception: Cannot create a file when that file already exists.
logrotate: StackTrace: at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.__Error.WinIOError()
at System.IO.File.InternalMove(String sourceFileName, String destFileName, Boolean checkHost)
at logrotate.Program.AgeOutRotatedFiles(logrotateconf lrc, FileInfo fi, String rotate_path)
at logrotate.Program.RotateFile(logrotateconf lrc, FileInfo fi)
at logrotate.Program.Main(String[] args)
E:\logrotate>E:\logrotate\logrotate.exe logrotate.conf -f
logrotate: Force option set to true
logrotate: Exception: Access to the path 'E:\MongoDB\Server\4.4\log\mongod.log.10' is denied.
logrotate: StackTrace: at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.File.InternalDelete(String path, Boolean checkHost)
at System.IO.File.Delete(String path)
at logrotate.Program.DeleteRotateFile(String m_filepath, logrotateconf lrc)
at logrotate.Program.AgeOutRotatedFiles(logrotateconf lrc, FileInfo fi, String rotate_path)
at logrotate.Program.RotateFile(logrotateconf lrc, FileInfo fi)
at logrotate.Program.Main(String[] args)
E:\logrotate>
Quickfix for now is to set a maximum of 9 files.

Answer 3

Create a python file logrotate.py with content:
import os
# path to your mongod.log file
path = 'D:/MongoDB/Server/5.0/log/'
# you can insert a condition here that checks the size of mongod.log first
# ...
# send logrotate command through terminal
# if you don't have mongosh(ell) installed, replace by mongo / mongod
os.system('cmd /c "mongosh --eval "db.adminCommand({ logRotate: 1 })"')
# delete rotated log file
for file in os.listdir(path):
if file.startswith('mongod.log.'):
os.remove(path+file)
quit()
Create a batch file logrotate.bat with content:
python logrotate.py
Create a task in Windows task scheduler that runs this program every X hours.