in the process of rotating kafka logs(kafka 0.8.1.1), I've managed to rotate all of them, except kafkaServer.out.
this is not handled by log4j.
after looking in kafka-run-class.sh which is the script that starts the java process, it runs with redirection to a file.
i cannot rotate the file unless kafka is restarted.
truncating it, the process just ignores it and keeps on writing to the file.
is there a way to force it to rotate this log file ?
thanks.
I have written a C++ tool called 'rotate' to solve the problem. Source code can be found here, https://github.com/peihanw/rotate
After compile and deploy to $APP_HOME/bin/rotate, please modify the start command in kafka-run-class.sh, here is my example:
nohup $JAVA $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$#" < /dev/null | $APP_HOME/bin/rotate -o "$CONSOLE_OUTPUT_FILE" -t 1 &
And some additional dustman like maintenance tools or scripts can be configured to compress or remove expire 'kafkaServer.out.YYYYmmddHHMMSS' files.
Related
I have a script that runs in the background which uses watch to monitor a directory for changes. This works just fine. However, I had a need for the script which runs the monitoring script to be run as daemon. The daemon is running as the same user, but now watch is returning "1 Various failures."
I suspect that there is some environment variable that is not set right, but there are too many to use trial and error to diagnose the issue. And unfortunately, "Various failures" is not very helpful. Any ideas how I might diagnose this?
The command is
watch -d -t -g ls -l
I think something like the following can be used as a work-around.
diff <(ls -l) <(sleep 1; ls -l)
I finally discovered the following on stderr.
Error opening terminal: unknown.
It was easily resolved by the answer to error opening terminal. So it did turn out to be an environment variable. I don't see this error message in the code. Perhaps it occurs when making a system call that needs the terminal.
I have a little Minecraft server running on my server for my little brother.
I am using Screen to run it in the background but I would like to make a service/daemon of it so it does not stop any more. I have looked all around the internet trying to find a script that does this for me but I cannot find one.
My start-up script for the Minecraft server is:
#!/bin/sh
java -Xms512M -Xmx1G -XX:+UseConcMarkSweepGC -jar spigot.jar nogui
One good way (especially with Ubuntu-based OS's) is to have a script run that instantly starts the server if it happens to close down/stop.
Here's a nice little sh script I've put together:
while true;
do
echo "Starting server now!";
java -jar yourServerJar.jar;
echo "Server restarting in 5 seconds! Press control+c to stop!";
sleep 5;
done;
What this script does is create an infinite loop, in which the Server instance is started (java -jar yourServerJar.jar) and restarts your server after 5 seconds of downtime.
Save the above code as start.sh, then type in your command prompt something along the lines of:
sudo sh start.sh
This will run the start.sh script file. (Make sure to change the .jar to your servers .jar file!)
I'm running wildfly as a service on Linux.
I used the well-written instruction on http://developer-should-know.tumblr.com/post/112230363742/how-to-install-wildfly-as-a-service-on-linux that is based on a script [wildflyhome/bin/init.d/wildfly-init-redhat.sh] contained in the wildfly distribution. This script uses the declaration
JBOSS_CONSOLE_LOG="/var/log/wildfly/console.log"
Problem: This configuration logs twice: Firstly in server.log (in wildflyhome/standalone/log) and secondly in console.log. This wastes storage (and maybe some performance).
Therefore I set
JBOSS_CONSOLE_LOG="wildflyhome/standalone/log/server.log"
But now each log entry is written twice into server.log -:)
Question: How can I configure wildfly such that it logs just once ?
You can remove the console-handler from the servers configuration. By default WildFly logs to the stdout and the server.log. The JBOSS_CONSOLE_LOG="/var/log/wildfly/console.log" is seeing the output from stdout.
To remove the console handler you can execute the following CLI command
/subsystem=logging/root-logger=ROOT:remove-handler(name=CONSOLE)
If you want you could also remove the console-handler itself.
/subsystem=logging/console-handler=CONSOLE:remove
I had similar problem with Windows Service on Wildfly 11.0.0.Final. Wildfly service created two additional log files. Example:
wildfly-stderr.2017-11-22.log
wildfly-stdout.2017-11-22.log
It saved all logs both to stdout file and to server.log.
I couldn't turn of console-handlers due to Spring Boot logging issues on old version. Instead, i edited service.bat and changed this lines:
if "%STDOUT%"=="" set STDOUT=auto
if "%STDERR%"=="" set STDERR=auto
To this:
if "%STDOUT%"=="" set STDOUT=""
if "%STDERR%"=="" set STDERR=""
It looks that after this logging work correctly(remember to uninstall and install service once again). Now it saves logs only to server.log. I tested this for a while and don't see any missing logs.
Regarding the question whether the service scripts need the consol.log, I would say "yes" the default init.d scripts do need the console handler, because it greps the output to figure out when the service is up and running:
cat /dev/null > "$JBOSS_CONSOLE_LOG"
if [ "$JBOSS_MODE" = "standalone" ]; then
start-stop-daemon --start --user "$JBOSS_USER" \
--chuid "$JBOSS_USER" --chdir "$JBOSS_HOME" --pidfile "$JBOSS_PIDFILE" \
--exec "$JBOSS_SCRIPT" -- -c $JBOSS_CONFIG $JBOSS_OPTS >> "$JBOSS_CONSOLE_LOG" 2>&1 &
else
start-stop-daemon --start --user "$JBOSS_USER" \
--chuid "$JBOSS_USER" --chdir "$JBOSS_HOME" --pidfile "$JBOSS_PIDFILE" \
--exec "$JBOSS_SCRIPT" -- --domain-config=$JBOSS_DOMAIN_CONFIG \
--host-config=$JBOSS_HOST_CONFIG $JBOSS_OPTS >> "$JBOSS_CONSOLE_LOG" 2>&1 &
fi
count=0
launched=0
until [ $count -gt $STARTUP_WAIT ]
do
grep 'JBAS015874:' "$JBOSS_CONSOLE_LOG" > /dev/null
if [ $? -eq 0 ] ; then
launched=1
break
fi
sleep 1
count=$((count + 1));
done
Looking at it, I would say storage is not a major issue, since the script copies /dev/null into the log every time Wildfly starts. And since it greps for a given code 'JBAS015874' to know the server is up, unless you want to delete your entire server.log on every boot, you are going to have to rewrite that bit too (or it will find this string from previous startups!).
So, unless you want to rewrite all the init scripts, I'd just live with it.
Applications should not log to standard out anyways. The only thing I see there after wildfly is up and running are uncaught runtime exceptions...
In the root-logger under domain:logging:3.0 subsystem, Comment out the
<handler name="CONSOLE"/> as below
<!--handler name="CONSOLE"/-->
Afterward, restart your Wildfly Service. The stdout log won't be written to any longer. Hope this helps
I'm having trouble to start a service as a specific user (under Ubuntu 14.4) and I'm unsure what the problem is. I use the following command to autostart a jar-file on startup:
nohup ${JAVA_EXEC} -jar ${MICROSERVICE_HOME}/bin/${MICROSERVICE_JAR} server ${MICROSERVICE_CONF} 2>> /dev/null &
That works perfectly, therefore there is no problem with the variables and so on. Well, this script get's executed by the actual user, which is in this case, the root. Since I don't want to take any risks, I do want to execute it as a specific (already existing) user. Normally my approach would be to change the to command to:
nohup su some_user -c "${JAVA_EXEC} -jar ${MICROSERVICE_HOME}/bin/${MICROSERVICE_JAR} server ${MICROSERVICE_CONF}"
But this doesn't work. I don't get any error messages (of course I left out the redirection of stderr for test purposes) and the nohup.out is empty.
I already have tried different versions, e.g. replacing the double quotes with single quotes and masking the "$" inside the command. According to this thread it should work with the syntax.
None of the solutions in that thread do work. E.g.
su some_user -c "nohup ${JAVA_EXEC} -jar ${MICROSERVICE_HOME}/bin/${MICROSERVICE_JAR} server ${MICROSERVICE_CONF}" -> doesn't work
nohup runuser some_user c "nohup ${JAVA_EXEC} -jar ${MICROSERVICE_HOME}/bin/${MICROSERVICE_JAR} server ${MICROSERVICE_CONF}"-> doesn't work (the runuser commands doesn't exist).
What do I miss?
Any help is very appreciated!
I'm trying to make a simple file so I can call it in SSH and it will start my minecraft server.
I tried making a batch file called start.bat with this code:
java -Xmx512M -Xmx512M -jar craftbukkit-1.2.5-R1.0.jar nogui
However, when I run it in SSH:
$ cd /Minecraft/server_1/
$ start.bat
The SSH returns that it is an invalid or unknown command. Is there any other way I can make a quick command/file to start my server? What file extensions would I use to get this working? It works if I paste that java command in SSH and run it, but I'd rather have a file.
The current working directory is not included in your PATH by default because it is a security risk on multiuser systems. (And a potential annoyance even on machines that are single user.) You would use ./start.bat to start the program.
Since you're using Windows naming conventions, I presume you also forgot to set the execution mode bit -- and you probably also forgot the shebang line at the top of the file.
Try this:
#!/bin/sh
java -Xmx512M -Xmx512M -jar craftbukkit-1.2.5-R1.0.jar nogui
Run chmod 500 on this file. (Strictly speaking, 555 could also work, if you didn't mind other people on the machine executing the file. But they don't need to, so don't let them.) See the chmod(1) manpage for more details on the modes -- 1 bits mean executable, 2 bits means writable, and 4 bits means readable -- thus, 5 is executable and readable.
Then, when you want to run the script, run it like this:
cd /Minecraft/server_1
./start.bat
Note the ./ -- that means the shell should start the search for the executable program in the current working directory. (It could be ./bin/start.bat if your current working directory had a bin subdirectory with a start.bat executable file.)
Is start.bat executable? Make sure you have #!/bin/sh as the first line of the file. Also the directory is probably not in in the path, so try this:
$ chmod 555 start.bat
$ ./start.bat