Aem Instance is not stating up - aem

Aem Author instance is not coming up after the automatic backup process. We have automatic backup process scheduled daily at 10:00 Pm PST. We are running a Shell script(backuprepo.sh) which in-turn triggers the stop.sh to bring the server down, do the backup process and then trigger the start.sh script to start the instance again.
The instance is brought down, but sometimes it is not started with the start script and has to be started manually. We have noticed that the below crx java process is not killed when the server is stopped though the stop script.
adobeam6 8454210 1 2 22:27:57 - 187:34 java -server
-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xmx4096m -Doak.queryLimitInMemory=500000 -Doak.queryLimitReads=100000 -Dupdate.limit=250000 -Doak.fastQuerySize=true -XX:MaxPermSize=512M -Djava.awt.headless=true -javaagent:/app/AEM6/author/crx-quickstart/app/newrelic/newrelic.jar -Djava.io.tmpdir=/app/AEM6/author/tmp -Duser.timezone=America/Los_Angeles -Dsling.run.modes=author,sit -jar crx-quickstart/app/cq-quickstart-6.0.0-standalone.jar start -c
crx-quickstart -i launchpad -p 4502
-Dsling.properties=conf/sling.properties
So we have to manually kill the process and trigger the start script to start the instance. Trying to figure out why the server is not stopped properly sometimes and why the above mentioned process is not killed even when the server is stopped through stop script.
We have verified the error logs which only stated the Instance has stopped as last Info.
Below are the steps executed by the backuprepo.sh script:
1)Trigger the stop.sh script
2)Wait for the server to stop automatically and backup process to complete.
3)After the backupprocess is complete, check if the server is started automatically.
4)Sometimes the server is not started automatically which is the issue.
Attached the start.sh and stop.sh scripts.
https://drive.google.com/open?id=0B67j43IdLr8mLXVCWTYxd1NialE
https://drive.google.com/open?id=0B67j43IdLr8mbDNPWndRdE5SWEE
Question:
Do we have any scenarios when we stop the AEM server, the crx java background process would still be running?
Any clues for our case which is preventing the java process to be release when the stop script is triggered?

Related

How can I launch postgres server headless (without terminal) on Windows?

Using Postgres 9.5 and the libpqxx c++ bindings, I want to launch a copy of postgres that is not installed on the users machine, but is instead packaged up in my application directory.
Currently, I am using pg_ctl.exe to start and stop the server, however when we do this, pg_ctl.exe seems to launch postgres.exe in a new terminal window.
I want it to launch postgres.exe in a headless state, but can't work out how.
I have tried enabling/disabling the logging collector, setting the logging method to a csv file (instead of stdout/stderr), and a couple of other logging related things, but I don't think the issue is the logging.
I have also tried running postgres.exe manually (without pg_ctl) and can get that to run headless by spawning it as a background process and redirecting the logs, but I would prefer to use the "pg_ctl start" api for the "wait for startup" (-w), and "timeout" (-t) options that it provides.
I believe you won't be able to do that with pg_ctl.
It is perfectly fine to start PostgreSQL directly through the server executable postgres.exe. Alternatively, you can use pg_ctl register to create a service and start the service.
In my use case, I was able to resolve the issue by running pg_ctl.exe using
CreateProcess, and providing the dwCreationFlags CREATE_NEW_PROCESS_GROUP | CREATE_NO_WINDOW.
I was originally using CREATE_NEW_PROCESS_GROUP | DETACHED_PROCESS, but DETACHED_PROCESS still allowed a postgres terminal to appear. This is because DETACHED_PROCESS will spawn the pg_ctl without a console, but any process that inherits stdin/stdout from pg_ctl will try to use it's console, and since there isn't one, one will be spawned. CREATE_NO_WINDOW however will launch the process with a conhost.exe, however the console will have no window. When the executables spawned by pg_ctl try to write to the terminal, they will successfully write to the console created by the conhost.exe which has no window.
I am now able to run pg_ctl from code with no console appearing.

Deploy jar on a remote server via Jenkins

I have an akka-scala app that I'm able to build as a jar. I can then send the jar to a remote server via Jenkins. However, I don't know how to properly deploy the app.
The first problem is killing previous instances of my app. If in Jenkins I try to execute such commands as pkill -f %proc% or ps ax | grep ... | awk ... | xargs kill -9 via ssh, Jenkins finishes with return code -1 even if I add || true to the end of these commands.
The second problem is starting the app. I'm able to successfully run the jar with nohup java -jar ... & command, but it doesn't really start though I'm able to see it in the process list.
None of these problems occur if I execute commands manually (even via ssh).
My question is - what am I doing wrong? What is the proper way to perform such task? Am I using the wrong tool?
Consider using something like supervisord for starting/stoping your app and also restarting it in case of crash or server reboot.

Jboss EAP 6 CLI Script with commands that require reload

I'm working on a cli script for JBoss EAP 6 (just a bunch of commands in a .bat file).
Now I'm running into issues with commands that rely on each other, where the former sets the server to a "reload-required" state.
For example:
first command: remove default-datasource ExampleDS. Outcome: success, server in state "reload required"
second command: remove h2-driver (required by default datasource). Outcome: failed, since server has not been restarted yet.
I've tried to place a reload command in between, but it seems as if the server is not completely up when the second command gets executed. Outcome still is failed, although if I try it a few seconds later, it works.
/subsystem=datasources/data-source=ExampleDS:remove
reload
/subsystem=datasources/jdbc-driver=h2:remove
Any suggestions how to make the CLI wait until JBoss is completely up again?
Try batching those commands together and doing a reload after you run the batch. For example:
batch
/subsystem=datasources/data-source=ExampleDS:remove
/subsystem=datasources/jdbc-driver=h2:remove
:reload
run-batch
Another option is to run the server to start the server in admin-only mode while configuring it. You should need the reload command in that case, but if you do make sure you use :reload(admin-only=true) until you're done configuring the server.
%JBOSS_HOME%\bin\standalone.bat --admin-only
rem Wait until server is started, then execute the CLI commands
%JBOSS_HOME%\bin\jboss-cli.bat -c --commands="/subsystem=datasources/data-source=ExampleDS:remove,/subsystem=datasources/jdbc-driver=h2:remove"

Can't make unicorn server start on CentOS boot

I have an unicorn init script in /etc/init.d
I have added description to make it centos-compatible.
It is configured in chkconfig:
unicorn 0:off 1:off 2:on 3:on 4:on 5:on 6:off
It starts normally if I run /etc/init.d/unicorn start
But when i reboot my system, it doesn't work.
Any ideas?
Look for the unicorn log file to see what the problem is. It is possible that not all of the services that unicorn requires have started by the time unicorn is started, which would explain why unicorn starts when started manually, but not when started automatically. If that is the problem, it should be reflected in the unicorn error log.
Also, check the runlevel with the runlevel command (or who -r) to make sure your system is at the runlevel you think it is.

How to make a server daemon which re-runs automatically when they're terminated unexpectedly?

I'm trying to running OrientDB on Ubuntu. Currently, I'm running with bin/server.sh. This works fine except it runs foreground on shell. I can make it work background by Ctrl+Z and bg command, but this doesn't mean it's running as daemon.
I wish the program will keep running after I logout. And will be started again when it terminated unexpectedly or OS restarts. Like MS Windows Services. But the problem is I don't know how can I do this.
How can I run a program as a long-running service?
If you do not own the server, look into using the "screen" command. It will allow you to run a command, detach from the console where the command is running, then log out while leaving it running. You may reconnect to the running screen to see output or restart the script. Here's more info about the screen command:
http://www.manpagez.com/man/1/screen/
If you own the server, you should write an init script. It's not very hard, and you can set it up to run automatically on startup. The system will run the script with a "start" parameter when you want it to start, and a "stop" parameter when you want it to stop. Here's more detailed information:
http://www.novell.com/coolsolutions/feature/15380.html
If the command doesn't already detach from the console (run in daemon mode) then in the init script place the command in parenthesis to run in it's own shell. You will not see any output unless you pipe it to a file within the parenthesis.
(bin/server.sh >> /var/log/server.log)