When run
etcd -peer-addr 127.0.0.1:7001 -addr 127.0.0.1:4001 -data-dir machines/machine1 -name machine1 &
I must to press enter in order to get back the console and etcd run in background
ANSIBLE
How can I recreate this purpose with ansible?
I want start etcd with ansible in this way
- name: run etcd like leader
shell: "etcd -peer-addr 127.0.0.1:7001 -addr 127.0.0.1:4001 -data-dir machines/machine1 -name machine2 &"
ansible keep holding the return of command and no continue for next task.
maybe a solution is running etcd in background. I don't know how
using
ansible 1.6.6
etcd 0.4.6
Check the docs on async mode. You can set async to some really long value, say 10 years in seconds (315569260), and set poll to 0.
This is a bit of a hack though. The better approach would be a systemd or init script that runs the process as a properly as a daemon.
Related
I am using Postgres with repmgr, one of the small problems I am having is that sometimes repmgr will have to stop and start the Postgres service and that will just kill the container, I tried some of the solutions online in the Dokcerfile but none seems to work, is there something I can add in the docker-compose file to prevent docker from exiting immediately, I don't want to stay alive forever, but maybe couple minutes?
Remember that docker-composer is mostly development thing. For production there are other ways, like kubernetes.
The only solution i know is to run our own .sh script as main process, which would have infinite loop with necessary checks in it.
This way you can control how to check - like ps aux and grep what you need. and exit main process if you need to by doing logic.
sh scrip would look something like:
while sleep 180; do
ps aux |grep postgres_service_name |grep -v grep
POSTGRESS=$?
if [ $POSTGRESS -ne 0 ]; then
# do what you need before exiting whole container
exit 1
fi
done
make sure you replace postgres_service_name with real name of Postgres service on linux.
Use that script as a startup script in docker compose or whatever you would use in prod.
if you really need 2 minutes before it is off - i would implement logic which would measure time after first time process is not there
The way docker is designed it will start a new container by starting the command specified as entrypoint/command to it and when this process terminates docker will kill all remaining processes in that container and shut it down.
So to keep the container running while the Postgres process is restarted you need to have another command running as the root process in the container.
You can achieve this by writing a simple shell script as a wrapper which will only exit when no Postgres process is running anymore or by using a dedicated init tool such as supervisord.
In Kubernetes, when a Pod repeatedly crashes and is in CrashLoopBackOff status, it is not possible to shell into the container and poke around to find the problem, due to the fact that containers (unlike VMs) live only as long as the primary process. If I shell into a container and the Pod is restarted, I'm kicked out of the shell.
How can I keep a Pod from crashing so that I can investigate if my primary process is failing to boot properly?
Redefine the command
In development only, a temporary hack to keep a Kubernetes pod from crashing is to redefine it and specify the container's command (corresponding to a Docker ENTRYPOINT) and args to be a command that will not crash. For instance:
containers:
- name: something
image: some-image
# `shell -c` evaluates a string as shell input
command: [ "sh", "-c"]
# loop forever, outputting "yo" every 5 seconds
args: ["while true; do echo 'yo' && sleep 5; done;"]
This allows the container to run and gives you a chance to shell into it, like kubectl exec -it pod/some-pod -- sh, and investigate what may be wrong.
This needs to be undone after debugging so that the container will run the command it's actually meant to run.
Adapted from this blog post.
There are also other methods used for debugging pods that are worth noting in your use case scenario:
If your container has previously crashed, you can access the previous container's crash log with: kubectl logs --previous ${POD_NAME} ${CONTAINER_NAME}
Debugging with an ephemeral debug container: Ephemeral containers are useful for interactive troubleshooting when kubectl exec is insufficient because a container has crashed or a container image doesn't include debugging utilities, such as with distroless images. kubectl has an alpha command that can create ephemeral containers for debugging beginning with version v1.18. An example for this method can be found here.
in my case I did build using mac m1/silicon. In this case pod crashes and there is no explicit message about this.
The problem was that I a also debugged using docker on the same m1 so could not really see what is wrong.
I would need to build image using docker build --platform linux/amd64.
I've got a elasticbeanstalk environment that needs to run a powershell script and restart before the application is deployed. According to the documentation this is supported as per the documentation
If the system requires a reboot after the command completes, the system reboots after the specified number of seconds elapses. If the system reboots as a result of a command, Elastic Beanstalk will recover to the point after the command in the configuration file. The default value is 60 seconds. You can also specify forever, but the system must reboot before you can run another command.
However when I add a reboot command to a ebextensions .config file I get the following exception from elasticbeanstalk
Error occurred during build: [Errno 4] Interrupted function call
The logs on the server after it has rebooted show that the command was executed so I assume the error is caused by a restart during the app deploy stage.
If I remove the restart command, deploy, wait for it to be ready then trigger a restart manually it works fine. But this is obviously not acceptable.
I've looked into the deployment hooks file system approach but that doesn't work either, and seems unessesary given it sounds like it should support this requirement out of the box.
Does anybody have any ideas?
We've had the same issue. We needed to disable SSL and TLS < 1.2, which requires registry changes and a reboot. Our workaround is to do the reboot in the container_commands section with a wait of forever. This seems to properly reboot and then trigger success in the deployment. However, it never actually does any of the steps after the reboot, which includes the built-in deployment of the code from the staging location to the actual final file destination (inetpub/wwwroot most likely). To get around this, have a step just before the reboot to copy the files from the local staging directory to the web root yourself.
We also needed to set a registry value and reboot. Our solution was to put the script in the command section and set waitAfterCompletion to foreve. There is a restart-computer --Force in our powershell script to cause the reboot.
disable_secure_time_seeding:
command: powershell.exe -ExecutionPolicy Bypass -File "C:\\scripts\\DisableSecureTimeSeeding.ps1" #This will cause a reboot
waitAfterCompletion: forever
I have an ansible playbook which deploys a jboss eap instance alongside some other things. The playbook runs fine until it gets to the point to start jboss using the provided standalone.sh script. I am using the shell module to start this script and it works fine in that jboss starts however when the task is executed ansible does not return any status message like a changed or OK and just seems to hang.
Is there a way I can force ansible to see this as something which has changed the system state ?
I don't personally use jboss, but this sounds to me like the startup.sh script simply isn't launching jboss in the background, so ansible is simply waiting for it to end and it never does.
There are a few potential ways you can address this. Given the information in this thread you might want to try a task like this:
- name: start jboss
shell: nohup standalone.sh > /dev/null
async: True
poll: 0
The poll: 0 tells Ansible to 'fire and forget' the command. If you don't include this then Ansible will kill the process when it returns from the remote server.
Another possibility is to use an init script. The thread I linked to above points to a location where you should be able to find an init script. You could install that (and leave it disabled if you don't want jboss to start up when the system reboots), and then simply run it via the service command from Ansible:
- name: start jboss
service: name=jboss state=started
Edit: If you're unwilling or unable to use nohup or an init script that puts jboss into the background then another alternative is to make use of screen if you have that installed and available to you. I regularly do this when invoking long-running commands from Ansible so that I can check back well after the playbook has been run to check on the status of the command:
- name: Invoke long running command
command: /usr/bin/screen -d -m /path/to/my/command
async: True
poll: 0
This will launch a detached screen session and invoke your command inside it. Ansible will promptly forget about it, leaving the command to run on its own.
I've completed this tutorial and successfully deployed a 3 node replica set. I can connect to it from other hosts and all is good. The question I have is that in the tutorial it states
Start MongoDB
Once the configuration files have been edited, start the database process manual:mongod on each instance by:
Log on onto the instance
Run the following command to start the process:
mongod --config /etc/mongod.conf
This should start the manual:mongod process
To me this seems as though the replica set is running as a user process and not as a system service as in the command
sudo service mongodb start
So what happens if one of the machines reboots? Is that process dead? How can I configure the whole replica set to run as a service?
On machine reboots, the mongod process will stop and you have to restart it.
In system scripts, I am not sure if on box restarts, mongod restart is automatically taken care of or not. But you can have service scripts for mongod process, which you get automatically, when you install using mongodb apt-get/yum packages.