upstart script. shell arithmetic in script stanza producing incorrect values. equivalent /bin/sh script works - sh

I have an upstart init script, but my dev/testing/production have different numbers of cpus/cores. I'd like to compute the number of worker processes to be 4 * number of cores within the init script
The upstart docs say that the script stanzas use /bin/sh syntax.
I created /bin/sh script to see what was going on. I'm getting drastically different results than my upstart script.
script stanza from my upstart script:
script
# get the number of cores
CORES=`lscpu | grep -v '#' | wc -l`
# set the number of worker processes to 4 * num cores
WORKERS=$(($CORES * 4))
echo exec gunicorn -b localhost:8000 --workers $WORKERS tutalk_site.wsgi > tmp/gunicorn.txt
end script
which outputs:
exec gunicorn -b localhost:8000 --workers 76 tutalk_site.wsgi
my equivalent /bin/sh script
#!/bin/sh
CORES=`lscpu -p | grep -v '#' | wc -l`
WORKERS=$(($CORES * 4))
echo exec gunicorn -b localhost:8000 --workers $WORKERS tutalk_site.wsgi
which outputs:
exec gunicorn -b localhost:8000 --workers 8 tutalk_site.wsgi
I'm hoping this is a rather simple problem and a few other pairs of eyes will locate the issue.
Any help would be appreciated.

I suppose I should have answered this several days ago. I first attempted using environment variables instead but didn't have any luck.
I solved the issue by replacing the computation with a python one-liner
WORKERS=$(python -c "import os; print os.sysconf('SC_NPROCESSORS_ONLN') * 2")
and that worked out just fine.
still curious why my bourne-shell script came up with the correct value while the upstart script, whose docs say use bourne-shell syntax didn't

Related

Perl Returning error for ls -l command

I am running a perl script from Nagios to check some files for certain characteristics on a windows machine. When I run the script from Nagios it responds with a result of:
UNKNOWN ERROR - execution of LANG=C ls -l resulted in an error 32512 -
My Code is from this GitHub with a single modification of line 168 so I can use it with windows:
use lib 'C$\Progra~1\Nagios\NRDS_Win\plugins';
The odd thing is the program actually outputs the expected result from the command line on the windows machine.
Here is the command:
check_files.pl -D c:\logs -F Health.log -a '~,300'
Here is an example:
CRITICAL - Health.log is 10703 (more than 300) seconds old - 1
Health.log files found
I modified line that defined LANG=C ls -l in the code but now i just get:
UNKNOWN ERROR - could not execute ls -l - No such file or directory
ls is unix command and by default there is no such command in windows.
If you need it, you can install it e.g. from GNU CoreUtils
You also need to change shell command on line 639 from LANG=C ls -l to just ls -l because in windows you can't set environment variables like that.

Start shrew vpn client (iked & ikec) on start-up of OSMC on Raspberry 2

I would like to connect to a VPN on start-up of OSMC.
Environment:
installed OSMC on Raspberry 2
downloaded, compiled and installed shrew soft vpn on the device
As user 'osmc' with ssh
> sudo iked starts the daemon successfully
> ikec -r "test.vpn" -a starts the client, loads the config and connects successfully
rc.local:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
sudo iked >> /home/osmc/iked.log 2>> /home/osmc/iked.error.log &
ikec -a -r "test.vpn" >> /home/osmc/ikec.log 2>> /home/osmc/ikec.error.log &
exit 0
after start of raspberry iked is as process visible with ps -e
but ikec is not running
osmc#osmc:~$ /etc/rc.local starts the script and connects to vpn successfully
Problem:
Why does the script not working correctly on start-up?
Thank you for your help!
I was also looking to do the same thing as you and ran into the same problem. I'm no linux expert, but I did figure out a workaround.
I created a script called ikec_after_reboot.sh and it looks like this...
$ cat ikec_after_reboot.sh
#!/bin/bash
echo "Starting ikec"
ikec -r test.vpn -a
I then installed cron.
sudo apt-get update
sudo apt-get install cron
Edit the cron job as root and run the ikec script 60 seconds after reboot.
sudo crontab -e
SHELL=/bin/bash
#reboot sleep 60 && /home/osmc/ikec_after_reboot.sh & >> /home/osmc/ikec.log 2>&1
Now edit your /etc/rc.local file and add the following.
sudo iked >> //home/osmc/iked.log 2>> /home/osmc/iked.error.log &
exit 0
Hopefully, this is helpful to you.

howto: elastic beanstalk + deploy docker + graceful shutdown

Hi great people of stackoverflow,
Were hosting a docker container on EB with an nodejs based code running on it.
When redeploying our docker container we'd like the old one to do a graceful shutdown.
I've found help & guides on how our code could receive a sigterm signal produced by 'docker stop' command.
However further investigation into the EB machine running docker at:
/opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh
shows that when "flipping" from current to the new staged container, the old one is killed with 'docker kill'
Is there any way to change this behaviour to docker stop?
Or in general a recommended approach to handling graceful shutdown of the old container?
Thanks!
Self answering as I've found a solution that works for us:
tl;dr: use .ebextensions scripts to run your script before 01flip, your script will make sure a graceful shutdown of whatevers inside the docker takes place
first,
your app (or whatever your'e running in docker) has to be able to catch a signal, SIGINT for example, and shutdown gracefully upon it.
this is totally unrelated to Docker, you can test it running wherever (locally for example)
There is a lot of info about getting this kind of behaviour done for different kind of apps on the net (be it ruby, node.js etc...)
Second,
your EB/Docker based project can have a .ebextensions folder that holds all kinda of scripts to execute while deploying.
we put 2 custom scripts into it, gracefulshutdown_01.config and gracefulshutdown_02.config file that looks something like this:
# gracefulshutdown_01.config
commands:
backup-original-flip-hook:
command: cp -f /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh /opt/elasticbeanstalk/hooks/appdeploy/01flip.sh.bak
test: '[ ! -f /opt/elasticbeanstalk/hooks/appdeploy/01flip.sh.bak ]'
cleanup-custom-hooks:
command: rm -f 05gracefulshutdown.sh
cwd: /opt/elasticbeanstalk/hooks/appdeploy/enact
ignoreErrors: true
and:
# gracefulshutdown_02.config
commands:
reorder-original-flip-hook:
command: mv /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh /opt/elasticbeanstalk/hooks/appdeploy/enact/10flip.sh
test: '[ -f /opt/elasticbeanstalk/hooks/appdeploy/enact/01flip.sh ]'
files:
"/opt/elasticbeanstalk/hooks/appdeploy/enact/05gracefulshutdown.sh":
mode: "000755"
owner: root
group: root
content: |
#!/bin/sh
# find currently running docker
EB_CONFIG_DOCKER_CURRENT_APP_FILE=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_file)
EB_CONFIG_DOCKER_CURRENT_APP=""
if [ -f $EB_CONFIG_DOCKER_CURRENT_APP_FILE ]; then
EB_CONFIG_DOCKER_CURRENT_APP=`cat $EB_CONFIG_DOCKER_CURRENT_APP_FILE | cut -c 1-12`
echo "Graceful shutdown on app container: $EB_CONFIG_DOCKER_CURRENT_APP"
else
echo "NO CURRENT APP TO GRACEFUL SHUTDOWN FOUND"
exit 0
fi
# give graceful kill command to all running .js files (not stats!!)
docker exec $EB_CONFIG_DOCKER_CURRENT_APP sh -c "ps x -o pid,command | grep -E 'workers' | grep -v -E 'forever|grep' " | awk '{print $1}' | xargs docker exec $EB_CONFIG_DOCKER_CURRENT_APP kill -s SIGINT
echo "sent kill signals"
# wait (max 5 mins) until processes are done and terminate themselves
TRIES=100
until [ $TRIES -eq 0 ]; do
PIDS=`docker exec $EB_CONFIG_DOCKER_CURRENT_APP sh -c "ps x -o pid,command | grep -E 'workers' | grep -v -E 'forever|grep' " | awk '{print $1}' | cat`
echo TRIES $TRIES PIDS $PIDS
if [ -z "$PIDS" ]; then
echo "finished graceful shutdown of docker $EB_CONFIG_DOCKER_CURRENT_APP"
exit 0
else
let TRIES-=1
sleep 3
fi
done
echo "failed to graceful shutdown, please investigate manually"
exit 1
gracefulshutdown_01.config is a small util that backups the original flip01 and deletes (if exists) our custom script.
gracefulshutdown_02.config is where the magic happens.
it creates a 05gracefulshutdown enact script and makes sure flip will happen afterwards by renaming it to 10flip.
05gracefulshutdown, the custom script, does this basically:
find current running docker
find all processes that need to be sent a SIGINT (for us its processes with 'workers' in its name
send a sigint to the above processes
loop:
check if processes from before were killed
continue looping for an amount of tries
if tries are over, exit with status "1" and dont continue to 10flip, manual interference is needed.
this assumes you only have 1 docker running on the machine, and that you are able to manually hop on to check whats wrong in the case it fails (for us never happened yet).
I imagine it can also be improved in many ways, so have fun.

Upstart / init script not working

I'm trying to create a service / script to automatically start and controll my nodejs server, but it doesnt seem to work at all.
First of all, I used this source as main reference http://kvz.io/blog/2009/12/15/run-nodejs-as-a-service-on-ubuntu-karmic/
After testing around, I minimzed the content of the actual file to avoid any kind of error, resulting in this (the bare minimum, but it doesnt work)
description "server"
author "blah"
start on started mountall
stop on shutdown
respawn
respawn limit 99 5
script
export HOME="/var/www"
exec nodejs /var/www/server/server.js >> /var/log/node.log 2>&1
end script
The file is saved in /etc/init/server.conf
when trying to start the script (as root, or normal user), I get:
root#iof304:/etc/init# start server
start: Job failed to start
Then, I tried to check my syntax with init-checkconf, resulting in:
$ init-checkconf /etc/init/server.conf
File /etc/init/server.conf: syntax ok
I tried different other things, like initctl reload-configuration with no result.
What can I do? How can I get this to work? It can't be that hard, right?
This is what our typical startup script looks like. As you can see we're running our node processes as user nodejs. We're also using the pre-start script to make sure all of the log file directories and .tmp directories are created with the right permissions.
#!upstart
description "grabagadget node.js server"
author "Jeffrey Van Alstine"
start on started mysql
stop on shutdown
respawn
script
export HOME="/home/nodejs"
exec start-stop-daemon --start --chuid nodejs --make-pidfile --pidfile /var/run/nodejs/grabagadget.pid --startas /usr/bin/node -- /var/nodejs/grabagadget/app.js --environment production >> /var/log/nodejs/grabagadget.log 2>&1
end script
pre-start script
mkdir -p /var/log/nodejs
chown nodejs:root /var/log/nodejs
mkdir -p /var/run/nodejs
mkdir -p /var/nodejs/grabagadget/.tmp
# Git likes to reset permissions on this file, but it really needs to be writable on server start
chown nodejs:root /var/nodejs/grabagadget/views/layout.ejs
chown -R nodejs:root /var/nodejs/grabagadget/.tmp
# Date format same as (new Date()).toISOString() for consistency
sudo -u nodejs echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Starting" >> /var/log/nodejs/grabagadget.log
end script
pre-stop script
rm /var/run/nodejs/grabagadget.pid
sudo -u nodejs echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Stopping" >> /var/log/nodejs/grabgadget.log
end script
As of Ubuntu 15, upstart is no longer being used, see systemd.

perl run two system commands error

So in my script I need to make to calls to unix, and I do it via the system command like so:
system "bash -i -c 'addmothernode'";
...
perl code ...
...
system "bash -i -c 'addnode -ip=$_'";
However, whenever I run both of these commands in the same script, for some reason my process is stopped like this:
[1]+ Stopped perl boot.pl
And the script can only be finished when I run fg %1. When I only have one of these system calls in, the perl script finishes successfully. But I need both commands because they depend on each other. Anyone have any ideas about what's going on? Thanks!
UPDATE:
A lot of answers below are saying I don't need to use bash -i to run a system command, and I know typically this is true but I need to use aliases that I have created and if I do not use this the aliases won't be recognized. So I do need bash -i.
This problem is unrelated to perl. You can easily reproduce the situation if you start two bashes in the interactive mode (-i) one after another:
$ cat 1.sh
bash -i -c 'sleep 1'
bash -i -c 'sleep 1'
$ bash 1.sh
[1]+ Stopped bash 1.sh
Of course it would be better to run bash in the non-interactive mode (without -i) or run the program directly, without bash, but if you need for some reason bash -i you can protect its run with setsid:
$ cat 1.sh
setsid bash -i -c 'sleep 1'
setsid bash -i -c 'sleep 1'
echo done
$ bash 1.sh
done
The bash -i means run an interactive shell; so you have two shells both reading from the terminal.
Try removing the -i options.
system "addmothernode";
should work.
To execute a command, bash is not needed. The Perl system function is like the system C function, it calls by default sh.
man system
exec
The standard to which the caller conforms determines which shell is used. See standards(5).
Standard Shell Used
______________________________________________________________
1989 ANSI C, 1990 ISO C, 1999 ISO C, /usr/xpg4/bin/sh
POSIX.1 (1990-2001), SUS, SUSv2, SUSv3,
XPG4
POSIX.1 (1988), SVID3, XPG3, no standard /usr/bin/sh
specified