Create a SysV Init script for Prometheus node exporter in old linux system (RHL6) - service

I have a server machine that has RHL6 (Red Hat Linux 6) and is based on SysV initialization (does not have systemd package), and I want to make my prometheus node exporter collect metrics from this machine.
All I can find online is how to create a node exporter service with systemctl (systemd): basically you create a .service file under /etc/systemd/system and then write something like this in it:
[Unit]
Description=Node Exporter
After=network.target
[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter
[Install]
WantedBy=multi-user.target
And then you start the service, enable it at startup, etc with systemctl command like this
sudo systemctl start node_exporter
sudo systemctl status node_exporter
sudo systemctl enable node_exporter
But the problem is that I don't have systemd installed and I don't have the right to update the server machine system so I am trying to find a way how to write an init script for node exporter to be placed under /etc/rd.d/init.d in my case.
It seems that all scripts under init.d are shell scripts that declare many methods like start(), stop(), restart(), reload(), force_reload(), ...
So it's not as easy as writing the service based on systemd.
Anyone have an idea how to do that with SysV init ???
Thanks,

I managed to found a solution for my problem.
Here is how the script looks like:
#!/bin/bash
#
# chkconfig: 2345 90 12
# description: node-exporter server
#
# Get function from functions library
. /etc/init.d/functions
# Start the service node-exporter
start() {
echo -n "Starting node-exporter service: "
/usr/sbin/node_exporter_service &
### Create the lock file ###
touch /var/lock/subsys/node-exporter
success $"node-exporter service startup"
echo
}
# Restart the service node-exporter
stop() {
echo -n "Shutting down node-exporter service: "
killproc node_exporter_service
### Now, delete the lock file ###
rm -f /var/lock/subsys/node-exporter
echo
}
### main logic ###
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status node_exporter_service
;;
restart|reload)
stop
start
;;
*)
echo $"Usage: $0 {start|stop|restart|reload|status}"
exit 1
esac
exit 0
We place the above script under /etc/init.d named "node-exporter" (without .sh) and we place the binary for the node exporter under /usr/sbin (with systemd we place binaries under /usr/local/bin).
You can download the binary file for node exporter from here https://github.com/prometheus/node_exporter/releases.
Then we add the script file to the list of services with command chkconfig --add node-exporter (to check if it already exists use command chkconfig --list node-exporter).
Enable the service with command chkconfig node-exporter on.
And then to start/stop/restart ... we use command /etc/init.d/node-exporter start/stop/restart ....
In the start script we basically run the binary file and in the stop script we kill the process by its name.
I hope this will be useful.

Related

systemd service redirect stdout to custom filename

I am working with systemd services in order to start an application. Stdout should be redirected to a file containing the current date (when the service was started). Logging to a file works fine, however, I don't know how to provide the date for the filename within the service. Any ideas?
...
[Service]
ExecStart=/bin/mybin
StandardOutput=file:/my/path/<filename should contain date>.log
...
systemd cannot generate the file name dynamically. But you can use bash redirection and date to create such a logfile.
[Service]
ExecStart=/bin/bash -c "/bin/mybin >/my/path/filename-$(date %%y-%%d-%%m).log"
I have the same needs and I have found a nice solution for me. It works well. Hope it can help you too.
1. Create an script.
You must put the script under /usr/bin. It's /usr/bin/ruoyi-gen.sh for me.
2. Add below contents.
#!/bin/bash
java -jar /root/xf-service/ruoyi-modules-gen-2.3.0.jar > /root/xf-service/ilogs/modules-gen-`date "+%Y-%m-%d"`.log 2>&1 &
Make the script executable -> chmod +x /usr/bin/ruoyi-gen.sh.
3. Add service description
Run vi /etc/systemd/system/ruoyi-gen.service, Add desciption like below:
[Unit]
Description=ruoyi-gen
[Service]
Type=forking
ExecStart=/usr/bin/ruoyi-gen.sh
[Install]
WantedBy=multi-user.target
4. Reload all systemd service files
systemctl daemon-reload
5.Start your service
systemctl start ruoyi-gen
It works on CentOS 7.

how to change systemd unit files directory?

Since I want to put all my service unit files in my own directory like /opt/myservice/, I found the way that use $SYSTEMD_UNIT_PATH in question https://unix.stackexchange.com/questions/224992/where-do-i-put-my-systemd-unit-file/367237#367237, however systemdctl can't find my service file in /opt/myservice/ after I setSYSTEMD_UNIT_PATH with shell command SYSTEMD_UNIT_PATH=/opt/myservice/, anyone knows how can it work? thx
[root#localhost system]# ls /opt/myservice/
test.service
[root#localhost system]# export SYSTEMD_UNIT_PATH=/opt/myservice/
[root#localhost system]# echo $SYSTEMD_UNIT_PATH
/opt/myservice/
[root#localhost system]# systemctl daemon-reload
[root#localhost system]# systemctl status test.service
Unit test.service could not be found.
From the document, the environment variable must be set as the kernel environement.
So, if the pid of systemd is 1, the environment must be set as kernel option, any others are invalid (e.g. /etc/profile, systemctl set-envionrment)
If you use grub-driven system, you can set it in /etc/default/grub, change the line starts with GRUB_CMDLINE_LINUX= and append SYSTEMD_UNIT_PATH=/absolute/path/to/your/services:. The last colon is required if you just want to append your services path.

Strace daemon process started using service

Strace can be used for tracing process by passing command for the process as below
strace -f -tt -o strace.log -D <SOME_COMMAND>
But below command fails to trace the syscalls of started daemon process
strace -f -tt -o strace.log -D service nginx start
In this case the strace just traces syscall for /usr/sbin/service and terminates. It does not trace syscalls on nginx process which is started as result of service nginx start
How do a I trace the process started by /usr/sbin/service? Specifically looking for solution with daemon process only!
Instead of running the nginx from service. Run service nginx stop and then run
strace nginx -g "daemon off;"
this will make sure that you get the trace of the process. The -g "daemon off;" will make sure the nginx is not run as a daemon process, else again the strace would end
Service command is just activating a process and if you want to strace it the best is to launch the process directly.
In case you are still interested in debugging the process started using the service command. Then do below
service nginx start
ps aux | grep nginx
Capture the pid from the nginx process and then attach to it using
strace -p <pid>
Forking Processes
To trace processes which fork, you need to use the -f flag
strace -f nginx
Service tracing
When you call service start nginx, assuming the system uses systemd, the call gets translated to systemctl start nginx. Now if you look at the source code of systemd
https://github.com/systemd/systemd/blob/cf45dd36282368d5cdf757cac2cf1fc2b562dab2/src/systemctl/systemctl.c#L3100
r = sd_bus_call_method_async(
bus,
NULL,
"org.freedesktop.systemd1",
"/org/freedesktop/systemd1",
"org.freedesktop.systemd1.Manager",
"Subscribe",
NULL, NULL,
NULL);
It doesn't spawn/fork the process. It sends the message to the systemd service which then starts nginx process.
So in short, NO you can't strace through your service nginx start command.
Change the ExecStart property of the service to include "strace". For example (tested on Debian Buster):
# grep ExecStart /lib/systemd/system/nginx.service
ExecStartPre=/usr/sbin/nginx -t -q -g 'daemon on; master_process on;'
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
# cd /etc/systemd/system/
# mkdir nginx.service.d
# cat > nginx.service.d/strace.conf <<-EOD
[Service]
ExecStart=
ExecStart=/usr/bin/strace -f -tt -o /tmp/strace.log -D /usr/sbin/nginx -g 'daemon on; master_process on;'
EOD
# systemctl daemon-reload
# systemctl restart nginx.service

Starting services at container startup

I'm trying to run 3 services at my container startup (snmpd, sshd and centengine)
As runlevel is unknown in the container, services won't start.
I built an image with this Dockerfile :
FROM centos:6.7
MAINTAINER nael <me#mail>
# Update CentOS
RUN yum -y update
# Install wget
RUN yum install -y wget
# Get Centreon Repo
RUN wget http://yum.centreon.com/standard/3.0/stable/ces-standard.repo -O /etc/yum.repos.d/ces-standard.repo
# Install Packages (SSH, sudo, Centreon Poller & Engine, SNMP)
RUN yum install -y --nogpgcheck openssh-clients openssh-server centreon-poller-centreon-engine sudo net-snmp net-snmp-utils
# Install supervisord
RUN rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
RUN yum --enablerepo=epel install -y supervisor
RUN mv -f /etc/supervisord.conf /etc/supervisord.conf.org
ADD supervisord.conf /etc/
# For sshd & centengine
EXPOSE 22 5669
# Change user password
RUN echo -e "password" | (passwd --stdin user)
# Disable PAM (causing issues while ssh login)
RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
RUN sed -ri 's/#UsePAM no/UsePAM no/g' /etc/ssh/sshd_config
# Start supervisord
CMD ["/usr/bin/supervisord"]
Here is the supervisord.conf file
[supervisord]
nodaemon=true
pidfile=/var/run/supervisord.pid
logfile=/var/log/supervisor/supervisord.log
[program:centengine]
command=service centengine start
[program:snmpd]
command=service snmpd start
[program:sshd]
command=service sshd start
But with this Dockerfile and supervisord.conf, when I start my container theses services aren't running.
What could be the problem ?
Not anymore using supervisord.
I just include a script with all the services ... start commands in the Dockerfile. When I create my container with docker run ... I just specify that I want to start it with my script.
& that's working very well.
Thanks #warmoverflow for trying to solve this.
You may find my dockerfy utility useful starting services, pre-running initialization commands before the primary command starts. See https://github.com/markriggins/dockerfy
For example:
RUN wget https://github.com/markriggins/dockerfy/releases/download/0.2.4/dockerfy-linux-amd64-0.2.4.tar.gz; \
tar -C /usr/local/bin -xvzf dockerfy-linux-amd64-*tar.gz; \
rm dockerfy-linux-amd64-*tar.gz;
ENTRYPOINT dockerfy
COMMAND --start bash -c "while false; do echo 'Ima Service'; sleep 1; done" -- \
--reap -- \
nginx
Would run a bash script as a service, echoing "Ima Service" every second, while the primary command nginx runs. If nginx exits, then the "Ima Service" script will automatically be stopped.
As an added benefit, any zombie processes left over by nginx will be automatically cleaned up.
You can also tail log files such as /var/log/nginx/error.log to stderr, edit nginx's configuration prior to startup and much more

How can I have a custom restart script for runit?

I'm using runit to manage an HAProxy and want to do a safe restart to reload a configuration file (specifically: haproxy -f /etc/haproxy/haproxy.cfg -sf $OLD_PROCESS_ID). I figure that I could run sv restart haproxy and tried to add a custom script named /etc/service/haproxy/restart, but it never seems to execute. How do I have a special restart script? Is my approach even good here? How do I reload my config with minimal impact using runit?
HAProxy runit service script
/etc/service/haproxy/run
#!/bin/sh
#
# runit haproxy
#
# forward stderr to stdout for use with runit svlogd
exec 2>&1
PID_PATH=/var/run/haproxy/haproxy.pid
BIN_PATH=/opt/haproxy/sbin/haproxy
CFG_PATH=/opt/haproxy/etc/haproxy.cfg
exec /bin/bash <<EOF
$BIN_PATH -f $CFG_PATH -D -p $PID_PATH
trap "echo SIGHUP caught; $BIN_PATH -f $CFG_PATH -D -p $PID_PATH -sf \\\$(cat $PID_PATH)" SIGHUP
trap "echo SIGTERM caught; kill -TERM \\\$(cat $PID_PATH) && exit 0" SIGTERM SIGINT
while true; do # Iterate to keep job running.
sleep 1 # Wake up to handle signals
done
EOF
Graceful reload that keeps things up and running.
sv reload haproxy
Full stop and start.
sv restart haproxy
This solution was inspired by https://gist.github.com/gfrey/8472007