Blackbox exporter of prometheus is not coming up as service.
When I run command from ExecStart it just works fine.
Would like to know what's missing from below configuration.
pwd ; cat blackbox_exporter.service
/usr/lib/systemd/system
[Unit]
Description=Blackbox Exporter Service
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
Restart=always
User=root
Group=root
ExecStart=/opt/blackbox_exporter/blackbox_exporter --config.file=/etc/prometheus/blackbox.yml --web.listen-address="<hostNameGoesHere>:9115"
[Install]
WantedBy=multi-user.target
And all files are with root privilege only.
ls -l blackbox_exporter.service /opt/blackbox_exporter/blackbox_exporter /etc/prometheus/blackbox.yml
-rw-r--r-- 1 root root 343 Oct 4 06:55 blackbox_exporter.service
-rw-r--r-- 1 root root 775 May 3 05:46 /etc/prometheus/blackbox.yml
-rwxr-xr-x 1 root root 16074005 Feb 27 2018 /opt/blackbox_exporter/blackbox_exporter
Error message is as per below.
systemctl status blackbox_exporter.service
● blackbox_exporter.service - Blackbox Exporter Service
Loaded: loaded (/usr/lib/systemd/system/blackbox_exporter.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since <TIMESTAMP>; 2s ago
Process: 101522 ExecStart=/opt/blackbox_exporter/blackbox_exporter --config.file=/etc/prometheus/blackbox.yml --web.listen-address="<hostNameGoesHere>:9115" **(code=exited, status=1/FAILURE)
Main PID: 101522 (code=exited, status=1/FAILURE)**
systemd[1]: Unit blackbox_exporter.service entered failed state.
systemd[1]: blackbox_exporter.service failed.
systemd[1]: blackbox_exporter.service holdoff time over, scheduling restart.
systemd[1]: Stopped Blackbox Exporter Service.
systemd[1]: start request repeated too quickly for blackbox_exporter.service
systemd[1]: Failed to start Blackbox Exporter Service.
systemd[1]: Unit blackbox_exporter.service entered failed state.
systemd[1]: blackbox_exporter.service failed.
Removed double quotes : "" from ExecStart line & it just worked like a charm!!
ExecStart=/opt/blackbox_exporter/blackbox_exporter --config.file=/etc/prometheus/blackbox.yml --web.listen-address=<hostNameGoesHere>:9115
Related
Trying to start a service to run gunicorn as backend server for Flask, not working. Running nginx as frontend server for React, working.
Server:
Virtualization: vmware
Operating System: Red Hat Enterprise Linux 8.4 (Ootpa)
CPE OS Name: cpe:/o:redhat:enterprise_linux:8.4:GA
Kernel: Linux 4.18.0-305.3.1.el8_4.x86_64
Architecture: x86-64
Service file in /etc/systemd/system/myservice.service:
[Unit]
Description="Description"
After=network.target
[Service]
User=root
Group=root
WorkingDirectory=/home/project/app/api
ExecStart=/home/project/app/api/venv/bin/gunicorn -b 127.0.0.1:5000 api:app
Restart=always
[Install]
WantedBy=multi-user.target
/app/api:
-rwxr-xr-x. 1 root root 2018 Jun 9 20:06 api.py
drwxrwxr-x+ 5 root root 100 Jun 7 10:11 venv
Error message:
● myservice.service - "Description"
Loaded: loaded (/etc/systemd/system/myservice.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Thu 2021-06-10 19:01:01 CEST; 5s ago
Process: 18307 ExecStart=/home/project/app/api/venv/bin/gunicorn -b 127.0.0.1:5000 api:app (code=exited, status=203/EXEC)
Main PID: 18307 (code=exited, status=203/EXEC)
Jun 10 19:01:01 xxxx systemd[1]: myservice.service: Service RestartSec=100ms expired, scheduling restart.
Jun 10 19:01:01 xxxx systemd[1]: myservice.service: Scheduled restart job, restart counter is at 5.
Jun 10 19:01:01 xxxx systemd[1]: Stopped "Description".
Jun 10 19:01:01 xxxx systemd[1]: myservice.service: Start request repeated too quickly.
Jun 10 19:01:01 xxxx systemd[1]: myservice.service: Failed with result 'exit-code'.
Jun 10 19:01:01 xxxx systemd[1]: Failed to start "Description".
Tried, not working:
Adding Environment="PATH=/home/project/app/api/venv/bin" under [Service]
$ systemctl reset-failed myservice.service
$ systemctl daemon-reload
Reboot, ofc.
Tried, working:
Running (as root) /home/project/app/api/venv/bin/gunicorn -b 127.0.0.1:5000 api:app while in /app/api directory
Does anyone know how to fix this problem?
Typically enough, I figured it out shortly after posting this issue.
SELinux is messing with permissions for files and directories, so for anyone experiencing the same issue, make sure to test with the following alterings (as root):
$ setsebool -P httpd_can_network_connect on
$ chcon -Rt httpd_sys_content_t /path/to/your/Flask/dir
In my case: $ chcon -Rt httpd_sys_content_t /home/project/app/api
While this is NOT a permanent fix, it's worth a try. Check out the SELinux docs for more permanent solutions.
I have made this service file to start a python script when my raspberry pi (4) boots up:
/etc/systemd/system/plants.service
[Unit]
Description=plant-sender
After=network.target
[Service]
Type=simple
User=root
Group=root
WorkingDirectory=/home/theo/Repos/plants-monitor/remote
ExecStart=/usr/bin/python main.py
Restart=on-failure
[Install]
WantedBy=multi-user.target
However, once the pi is on, I run sudo systemctl status plants, and get:
* plants.service - plant-sender
Loaded: loaded (/etc/systemd/system/plants.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2020-03-30 20:22:43 EDT; 1min 45s ago
Process: 323 ExecStart=/usr/bin/python main.py (code=exited, status=1/FAILURE)
Main PID: 323 (code=exited, status=1/FAILURE)
Mar 30 20:22:43 arpi systemd[1]: plants.service: Scheduled restart job, restart counter is at 5.
Mar 30 20:22:43 arpi systemd[1]: Stopped plant-sender.
Mar 30 20:22:43 arpi systemd[1]: plants.service: Start request repeated too quickly.
Mar 30 20:22:43 arpi systemd[1]: plants.service: Failed with result 'exit-code'.
Mar 30 20:22:43 arpi systemd[1]: Failed to start plant-sender.
But, after running sudo systemctl restart plants, the service starts up and everything is fine.
If it doesn't start on boot but does on systemctl restart, I'd be looking at whether /home/theo/Repos/plants-monitor/remote is mounted at that point.
There may be something automounting or home-mounting your home directory when you log in.
If so, you could change the working directory to something that exists always, even if only a test.
Additionally, using journalctl -n 9999 -u plants will get you more log messages, so you can see why it's failing, rather than just seeing the "tried too many times, giving up" messages.
When trying to start zookeeper service I get the following
● zookeeper.service
Loaded: loaded (/etc/systemd/system/zookeeper.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Thu 2020-04-02 16:19:24 EDT; 5min ago
Process: 5201 ExecStop=/usr/local/kafka/kafka_2.13-2.4.1/bin/zookeeper-server-stop.sh (code=exited, status=1/FAILURE)
Process: 4882 ExecStart=/usr/local/kafka/kafka_2.13-2.4.1/bin/zookeeper-server-start.sh /usr/local/kafka/kafka_2.13-2.4.1/config/zookeeper.properties (code=exited, status=127)
Main PID: 4882 (code=exited, status=127)
Apr 02 16:19:24 centos.localdomain systemd[1]: Started zookeeper.service.
Apr 02 16:19:24 centos.localdomain systemd[1]: zookeeper.service: main process exited, code=exited, status=127/n/a
Apr 02 16:19:24 centos.localdomain systemd[1]: zookeeper.service: control process exited, code=exited status=1
Apr 02 16:19:24 centos.localdomain systemd[1]: Unit zookeeper.service entered failed state.
Apr 02 16:19:24 centos.localdomain systemd[1]: zookeeper.service failed.
The zookeeper.service file is configured as follows
[Unit]
Requires=network.target remote-fs.target
After=network.target remote-fs.target
[Service]
Type=simple
User=specadmin
ExecStart=/usr/local/kafka/kafka_2.13-2.4.1/bin/zookeeper-server-start.sh /usr/local/kafka/kafka_2.13-2.4.1/config/zookeeper.properties
ExecStop=/usr/local/kafka/kafka_2.13-2.4.1/bin/zookeeper-server-stop.sh
Restart=on-abnormal
[Install]
WantedBy=multi-user.target
When trying to run zookeeper manually with the same user configured in the service file everything works fine.
Please advise
Turns out the issue was related to the environment variables systemd uses.
Systemd uses a fixed $PATH variable and the changes that are made to the /etc/profile /etc/bashrc and the like are not applied to systemd.
Zookeeper runs java which needs to be part of the search path, but since systemd doesn't use the files where the search path is set, zookeeper start script couldn't find java.
I solved it by overriding the search path by adding Environment=PATH=... parameter in the zookeeper service file and adding all the required directories.
I have a trouble/problem with my Filebeat installation.
When I try it to start with "service filebeat start", it says "Starting Filebeat". After "service filebeat status" I get 4 PIDs (until here everything looks "normal"):
[root#(Server) run]# service filebeat status
Filebeat is running with pid: 30650 30657 30658 30659
But after checking the PID, we see that it is not running:
[root#(Server) run]# ps -ef | grep 30650
root 30665 31360 0 16:27 pts/0 00:00:00 grep --color=auto 30650
Trying to start it with systemctl doesn't help:
[root#(Server) run]# systemctl start filebeat
Job for filebeat.service failed because a configured resource limit was exceeded. See "systemctl status filebeat.service" and "journalctl -xe" for details.
Status says:
[root#Server run]# systemctl status filebeat
● filebeat.service - LSB: start and stop filebeat
Loaded: loaded (/etc/rc.d/init.d/filebeat; bad; vendor preset: disabled)
Active: failed (Result: resources) since Tue 2017-09-26 16:30:33 CEST; 1min 41s ago
Docs: man:systemd-sysv-generator(8)
Process: 32118 ExecStart=/etc/rc.d/init.d/filebeat start (code=exited, status=0/SUCCESS)
Sep 26 16:30:33 Server... systemd[1]: Starting LSB: start and stop filebeat...
Sep 26 16:30:33 Server... filebeat[32118]: Starting Filebeat
Sep 26 16:30:33 Server... su[32119]: (to user) root on none
Sep 26 16:30:33 Server... systemd[1]: PID file /var/run/filebeat.pid not readable (yet?) after start.
Sep 26 16:30:33 Server... systemd[1]: Failed to start LSB: start and stop filebeat.
Sep 26 16:30:33 Server... systemd[1]: Unit filebeat.service entered failed state.
Sep 26 16:30:33 Server... systemd[1]: filebeat.service failed.
Does somebody has any idea?
Regards
Problem was "chown permissions". I installed filebeat not as root and the "data" directory had root user & group ownership. After changing that, it runs and starts automatically after boot.
Regards
I have a RHEL7 server that is part of a Mongo cluster. There are three mongo processes that I would like to be automatically started on system boot. One mongod, one arbiter and one mongos:
/usr/bin/mongod -f /etc/mongo_shard001.conf
/usr/bin/mongod -f /etc/mongoarb.conf
/usr/bin/mongos -f /etc/mongos.conf
I have been trying to create systemd services for these commands i.e
[Unit]
Description=mongo configuration server
After=network.target
[Service]
User=mongod
Group=mongod
ExecStart=/usr/bin/mongod -f /etc/mongoconf.conf
[Install]
WantedBy=multi-user.target
When I try to do sudo systemctl daemon-reload && sudo systemctl start mongoconf, I get this error
● mongoconf.service - mongo configuration server
Loaded: loaded (/etc/systemd/system/mongoconf.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Thu 2017-02-02 14:38:34 AWST; 20s ago
Process: 5114 ExecStart=/usr/bin/mongod -f /etc/mongoconf.conf (code=exited, status=1/FAILURE)
Main PID: 5114 (code=exited, status=1/FAILURE)
Feb 02 14:38:34 mdb1 systemd[1]: Started mongo configuration server.
Feb 02 14:38:34 mdb1 systemd[1]: Starting mongo configuration server...
Feb 02 14:38:34 mdb1 systemd[1]: mongoconf.service: main process exited, code=exited, status=1/FAILURE
Feb 02 14:38:34 mdb1 systemd[1]: Unit mongoconf.service entered failed state.
Feb 02 14:38:34 mdb1 systemd[1]: mongoconf.service failed.
I have also tried using a forked type with pid file:
[Unit]
Description=mongo configuration server
After=network.target
[Service]
User=mongod
Group=mongod
ExecStart=/usr/bin/mongod -f /etc/mongoconf.conf --pidfilepath /var/lib/mongoconf/pid --fork
Type=forking
PIDFile=/var/run/mongodb/mongoconf/pid
[Install]
WantedBy=multi-user.target
But gives this error
● mongoconf.service - mongo configuration server
Loaded: loaded (/etc/systemd/system/mongoconf.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Thu 2017-02-02 14:45:36 AWST; 4s ago
Process: 5256 ExecStart=/usr/bin/mongod -f /etc/mongoconf.conf --pidfilepath /var/lib/mongoconf/pid --fork (code=exited, status=1/FAILURE)
Main PID: 5114 (code=exited, status=1/FAILURE)
Feb 02 14:45:36 mdb1 systemd[1]: Starting mongo configuration server...
Feb 02 14:45:36 mdb1 mongod[5256]: about to fork child process, waiting until server is ready for connections.
Feb 02 14:45:36 mdb1 mongod[5256]: forked process: 5258
Feb 02 14:45:36 mdb1 systemd[1]: mongoconf.service: control process exited, code=exited status=1
Feb 02 14:45:36 mdb1 systemd[1]: Failed to start mongo configuration server.
Feb 02 14:45:36 mdb1 systemd[1]: Unit mongoconf.service entered failed state.
Feb 02 14:45:36 mdb1 systemd[1]: mongoconf.service failed.
Starting the mongo config manually works fine and creates the pid file
/usr/bin/mongod -f /etc/mongoconf.conf --pidfilepath /var/lib/mongoconf/pid --fork
The version of mongod I am using is the one from mongodb.com, and I installed it following their install guide.
db version v3.4.1
git version: 5e103c4f5583e2566a45d740225dc250baacfbd7
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
allocator: tcmalloc
modules: none
build environment:
distmod: rhel70
distarch: x86_64
target_arch: x86_64
from this repo
[mongodb-org-3.4]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-3.4.asc
I am wondering if I am going about this the wrong way, is there a better way to do this?
I know you said rhel7 but since it's the only answer coming up on duckduckgo for this question, this can be useful. Under Ubuntu 15 and up:
sudo systemctl enable mongod.service
Here is my solution
make a bash script with these lines
/usr/bin/mongod -f /etc/mongo_shard001.conf
/usr/bin/mongod -f /etc/mongoarb.conf
/usr/bin/mongos -f /etc/mongos.conf
and then add this line to your crontab
#reboot root cd /foldername && ./scriptname.sh
systemd would be a better solution, if anyone knows how to set it up.
the mongo documentation is no help