i am using ubuntu 16 and systemd for running celery as a daemon.i have created the unit file also but i am not able to run celery as a service. why is this error?
/etc/systemd/system/celery.service
[Unit]
Description=Celery Service
After=network.target
[Service]
Type=forking
User=celery
Group=celery
EnvironmentFile=-/etc/default/celery
WorkingDirectory=/srv/weaver/src
ExecStart=/bin/sh -c '${CELERY_BIN} multi start ${CELERYD_NODES} \
-A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
--logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
ExecStop=/bin/sh -c '${CELERY_BIN} multi stopwait ${CELERYD_NODES} \
--pidfile=${CELERYD_PID_FILE}'
ExecReload=/bin/sh -c '${CELERY_BIN} multi restart ${CELERYD_NODES} \
-A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} \
--logfile=${CELERYD_LOG_FILE} --loglevel=${CELERYD_LOG_LEVEL} ${CELERYD_OPTS}'
[Install]
WantedBy=multi-user.target
file at /etc/default/celery
ENABLED="true"
CELERYD_NODES="worker1"
#CELERYD_NODES="worker1 worker2 worker3"
CELERY_BIN="/usr/local/bin/celery"
CELERY_APP="main:celery_app"
CELERYD_CHDIR="/srv/weaver/src"
CELERYD_OPTS=" --queue=weaver --time-limit=100000 --concurrency=2"
CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery2/%N.pid"
CELERYD_USER="celery"
CELERYD_GROUP="celery"
CELERY_CREATE_DIRS=1
# Change Celery Beat
CELERYBEAT_CHDIR="/srv/weaver/src"
# Log files
CELERYBEAT_LOG_FILE="/var/log/celery/celerybeat.log"
# Celery Beat Log files
CELERYBEAT_PID_FILE="/var/run/celery/celerybeat.pid"
# Scheduler for celery
CELERYBEAT_OPTS=" --pidfile=/var/run/celery/celerybeat.pid --sch
OUTPUT OF RUNNING SERVICE
● celery.service - Celery Service
Loaded: loaded (/etc/systemd/system/celery.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2017-01-12 17:12:32 IST; 2min 17s ago
Process: 18561 ExecStop=/bin/sh -c ${CELERY_BIN} multi stopwait ${CELERYD_NODES} --pidfile=${CELERYD_PID_FILE} (code=exited, status=0/SUCCESS)
Process: 18540 ExecStart=/bin/sh -c ${CELERY_BIN} multi start ${CELERYD_NODES} -A ${CELERY_APP} --pidfile=${CELERYD_PID_FILE} --logfile=${CELERYD_LOG_FIL
Main PID: 18555 (code=exited, status=1/FAILURE)
Jan 12 17:12:30 fb01 systemd[1]: Starting Celery Service...
Jan 12 17:12:31 fb01 sh[18540]: celery multi v3.1.23 (Cipater)
Jan 12 17:12:31 fb01 sh[18540]: > Starting nodes...
Jan 12 17:12:31 fb01 sh[18540]: > worker1#fb01: OK
Jan 12 17:12:31 fb01 systemd[1]: Started Celery Service.
Jan 12 17:12:32 fb01 systemd[1]: celery.service: Main process exited, code=exited, status=1/FAILURE
Jan 12 17:12:32 fb01 sh[18561]: celery multi v3.1.23 (Cipater)
Jan 12 17:12:32 fb01 sh[18561]: > worker1#fb01: DOWN
Jan 12 17:12:32 fb01 systemd[1]: celery.service: Unit entered failed state.
Jan 12 17:12:32 fb01 systemd[1]: celery.service: Failed with result 'exit-code'.
I just hit this exact issue. My problem was a configuration issue. In particular, I wasn't setting
CELERYD_LOG_LEVEL
in my environment file. (/etc/default/celeryd in your case). It looks like you have made the same mistake.
(I also had a few other configuration issues that I needed to resolve. I discovered these by running celery on the commandline.)
I had this same symptom, turned out to be permissions.
In my case something like:
chmod o+x /srv/weaver/src
sorted it.
Note: That this is not the best way to enable the required permission but that's not pertinent to this answer.
Related
Trying to start a service to run gunicorn as backend server for Flask, not working. Running nginx as frontend server for React, working.
Server:
Virtualization: vmware
Operating System: Red Hat Enterprise Linux 8.4 (Ootpa)
CPE OS Name: cpe:/o:redhat:enterprise_linux:8.4:GA
Kernel: Linux 4.18.0-305.3.1.el8_4.x86_64
Architecture: x86-64
Service file in /etc/systemd/system/myservice.service:
[Unit]
Description="Description"
After=network.target
[Service]
User=root
Group=root
WorkingDirectory=/home/project/app/api
ExecStart=/home/project/app/api/venv/bin/gunicorn -b 127.0.0.1:5000 api:app
Restart=always
[Install]
WantedBy=multi-user.target
/app/api:
-rwxr-xr-x. 1 root root 2018 Jun 9 20:06 api.py
drwxrwxr-x+ 5 root root 100 Jun 7 10:11 venv
Error message:
● myservice.service - "Description"
Loaded: loaded (/etc/systemd/system/myservice.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Thu 2021-06-10 19:01:01 CEST; 5s ago
Process: 18307 ExecStart=/home/project/app/api/venv/bin/gunicorn -b 127.0.0.1:5000 api:app (code=exited, status=203/EXEC)
Main PID: 18307 (code=exited, status=203/EXEC)
Jun 10 19:01:01 xxxx systemd[1]: myservice.service: Service RestartSec=100ms expired, scheduling restart.
Jun 10 19:01:01 xxxx systemd[1]: myservice.service: Scheduled restart job, restart counter is at 5.
Jun 10 19:01:01 xxxx systemd[1]: Stopped "Description".
Jun 10 19:01:01 xxxx systemd[1]: myservice.service: Start request repeated too quickly.
Jun 10 19:01:01 xxxx systemd[1]: myservice.service: Failed with result 'exit-code'.
Jun 10 19:01:01 xxxx systemd[1]: Failed to start "Description".
Tried, not working:
Adding Environment="PATH=/home/project/app/api/venv/bin" under [Service]
$ systemctl reset-failed myservice.service
$ systemctl daemon-reload
Reboot, ofc.
Tried, working:
Running (as root) /home/project/app/api/venv/bin/gunicorn -b 127.0.0.1:5000 api:app while in /app/api directory
Does anyone know how to fix this problem?
Typically enough, I figured it out shortly after posting this issue.
SELinux is messing with permissions for files and directories, so for anyone experiencing the same issue, make sure to test with the following alterings (as root):
$ setsebool -P httpd_can_network_connect on
$ chcon -Rt httpd_sys_content_t /path/to/your/Flask/dir
In my case: $ chcon -Rt httpd_sys_content_t /home/project/app/api
While this is NOT a permanent fix, it's worth a try. Check out the SELinux docs for more permanent solutions.
I have made this service file to start a python script when my raspberry pi (4) boots up:
/etc/systemd/system/plants.service
[Unit]
Description=plant-sender
After=network.target
[Service]
Type=simple
User=root
Group=root
WorkingDirectory=/home/theo/Repos/plants-monitor/remote
ExecStart=/usr/bin/python main.py
Restart=on-failure
[Install]
WantedBy=multi-user.target
However, once the pi is on, I run sudo systemctl status plants, and get:
* plants.service - plant-sender
Loaded: loaded (/etc/systemd/system/plants.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2020-03-30 20:22:43 EDT; 1min 45s ago
Process: 323 ExecStart=/usr/bin/python main.py (code=exited, status=1/FAILURE)
Main PID: 323 (code=exited, status=1/FAILURE)
Mar 30 20:22:43 arpi systemd[1]: plants.service: Scheduled restart job, restart counter is at 5.
Mar 30 20:22:43 arpi systemd[1]: Stopped plant-sender.
Mar 30 20:22:43 arpi systemd[1]: plants.service: Start request repeated too quickly.
Mar 30 20:22:43 arpi systemd[1]: plants.service: Failed with result 'exit-code'.
Mar 30 20:22:43 arpi systemd[1]: Failed to start plant-sender.
But, after running sudo systemctl restart plants, the service starts up and everything is fine.
If it doesn't start on boot but does on systemctl restart, I'd be looking at whether /home/theo/Repos/plants-monitor/remote is mounted at that point.
There may be something automounting or home-mounting your home directory when you log in.
If so, you could change the working directory to something that exists always, even if only a test.
Additionally, using journalctl -n 9999 -u plants will get you more log messages, so you can see why it's failing, rather than just seeing the "tried too many times, giving up" messages.
When I do journalctl -f after systemctl start celery.service I get
Mar 21 19:14:21 ubuntu-2gb-nyc3-01 systemd[1]: Reloading.
Mar 21 19:14:21 ubuntu-2gb-nyc3-01 systemd[1]: Started ACPI event daemon.
Mar 21 19:14:25 ubuntu-2gb-nyc3-01 systemd[21431]: celery.service: Failed at step USER spawning /home/mike/movingcollage/movingcollageenv/bin/celery: No such process
Mar 21 19:14:25 ubuntu-2gb-nyc3-01 systemd[1]: Starting celery service...
Mar 21 19:14:25 ubuntu-2gb-nyc3-01 systemd[1]: celery.service: Control process exited, code=exited status=217
Mar 21 19:14:25 ubuntu-2gb-nyc3-01 systemd[1]: Failed to start celery service.
Mar 21 19:14:25 ubuntu-2gb-nyc3-01 systemd[1]: celery.service: Unit entered failed state.
Mar 21 19:14:25 ubuntu-2gb-nyc3-01 systemd[1]: celery.service: Failed with result 'exit-code'.
This is my celery.service configuration:
[Unit]
Description=celery service
After=network.target
[Service]
#PIDFile=/run/celery/pid
Type=forking
User=celery
Group=celery
#RuntimeDirectory=celery
WorkingDirectory=/home/mike/movingcollage
ExecStart=/home/mike/movingcollage/movingcollageenv/bin/celery multi start 3 -A movingcollage "-c 5 -Q celery -l INFO"
ExecReload=/home/mike/movingcollage/movingcollageenv/bin/celery multi restart 3
ExecStop=/home/mike/movingcollage/movingcollageenv/bin/celery multi stopwait 3
[Install]
WantedBy=multi-user.target
Does anyone know what is wrong? Thanks in advance
For celery multi I think it is better to use Type=oneshot. Celery can start many workers processes and each will have its own PID.
I start my celery like this:
celery multi start 2\
-A my_app_name\
--uid=1001 --gid=1001\
-f /var/log/celery/celery.log\
--loglevel="INFO"\
--pidfile:1=/run/celery1.pid\
--pidfile:2=/run/celery2.pid
Of course in your case uid, gid and all paths will be different.
You need to change:
User=celery
Group=celery
to your user and group, in my case:
User=ubuntu
Group=ubuntu
I am trying to configure New Relic to work with Celery. I am working on a Django application hosted on Amazon EC2 w/ CentOS 7.
I thought all I needed to do to configure New Relic for celery was to edit the following line in /etc/systemd/system/celery.service
:
ExecStart=/home/myuser/project/venv/bin/celery -A project worker -l info -c 4
and change it to:
ExecStart=/home/myuser/project/newrelic.ini newrelic-admin run-program celery -A project worker -l info -c 4
But I see the following errors:
[root#ip-172-31-60-222 system]# systemctl daemon-reload
[root#ip-172-31-60-222 system]# systemctl restart celery
[root#ip-172-31-60-222 system]# systemctl status celery.service -l
● celery.service - datasidekick celery service
Loaded: loaded (/etc/systemd/system/celery.service; enabled; vendor preset: disabled)
Active: failed (Result: start-limit) since Wed 2017-03-01 04:16:33 UTC; 900ms ago
Process: 22969 ExecStart=/home/datasidekick/datasidekick/newrelic.ini newrelic-admin run-program /home/datasidekick/datasidekick/venv/bin/celery -A datasidekick worker -l info -c 4 (code=exited, status=203/EXEC)
Main PID: 22969 (code=exited, status=203/EXEC)
Mar 01 04:16:33 ip-172-31-60-222.ec2.internal systemd[1]: Unit celery.service entered failed state.
Mar 01 04:16:33 ip-172-31-60-222.ec2.internal systemd[1]: celery.service failed.
Mar 01 04:16:33 ip-172-31-60-222.ec2.internal systemd[1]: celery.service holdoff time over, scheduling restart.
Mar 01 04:16:33 ip-172-31-60-222.ec2.internal systemd[1]: start request repeated too quickly for celery.service
Mar 01 04:16:33 ip-172-31-60-222.ec2.internal systemd[1]: Failed to start datasidekick celery service.
Mar 01 04:16:33 ip-172-31-60-222.ec2.internal systemd[1]: Unit celery.service entered failed state.
Mar 01 04:16:33 ip-172-31-60-222.ec2.internal systemd[1]: celery.service failed.
I'm not sure what I'm doing wrong. Any help is greatly appreciated!
It's been a while since you asked the question but the reason of the fail is probably the wrong ExecStart command.
You use /home/myuser/project/newrelic.ini newrelic-admin run-program celery -A project worker -l info -c 4 as the command to start the service. The first part of the command will try to execute /home/myuser/project/newrelic.ini, which is a text file, and fail with Permission denied error as text files don't have exec permission by default. Or it will cause a bash syntax error somewhere and fail as well.
Insead, use
Environment="NEW_RELIC_CONFIG_FILE=/home/myuser/project/newrelic.ini"
ExecStart=newrelic-admin run-program celery -A project worker -l info -c 4
run journalctl -u celery -fand paste logs here
also have look at this links
https://docs.newrelic.com/docs/agents/python-agent/back-end-services/python-agent-celery
https://discuss.newrelic.com/t/need-some-help-setting-up-new-relic-with-our-celery-workers/30181
I created a service script named "puma.service" in /etc/systemd/system/ with the following contents:
[Unit]
Description=Puma HTTP Server
After=network.target
[Service]
Type=simple
User=ubuntu
WorkingDirectory=/home/username/appdir/current
ExecStart=/bin/bash -lc "/home/username/appdir/current/sbin/puma -C /home/username/appdir/current/config/puma.rb /home/username/appdir/current/config.ru"
Restart=always
[Install]
WantedBy=multi-user.target
I enabled the service and when started, I'm getting the following log from systemctl:
● puma.service - Puma HTTP Server
Loaded: loaded (/etc/systemd/system/puma.service; enabled; vendor preset: enabled)
Active: inactive (dead) (Result: exit-code) since Wed 2016-12-14 10:09:46 UTC; 12min ago
Process: 16889 ExecStart=/bin/bash -lc cd /home/username/appdir/current && bundle exec puma -C /home/username/appdir..
Main PID: 16889 (code=exited, status=127)
Dec 14 10:09:46 ip-172-31-29-40 systemd[1]: puma.service: Main process exited, code=exited, status=127/n/a
Dec 14 10:09:46 ip-172-31-29-40 systemd[1]: puma.service: Unit entered failed state.
Dec 14 10:09:46 ip-172-31-29-40 systemd[1]: puma.service: Failed with result 'exit-code'.
Dec 14 10:09:46 ip-172-31-29-40 systemd[1]: puma.service: Service hold-off time over, scheduling restart.
Dec 14 10:09:46 ip-172-31-29-40 systemd[1]: Stopped Puma HTTP Server.
Dec 14 10:09:46 ip-172-31-29-40 systemd[1]: puma.service: Start request repeated too quickly.
Dec 14 10:09:46 ip-172-31-29-40 systemd[1]: Failed to start Puma HTTP Server.
Although, when I give the command in SSH terminal the server started and was running perfect. Is there any changes I have to make in the service file?
Note:
I have changed the dirnames for your convenience.
I did some research and the cause of status 127 is due to the executable not in the path. But, I guess that won't be a problem.
Can you shed some light?
I found the problem and changed the ExecStart as mentioned below and it worked like a charm:
ExecStart=/home/username/.rbenv/shims/bundle exec puma -e production -C ./config/puma.rb config.ru
PIDFile=/home/username/appdir/shared/tmp/pids/puma.pid
bundle should be taken from the rbenv shims and also the puma's config file (config/puma.rb) and application's config file (config.ru) can be given in relative path.
One way to solve it is to specify a PID file and systemd will take a look at that file to check on service status.
Here's how we use this in our scripts (adapted to your given sample)
ExecStart=/bin/bash -lc '/home/username/appdir/current/sbin/puma -C /home/username/appdir/current/config/puma.rb /home/username/appdir/current/config.ru --pidfile /home/username/appdir/current/tmp/pids/puma.pid'
PIDFile=/home/username/appdir/current/tmp/pids/puma.pid
Take note that you might have to configure --pidfile via your -C puma.rb file instead of passing it an as parameter. I'm just showing it here to illustrate that --pidfile (in puma config) should be the same as PIDFile in the service file.
As for why the error message is that way, I'm not sure myself and is interested in the answer too.
For rvm users try with
[Unit]
Description=Puma HTTP Server
After=network.target
[Service]
Type=simple
User=user_name
Group=user_name
WorkingDirectory=/home/user_name/apps/app_name/current
Environment=RAILS_ENV=production
ExecStart=/home/user_name/.rvm/bin/rvm ruby-3.1.2#app_name do bundle exec --keep-file-descriptors puma -C /home/user_name/apps/app_name/current/config/puma/production.rb
ExecReload=/bin/kill -USR1 $MAINPID
StandardOutput=append:/home/user_name/apps/app_name/current/log/puma_access.log
StandardError=append:/home/user_name/apps/app_name/current/log/puma_error.log
SyslogIdentifier=app_name-puma
Restart=always
KillMode=process
[Install]
WantedBy=multi-user.target
if you are not using a gemset change
ExecStart=/home/user_name/.rvm/bin/rvm ruby-3.1.2#app_name ....
to
ExecStart=/home/user_name/.rvm/bin/rvm default