I've been running celery on server for a while and today when I tried to restart celery by using
sudo systemctl restart celery
The process somehow failed. I did journalctl -xe and this is the error message that i got:
This error might have occured when I reboot my ubuntu server.
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: File "/home/baefloral/baefloral/fandeedv3/lib/python3.7/site-packages/celery/apps/beat.py", line 77, in run
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: self.start_scheduler()
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: File "/home/baefloral/baefloral/fandeedv3/lib/python3.7/site-packages/celery/apps/beat.py", line 88, in start_s
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: platforms.create_pidlock(self.pidfile)
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: File "/home/baefloral/baefloral/fandeedv3/lib/python3.7/site-packages/celery/platforms.py", line 260, in create
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: pidlock = _create_pidlock(pidfile)
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: File "/home/baefloral/baefloral/fandeedv3/lib/python3.7/site-packages/celery/platforms.py", line 270, in _creat
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: pidlock.acquire()
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: File "/home/baefloral/baefloral/fandeedv3/lib/python3.7/site-packages/celery/platforms.py", line 151, in acquir
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: reraise(LockFailed, LockFailed(str(exc)), sys.exc_info()[2])
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: File "/home/baefloral/baefloral/fandeedv3/lib/python3.7/site-packages/celery/exceptions.py", line 108, in rerai
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: raise value.with_traceback(tb)
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: File "/home/baefloral/baefloral/fandeedv3/lib/python3.7/site-packages/celery/platforms.py", line 149, in acquir
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: self.write_pid()
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: File "/home/baefloral/baefloral/fandeedv3/lib/python3.7/site-packages/celery/platforms.py", line 218, in write_
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: pidfile_fd = os.open(self.path, PIDFILE_FLAGS, PIDFILE_MODE)
Aug 09 15:19:01 OdinAsk-nyc1-Ubuntu-01 sh[14937]: celery.platforms.LockFailed: [Errno 13] Permission denied: '/var/run/celery/beat.pid'
Here is my configuration file
# Name of nodes to start
# here we have a single node
CELERYD_NODES="w1"
# or we could have three nodes:
#CELERYD_NODES="w1 w2 w3"
# Absolute or relative path to the 'celery' command:
CELERY_BIN="/home/baefloral/baefloral/fandeedv3/bin/celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"
# App instance to use
# comment out this line if you don't use an app
CELERY_APP="Reddit_site"
# or fully qualified:
#CELERY_APP="proj.tasks:app"
# How to call manage.py
CELERYD_MULTI="multi"
# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=2700 --concurrency=8"
# - %n will be replaced with the first part of the nodename.
# - %I will be replaced with the current child process index
# and is important when using the prefork pool to avoid race conditions.
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_LOG_LEVEL="INFO"
# you may wish to add these options for Celery Beat
CELERYBEAT_PID_FILE="/var/run/celery/beat.pid"
CELERYBEAT_LOG_FILE="/var/log/celery/beat.log"
Before this error, I was facing ERO20: [Errno 20] Not a directory: '/var/run/celery/beat.pid' so, I've checked /var/run/celery and notice that celery is not a directlyon /var/run for some reasons, but it is rather an empty file. I remove the file and made it into an empty directory
I've tried some suggestions from previous questions, I tried to type and didnt work
sudo chmod 755 /var/log/celery/ /var/run/celery/
sudo chown root:root /var/log/celery/ /var/run/celery/
I am running the tasks on a ubuntu remote server on digitalocean and using a virtual environment and not a root user to run it
I ran into similar situation you had. Here's the fix I had on the Ubuntu server. Since Ubuntu rebuild /var/run on every reboot, the config below will automatically create directories needed by apps on every reboot.
Add a file called celery.conf in /etc/tmpfiles.d.
In this file, add one line:
d /var/run/celery 0755 <user> <group>
Replace user and group with what is configured on your system for celery systemd instance.
Related
I have entries like these in apache2 error.log
[Thu Jan 12 09:18:51.078445 2023] [core:error] [pid 47992] [client 152.89.196.211:53158] AH10244: invalid URI path (/cgi-bin/.%2e/.%2e/.%2e/.%2e/bin/sh)
[Wed Jan 11 06:01:09.820582 2023] [core:error] [pid 30833] [client 185.225.74.55:39856] AH10244: invalid URI path (/cgi-bin/.%%%%32%%65/.%%%%32%%65/.%%%%32%%65/.%%%%32%%65/.%%%%32%%65/bin/sh)
[Wed Jan 11 17:16:49.643509 2023] [core:error] [pid 41882] [client 152.89.196.211:52746] AH10244: invalid URI path (/cgi-bin/.%2e/.%2e/.%2e/.%2e/bin/sh)
I got this to work on the command line:
fail2ban-regex test.log '.*\[client <HOST>:\d+\] AH10244.*$'
Every time I try to stick the regex into a .conf file like so:
[Definition]
failregex = .*\[client <HOST>:\d+\] AH10244.*$
ignoreregex =
fail2ban complains:
Running tests
=============
Use failregex line : filter.conf
ERROR: No failure-id group in 'filter.conf'
I've looked in the man pages and online but I can't find an explanation of what this message is trying to say, or how to fix it.
The Questions
How do I wrap a .conf file around this regex?
What does that error mean?
Could I (how would I) use the pre-defined stuff in apache-common.conf to make this regex more robust?
This fixed it:
fail2ban-regex test.log ./filter.conf
I had my test files (test.log and filter.conf) in my home dir. When I ( from the home dir ) issued the command:
fail2ban-regex test.log filter.conf
I assumed that I was referencing ./test.log and ./filter.conf but I think that fail2ban was looking in the filter.d/ folder to try to find filter.conf.
I found that if filter.conf was in the /etc/fail2ban/filter.d/ folder, then fail2ban-regex test.log filter.conf succeeded.
I'm porting the lirc_serial kernel module to work on our embedded board. We only need to implement the IR transmitter function.
For only the transmitter, the custom driver need only control the RTS pin on /dev/ttyS0, from within the module.
On standard hardware, the driver loads:
00:05: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
NEW APPROACH
I have not figured out how to deactivate the current driver for the serial port, so rather than create a new driver, how would you use the current 8250-dw driver to change the RTS pin on behalf of my kernel module? Something like this?
Using line disciplines per this article on the slip driver looked promising, Just take slip.c and remove the network side of the code. But it needs a user space program (slattach or dip) to open /dev/ttyS0 and activate the line discipline.
Is that possible (or a good idea) from within a kernel module?
In this similar question, How do I open/write/read a uart device from a kernel module?, Ian Abbott suggested backporting serdev to kernel 4.9.
That's getting a bit involved and we're already behind schedule. Is there an easier way?
ORIGINAL QUESTION
However, the embedded board (based on the BayTrail Atom E3845) has the serial port controller in memory mapped I/O:
80860F0A:00: ttyS0 at MMIO 0x90a0c000 (irq = 39, base_baud = 2764800) is a 16550A
80860F0A:01: ttyS1 at MMIO 0x90a0e000 (irq = 40, base_baud = 2764800) is a 16550A
I'm new to driver development. I guess 0x90a0c000 is the physical address of the controller?
To probe the module, I first remapped 0x90a0c000 to a virtual address using ioremap_nocache and then tried to reserve the memory using request_mem_region. That failed.
ioVirtBase = ioremap_nocache(iommap, 8);
TQTRACE("ecp_serial_probe: devm_ioremap for MMIO 0x%X returned 0x%X\n", (uint32_t)iommap, (uint32_t)ioVirtBase);
if (ioVirtBase != NULL)
{
tqDumpBuffer(ioVirtBase, 8);
}
tqRes = request_mem_region((uint32_t)ioVirtBase, 8, ECP_DRIVER_NAME);
TQTRACE("ecp_serial_probe: request_mem_region for 0x%X returned 0x%X\n", (uint32_t)ioVirtBase, (uint32_t)tqRes);
if (!tqRes)
{
TQTRACE("ecp_serial_probe: Cannot request memory at 0x%X\n", (uint32_t)iommap);
return -ENXIO;
}
Is this the correct order of the functions?
Also, it seems request_mem_region fails because the device is under control of 80860F0A ?? There is no such entry in lsmod but there is an entry in /sys/devices.
Do I need to unload that driver to control the USART? How?
# ls -l /sys/devices/platform/80860F0A\:00
lrwxrwxrwx 1 root root 0 Jul 8 23:30 driver -> ../../../bus/platform/drivers/dw-apb-uart
-rw-r--r-- 1 root root 4096 Jul 9 17:02 driver_override
lrwxrwxrwx 1 root root 0 Jul 9 17:14 firmware_node -> ../../LNXSYSTM:00/LNXSYBUS:00/80860F0A:00
-r--r--r-- 1 root root 4096 Jul 9 17:02 modalias
drwxr-xr-x 2 root root 0 Jul 9 17:02 power
lrwxrwxrwx 1 root root 0 Jul 8 23:30 subsystem -> ../../../bus/platform
drwxr-xr-x 3 root root 0 Jul 8 23:30 tty
-rw-r--r-- 1 root root 4096 Jul 9 17:02 uevent
drwxr-xr-x 3 root root 0 Jul 8 23:30 VCOM0001:00
dmesg output below. Dumping the data at the remapped virtual address is not consistent. Sometimes all 0xFF, other times, 00 00 00 00 41 02 1C 48. I don't understand that either...
MARK Tue Jul 9 17:45:35 SGT 2019
ecp_serial: ecp_serial_exit_module()
Spectre V2 : System may be vulnerable to spectre v2
ecp_serial: loading module not compiled with retpoline compiler.
ecp_serial: ecp_serial_init_module()
ecp_serial: ecp_serial_init()
ecp_serial: ecp_serial_probe() iommap=0x90A0C000
ecp_serial: ecp_serial_probe: devm_ioremap for MMIO 0x90A0C000 returned 0xE3296000
ecp_serial: Dump address 0xE3296000:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0000 FF FF FF FF FF FF FF FF
ecp_serial: ecp_serial_probe: request_mem_region for 0xE3296000 returned 0x0
ecp_serial: ecp_serial_probe: Cannot request memory at 0x90A0C000
platform ecp_serial.0: lirc_dev: driver ecp_serial registered at minor = 0
MARK Tue Jul 9 17:46:08 SGT 2019
ecp_serial: ecp_serial_exit_module()
Spectre V2 : System may be vulnerable to spectre v2
ecp_serial: loading module not compiled with retpoline compiler.
ecp_serial: ecp_serial_init_module()
ecp_serial: ecp_serial_init()
ecp_serial: ecp_serial_probe() iommap=0x90A0C000
ecp_serial: ecp_serial_probe: devm_ioremap for MMIO 0x90A0C000 returned 0xE32A2000
ecp_serial: Dump address 0xE32A2000:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
0000 00 00 00 00 41 02 1C 48
ecp_serial: ecp_serial_probe: request_mem_region for 0xE32A2000 returned 0x0
ecp_serial: ecp_serial_probe: Cannot request memory at 0x90A0C000
platform ecp_serial.0: lirc_dev: driver ecp_serial registered at minor = 0
What proc/iomem has to say
90a0c000-90a0cfff : 80860F0A:00
90a0e000-90a0efff : 80860F0A:01
So indeed, the memory is under control of another driver... But how to unload it if it is not listed in lsmod ?
# rmmod 80860F0A:00
ERROR: Module 80860F0A:00 does not exist in /proc/modules
# rmmod 80860F0A
ERROR: Module 80860F0A does not exist in /proc/modules
OS INFO
# uname -a
Linux ecp 4.4.127-1.el6.elrepo.i686 #1 SMP Sun Apr 8 09:44:43 EDT 2018 i686 i686 i386 GNU/Linux
# cat /etc/centos-release
CentOS release 6.6 (Final)
I'm trying, without success, to ban this line using fail2ban-regex :
root#XXXX:/etc/fail2ban#fail2ban-regex '[Mon Jun 22 09:55:16.780767 2015] [:error] [pid 1962] [client 1.2.3.4:38909] script '/var/html/abc.php' not found or unable to stat' filter.d/apache-noscript.conf
using the original failregex in the apache-noscript
failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): /\S*(\.php|\.a$
^%(_apache_error_client)s script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat\s*$
the result is line 'missed' (i was waiting for line 'matched' )
Lines: 1 lines, 0 ignored, 0 matched, 1 missed
|- Missed line(s):
| [Mon Jun 22 09:55:16.780767 2015] [:error] [pid 1962] [client 1.2.3.4:38909] script /var/html/abc.php not found or unable to stat
Please do you know the The_good_failregex to match this line (as like in std log) ? (Fail2Ban v0.8.11 Apache/2.4.7 (Ubuntu)) :
fail2ban-regex '[Mon Jun 22 09:55:16.780767 2015] [:error] [pid 1962] [client 1.2.3.4:38909] script '/var/html/abc.php' not found or unable to stat' The_good_failregex
Many Thanks for your help.
It seems to be version issue.
Fail2Ban v0.8.11 was currently used,
need to be be updated to v0.8.12 (:? before 'error' ...)
_apache_error_client = [[^]]*] [(:?error|\S+:\S+)]( [pid \d+(:\S+ \d+)?])? [client (:\d{1,5})
Thanks.
I downloaded a mongodb dump. Now i am trying to move files from the dump to my mongodb location. I'm using mongorestore --dbpath [mongodb path] [dump path] cmd to do this. But i am seeing the too many positional options error in the command prompt. Can anyone help me with this issue.
This is the error from the command line:
C:\Documents and Settings\>mongorestore -dbpath C:\Program Files\Mon
goDb\data\db C:\Documents and Settings\My Documents\localadventure
s\localadventures
ERROR: too many positional options
Import BSON files into MongoDB.
I tried again today and saw the following message in my cmd prompt:
C:\>mongorestore --dbpath "C:\Program Files\MongoDb\data\db" "C:\Documents and S
ettings\seemanapallik\My Documents\localadventures\localadventures"
Mon Dec 23 09:05:01.515 [tools] C:\Documents and Settings\seemanapallik\My Docum
ents\localadventures\localadventures\adventure.bson
Mon Dec 23 09:05:01.515 [tools] going into namespace [localadventures.ad
venture]
Mon Dec 23 09:05:01.531 [tools] warning: Restoring to localadventures.adventure
without dropping. Restored data will be inserted without raising errors; check y
our server log
1 objects found
Mon Dec 23 09:05:01.546 [tools] Creating index: { key: { _id: 1 }, ns: "
localadventures.adventure", name: "_id_" }
Mon Dec 23 09:05:01.546 [tools] C:\Documents and Settings\seemanapallik\My Docum
ents\localadventures\localadventures\patch_version.bson
Mon Dec 23 09:05:01.546 [tools] going into namespace [localadventures.pa
tch_version]
Mon Dec 23 09:05:01.546 [tools] warning: Restoring to localadventures.patch_vers
ion without dropping. Restored data will be inserted without raising errors; che
ck your server log
2 objects found
Mon Dec 23 09:05:01.562 [tools] Creating index: { key: { _id: 1 }, ns: "
localadventures.patch_version", name: "_id_" }
Mon Dec 23 09:05:01.562 [tools] C:\Documents and Settings\seemanapallik\My Docum
ents\localadventures\localadventures\PaxHeader\adventure.bson
Mon Dec 23 09:05:01.562 [tools] going into namespace [PaxHeader.adventur
e]
assertion: 16619 code FailedToParse: FailedToParse: Expecting '{': offset:0
Mon Dec 23 09:05:01.578 dbexit:
Mon Dec 23 09:05:01.578 [tools] shutdown: going to close listening sockets...
Mon Dec 23 09:05:01.578 [tools] shutdown: going to flush diaglog...
Mon Dec 23 09:05:01.578 [tools] shutdown: going to close sockets...
Mon Dec 23 09:05:01.578 [tools] shutdown: waiting for fs preallocator...
Mon Dec 23 09:05:01.578 [tools] shutdown: closing all files...
Mon Dec 23 09:05:01.609 [tools] closeAllFiles() finished
Mon Dec 23 09:05:01.609 [tools] shutdown: removing fs lock...
Mon Dec 23 09:05:01.609 dbexit: really exiting now
C:\Documents and Settings>mongorestore -dbpath C:\Program Files\MongoDb\data\db C:\Documents and Settings\My Documents\localadventures\localadventures
ERROR: too many positional options
If you have filenames or directory paths containing spaces, these need to be enclosed in double quotes. Otherwise command line programs such as mongorestore may reasonably expect that a space character is the start of a new parameter rather than a continuation of the previous value.
Tip: If you use TAB to autocomplete filenames in the Windows cmd.exe shell the correct quoting should be added automatically.
Assuming that C:\Documents and Settings\My Documents\localadventures\ contains your mongodump files, the expectedmongorestore usage would be:
> mongorestore "C:\Documents and Settings\My Documents\localadventures\localadventures"
Alternatively, you can also change to the dump directory first to save a bit of typing/quoting. A relative path works just fine:
> cd "C:\Documents and Settings\My Documents\localadventures\"
> mongorestore localadventures
You should only include the --dbpath parameter if you are restoring directly to data files and not into a running MongoDB server instance.
There are some more examples of mongorestore usage in the MongoDB manual.
Forewarning to others! I encountered a similar issue that was caused by saving commands in TextEdit and then just copying and pasting. TE must have been affecting spaces and dashes.
In my case, I use
mongorestore --db [dbname] [dumppath]
and it works.
My issue was no setting flags with double hyphens. Instead of setting the db like this --d dbname, I did this -d dbname. Oops!
I have the following Scala (2.10.1) script:
#!/bin/bash
export CLASSPATH=${WORKSPACE}/build/classes/main:${WORKSPACE}/build/libs/setter-for-catan.jar
exec ${SCALA_HOME}/bin/scala "$0" "$#" 2>&1
!#
import sfc.board.SmallBoard
and:
$ cd ${WORKSPACE}
$ find build/classes/main -name 'SmallBoard*.class'
build/classes/main/sfc/board/SmallBoard$$anonfun$1.class
build/classes/main/sfc/board/SmallBoard$$anonfun$2.class
build/classes/main/sfc/board/SmallBoard$$anonfun$3.class
build/classes/main/sfc/board/SmallBoard$$anonfun$4.class
build/classes/main/sfc/board/SmallBoard$$anonfun$5.class
build/classes/main/sfc/board/SmallBoard$$anonfun$6.class
build/classes/main/sfc/board/SmallBoard$.class
build/classes/main/sfc/board/SmallBoard.class
$ jar tvf build/libs/setter-for-catan.jar | grep SmallBoard
1272 Sun Jun 30 20:40:54 EDT 2013 sfc/board/SmallBoard$$anonfun$1.class
1272 Sun Jun 30 20:40:54 EDT 2013 sfc/board/SmallBoard$$anonfun$2.class
1276 Sun Jun 30 20:40:54 EDT 2013 sfc/board/SmallBoard$$anonfun$3.class
1581 Sun Jun 30 20:40:54 EDT 2013 sfc/board/SmallBoard$$anonfun$4.class
1103 Sun Jun 30 20:40:54 EDT 2013 sfc/board/SmallBoard$$anonfun$5.class
1527 Sun Jun 30 20:40:54 EDT 2013 sfc/board/SmallBoard$$anonfun$6.class
8097 Sun Jun 30 20:40:54 EDT 2013 sfc/board/SmallBoard$.class
3722 Sun Jun 30 20:40:54 EDT 2013 sfc/board/SmallBoard.class
yet the script isn't finding the sfc package much less the class:
$ ./setter-for-catan.scala
./setter-for-catan.scala:10: error: not found: object sfc
import sfc.board.SmallBoard
^
one error found
What's going on and what needs to be done to fix it?
Use scala -cp ${CLASSPATH}, not scala -classpath ${CLASSPATH}.