Everyday I get an error email from my python flask app with the following error:
WriteError: error 5 from memcached_set: (54648992) SERVER HAS FAILED
AND IS DISABLED UNTIL TIMED RETRY, host: 127.0.0.1:11211 ->
libmemcached/connect.cc:612
Memcached is running and the address is set up to 127.0.0.1 on both the client and the server. When I manually check connection it seems just fine. It only happens periodically.
~ cat /etc/memcached.conf:
# memcached default config file
# 2003 - Jay Bonci <jaybonci#debian.org>
# This configuration file is read by the start-memcached script provided as
# part of the Debian GNU/Linux distribution.
# Run memcached as a daemon. This command is implied, and is not needed for the
# daemon to run. See the README.Debian that comes with this package for more
# information.
-d
# Log memcached's output to /var/log/memcached
logfile /var/log/memcached.log
# Be verbose
# -v
# Be even more verbose (print client commands as well)
# -vv
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m 64
# Default connection port is 11211
-p 11211
# Run the daemon as root. The start-memcached will default to running as root if no
# -u command is present in this config file
-u memcache
# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
# it's listening on a firewalled interface.
-l 127.0.0.1
# Limit the number of simultaneous incoming connections. The daemon default is 1024
# -c 1024
# Lock down all paged memory. Consult with the README and homepage before you do this
# -k
# Return error when memory is exhausted (rather than removing items)
# -M
# Maximize core file limit
# -r
/var/log/memcached.log is empty.
Related
I recently installed memcached via the commands sudo apt-get install memcached and sudo apt-get install python-memcache.
I next did some quick settings in /etc/memcached.conf (attached at the end of the question). For the record, I'm connecting via unix socket. Next I tried sudo service memcached restart to get it up and running.
However, it didn't work. I then manually created a dir in /var/run called memcached and ran chown memcache:root /var/run/memcached.
Then doing sudo service memcached restart was a success.
However, if I do sudo reboot and log back into the machine after the reboot,the sock file (and the folder with it) are gone.
How do I get memcached to persist in this scenario? I'm on Ubuntu 14.04, and the memcached config file is as follows:
# memcached default config file
# 2003 - Jay Bonci <jaybonci#debian.org>
# This configuration file is read by the start-memcached script provided as
# part of the Debian GNU/Linux distribution.
# Run memcached as a daemon. This command is implied, and is not needed for the
# daemon to run. See the README.Debian that comes with this package for more
# information.
-d
# Log memcached's output to /var/log/memcached
logfile /var/log/memcached.log
# Be verbose
-v
# Be even more verbose (print client commands as well)
# -vv
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
#-m 64
-m 320
# Default connection port is 11211
#-p 11211
# Run the daemon as root. The start-memcached will default to running as root if no
# -u command is present in this config file
-u memcache
# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
# it's listening on a firewalled interface.
#-l 127.0.0.1
-s /var/run/memcached/memcached.sock
-a 0777
# Limit the number of simultaneous incoming connections. The daemon default is 1024
# -c 1024
# Lock down all paged memory. Consult with the README and homepage before you do this
# -k
# Return error when memory is exhausted (rather than removing items)
# -M
# Maximize core file limit
# -r
Please ask for more information in case you need it.
I solved my problem by delving into memcached's init.d startup script.
I added the following lines at the head of the script (they weren't there already):
if [ ! -d /var/run/memcached ]; then
mkdir /var/run/memcached
chown memcache /var/run/memcached
fi
I saved, and then set it to start automatically on boot with:
sudo update-rc.d memcached defaults
sudo update-rc.d memcached enable
I have nginx configured to use syslog and unix sockets:
access_log syslog:server=unix:/var/run/syslog-ng.sock,tag=ngw_access,facility=local7,severity=info;
The syslog service then sends the logs to a remote location.
I would like to read the logs from the socket file directly rather than changing the configuration of nginx to put logs into a text file each time I have to debug the message.
I tried using NC (netcat) both traditional and BSD versions:
$ nc -luD /var/run/syslog-ng.sock
nc: getaddrinfo: Servname not supported for ai_socktype
Let me know if this is possible to do.
P.S.
$ sudo update-alternatives --config nc
There are 2 choices for the alternative nc (providing /bin/nc).
Selection Path Priority Status
------------------------------------------------------------
* 0 /bin/nc.openbsd 50 auto mode
1 /bin/nc.openbsd 50 manual mode
2 /bin/nc.traditional 10 manual mode
I have classic situation. Need to configure monit for memcached on CentOS7. The problem is, that all configs i can find in google contains this row:
check process memcached with pidfile /var/run/memcached/memcached.pid
However, There is no memcached.pid file in /var/run and no /var/run/memcached folder. I've checked /usr/lib/systemd/system/memcached.service
[Service]
Type=simple
EnvironmentFile=-/etc/sysconfig/memcached
ExecStart=/usr/bin/memcached -u $USER -p $PORT -m $CACHESIZE -c $MAXCONN $OPTIONS
So, there is no path to .pid file.
The question is can I check memcached without .pid file?
The second question - can be this .pid file in another location?
Replace in your monit config
check process memcached with pidfile /var/run/memcached/memcached.pid
with
check process memcached with match memcached
My config for memcached:
check process memcached with match memcached
start program = "/usr/bin/systemctl start memcached"
stop program = "/usr/bin/systemctl stop memcached"
if failed host 127.0.0.1 port 11211 protocol MEMCACHE then restart
if cpu > 70% for 2 cycles then alert
if cpu > 98% for 5 cycles then restart
if 2 restarts within 3 cycles then timeout
Centos 7, monit 5.14
I'm using mikoomi plugin to monitor mongodb, in that there is shell script which is calling php script and returning 0 to zabbix and parallely writing a data file and log file in /tmp/ directory.
Now my question : How is zabbix reading that data/log file and how are keys fetching information from that file? I need to debug this because zabbix is not rendering any data.
How do I set this up correctly?
Format of data file is as follows:
<Mongo Hostname> mem_virtual 39484
<Mongo Hostname> connections_current 34
<Mongo Hostname> mem_resident 1018
Mikoomi script is using zabbix_sender utility to notify zabbix server about the data collected.
exec("zabbix_sender -vv -z 127.0.0.1 -i $data_file_name 2>&1", $log_file_data) ;
Zabbix server will read the key values from data file and update the db.
Which OS are you using? Check for selinux policy related errors. If selinux policies are enabled in your server, disable and check whether data file is getting updated.
For checking selinux policy enable/disable, run below command
getenforce
If the output is Enforcing then selinux policies are enabled. Disable using following command.
setenforce 0
And again run getenforce to check whether the output is Permissive.
make sure you configured a cron job to run the script for every MongoDB host.
something along the lines of:
* * * * * /usr/lib/zabbix/externalscripts/mikoomi-mongodb-plugin.sh -h [ip address] -p 27017 -z [hostname]
the hostname has to be as it's configured in Zabbix for the target MongoDB server
I have just installed varnish and its ready to serve web pages.
Problem is that we are hitting open files limit.
We have already set ulimit to varnish user.
We want to find a file where we can write this configuration:
# Maximum number of open files
NFILES=131072
# Locked shared memory, default log size is 82MB + header
MEMLOCK=82000
## Configuration with VCL
#
# Listen on port 80, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request. Use a
# fixed-size cache file.
#
# Note: you must replace "example.org" with the outside IP address of your server
# - this is the address at which Varnish receives incoming requests.
# $wgSquidServers in MediaWiki's LocalSettings.php will also need to list all addresses for this Varnish cache.
#
DAEMON_OPTS="-a example.org:80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-u varnish -g varnish \
-s file,/var/lib/varnish/varnish_storage.bin,4G"
The file you're looking for is the Varnish daemon config script. You can find this (usually) in the /etc/sysconfig/varnish if you're on Redhat/RHEL, or /etc/default/varnish if you're on Debian/Ubuntu
There are two places that you will need to do varnish configuration:
/etc/default/varnish
/etc/varnish/default.vcl
This is on centos linux server
/etc/sysconfig/varnish
/etc/varnish/default.vcl
you will also have to change httpd config file to change the port if you want to put varnish in front of apache
/etc/httpd/conf/httpd.conf
Source: How to install and configure Varnish with Apache on Centos 6
It's also worth noting that reloading the varnish config after editing clears it's cache.