Logstash only pick up syslog if running with -f option - elastic-stack

While trying to get ELK to process syslog messages, I can only get it to listen on UDP port 514 if using the -f option:
sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/syslog.conf
Even though logstash is running in the background and I have placed the configuration in the conf.d folder, it does not listen on port 514 unless I run the above command. Most of the configuration is default.
How do I get Logstash to automatically pick up the configuration?

Related

Memcached servers keeps failing

Everyday I get an error email from my python flask app with the following error:
WriteError: error 5 from memcached_set: (54648992) SERVER HAS FAILED
AND IS DISABLED UNTIL TIMED RETRY, host: 127.0.0.1:11211 ->
libmemcached/connect.cc:612
Memcached is running and the address is set up to 127.0.0.1 on both the client and the server. When I manually check connection it seems just fine. It only happens periodically.
~ cat /etc/memcached.conf:
# memcached default config file
# 2003 - Jay Bonci <jaybonci#debian.org>
# This configuration file is read by the start-memcached script provided as
# part of the Debian GNU/Linux distribution.
# Run memcached as a daemon. This command is implied, and is not needed for the
# daemon to run. See the README.Debian that comes with this package for more
# information.
-d
# Log memcached's output to /var/log/memcached
logfile /var/log/memcached.log
# Be verbose
# -v
# Be even more verbose (print client commands as well)
# -vv
# Start with a cap of 64 megs of memory. It's reasonable, and the daemon default
# Note that the daemon will grow to this size, but does not start out holding this much
# memory
-m 64
# Default connection port is 11211
-p 11211
# Run the daemon as root. The start-memcached will default to running as root if no
# -u command is present in this config file
-u memcache
# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
# it's listening on a firewalled interface.
-l 127.0.0.1
# Limit the number of simultaneous incoming connections. The daemon default is 1024
# -c 1024
# Lock down all paged memory. Consult with the README and homepage before you do this
# -k
# Return error when memory is exhausted (rather than removing items)
# -M
# Maximize core file limit
# -r
/var/log/memcached.log is empty.

Reading syslog-ng logs from the unix socket diretly

I have nginx configured to use syslog and unix sockets:
access_log syslog:server=unix:/var/run/syslog-ng.sock,tag=ngw_access,facility=local7,severity=info;
The syslog service then sends the logs to a remote location.
I would like to read the logs from the socket file directly rather than changing the configuration of nginx to put logs into a text file each time I have to debug the message.
I tried using NC (netcat) both traditional and BSD versions:
$ nc -luD /var/run/syslog-ng.sock
nc: getaddrinfo: Servname not supported for ai_socktype
Let me know if this is possible to do.
P.S.
$ sudo update-alternatives --config nc
There are 2 choices for the alternative nc (providing /bin/nc).
Selection Path Priority Status
------------------------------------------------------------
* 0 /bin/nc.openbsd 50 auto mode
1 /bin/nc.openbsd 50 manual mode
2 /bin/nc.traditional 10 manual mode

Monit memcached config without pidfile

I have classic situation. Need to configure monit for memcached on CentOS7. The problem is, that all configs i can find in google contains this row:
check process memcached with pidfile /var/run/memcached/memcached.pid
However, There is no memcached.pid file in /var/run and no /var/run/memcached folder. I've checked /usr/lib/systemd/system/memcached.service
[Service]
Type=simple
EnvironmentFile=-/etc/sysconfig/memcached
ExecStart=/usr/bin/memcached -u $USER -p $PORT -m $CACHESIZE -c $MAXCONN $OPTIONS
So, there is no path to .pid file.
The question is can I check memcached without .pid file?
The second question - can be this .pid file in another location?
Replace in your monit config
check process memcached with pidfile /var/run/memcached/memcached.pid
with
check process memcached with match memcached
My config for memcached:
check process memcached with match memcached
start program = "/usr/bin/systemctl start memcached"
stop program = "/usr/bin/systemctl stop memcached"
if failed host 127.0.0.1 port 11211 protocol MEMCACHE then restart
if cpu > 70% for 2 cycles then alert
if cpu > 98% for 5 cycles then restart
if 2 restarts within 3 cycles then timeout
Centos 7, monit 5.14

MongoDB cannot remote access

I'm new to linux server. I install mongodb on centos 6.3. And I run the mongodb server in this command:
mongod -config /etc/mongodb.conf &
And i'm sure that I have make bind_ip to listen all ip:
# mongodb.conf
# Where to store the data.
dbpath=/var/lib/mongodb
#where to log
logpath=/var/log/mongodb/mongodb.log
logappend=true
rest = true
bind_ip = 0.0.0.0
port = 27017
But, I cannot make mongodb remote access either. my server ip is 192.168.2.24,and I run mongo in my local pc to access this mongodb, it show me this error:
Error: couldn't connect to server 192.168.2.24:2701
7 (192.168.2.24), connection attempt failed at src/mongo/shell/mongo.js:148
exception: connect failed
But, I can access this mongodb in server where mongodb install using this command:
mongo --host 192.168.2.24
So, I think it may success to make mongo remote access, but maybe something wrong with linux server,maybe firewall? So,I try to use the command to check the port whether open for remote access:
iptables -L -n | grep 27017
nothing is returned, then I add port to iptalbes using this command:
iptables -A INPUT -p tcp --dport 27017 -j ACCEPT
iptables -A OUTPUT -p tcp --source-port 27017 -m state --state ESTABLISHED -j ACCEPT
and save the iptables & restart it:
iptables-save | sudo tee /etc/sysconfig/iptables
service iptables restart
I can see port of 27017 is added to iptables list, but it still not work at all. I think it may not success in opening the port of 27017. How should I do for it? I'm new to linux server,by the way my linux server pc is offline. So it can't use the command about "yum". please give me solution in detail. Thanks so much.
It seems like the firewall is not configured correctly.
Disclaimer: Fiddling with firewall settings has security implications. DO NOT USE THE FOLLOWING PROCEDURE ON PRODUCTION SYSTEMS UNLESS YOU KNOW WHAT YOU ARE DOING!!! If in the slightest doubt, get back to a sysadmin or DBA.
The problem
Put simply, a firewall limits the access to services like MongoDB running on the protected machine by unauthorized parties.
CentOS only allows access to ssh by default. We need to configure the firewall so that you can access the MongoDB service.
The solution
We will install a small tool provided by CentOS < 7 (version 7 provides different means), which simplifies the use of iptables, which in turn configures netfilter, the framework of the Linux kernel allowing manipulation of network packets – thus providing firewall functionality (amongst other cool things).
Then, we will use said tool to configure the firewall functionality so that MongoDB is accessible from everywhere. I can't give you a more secure configuration, since I do not know your network setup. Again, use this procedure on production systems at your own risk. You have been warned!
Installation of system-config-firewall-tui
First, you have to log into your CentOS box as root, which allows installation and deinstallation of packages and change system-wide configurations.
Then, you need to issue (the dollar sign denotes the shell prompt)
$ yum -y install system-config-firewall-tui
The result should look something like this
Configuration of the firewall
Next, you need to start the tool we just installed
$ system-config-firewall-tui
which will create a small command line GUI:
Do not simply disable the firewall!.
Press Tab or →| respectively, until the "Customize" button is highlighted. Now press ↵. In the next screen, highlight "Forward" and press ↵. You now should be in a screen called "Other Ports",
in which you highlight "Add" and press↵. This brings you to a screen "Port and Protocol" which you fill like shown below
The configuration explained: MongoDB uses TCP for communicating with the clients and it listens on port 27017 by default for a standalone instance. Note that you might need to change the port according to the referenced list in case you do not run a standalone instance or replica set.
The next step is to highlight "OK" and press ↵, which will seemingly clear the inputs. However, the configuration we just made is saved. So we will press "Cancel" and return to the "Other Ports" screen, which should now look like this:
Now, we press "Close" and return to the main screen of "system-config-firewall-tui". Here, we press "Ok" and the tool asks you if you really want to apply those the changes you just made. Take the time to really think about that. ;)
Pressing "Yes" will now modify the firewall rules executed by the Linux kernel.
We can verify that by issuing
$ iptables -L -n | grep 27017
which should result in the output below:
ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:27017
Now you should be able to connect to your MongoDB server.

redis server does not start

Hi I have the following problem with redis,
I have installed redis on Ubuntu 12.10 with
sudo apt-get install redis-server
However, then comes the start of the message server
[6793] February 6 21:46:54 # Warning: no config file specified, using the default config. In order to specify a config file use 'redis-server / path / to / redis.conf'
[6793] 6 February 6379 21:46:54 # Opening port: bind: Address already in use
What can I do that the server starts?
You should stop the current instance by using the following command.
/etc/init.d/redis-server stop
The default port for Redis, 6793, is already being used, so you'll need to find out what is using that port.
You can check by running: lsof -i :6793
You'll then need to either kill whatever is using that port or specify a different port for Redis using a config file.
pidof redis-server can get you the process id of redis-server;
OR lsof -i:6379 can get you the process id of the running instance with the default listening port 6379.
I would recommend you to check the log file for the troubleshooting possible errors.
Usually located at /var/log/redis/redis.log