I have this options in defaults section in haproxy.cfg:
option dontlog-normal
option dontlognull
But still have this type of log lines in the logfile:
localhost haproxy[28330]: Connect from XXX.XXX.XXX.XXX:2536 to YYY.YYY.YYY.YYY:80 (http-in/HTTP)
How can I disable this lines in the logs?
specify a min log level to notice, eg:
log 127.0.0.1 local0 notice
Related
I’ve been working on creating a new syslog setup and have run into an issue, that i cannot find a solution for, so i thought maybe someone here could help me out.
I have a setup with 2 syslog servers and 2 haproxy nodes(in HA with keepalived). i have 2 endpoints on configured on the haproxy nodes: “endpoint_X” and “endpoint_Y” for different types of logs. I would like to control the flow of syslog messages, so that when syslog is send to “endpoint_X”:514 its send to syslog01 and when “endpoint_Y”:514 its send to syslog02. this is normally done with the use of ACL’s for normal frontends. But for syslog I use HAproxy’s “log-forward” function, where ACL’s is not supported for.
Below is an example of my config:
ring syslog01
description " "
format rfc3164
maxlen 1200
size 357913941
server syslog01 XXXXX_01:514 source YYYYY check
timeout client 90s
timeout connect 10s
timeout server 90s
timeout check 10s
ring syslog02
description " "
format rfc3164
maxlen 1200
size 357913941
server syslog02 XXXXX_02:514 source YYYYYY check
timeout client 90s
timeout connect 10s
timeout server 90s
timeout check 10s
log-forward syslog
bind 0.0.0.0:514
bind [::]:514
dgram-bind 0.0.0.0:514
dgram-bind [::]:514
log ring#syslog01 local0
log ring#syslog02 local0
does anyone have an idea if there is something i can do to get around this issue , so i can control the data flow in log-forward, other than using differen ports? I use haproxy version 2.6
i have tried some like the following, but as stated ACL does not work with log-forward:
acl acl_endpoint_X hdr(host) -i endpoint_X
acl acl_endpoint_X hdr(host) -i endpoint_Y
log ring#syslog01 local0 if endpoint_X hdr(host)
log ring#syslog02 local0 if endpoint_Y hdr(host)
I have an installation with 2 webservices behind a load balancer with HAProxy. While on service run by 3 servers responds quite fine, the other service with just one server doesn't.
So basically here's what should happen:
loadbalancer --> rancherPlatformAdministration if certain url is used
loadbalancer --> rancherServices for all other requests
Here's my haproxy.cfg:
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
frontend http-in
bind *:80
# Define hosts
acl host_rancherAdmin hdr(host) -i admin.mydomain.tech
use_backend rancherPlatformAdministration if host_rancherAdmin
default_backend rancherServices
backend rancherServices
balance roundrobin
server rancherserver91 192.168.20.91:8080 check
server rancherserver92 192.168.20.92:8080 check
server rancherserver93 192.168.20.93:8080 check
backend rancherPlatformAdministration
server rancherapi01 192.168.20.20:8081 check
wget --server-response foo.mydomain.tech answers with a 401 which is respected behaviour as I am not providing a username nor a password. I can also open up foo.mydomain.tech with my browser an log in. So this part works as I said before.
wget --server-response 192.168.20.20:8081 (yes, this Tomcat really is running under 8081) locally from the loadbalancer responds with 200 and thus works just fine, while trying wget --server-response admin.mydomain.tech results in the following:
--2018-06-10 20:51:56-- http://admin.mydomain.tech/
Aufl"osen des Hostnamens admin.mydomain.tech (admin.mydomain.tech)... <PUBLIC IP>
Verbindungsaufbau zu admin.mydomain.tech (admin.mydomain.tech)|<PUBLIC IP>|:80 ... verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet ...
HTTP/1.0 503 Service Unavailable
Cache-Control: no-cache
Connection: close
Content-Type: text/html
2018-06-10 20:51:56 FEHLER 503: Service Unavailable.
I am pretty sure I am missing something here; I am aware of the differences in forwarding the request as a layer 4 or a layer 7 request – which seems to work just fine. I am providing mode http so I am on layer7...
Any hints on what's happening here or on how I can debug this?
Turns out that in my case the selinux was the showstopper – after putting it to permissive mode by setenforce 0, it just worked...
Since this change is not restart-persistent, I had to follow the instructions found here: https://www.tecmint.com/disable-selinux-temporarily-permanently-in-centos-rhel-fedora/
In Ejabberd 18, the parameter timeout under ejabberd_c2s in configuration seems like not working. When I checked the Log,
unknown listen option 'timeout' for 'ejabberd_c2s'
Now the problem I am facing is, the client connections are getting closed immediately.
Looking at https://docs.ejabberd.im/admin/configuration/#listening-ports right now I see that the ejabberd_c2s listener does not support any option named timeout.
See:
ejabberd_c2s: Handles c2s connections.
Options: access, certfile, ciphers, dhfile, protocol_options, max_fsm_queue, max_stanza_size, shaper, starttls, starttls_required, tls, zlib, tls_compression
Where did you read about such an option for that listener?
I use the following configuration to access internet from local 127.0.0.1:2000 proxy to the internet.:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
#log loghost local0 info
maxconn 4096
#chroot /usr/share/haproxy
user haproxy
group haproxy
daemon
#debug
#quiet
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000
listen appname 0.0.0.0:2000
mode http
stats enable
acl white_list src 127.0.0.1
tcp-request content accept if white_list
tcp-request content reject
stats uri /haproxy?stats
stats realm Strictly\ Private
stats auth special_admin:special_username
balance roundrobin
option httpclose
option forwardfor
server lamp1 23.123.1.110:3128 check
Unfortunately I need to authenticate to my external proxy 23.123.1.110 via http basic authentication "special_admin:special_username".
My question is, is there any way to use basic authentication like :
server lamp1 http://special_admin:special_username#23.123.1.110:3128 check
Thanks
In your example you only need to add the necessary Authorization header with the authorization method and the username:password encoded as base64 like this:
reqadd Authorization:\ Basic\ c3BlY2lhbF9hZG1pbjpzcGVjaWFsX3VzZXJuYW1l
I created the base64 encoded string like this:
echo -n "special_admin:special_username" | base64
For more details about HTTP Basic authorization see https://en.wikipedia.org/wiki/Basic_access_authentication#Client_side
Below listed steps have worked for me.
# haproxy conf
global
log 127.0.0.1 local1
maxconn 4096
defaults
mode http
maxconn 2048
userlist AuthUsers
user admin password $6$SydPP/et7BGN$C5VIhcn6OxuIaLPhCDCmzJyqDYQF8skik3J6sApkXPa6YPSVGutcgQPpdX/VEycGNi3sw7NxLSflEb53gzJtA1
frontend nginx-frontend
bind *:5000
mode http
timeout connect 5s
timeout client 5s
timeout server 5s
default_backend nginx-backend
# For Path based basic authentication use this commented example
#acl PATH_cart path_beg -i /testing
#acl authusers_acl http_auth(AuthUsers)
#http-request auth realm nginx-backend if PATH_cart !authusers_acl
acl authusers_acl http_auth(AuthUsers)
http-request auth realm nginx-backend if !authusers_acl
backend nginx-backend
server nginx nginx:80 check inter 5s rise 2 fall 3
Install below package to generate hash password
sudo apt-get install whois
mkpasswd -m sha-512 'your_password'
mkpasswd -m sha-512 admin#456
expected output
$6$gnGNapo/XeXYg39A$T/7TDfMrZXUDPbv5UPYemrdxdh5xEwqBrzSbpJYs9rfxLbQtgQzxyzkSGWIVOEGze8KrsA0urh3/dG.1xOx3M0
Copy the generated password and paste in haproxy.cfg file
#Deploy the containers to test configuration
sudo docker run -d --name nginx nginx
sudo docker run -d -p 5000:5000 --name haproxy --link nginx:nginx -v /home/users/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg haproxy
Check in the browser, username and password will be prompted.
I want to create a configuration such that the heartbeat between haproxy and the backend is based on HTTP POST.
Does anyone have any idea about this?
I have tried the below configuration, but it only sent the http HEAD to the backend server (I want HTTP POST):
backend mlp
mode http
balance roundrobin
server mlp1 192.168.12.165:9210 check
server mlp2 192.168.12.166:9210 check
Thanks for your help.
#Mohsin,
Thank you so much. I indeed work.
But I want to specify the request message, seems my configure doesn't work. I appreciate that if you can help too.
[root#LB_vAPP_1 tmp]# more /var/www/index.txt
POST / HTTP/1.1\r\nHost: 176.16.0.8:2234\r\nContent-Length: 653\r\n\r\n<?xml version=\"1.0\" encoding=\"gb2312\"?>\r\n<svc_init ver=\"3.2.0\">\r\n<hdr ver=\"3.2.0\">\r\n<client>\r\n<id>915948</id>\r\n<pwd>915948</pwd>\r\n<serviceid></serviceid>\r\n</client>\r\n<requestor><id>13969041845</id></requestor>\r\n</hdr>\r\n<slir ver=\"3.2.0\" res_type=\"SYNC\">\r\n<msids><msid enc=\"ASC\" type=\"MSISDN\">00000000000</msid></msids>\r\n<eqop>\r\n<resp_req type=\"LOW_DELAY\"/>\r\n<hor_acc>200</hor_acc>\r\n</eqop>\r\n<geo_info>\r\n<CoordinateReferenceSystem>\r\n<Identifier
>\r\n<code>4326</code>\r\n<codeSpace>EPSG</codeSpace>\r\n<edition>6.1</edition>\r\n</Identifier\r\n</CoordinateReferenceSystem>\r\n</geo_info>\r\n<loc_type type=\"CURRENT_OR_LAST\"/>\r\n<prio type=\"HIGH\"/>\r\n</slir>\r\n</svc_init>\r\n\r\n\r\n\r\n
my haproxy.conf file is as bellowing:
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local7
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
ulimit-n 65536
daemon
nbproc 1
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
defaults
mode tcp
retries 3
log global
option redispatch
# option abortonclose
retries 3
timeout queue 28s
timeout connect 28s
timeout client 28s
timeout server 28s
timeout check 1s
maxconn 32000
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend mlp
mode tcp
option persist
# bind 10.68.97.42:9211 ssl crt /etc/ssl/server.pem
#bind 10.68.97.42:9211
bind 10.68.97.42:9210
default_backend mlp
frontend supl
mode tcp
option persist
bind 10.68.97.42:7275
default_backend supl
#-------------
# option1 http check
#------------
backend mlp
mode http
balance roundrobin
option httpchk POST / HTTP/1.1\r\nHost: 176.16.0.8:2234\r\nContent-Length: 653\r\n\r\n{<?xml version=\"1.0\" encoding=\"gb2312\"?>\r\n<svc_init ver=\"3.2.0\">\r\n<hdr ver=\"3.2.0\">\r\n<client>\r\n<id>915948</id>\r\n<pwd>915948</pwd>\r\n<serviceid></serviceid>\r\n</client>\r\n<requestor><id>13969041845</id></requestor>\r\n</hdr>\r\n<slir ver=\"3.2.0\" res_type=\"SYNC\">\r\n<msids><msid enc=\"ASC\" type=\"MSISDN\">00000000000</msid></msids>\r\n<eqop>\r\n<resp_req type=\"LOW_DELAY\"/>\r\n<hor_acc>200</hor_acc>\r\n</eqop>\r\n<geo_info>\r\n<CoordinateReferenceSystem>\r\n<Identifier>\r\n<code>4326</code>\r\n<codeSpace>EPSG</codeSpace>\r\n<edition>6.1</edition>\r\n</Identifier>\r\n</CoordinateReferenceSystem>\r\n</geo_info>\r\n<loc_type type=\"CURRENT_OR_LAST\"/>\r\n<prio type=\"HIGH\"/>\r\n</slir>\r\n</svc_init>\r\n\r\n\r\n\r\n}
http-check expect rstring <result resid=\"4\">UNKNOWN SUBSCRIBER</result>
server mlp1 192.168.12.165:9210 check
server mlp2 192.168.12.166:9210 check
#server mlp2 192.168.12.166:9210 check
backend supl
mode tcp
source 0.0.0.0 usesrc clientip
balance roundrobin
server supl1 192.168.12.165:7275 check
server supl2 192.168.12.166:7275 check
#server supl2 192.168.12.166:7275 check
#Mohsin,
Thanks for your answer, it gave me the critical clue to resolve this issue.
However, my message is as bellowing, right now it can work as I want(send the specified request and check the specified response). I post it, hopefully, it may help others also. One point is, the content-length is very important.
backend mlp
mode http
balance roundrobin
option httpchk POST / HTTP/1.1\r\nUser-Agent:HAProxy\r\nHost:176.16.0.8:2234\r\nContent-Type:\ text/xml\r\nContent-Length:516\r\n\r\n91594891594813969041845000000000003200
http-check expect rstring <result resid=\"4\">UNKNOWN SUBSCRIBER</result>
server mlp1 192.168.12.165:9210 check
server mlp2 192.168.12.166:9210 check
I was able to get this working after a bit of experimenting.
This was my setup
HAProxy -> NGINX -> Backend
I was sniffing the requests at the NGINX stage with tcpdump to see what was actually happening.
In order to change the health check request we have to follow a hack described in the documentation to change the HTTP version and send headers:
It is possible to send HTTP headers after the string by concatenating them using rn and backslashes spaces. This is useful to send Host headers when probing a virtual host
This is the raw http check I want to send:
POST ${ENDPOINT} HTTP/1.0
Content-Type: application/json
{"body": "json"}
The big issue here is that HAProxy adds a new header by itself: Connection: close, so this is what NGINX gets:
POST ${ENDPOINT} HTTP/1.0
Content-Type: application/json
{"body": "json"}
Connection: close
This leads, at least in my case to error 400s due to a malformed request.
The fix is to add a Content-Length header:
POST ${ENDPOINT} HTTP/1.0
Content-Type: application/json
Content-Length: 16
{"body": "json"}
Connection: close
Since the Content-Length should take precedence over the actual length, this forces the last header to be ignored. This is what NGINX passes to the backend:
POST ${ENDPOINT} HTTP/1.0
Host: ~^(.+)$
X-Real-IP: ${IP}
X-Forwarded-For: ${IP}
Connection: close
Content-Length: 16
Content-Type: application/json
{"body": "json"}
This is my final check:
option httpchk POST ${ENDPOINT} HTTP/1.0\r\nContent-Type:\ application/json\r\nContent-Length:\ 16\r\n\r\n{\"body\":\"json\"}
If it's just JSON you should be ok copying and pasting this and adjusting the content length.
However, I do recommend that you follow the same procedure and sniff the actual health checks, because, with the characters one has to escape in the config file, creating the request properly can be tricky.
Open haproxy/conf/haproxy.conf file. Goto end of the page, you will see that there is a line 'option httpchk GET /', change GET to POST and you are done.
Let me know if you face any problem.