In HAProxy, is it possible to stop routing traffic to a specific server with active sessions by disabling the server? - haproxy

I am trying implement the following setup
HA -|
|- Redis1
|- Redis2
At any time only one of the redis instances should serve the incoming requests.
Going by the documentation, it seems that you can disable a server dynamically and HA would stop directing the traffic to the disabled server.
While this worked for new client connections, existing client connections are still served content from the disabled server.
But if I kill the redis instance, even the existing client connections are redirected to the other instance.
Is it possible to achieve this behavior without killing the instance?
Heres my HA config:
global
stats socket /opt/haproxy/admin.sock mode 660 level admin
stats socket ipv4#*:19999 level admin
defaults
log global
mode tcp
listen myproxy
mode tcp
bind *:4444
balance roundrobin
server redis1 127.0.0.1:6379 check
server redis2 127.0.0.1:7379 check

Found the answer. Need to add the following directives:
on-marked-down shutdown-sessions
This closes any existing sessions. Eg:
server redis1 127.0.0.1:6379 check on-marked-down shutdown-sessions
server redis2 127.0.0.1:7379 check on-marked-down shutdown-sessions

Related

HA Proxy not passing windows authentation?

We have a simple HA Proxy (13.5) and an IIS Server behind it. The IIS Server itself requires parallel services on the same box, all of which require Windows Authentication. But, it appears that while on "server" and trying to route traffic to the HA Proxy, back to the same server doesn't pass authentication.
frontend VipTst-M-TCPMode
bind 10.5.30.128:80 name http
bind 10.5.30.128:443 name https
timeout client 180s
option tcplog
mode tcp
log global
default_backend M-TcpMode
####### TCP MODE
backend M-TcpMode
balance roundrobin
mode tcp
log global
timeout server 180s
timeout connect 3s
default-server inter 3s rise 2 fall 3
server ServerA 10.20.30.104 maxconn 1000 weight 10 check port 443 inter 5000
So, from ServerA->HAProxy->ServerA/someservice doesn't seem to work. Ironically, if I go from my desktop like this: Desktop-HAproxy->ServerA/someservice it works fine.
And if I just go ServerA/someservice the page also renders.
In ServerA-HAProxy->ServerA, I'm prompted for credentials.
So what did I miss?
Thanks,
Nick

Haproxy still dispatches connections to backend server when it's graceful restarting

We are using haproxy for thrift(rpc) server load balancing in tcp mode. But we've encountered one problem when backend server restarts.
When our thrift(rpc) server restarts, it first stop listening on the port which haproxy is configured to connect, but will still process running requests until they are all done(graceful restart).
So during restarting period, there are still connected sockets made from client to backend server via haproxy while backend server is not accepting any new connections, but haproxy still treats this backend server as healthy, and will dispatch new connections to this server. Any new connections dispatched to this server will take quite a long time to connect, and then timeout.
Is there any way to notify haproxy that server has stop listening and not to dispatch any connection to it?
I've tried following:
timeout connect set to very low + redispatch + retry 3
option tcp-check
Both not solve the problem.

HAProxy - Request getting Broadcast to every server

I am hosting two different application versions on same servers on different ports. In basic version i expect that following configuration should send request in RoundRobin fashion to different ports. But what i am observing is the request is getting broadcasted to ALL of my server endpoints. Meaning in below example my main request to port 8080 gets FWD to both www.myappdemo.com:5001 and www.myappdemo.com:5002... although the response send by proxy is ALWAYS from www.myappdemo.com:5001.
Can anyone tell what is wrong here?
global
debug
maxconn 256
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend http-in
bind *:8080
default_backend servers
backend servers
balance roundrobin
server svr_50301 www.myappdemo.com:5001 maxconn 32 check
server svr_50302 www.myappdemo.com:5002 maxconn 32 check
i can advise you to enable logs and web interface, after that you can provide us more logs and you can check in web interface also if haproxy detects you second server(svr_50302) to be alive.
Reference to HAProxy 1.5 Doc's :
Web Interface - http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#4.2-stats%20admin
Good info how to enable login - http://webdevwonders.com/haproxy-load-balancer-setup-including-logging-on-debian/
Best Regards,
Dani

Combine HAProxy stats?

I have two instances of HAProxy. Both instances have stats enabled and are working fine.
I am trying to combine the stats from both instances into one so that I can use a single HAProxy to view the front/backends stats. I've tried to have the stats listener on the same port for both haproxy instances but this isn't working. I've tried using the sockets interface but this only reports on one of the interfaces as well.
Any ideas?
My one haproxy config file looks like this:
global
daemon
maxconn 256
log 127.0.0.1 local0 debug
log-tag haproxy
stats socket /tmp/haproxy
defaults
log global
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend http-in
bind *:8000
default_backend servers
log global
option httplog clf
backend servers
balance roundrobin
server ws8001 localhost:8001
server ws8002 localhost:8002
log global
listen admin
bind *:7000
stats enable
stats uri /
The other haproxy config is the same except the front/backend server IPs are different.
While perhaps not an exact answer to this specific question, I've seen this kind of question enough that I think it deserves to be answered.
When running with nbproc greater than 1, the Stack Exchange guys have a unique solution. They have a listen section that receives SSL traffic and then uses send-proxy to 127.0.0.1:80. They then have a frontend that binds to 127.0.0.1:80 like this: bind 127.0.0.1:80 accept-proxy. Inside of that frontend they then bind that frontend, e.g. bind-process 1 and in the globals section the do the following:
global
stats socket /var/run/haproxy-t1.stat level admin
stats bind-process 1
The advantage of this is that they get multiple cores for SSL offloading and then a single core dedicated to load balancing traffic. All traffic ultimately flows through this frontend and therefore they can accurately measure stats from that frontend.
This can't work. Haproxy keeps stats separated in each process. It has no capabilities to combine stats of multiple processes.
That said, you are of course free to use external monitoring tools like (munin, graphite or even nagios) which can aggregate the CSV data from multiple stats sockets and display them in unified graphs. These tools are however out-of-scope of core haproxy.

how to disable haproxy after backend servers are down

Can you any help me with this issue. I have installed haproxy loadbalancer. it is working perfect, but the problem is other. When the application connect to the backend server direct without loadbalancer and the server is down, the application say "trying to reconnect" - this is good, because a user know that the server is down. But wenn application is connect to loadbalancer and server is down, the application staying open and don't say "trying to reconnect". This is because the app is connect direct to haproxy and the app think, that everything is ok with connection. Do you have any ideas how to make haproxy to be disable or service to be shutdown when all backend servers are down and of course when some of the servers are up, haproxy to be up also
I think you're asking the same question as How can I make HAProxy reject TCP connections when all backend servers are down
You want to explicitly reject the connection if backend servers are down:
acl site_dead nbsrv lt 1
tcp-request reject if site_dead
Or acl site_dead nbsrv(backend_name) lt 1 where backend_name is the name of a different backend.
nbsrv documentation
acl documentation
tcp-reject documentation