Append or Strip path to individual server in backend - HAProxy - haproxy

First, a thank you for taking the time
I would like to add/remove a path /somepath in the backend for just one server where there are two servers LB. Let me give you an example. In the below one server 10.0.0.10 accepts traffic with /somepath but the other server 10.0.0.20 doesn’t require the /somepath. If I use a reqrep and strip /somepath it would strip it for both servers, however, I would want it to be stripped/add to only one server in the backend. I am using Haproxy version 1.5.
mode http
balance roundrobin
server webtest1 10.0.0.10:80 weight 90
server webtest2 10.0.0.20:80 weight 10

Related

HAProxy redirect port and mask url

I have a couple of webservers that are reachable directly through the following URL:
https://abcd.example.com:8445/desktop/container/landing.jsp?locale=en_US
https://wxyz.example.com:8445/desktop/container/landing.jsp?locale=en_US
I need to use HAProxy to loadbalance between the two and use the following URLs instead when hitting the frontend:
http://1234.example.com/desktop/container/landing.jsp?locale=en_US
or
https://1234.example.com:8445/desktop/container/landing.jsp?locale=en_US
So other requirements beside the two above:
If initial traffic is port 80, convert to port 8445
Mask the URL so that on the browser while it redirected to https and port to 8445, the host remains intact, like so: https://1234.example.com:8445/desktop/container/landing.jsp?locale=en_US
Here's my config so far:
frontend WebApp_frontend
mode http
bind 10.4.34.11:80
acl is80 dst_port 80
http-request set-uri https://%[req.hdr(Host)]:8445%[path]?%[query] if is80
default_backend WebApp-backend
backend WebApp_backend
description WebApp
balance roundrobin
mode http
server webserver1 10.2.89.222:8445 check inter 5s fall 3 rise 5 downinter 1m ssl verify none
server webserver2 10.4.89.223:8445 check inter 5s fall 3 rise 5 downinter 1m ssl verify none
The problem I'm facing right now is that when you access the frontend, HAProxy will redirect you to any of the webservers and force your client to hit the webserver directly instead of through the HAProxy. I need the connection to remain through the HAProxy.
If all your application is doing is redirecting to HTTPs then you should probably just handle that directly within HAProxy. You might want to also explore whether your application supports X-Forwarded-Proto and X-Forwarded-Host.
Another option is you can have HAProxy rewrite the redirects from the backend application to the hostname you choose. Using HAProxy 2.1 you would do something like this:
http-response replace-header location https?://[^:/]*(:?[0-9]+/.*) https://1234.example.com\1 if { status 301:302 }

Complex HAProxy Setup with HTTP2 and HTTP1 togather

We have a complex haproxy setup, where one https frontend were used with multiple backend based on path, host name, subdomians, sni, ssl termination etc. Now in one of our case we need to use http2 with all those http backends. and most importantly it should be support host name based routing to different backends. Our backend support both http2 and http1 we just need to forward traffic there after terminating ssl based on hostname.
Couldn't find much more examples related to http2 in Haproxy. All those i found used mode tcp in frontend like this. I am afraid using mode tcp could break my http routings.
So how could i may accomplish this?
Our Haproxy version is 1.7.1
OS Ubuntu16.04
originally asked on discourse.haproxy.org

haproxy: set timeout if <condition>

The question seems to be quite straight and easy, however I have not been able to find a proper answer.
In haproxy I have 1 backend, say:
backend-1
and 2 frontends, say:
frontend-1
frontend-2
In the backend stanza I want to set a "timeout server" parameter, but, only if the connection comes from frontend-1.
As I didn't find anything I tried to figure it out myself:
backend backend-1
bind *:80
option <blahblah_option>
timeout server 1d if frontend frontend-1
This syntax does not work, and I am mentioning it to let understand what I am trying to achieve.
This is not doable yet in HAProxy.
Later, you will be able to set timeouts using tcp-request and http-request rules.
What we usually do to workaround this for now, is that we setup 2 backends using the same parameters, but different timeout servers.
This is useful when a few urls only deserve a long server timeout.
Edit followup your comment about multiple health checks:
Well, that's why the server's 'track' directive exists:
backend my_app
server srv1 10.0.0.1:80 check
backend my_app_longtime
server srv1 10.0.0.1:80 track my_app/srv1
In the conf above, the server in my_app_longtime backend won't be checked. That said, it will follow up the same state than srv1 in the backend my_app.
Baptiste
Baptiste
I did it like this and it worked. It made it possible to extend timeout on specific app urls, which are more time consuming. Used that trace health check - thanks Babtiste.
frontend www-http
bind 10.0.0.1:80
default_backend app
acl long_url path_beg -i /long_url
use_backend app-extended if long_url
backend app
server web-1 10.0.0.2:80 check
backend app-extended
server web-1 10.0.0.2:80 trace app/web-1
timeout server 10m

Combine HAProxy stats?

I have two instances of HAProxy. Both instances have stats enabled and are working fine.
I am trying to combine the stats from both instances into one so that I can use a single HAProxy to view the front/backends stats. I've tried to have the stats listener on the same port for both haproxy instances but this isn't working. I've tried using the sockets interface but this only reports on one of the interfaces as well.
Any ideas?
My one haproxy config file looks like this:
global
daemon
maxconn 256
log 127.0.0.1 local0 debug
log-tag haproxy
stats socket /tmp/haproxy
defaults
log global
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend http-in
bind *:8000
default_backend servers
log global
option httplog clf
backend servers
balance roundrobin
server ws8001 localhost:8001
server ws8002 localhost:8002
log global
listen admin
bind *:7000
stats enable
stats uri /
The other haproxy config is the same except the front/backend server IPs are different.
While perhaps not an exact answer to this specific question, I've seen this kind of question enough that I think it deserves to be answered.
When running with nbproc greater than 1, the Stack Exchange guys have a unique solution. They have a listen section that receives SSL traffic and then uses send-proxy to 127.0.0.1:80. They then have a frontend that binds to 127.0.0.1:80 like this: bind 127.0.0.1:80 accept-proxy. Inside of that frontend they then bind that frontend, e.g. bind-process 1 and in the globals section the do the following:
global
stats socket /var/run/haproxy-t1.stat level admin
stats bind-process 1
The advantage of this is that they get multiple cores for SSL offloading and then a single core dedicated to load balancing traffic. All traffic ultimately flows through this frontend and therefore they can accurately measure stats from that frontend.
This can't work. Haproxy keeps stats separated in each process. It has no capabilities to combine stats of multiple processes.
That said, you are of course free to use external monitoring tools like (munin, graphite or even nagios) which can aggregate the CSV data from multiple stats sockets and display them in unified graphs. These tools are however out-of-scope of core haproxy.

HAProxy random HTTP 503 errors

We've setup 3 servers:
Server A with Nginx + HAproxy to perform load balancing
backend server B
backend server C
Here is our /etc/haproxy/haproxy.cfg:
global
log /dev/log local0
log 127.0.0.1 local1 notice
maxconn 40096
user haproxy
group haproxy
daemon
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
maxconn 2000
contimeout 50000
clitimeout 50000
srvtimeout 50000
stats enable
stats uri /lb?stats
stats realm Haproxy\ Statistics
stats auth admin:admin
listen statslb :5054 # choose different names for the 2 nodes
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth admin:admin
listen Server-A 0.0.0.0:80
mode http
balance roundrobin
cookie JSESSIONID prefix
option httpchk HEAD /check.txt HTTP/1.0
server Server-B <server.ip>:80 cookie app1inst2 check inter 1000 rise 2 fall 2
server Server-C <server.ip>:80 cookie app1inst2 check inter 1000 rise 2 fall 3
All of the three servers have a good amount of RAM and CPU cores to handle requests
Random HTTP 503 errors are shown when browsing: 503 Service Unavailable - No server is available to handle this request.
And also on server's console:
Message from syslogd#server-a at Dec 21 18:27:20 ...
haproxy[1650]: proxy Server-A has no server available!
Note that 90% times of the time there is no errors. These errors happens randomly.
I had the same issue. After days of pulling my hair out I found the issue.
I had two HAProxy instances running. One was a zombie that somehow never got killed during maybe an update or a haproxy restart. I noticed this when refreshing the /haproxy stats page and the PID would change between two different numbers. The page with one of the numbers had absurd connection stats. To confirm I did
netstat -tulpn | grep 80
Or
sudo lsof -i:80
and saw two haproxy processes listening to port 80.
To fix the issue I did a "kill xxxx" where xxxx is the pid with the suspicious statistics.
Adding my answer here for anyone else who encounters this exact same problem but none of the listed solutions above are applicable. Please note that my answer does not apply to the original code listed above.
For anyone else who may have this problem, check your config and see if you might have mistakenly put the same "bind" line in multiple sections of your config. Haproxy does not check this during startup, and I plan to submit this as a recommended validation check to the developers. In my case, I have 3 different sections of the config, and I mistakenly put the same IP binding in two different places. It was about a 50/50 shot on whether or not the correct section would be used or the incorrect section was used. Even when the correct section was used, about half of the requests still got a 503.
It is possible your servers share, perhaps, a common resource that is timing out at certain times, and that your health check requests are being made at the same time (and thus pulling the backend servers out at the same time).
You can try using the HAProxy option spread-checks to randomize health checks.
I had the same issue, due to 2 HAProxy services running in the linux box, but with different name/pid/resources. Unless i stop the unwanted one, the required instances throws 503 error randomly, say 1 in 5 times.
Was trying to use single linux box for multiple URL routing but looks a limitation in haproxy or the config file of haproxy i have defined.
Hard to say without more details, but is it possible you are exceeding the configured maxconn for each backend? The Stats UI shows these stats on both the frontend and on individual backends.
I resolved my intermittent 503s with HAProxy by adding option http-server-close to backend. Looks like uWSGI (which is upstream) is not doing well with keep-alive. Not sure what's really behind the problem, but after adding this option, haven't seen single 503 since.
don't use the "bind" line in multiple sections of your haproxy.cfg
for example, this would be wrong
frontend stats
bind *:443 ssl crt /etc/ssl/certs/your.pem
frontend Main
bind *:443 ssl crt /etc/ssl/certs/your.pem
fix like this
frontend stats
bind *:8443 ssl crt /etc/ssl/certs/your.pem
frontend Main
bind *:443 ssl crt /etc/ssl/certs/your.pem