Why does HAProxy show that a server's check URL is a 404 when running curl on this URL is successful? - haproxy

I'm setting up HAProxy to load-balance a resource between 3 back-ends. Here is the HAProxy config : (In the following snippets I replaced the actual domain name by example.net)
global
log 127.0.0.1 local2
log-send-hostname
maxconn 2000
pidfile /var/run/haproxy.pid
stats socket /var/run/haproxy.sock mode 600 level admin
stats timeout 30s
daemon
# SSL ciphers
...
defaults
mode http
option forwardfor
option contstats
option http-server-close
option log-health-checks
option redispatch
timeout connect 5000
timeout client 10000
timeout server 10000
...
frontend front
bind *:443 ssl crt /usr/local/etc/haproxy/front.pem
reqadd X-Forwarded-Proto:\ https if { ssl_fc }
stats uri /haproxy?stats
option httpclose
option forwardfor
default_backend back
balance source
backend back
balance roundrobin
option httpchk GET /healthcheck HTTP/1.0
server server1 xxx.xxx.xxx.xxx:80 check inter 5s fall 2 rise 1
server server2 yyy.yyy.yyy.yyy:8003 check backup
server mysite example.net:80 check backup
The issue is the following: even though the first 2 servers respond correctly, the domain-based one always shows as a 404:
What is counter-intuitive to me is that if I use curl to access this same healthcheck, I get an HTTP 200 (like I would expect to see in the HAProxy stats) :
curl -I http://example.net/healthcheck
HTTP/1.1 200 OK
When I ping my site, I get:
# ping example.net
PING example.net (217.160.0.195) 56(84) bytes of data.
64 bytes from 217-160-0-195.elastic-ssl.ui-r.com (217.160.0.195): icmp_seq=1 ttl=50 time=45.7 ms
Is it because the IP of my domain is shared with other domains (1&1 shared hosting) that HAProxy can't access it? Why is that and how to make HAProxy reach it correctly?

Related

haproxy.conf frontend/subdirectory to backend on a different host

I have a pretty basic digital ocean container set-up to hold a personal blog (jcress.org)
I'd like jcress.org/drobot/ to forward to my octoprint server, hosted on a raspberry pi in my basement. haproxy will handle http auth for requests originating outside the lan.
nginix serves a port i'm forwarding from the raspberry pi with ssh -R, all that seems to work.
When the request lands on the raspberry pi I see this render of the login page; filling the form and hitting log in doesn't work, and i don't see any activity in /var/log/haproxy.log
From the LAN I get:
Here's haproxy.conf
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
option http-server-close
option forwardfor
maxconn 2000
timeout connect 5s
timeout client 15min
timeout server 15min
frontend public
log /dev/log local0 debug
bind :::80 v4v6
bind :::443 v4v6 ssl crt /etc/ssl/snakeoil.pem
option forwardfor except 127.0.0.1
use_backend webcam if { path_beg /webcam/ }
use_backend octoprint_lan if { hdr_beg(host) -i 10.0 }
default_backend octoprint
backend octoprint_lan
reqrep ^([^\ :]*)\ /(.*) \1\ /\2
option forwardfor
server octoprint1 127.0.0.1:5000
errorfile 503 /etc/haproxy/errors/503-no-octoprint.http
backend octoprint
http-request set-header Host octopi-drobot.local
reqrep ^([^\ :]*)\ /drobot/?(.*) \1\ /\2
option forwardfor
server octoprint1 127.0.0.1:5000
errorfile 503 /etc/haproxy/errors/503-no-octoprint.http
acl ValidOctoPrintUser http_auth(OctoPrintUsers)
http-request auth realm OctoPrint if !ValidOctoPrintUser
userlist OctoPrintUsers
user USAR insecure-password PASSWARD
here's what shows up in the haproxy log:
Oct 15 17:10:43 octopi-drobot haproxy[3777]: ::1:57030 [15/Oct/2019:17:10:42.938] public octoprint/octoprint1 0/0/92/45/137 200 3074 - - ---- 9/9/1/1/0 0/0 "GET / HTTP/1.0"
Looks like it's not loading your CSS when you're not on the LAN. You need to add a redirect for the CSS/JavaScript files. Try taking a look at your OctoPrint login source HTML, and add the necessary redirects.

How to link frontend to backend when the path request are different?

I have an Haproxy set with https offloadin, and I'm trying to correctly point the requests made to frontend to it's corresponding backend, but bumped into some obstacles.
I have a backend server on http://:9000/abc (NOT in root of the webserver) and when I set a frontend with https:///abc the pointing works as expected and I see the login page.
But I also have another backend server, which is on http://:8888 (IN the root of webserver, it makes it's own redirect to http://:8888/def) and I want it to be accessible by https:///def. But in this case the pointing doesn't work.
How can I make https:///def point to http://:8888 ? Heres is my .cfg
Using HAproxy 1.7
# Automaticaly generated, dont edit manually.
# Generated on: 2019-01-28 13:59
global
maxconn 1000
stats socket /tmp/haproxy.socket level admin
uid 80
gid 80
nbproc 1
hard-stop-after 15m
chroot /tmp/haproxy_chroot
daemon
tune.ssl.default-dh-param 2048
server-state-file /tmp/haproxy_server_state
listen HAProxyLocalStats
bind 127.0.0.1:2200 name localstats
mode http
stats enable
stats refresh 10
stats admin if TRUE
stats show-legends
stats uri /haproxy/haproxy_stats.php?haproxystats=1
timeout client 5000
timeout connect 5000
timeout server 5000
frontend shared-frontend-merged
bind 200.129.168.14:443 name 200.129.168.14:443 no-sslv3 ssl crt-list /var/etc/haproxy/shared-frontend.crt_list
mode http
log global
option http-keep-alive
option forwardfor
acl https ssl_fc
http-request set-header X-Forwarded-Proto http if !https
http-request set-header X-Forwarded-Proto https if https
timeout client 30000
acl aclcrt_shared-frontend var(txn.txnhost) -m reg -i ^ifamcmc\.ddns\.net(:([0-9]){1,5})?$
acl ACL1 var(txn.txnpath) -m sub -i abc
acl ACL2 var(txn.txnpath) -m sub -i def
http-request set-var(txn.txnhost) hdr(host)
http-request set-var(txn.txnpath) path
use_backend glpi_ipvANY if ACL1
use_backend ciweb_ipvANY if ACL2
frontend http-to-https
bind 200.129.168.14:80 name 200.129.168.14:80
mode http
log global
option http-keep-alive
timeout client 30000
http-request redirect scheme https
backend abc_ipvANY
mode http
id 102
log global
timeout connect 30000
timeout server 30000
retries 3
option httpchk OPTIONS /
server abc 10.100.0.30:9000 id 103 check inter 1000
backend def_ipvANY
mode http
id 104
log global
timeout connect 30000
timeout server 30000
retries 3
option httpchk OPTIONS /
server def 10.100.0.40:8888 id 105 check inter 1000
I expect that access to https:///def correctly points to the backend at http://:8888
https://<my.address.com>/abc ------> http://<internal_ip>:9000/abc (OK)
https://<my.address.com>/def ------> http://<internal_ip_2>:8888 (NOT OK)
Have your HAProxy system do initially forwarding based on ports, and then wildcards on your directory.
Please see below:
frontend a-frontend-conf
# Declare an ACL using path_beg (Path Begins)
acl path_images path_beg /images
# Use backend server1 if acl condition path_images is fulfilled
use_backend server1 if path_images
backend server1
[...]
Source: https://serverfault.com/questions/659793/haproxy-how-to-balance-traffic-within-directory-reached

HAProxy environment refusing to connect

I have an installation with 2 webservices behind a load balancer with HAProxy. While on service run by 3 servers responds quite fine, the other service with just one server doesn't.
So basically here's what should happen:
loadbalancer --> rancherPlatformAdministration if certain url is used
loadbalancer --> rancherServices for all other requests
Here's my haproxy.cfg:
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
frontend http-in
bind *:80
# Define hosts
acl host_rancherAdmin hdr(host) -i admin.mydomain.tech
use_backend rancherPlatformAdministration if host_rancherAdmin
default_backend rancherServices
backend rancherServices
balance roundrobin
server rancherserver91 192.168.20.91:8080 check
server rancherserver92 192.168.20.92:8080 check
server rancherserver93 192.168.20.93:8080 check
backend rancherPlatformAdministration
server rancherapi01 192.168.20.20:8081 check
wget --server-response foo.mydomain.tech answers with a 401 which is respected behaviour as I am not providing a username nor a password. I can also open up foo.mydomain.tech with my browser an log in. So this part works as I said before.
wget --server-response 192.168.20.20:8081 (yes, this Tomcat really is running under 8081) locally from the loadbalancer responds with 200 and thus works just fine, while trying wget --server-response admin.mydomain.tech results in the following:
--2018-06-10 20:51:56-- http://admin.mydomain.tech/
Aufl"osen des Hostnamens admin.mydomain.tech (admin.mydomain.tech)... <PUBLIC IP>
Verbindungsaufbau zu admin.mydomain.tech (admin.mydomain.tech)|<PUBLIC IP>|:80 ... verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet ...
HTTP/1.0 503 Service Unavailable
Cache-Control: no-cache
Connection: close
Content-Type: text/html
2018-06-10 20:51:56 FEHLER 503: Service Unavailable.
I am pretty sure I am missing something here; I am aware of the differences in forwarding the request as a layer 4 or a layer 7 request – which seems to work just fine. I am providing mode http so I am on layer7...
Any hints on what's happening here or on how I can debug this?
Turns out that in my case the selinux was the showstopper – after putting it to permissive mode by setenforce 0, it just worked...
Since this change is not restart-persistent, I had to follow the instructions found here: https://www.tecmint.com/disable-selinux-temporarily-permanently-in-centos-rhel-fedora/

HAProxy not forwarding requests to backend server

I have HAProxy (v1.5.8) as a load balancer with keepalived (v1.2.2) to maintain high availability. I also have several Jetty servers on a different machine on the local network as backend server with a webapp.
The problem I have is: I can access to the web app through it IP (z.z.z.z in the code below), but when I try to access by the VIP(y.y.y.y in the code below) I get a connection refused error. I dont have any iptables rule set (Debian 7).
My haproxy.cfg file is this one:
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
chroot /var/lib/haproxy
#stats socket /run/haproxy/admin.sock mode 660 level admin
#stats timeout 30s
maxconn 4096
user haproxy
group haproxy
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
maxconn 2000
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
listen stats x.x.x.x:9000
mode http
balance
option httplog
timeout client 5000
timeout connect 4000
timeout server 30000
stats uri /haproxy_stats
stats realm HAProxy/Statistics
stats auth user:passwd
stats admin if TRUE
listen webfarm y.y.y.y:8080
mode http
stats enable
stats auth someuser:somepassword
balance roundrobin
cookie JSESSIONID prefix
option httpclose
option forwardfor
option httpchk HEAD /check.txt HTTP/1.0
server webA z.z.z.z:9400 check
In the keepalived conf file I have defined the VIP I will use (Is the same one defined in the haproxy.cfg :
virtual_ipaddress {
y.y.y.y
}
In the stats web I get the next results:
HAProxy Stats
After this, I believe that haproxy cannot get to the backends as they are displayed as frontend servers. I am missing something in the conf files?
Thank you
No, 'frontend' and 'backend' are summary lines not titles. It is normal for the servers to be between the two.
Your backend is down because it is returning a 404 to haproxy. You need to make /check.txt return a valid response.
I'd recommend simply disabling your checks until you get a bit more comfortable with HAProxy as a whole. Id also recommend you turn on the stats page, which will give you a better visualization of what HAProxy sees. See below:
###########################################
#
# HAProxy Stats page
#
###########################################
listen stats
bind *:9090
mode http
maxconn 10
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth -----:-----

HaProxy (cannot bind socket, select test failed)

Hello everyone,
Im working about an high availbility project, I had to put in production an haproxy for some applications. Everything was ok after some basics tests but I had some errors and cant fix it. Does someone have some ideas ?
here is the test
# /usr/sbin/haproxy -d -f /etc/haproxy/haproxy.cfg
Available polling systems :
sepoll : pref=400, test result OK
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result FAILED
Total: 4 (3 usable), will use sepoll.
Using sepoll() as the polling mechanism.
[ALERT] 174/160258 (22038) : Starting proxy mysql: cannot bind socket
[ALERT] 174/160258 (22038) : Starting proxy http: cannot bind socket
There is my file haproxy.cfg
global
log 127.0.0.1 local0 notice
user haproxy
group haproxy
maxconn 32000
ulimit-n 65536
defaults
log global
option dontlognull
retries 2
timeout connect 3000
timeout server 5000
timeout client 5000
option redispatch
listen mysql
bind *:3306
mode tcp
option tcplog
balance roundrobin
option mysql-check user haproxy_check
server mysql1 10.83.83.167:3306 check
server mysql2 10.83.83.168:3306 check
server mysql3 10.83.83.169:3306 check
listen http
mode http
bind *:80
stats enable
stats uri /stats
stats auth admin:HaProxy2014
acl app1_cluster_acl hdr_beg(host) -i app1
acl app2_cluster_acl hdr_beg(host) -i app2
acl mysql_cluster_acl hdr_beg(host) -i mysql
use_backend app1_cluster if app1_cluster_acl
use_backend app2_cluster if app2_cluster_acl
use_backend mysql_cluster if mysql_cluster_acl
backend app1_cluster
mode http
cookie SERVERID insert indirect nocache
option forwardfor header X-Real-IP
option http-server-close
option httplog
balance roundrobin
server serv1 10.83.83.203:80 check cookie serv1
server serv2 10.83.83.204:80 check cookie serv2
backend app2_cluster
mode http
cookie SERVERID insert indirect nocache
option forwardfor header X-Real-IP
option http-server-close
option httplog
balance roundrobin
server serv1 10.83.83.187:80 check cookie serv1
server serv2 10.83.83.188:80 check cookie serv2
backend mysql_cluster
mode http
cookie SERVERID insert indirect nocache
option forwardfor header X-Real-IP
option http-server-close
option httplog
balance roundrobin
server mysql1 10.83.83.167:80 check cookie serv1
server mysql2 10.83.83.168:80 check cookie serv2
server mysql3 10.83.83.169:80 check cookie serv2
I get the same error if there is already a mysql or http service running on my load balancer in addition to ruuning on the back ends.
For example if nginx/apache is already running on my load balancer.
$ netstat -anp | grep ":80"
tcp 0 0 127.0.0.1:80 0.0.0.0:* LISTEN 3646/nginx
And i try to start my load balancer with bind *:80 i get a similar error.
$ haproxy -d -f /etc/haproxy/haproxy.cfg
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result FAILED
Total: 3 (2 usable), will use epoll.
Using epoll() as the polling mechanism.
[ALERT] 195/001456 (1903) : Starting frontend www: cannot bind socket [0.0.0.0:80]
If you need to have a mysql or http instance listening on 127.0.0.1 then you can specify the ip of another interface in the bind call.
bind: 10.0.0.20:80
With the latest version of haproxy you can now even use variables.
bind ${LB1}:80
And export these from the haproxy startup script or /etc/default/haproxy
export LB1="10.0.0.20"
Otherwise perhaps there is an issue with your haproxy startup scripts.