Haproxy request gets timeout when doing ACL - haproxy

I created one simple application and i'm running it in 4 different containers.
I managed to proxy all the ports to the bind 9991, but when i create acl to use different backends it does'nt work, only the default back-end.
Any tips for using haproxy will be very usefull!
haproxy.cfg :
frontend tests
bind *:9991
timeout client 60s
mode http
acl is_servers_2 path_end /app2
use_backend servers_2 if is_servers_2
acl is_servers_3 path_end /app3
use_backend servers_3 if is_servers_3
default_backend servers_2
backend servers_2
timeout connect 10s
timeout server 60s
mode http
server server3001 127.0.0.1:3001
server server3002 127.0.0.1:3002
backend servers_3
timeout connect 10s
timeout server 60s
mode http
server server3003 127.0.0.1:3003
server server3004 127.0.0.1:3004
haproxy version: 2.4.4
ubuntu: 18.04

Indeed my api was the problem.
It wasn't configured to handle /app2 and /app3 path urls, just /

Related

HAProxy SSL passthrough and ACL rules not working

I'm trying to get SSL passthrough working so only my backends need SSL and not the HAProxy frontends.
I also want to use ACL rules to only allow certain domains to get sent to the backend and those that do not match will get another backend.
So far I have this, but it seems to not be working:
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon
maxconn 10240
nbproc 4
nbthread 1
cpu-map auto:1/1-4 0-3
defaults
log global
mode http
option httplog
option dontlognull
maxconn 2048
retries 3
timeout connect 10s
timeout client 30s
timeout server 30s
timeout http-request 10s
timeout http-keep-alive 2s
timeout queue 5s
timeout tunnel 2m
timeout client-fin 2s
timeout server-fin
frontend abc.com
bind AAA.BBB.CCC.DDD:80
bind AAA.BBB.CCC.DDD:443
acl ValidDomain hdr_dom(host) -i abc.com
use_backend abc.com if ValidDomain
default_backend fallback
backend abc.com
balance static-rr
server default ZZZ.YYY.XXX.WWW:443 ssl check verify none
backend fallback
balance static-rr
server fallback 127.0.0.1:8080
Can anyone figure out why it isn't working?
What you need is the mode tcp instead of mode http and the can your route based on SNI.
You can get a example solution in this answer Serving LDAPS lookups over HAProxy, unable to bind in testing

How to configure HAProxy with Certbot/letsencrypt

I have one domain with several sub-domains. All of them point to my VPS. I use docker in swarm mode to run my web services and my (HA)proxy.
I also use Certbot/Let’s Encrypt in standalone to get the certificate for my domain.
Everything seems to be working fine except the HAproxy configuration for certbot.
Here is my HAProxy config:
defaults
mode http
log global
option httplog
option dontlognull
timeout check 5s
timeout connect 5s
timeout client 50s
timeout client-fin 50s
timeout server 50s
timeout tunnel 1h
resolvers docker
nameserver dns1 127.0.0.11:53
resolve_retries 3
timeout resolve 1s
timeout retry 1s
hold other 10s
hold refused 10s
hold nx 10s
hold timeout 10s
hold valid 10s
hold obsolete 10s
# HTTP(S) frontend
frontend web-in
bind *:80
bind *:443 ssl crt /etc/ssl/haproxy.pem
# test URI to see if its a letsencrypt request
acl letsencrypt-acl path_beg /.well-known/acme-challenge/
use_backend be-letsencrypt if letsencrypt-acl
# sub-domains
acl sub1 hdr(host) -i sub1.mydomain.com
acl sub2 hdr(host) -i sub2.mydomain.com
acl sub3 hdr(host) -i sub3.mydomain.com
# figure out which one to use
use_backend be-sub1 if sub1
use_backend be-sub2 if sub2
use_backend be-sub3 if sub3
# Default
default_backend maintenance
# Backend | LE
backend be-letsencrypt
server letsencrypt dockerhost:8888 check resolvers docker init-addr none
# Sub1
backend be-sub1
option forwardfor
http-request add-header X-Forwarded-Proto https
redirect scheme https code 301 if !{ ssl_fc }
server sub1-server webservice1:8080 check resolvers docker init-addr libc,none
# ... Sub2, Sub3 ...
# Default backend
backend maintenance
balance roundrobin
I expect all the following urls:
mydomain.com/.well-known/acme-challenge/
sub1.mydomain.com/.well-known/acme-challenge/
sub2.mydomain.com/.well-known/acme-challenge/
sub3.mydomain.com/.well-known/acme-challenge/
to resolve to dockerhost:8888. But instead I always get 503 Service Unavailable.
To test the configuration I run dockercloud/hello-world on the port 8888
docker run --rm -d -p 8888:80 dockercloud/hello-world
I know that dockerhost:8888 resolution work fine as I can test it with the following configuration:
# Sub1
backend be-sub1
server letsencrypt dockerhost:8888 check resolvers docker init-addr none
Meaning that when I hit my syb-domain sub1.mydomain.com I end up on the dockercloud/hello-world
"Hello world!" page.
So why path based resolution doesn't work ? Am I missing something ?
acl letsencrypt-acl path_beg /.well-known/acme-challenge/
use_backend be-letsencrypt if letsencrypt-acl
I had a very similar issue.
My verification failed with the same 503 error you describe. As I was wondering why that is since I saw the OpenPort of the certbot dockerimage on my machine and the redirects in the HAproxy logs -> I found out that since I was using HAproxy also in an docker Image and the backend server config was connecting to 127.0.0.1 within the HAproxy docker image which of corse cant work as the Port of the certbot was in another docker image I changed the server backend config to the IP address of the Host on which the docker image was running all worked well.
Maybe it helps someone ;) cheers

Layer4 "Connection refused" with haproxy

I need some advise on how to setup haproxy. I have two web-servers up and running. For testing they run a simple node server on port 8080.
Now on my haproxy server I start haproxy which gives me the following:
$> /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg
[WARNING] 325/202628 (16) : Server node-backend/server-a is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[WARNING] 325/202631 (16) : Server node-backend/server-b is DOWN, reason: Layer4 timeout, check duration: 2001ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 325/202631 (16) : backend 'node-backend' has no server available!
Just one note: If I do:
haproxy$> wget server-a:8080
I get the response from the node server.
Here is my haproxy.cfg:
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode tcp
log global
option tcplog
option dontlognull
option http-server-close
# option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend www
bind *:80
default_backend node-backend
#---------------------------------------------------------------------
# round robin balancing between the various backends
#--------------------------------------------------------------------
backend node-backend
balance roundrobin
mode tcp
server server-a 172.19.0.2:8080 check
server server-b 172.19.0.3:8080 check
If I remove the check option it seems to work. Any suggestions how I can fix this checking mechanism of haproxy?
You need to get exact ip address of your server with the help of command
ifconfig
and correct the below address in your haproxy.cfg file:
172.19.0.2:8080
172.19.0.3:8080
or modify line like below
server server-a server-a:8080 check
server server-b server-b:8080 check
Remove "mode tcp" and change it to "mode http".
Im just guessing here but i suppose haproxy is doing a tcp check against your web server and the web server can not respond to it.
in "mode http" it checks the web server in http mode and expects a "response 200" for L4 check
and expects a string (whatever you defined) as a L7 check
eg. L4
backend node-backend
balance roundrobin
mode http #(NOT NEEDED IF DEFINED IN DEFAULTS)
option httpchk
server server-a 172.19.0.2:8080 check
server server-b 172.19.0.3:8080 check
eg. L7
backend node-backend
balance roundrobin
mode http #(NOT NEEDED IF DEFINED IN DEFAULTS)
option httpchk get /SOME_URI
http-check expect status 200
server server-a 172.19.0.2:8080 check
server server-b 172.19.0.3:8080 check
Another note related to #basickarl's comment on docker. If you are sending into a docker (docker-compose) instance (namely where you have multiple instances of service running) you likely need to define the docker resolver and use it for dns resolution on your backend:
resolver:
resolvers docker_resolver
nameserver dns 127.0.0.11:53
backend usage of resolver:
backend main
balance roundrobin
option http-keep-alive
server haproxyapp app:80 check inter 10s resolvers docker_resolver resolve-prefer ipv4
i tryied all this answers nothing works for me. only put the gateway IP of network work, for default bridge is 172.17.0.1.
In the servers put the : and with this haproxy connects with success.
My example of custom network with fixed ips and gateway:
----- haproxy config
backend be_pe_8545
mode http
balance roundrobin
server p1 172.20.0.254:18545 check inter 10s
server p2 172.20.0.254:28545 check inter 10s
----- docker app / network
docker_app: ...
networks:
public_network:
ipv4_address: 172.20.0.50
public_network:
name: public_network
driver: bridge
ipam:
driver: default
config:
- subnet: 172.20.0.0/24
gateway: 172.20.0.254

HAProxy hostname or URL backend

Is it possible to set up backend as Hostnames or URLs?
I want to use haproxy to balance both http and https requests to a pool of backends (TCP mode).
Since backends can change their IPs, I want to set backend servers as URLs.
...
defaults
mode tcp
option redispatch
...
frontend all
bind *:80
bind *:443
option tcplog
# DEFAULT
default_backend hostname_servers
backend hostname_severs
mode tcp
balance roundrobin
option ssl-hello-chk
server host1 host1.myapp.com check
server host2 host2.myapp.com check
...
server hostN hostN.myapp.com check
Thanks!
I do believe that functionality is now available in 1.6, http://blog.haproxy.com/2015/10/14/whats-new-in-haproxy-1-6/
From Link:
resolvers docker
nameserver dnsmasq 127.0.0.1:53
defaults
mode http
log global
option httplog
frontend f_myapp
bind :80
default_backend b_myapp
backend b_myapp
server s1 nginx1:80 check resolvers docker resolve-prefer ipv4

How do I find a list of healthy nodes in a TCP haproxy setup

I have an haproxy setup which is NOT fronting an HTTP cluster. How can I ask haproxy which nodes are currently in the healthy list, and which are considered unhealthy?
I found the "stats" command, but that only works for HTTP traffic
You can add an additional port to your frontend (e.g. 8080) and enable stats for only that backend.
frontend
bind *:123,*:8080
acl is_stats dst_port 8080
use_backend stats if is_stats
default_backend your_TCP_backend
backend stats
stats enable