I have a pretty basic digital ocean container set-up to hold a personal blog (jcress.org)
I'd like jcress.org/drobot/ to forward to my octoprint server, hosted on a raspberry pi in my basement. haproxy will handle http auth for requests originating outside the lan.
nginix serves a port i'm forwarding from the raspberry pi with ssh -R, all that seems to work.
When the request lands on the raspberry pi I see this render of the login page; filling the form and hitting log in doesn't work, and i don't see any activity in /var/log/haproxy.log
From the LAN I get:
Here's haproxy.conf
defaults
log global
mode http
option httplog
option dontlognull
retries 3
option redispatch
option http-server-close
option forwardfor
maxconn 2000
timeout connect 5s
timeout client 15min
timeout server 15min
frontend public
log /dev/log local0 debug
bind :::80 v4v6
bind :::443 v4v6 ssl crt /etc/ssl/snakeoil.pem
option forwardfor except 127.0.0.1
use_backend webcam if { path_beg /webcam/ }
use_backend octoprint_lan if { hdr_beg(host) -i 10.0 }
default_backend octoprint
backend octoprint_lan
reqrep ^([^\ :]*)\ /(.*) \1\ /\2
option forwardfor
server octoprint1 127.0.0.1:5000
errorfile 503 /etc/haproxy/errors/503-no-octoprint.http
backend octoprint
http-request set-header Host octopi-drobot.local
reqrep ^([^\ :]*)\ /drobot/?(.*) \1\ /\2
option forwardfor
server octoprint1 127.0.0.1:5000
errorfile 503 /etc/haproxy/errors/503-no-octoprint.http
acl ValidOctoPrintUser http_auth(OctoPrintUsers)
http-request auth realm OctoPrint if !ValidOctoPrintUser
userlist OctoPrintUsers
user USAR insecure-password PASSWARD
here's what shows up in the haproxy log:
Oct 15 17:10:43 octopi-drobot haproxy[3777]: ::1:57030 [15/Oct/2019:17:10:42.938] public octoprint/octoprint1 0/0/92/45/137 200 3074 - - ---- 9/9/1/1/0 0/0 "GET / HTTP/1.0"
Looks like it's not loading your CSS when you're not on the LAN. You need to add a redirect for the CSS/JavaScript files. Try taking a look at your OctoPrint login source HTML, and add the necessary redirects.
I tried to receive request and want to redirect it to other host using dns name and exposed with https protocol. For example, my server is http://8.8.8.8:10101/partnerA/getUser. I want haproxy redirect this to https://partner.com/partnerA/getUser (same path as the source).
I also want to filter by path for another redirect destination such as http://8.8.8.8:10101/partnerB/getMarketShare will redirected by HAProxy to https://subdomainb.differentpartner.com/partnerB/getMarketShare(notice the path also follow the same rule, but based on path it will give different host name.
I tried below haproxy.cfg
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
user haproxy
group haproxy
daemon
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main
bind *:10101
acl url_partnerA path_beg -i /partnerA
acl url_partnerB path_beg -i /partnerB
http-request redirect scheme https if url_partnerA
http-request redirect scheme https if url_partnerB
http-request redirect prefix https://partnerA.com if url_partnerA
http-request redirect prefix https://subdomainb.differentpartner.com/ if url_partnerA
default_backend app
#---------------------------------------------------------------------
# round robin balancing between the various backends
backend app
balance roundrobin
# server app1 127.0.0.1:11003 check
But everytime I access (I use http) POST http://8.8.8.8:10101/partnerA/getUser, the log from haproxy -f haproxy10101.cfg -d will give me this
00000000:main.accept(0005)=0009 from [8.8.8.8:48554] ALPN=<none>
00000000:main.clireq[0009:ffffffff]: POST /partnerA/getUser HTTP/1.1
00000000:main.clihdr[0009:ffffffff]: Host: 8.8.8.8:10101
00000000:main.clihdr[0009:ffffffff]: User-Agent: curl/7.47.0
00000000:main.clihdr[0009:ffffffff]: Accept: */*
00000000:main.clihdr[0009:ffffffff]: Authorization: Basic dGNhc2g6RzBqM2tmMHJsMWYzIQ==
00000000:main.clihdr[0009:ffffffff]: Content-Type: application/json
00000000:main.clihdr[0009:ffffffff]: Postman-Token: 45a236c-740a-4859-a13a-1c45195a99f2
00000000:main.clihdr[0009:ffffffff]: cache-control: no-cache
00000000:main.clihdr[0009:ffffffff]: Content-Length: 218
00000000:main.clicls[0009:ffffffff]
00000000:main.closed[0009:ffffffff]
Anything I miss to make it work? Thanks
I'm attempting to configure HAProxy to close a client TCP connection after it has been used to process N requests. My goal is to have our long-lived clients occasionally re-establish connections that are otherwise kept-alive by HTTP Keep-Alive.
Basically I'm trying to implement the equivalent of nginx's keepalive_requests (http://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_requests).
I currently have something like:
frontend https-in
bind *:443 ssl crt /etc/ssl/private/cert.pem
stick-table type binary len 32 size 1000 expire 75s store gpc0
acl close_connection sc0_get_gpc0 gt 3
acl exceeded_connection sc0_get_gpc0 gt 4
http-response set-header Connection Keep-Alive unless close_connection
http-response set-header Keep-Alive timeout=75\ max=3 unless close_connection
http-response set-header Connection Close if close_connection
timeout http-keep-alive 75s
timeout client 75s
tcp-request content track-sc0 ssl_fc_session_id
tcp-request content reject if exceeded_connection
http-request sc-inc-gpc0
default_backend https
backend https
option httpchk GET /health
server localhost 127.0.0.1:8080 maxconn 1000
But some problems with this include:
The SSL session ID may be re-used across connections
This ends abruptly closing a connection from a client when they've exceeded the threshold (assuming they ignore the Connection: Close)
Are there any recommended approaches for something like this? Ideally I would like to:
Track the counter based on individual TCP connection (not src to avoid the case where the same IP has multiple connections established)
Close the connection on the final response (at the same time I send Connection: Close)
But I haven't been able to track down ways to do either of those.
Thanks!
Edit
I was able to devise a better way to track unique TCP connections by creating hashing a tuple of the src,src_port,dst,dst_port:
http-request set-header X-Unique-Id %[src]:%[src_port]:%[dst]:%[dst_port]
http-request set-header X-Unique-Id-SHA %[req.fhdr(X-Unique-Id),sha1]
http-request track-sc0 req.fhdr(X-Unique-Id-SHA)
I'm not crazy about having to create the dummy headers, but this seems to work.
I'm setting up HAProxy to load-balance a resource between 3 back-ends. Here is the HAProxy config : (In the following snippets I replaced the actual domain name by example.net)
global
log 127.0.0.1 local2
log-send-hostname
maxconn 2000
pidfile /var/run/haproxy.pid
stats socket /var/run/haproxy.sock mode 600 level admin
stats timeout 30s
daemon
# SSL ciphers
...
defaults
mode http
option forwardfor
option contstats
option http-server-close
option log-health-checks
option redispatch
timeout connect 5000
timeout client 10000
timeout server 10000
...
frontend front
bind *:443 ssl crt /usr/local/etc/haproxy/front.pem
reqadd X-Forwarded-Proto:\ https if { ssl_fc }
stats uri /haproxy?stats
option httpclose
option forwardfor
default_backend back
balance source
backend back
balance roundrobin
option httpchk GET /healthcheck HTTP/1.0
server server1 xxx.xxx.xxx.xxx:80 check inter 5s fall 2 rise 1
server server2 yyy.yyy.yyy.yyy:8003 check backup
server mysite example.net:80 check backup
The issue is the following: even though the first 2 servers respond correctly, the domain-based one always shows as a 404:
What is counter-intuitive to me is that if I use curl to access this same healthcheck, I get an HTTP 200 (like I would expect to see in the HAProxy stats) :
curl -I http://example.net/healthcheck
HTTP/1.1 200 OK
When I ping my site, I get:
# ping example.net
PING example.net (217.160.0.195) 56(84) bytes of data.
64 bytes from 217-160-0-195.elastic-ssl.ui-r.com (217.160.0.195): icmp_seq=1 ttl=50 time=45.7 ms
Is it because the IP of my domain is shared with other domains (1&1 shared hosting) that HAProxy can't access it? Why is that and how to make HAProxy reach it correctly?
I want to create a configuration such that the heartbeat between haproxy and the backend is based on HTTP POST.
Does anyone have any idea about this?
I have tried the below configuration, but it only sent the http HEAD to the backend server (I want HTTP POST):
backend mlp
mode http
balance roundrobin
server mlp1 192.168.12.165:9210 check
server mlp2 192.168.12.166:9210 check
Thanks for your help.
#Mohsin,
Thank you so much. I indeed work.
But I want to specify the request message, seems my configure doesn't work. I appreciate that if you can help too.
[root#LB_vAPP_1 tmp]# more /var/www/index.txt
POST / HTTP/1.1\r\nHost: 176.16.0.8:2234\r\nContent-Length: 653\r\n\r\n<?xml version=\"1.0\" encoding=\"gb2312\"?>\r\n<svc_init ver=\"3.2.0\">\r\n<hdr ver=\"3.2.0\">\r\n<client>\r\n<id>915948</id>\r\n<pwd>915948</pwd>\r\n<serviceid></serviceid>\r\n</client>\r\n<requestor><id>13969041845</id></requestor>\r\n</hdr>\r\n<slir ver=\"3.2.0\" res_type=\"SYNC\">\r\n<msids><msid enc=\"ASC\" type=\"MSISDN\">00000000000</msid></msids>\r\n<eqop>\r\n<resp_req type=\"LOW_DELAY\"/>\r\n<hor_acc>200</hor_acc>\r\n</eqop>\r\n<geo_info>\r\n<CoordinateReferenceSystem>\r\n<Identifier
>\r\n<code>4326</code>\r\n<codeSpace>EPSG</codeSpace>\r\n<edition>6.1</edition>\r\n</Identifier\r\n</CoordinateReferenceSystem>\r\n</geo_info>\r\n<loc_type type=\"CURRENT_OR_LAST\"/>\r\n<prio type=\"HIGH\"/>\r\n</slir>\r\n</svc_init>\r\n\r\n\r\n\r\n
my haproxy.conf file is as bellowing:
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local7
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
ulimit-n 65536
daemon
nbproc 1
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
defaults
mode tcp
retries 3
log global
option redispatch
# option abortonclose
retries 3
timeout queue 28s
timeout connect 28s
timeout client 28s
timeout server 28s
timeout check 1s
maxconn 32000
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend mlp
mode tcp
option persist
# bind 10.68.97.42:9211 ssl crt /etc/ssl/server.pem
#bind 10.68.97.42:9211
bind 10.68.97.42:9210
default_backend mlp
frontend supl
mode tcp
option persist
bind 10.68.97.42:7275
default_backend supl
#-------------
# option1 http check
#------------
backend mlp
mode http
balance roundrobin
option httpchk POST / HTTP/1.1\r\nHost: 176.16.0.8:2234\r\nContent-Length: 653\r\n\r\n{<?xml version=\"1.0\" encoding=\"gb2312\"?>\r\n<svc_init ver=\"3.2.0\">\r\n<hdr ver=\"3.2.0\">\r\n<client>\r\n<id>915948</id>\r\n<pwd>915948</pwd>\r\n<serviceid></serviceid>\r\n</client>\r\n<requestor><id>13969041845</id></requestor>\r\n</hdr>\r\n<slir ver=\"3.2.0\" res_type=\"SYNC\">\r\n<msids><msid enc=\"ASC\" type=\"MSISDN\">00000000000</msid></msids>\r\n<eqop>\r\n<resp_req type=\"LOW_DELAY\"/>\r\n<hor_acc>200</hor_acc>\r\n</eqop>\r\n<geo_info>\r\n<CoordinateReferenceSystem>\r\n<Identifier>\r\n<code>4326</code>\r\n<codeSpace>EPSG</codeSpace>\r\n<edition>6.1</edition>\r\n</Identifier>\r\n</CoordinateReferenceSystem>\r\n</geo_info>\r\n<loc_type type=\"CURRENT_OR_LAST\"/>\r\n<prio type=\"HIGH\"/>\r\n</slir>\r\n</svc_init>\r\n\r\n\r\n\r\n}
http-check expect rstring <result resid=\"4\">UNKNOWN SUBSCRIBER</result>
server mlp1 192.168.12.165:9210 check
server mlp2 192.168.12.166:9210 check
#server mlp2 192.168.12.166:9210 check
backend supl
mode tcp
source 0.0.0.0 usesrc clientip
balance roundrobin
server supl1 192.168.12.165:7275 check
server supl2 192.168.12.166:7275 check
#server supl2 192.168.12.166:7275 check
#Mohsin,
Thanks for your answer, it gave me the critical clue to resolve this issue.
However, my message is as bellowing, right now it can work as I want(send the specified request and check the specified response). I post it, hopefully, it may help others also. One point is, the content-length is very important.
backend mlp
mode http
balance roundrobin
option httpchk POST / HTTP/1.1\r\nUser-Agent:HAProxy\r\nHost:176.16.0.8:2234\r\nContent-Type:\ text/xml\r\nContent-Length:516\r\n\r\n91594891594813969041845000000000003200
http-check expect rstring <result resid=\"4\">UNKNOWN SUBSCRIBER</result>
server mlp1 192.168.12.165:9210 check
server mlp2 192.168.12.166:9210 check
I was able to get this working after a bit of experimenting.
This was my setup
HAProxy -> NGINX -> Backend
I was sniffing the requests at the NGINX stage with tcpdump to see what was actually happening.
In order to change the health check request we have to follow a hack described in the documentation to change the HTTP version and send headers:
It is possible to send HTTP headers after the string by concatenating them using rn and backslashes spaces. This is useful to send Host headers when probing a virtual host
This is the raw http check I want to send:
POST ${ENDPOINT} HTTP/1.0
Content-Type: application/json
{"body": "json"}
The big issue here is that HAProxy adds a new header by itself: Connection: close, so this is what NGINX gets:
POST ${ENDPOINT} HTTP/1.0
Content-Type: application/json
{"body": "json"}
Connection: close
This leads, at least in my case to error 400s due to a malformed request.
The fix is to add a Content-Length header:
POST ${ENDPOINT} HTTP/1.0
Content-Type: application/json
Content-Length: 16
{"body": "json"}
Connection: close
Since the Content-Length should take precedence over the actual length, this forces the last header to be ignored. This is what NGINX passes to the backend:
POST ${ENDPOINT} HTTP/1.0
Host: ~^(.+)$
X-Real-IP: ${IP}
X-Forwarded-For: ${IP}
Connection: close
Content-Length: 16
Content-Type: application/json
{"body": "json"}
This is my final check:
option httpchk POST ${ENDPOINT} HTTP/1.0\r\nContent-Type:\ application/json\r\nContent-Length:\ 16\r\n\r\n{\"body\":\"json\"}
If it's just JSON you should be ok copying and pasting this and adjusting the content length.
However, I do recommend that you follow the same procedure and sniff the actual health checks, because, with the characters one has to escape in the config file, creating the request properly can be tricky.
Open haproxy/conf/haproxy.conf file. Goto end of the page, you will see that there is a line 'option httpchk GET /', change GET to POST and you are done.
Let me know if you face any problem.