How to set host-header depending on choosen backend when using HAProxy as loadbalancer - haproxy

Under certain circumstances it is required to modify the host-header based on the backend selected by HAProxy loadbalancing (an example is described here: https://www.claudiokuenzler.com/blog/919/haproxy-how-use-different-http-host-header-for-each-backend-server)
My internet research shows that there are basically two ways to achieve this:
a) use "http-send-name-header Host" (not recommended according the HAProxy docs: https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-http-send-name-header)
b) use " http-request set-header Host ... if { srv_id 1 }"
(the following article describes these two techniques: https://serverfault.com/questions/876871/configure-haproxy-to-include-host-headers-for-different-backends)
Option a) works as expected but it is discouraged to be use (see https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-http-send-name-header).
For this reason I am trying to option b). Unfortunately the config cowardly refuses to work:
backend loadbalanced-backends
mode http
balance roundrobin
option forwardfor
http-request set-header Host one.domain.com if { srv_id 1 }
http-request set-header Host two.domain.com if { srv_id 2 }
server one one.domain.com:8000
server two two.domain.com:8000
The warnings printed in the log:
[WARNING] (1) : config : parsing ....cfg:14] : anonymous acl will never match because it uses keyword 'srv_id' which is incompatible with 'backend http-request header rule'
[WARNING] (1) : config : parsing ....cfg:15] : anonymous acl will never match because it uses keyword 'srv_id' which is incompatible with 'backend http-request header rule'
Besids the warning the logged request-header show that the host is not changed to one.domain.com resp. two.domain.com.
What exactly means the warning "anonymous acl will never match..."? I do not see what is wrong with the configuration. Any ideas are welcome.

Related

HAProxy set authorization header from cookie

I have a backend that I don't control that works using magic links. If I use the magic link, I don't have to log on. I want to share the magic link externally without exposing the actual key as the key changes from time to time and I was hoping to use haproxy as a reverse proxy. I don't want the link to be open to the entire internet and would like to use basic auth as well. The problem I am facing is that the backend overwrites the Authorization header (it's required for the magic link to work) and I get stuck in a loop and need to log on every time
My workaround:
On first request I request the basic auth (works)
Then I write it to a cookie (this part works)
On each subsequent request, if the cookie exists, I read the cookie and set the Authorization header to the cookie value (this part does not work)
Then I run http_auth which in my mind should now work since I have overwritten the header
But it does not work. Any suggestions?
userlist auth-list
user myuser insecure-password mypass
frontend myfrontend
bind *:80
mode http
acl is-path path -i -m beg /publiclink
acl has_cookie req.hdr(X-MyAuth) -m found
http-request set-header Authorization %[req.hdr(X-MyAuth)] if has_cookie
http-request auth unless { http_auth(auth-list) }
http-request set-var(txn.myhostheader) req.hdr(Authorization) if { http_auth(auth-list) !has_cookie }
default_backend node
backend node
mode http
server dcnode1 192.168.0.1:8000 check
http-response set-header set-cookie "X-MyAuth=%[var(txn.myhostheader)]; Path=/" if { var(txn.myhostheader) -m found }
http-request replace-path /publiclink1(.*) /magiclink\1
http-request set-header Authorization "Key magiclink"
My answer is related to this point :
On each subsequent request, if the cookie exists, I read the cookie and set the Authorization header to the cookie value (this part does not work)
In your backend, you set the Set-Cookie: X-MyAuth=... header :
backend node
http-response set-header set-cookie "X-MyAuth=%[var(txn.myhostheader)]; Path=/" if { var(txn.myhostheader) -m found }
So the next request contains a Cookie header like this one : Cookie: .* X-MyAuth=.*.
But in your frontend, you use the X-MyAuth header (which probably does not exist) :
frontend myfrontend
acl has_cookie req.hdr(X-MyAuth) -m found
http-request set-header Authorization %[req.hdr(X-MyAuth)] if has_cookie
You may use the X-MyAuth cookie value like this :
frontend myfrontend
acl has_cookie cook(X-MyAuth) -m found
http-request set-header Authorization %[cook(X-MyAuth)] if has_cookie

Implement a rate-limit relating to the healthy servers count using haproxy

I want to implement a rate-limit system using the sticky table of HAProxy. Consider that I have 100 servers, and a limit of 10 requests per server, the ACL would be
http-request track-sc0 int(1) table GlobalRequestsTracker
http-request deny deny_status 429 if { sc0_http_req_rate(GlobalRequestsTracker),div(100) gt 10 }
Now if I want to make this dynamic depending on the healthy servers count, I need to replace the hardcoded 100 per the nbsrv converter.
http-request track-sc0 int(1) table GlobalRequestsTracker
http-request deny deny_status 429 if { sc0_http_req_rate(GlobalRequestsTracker),div(nbsrv(MyBackend)) gt 10 }
But I'm getting the error:
error detected while parsing an 'http-request deny' condition : invalid args in converter 'div' : expects an integer or a variable name in ACL expression 'sc0_http_req_rate(GlobalRequestsTracker),div(nbsrv(MyBackend))'.
Is there a way to use nbsrv as a variable inside the div operator?
HAProxy does no allow for nested function calls as far as I know. But you could store the number of backend servers in a variable and use it in the division (see http-request set-var in the HAProxy documentation). I have not tested it or used personaly, but I guess it could look like:
frontend <fe>
http-request track-sc0 int(1) table <tbl>
http-request set-var(req.<var>) nbsrv(<be>)
http-request deny deny_status <code> if { sc0_http_req_rate(<tbl>),div(req.<var>) gt <val> }
See the HAProxy documentation.

HAProxy frontend rule matching order

I have a haproxy configuration as follows. (haproxy 1.7) We want to catch all OPTIONS request and respond directly to them instead of routing the requests to backends (which have basic auth enabled).
This was working fine when we developed it but now it seems to not be matching the rules in order (not sure what we have/haven't done which has caused this):
global
log 127.0.0.1 local1
tune.ssl.default-dh-param 2048
lua-load /etc/haproxy/cors.lua
stats socket /var/run/haproxy.sock mode 400
# Default certificate and key directories
ca-base /etc/ssl/private
crt-base /etc/ssl/private
# User lists used to enforce HTTP Basic Authentication
userlist ul_100123-2ovt9rsu
user app1 password $6$lCjf6VnWhI$kcjmpWdV.odeYf4psUhcVKs49ZtPk3MDhg5wtLNUx658A3EWdDHJQqs9xCD1d.7zG05M2nwOxdkC6o/MSpifv0
userlist ul_100123-9uvsclqr
user app1 password $6$DlcLoDMMu$wDm3O0W1eiQuk8gI.GmpzI1.jbBf.UYQ.KM73nHa1tGZJNfzkDpVnLUhh7v7C9yPHB1oo0cRrFnfOdeyAf/eU1
# Front-end for public services which have SSL termination at the router.
frontend term
bind *:443 accept-proxy ssl no-sslv3 crt router/fred-external.pem crt router/fred-external.ace.pem crt router
reqadd X-Forwarded-Proto:\ https
rspidel ^(Server|X-Powered-By):
option forwardfor
mode http
http-request use-service lua.cors-response if METH_OPTIONS { req.hdr(origin) -m found }
acl host_match_100123-2ovt9rsu ssl_fc_sni -i 2ovt9rsu.fredurl.com
use_backend b_term_100123-2ovt9rsu if host_match_100123-2ovt9rsu
......
If I curl -X OPTIONS to 2ovt9rsu.fredurl.com it matches the 2nd rule and forwards me to the b_term_100123-2ovt9rsu backend which then fails as I haven't provided auth creds.
If I curl -X OPTIONS to Anything.fredurl.com it matches the first http-request and responds with the cors response as expected.
Why does the 2ovt9rsu.fredurl.com not match the first http-request rule and then return the cors-response?
In the logs we can see
Nov 7 18:24:09 localhost haproxy[37302]: 94.45.23.22:49853 [07/Nov/2017:18:24:09.807] term~ b_term_100123-2ovt9rsu/<lua.cors-response> -1/-1/-1/-1/73 401 249 - - PR-- 0/0/0/0/3 0/0 "OPTIONS / HTTP/1.1"
when the request gets forwarded to the backend
http-request gets executed before use_backend, the config looks good to me, have you set origin header when you curl ?

Setting a unique http request id with haproxy's http-request set-header

So I have some existing code that sets a unique request ID in our front-end load balancer:
unique-id-format %{+X}o\ %Ts_%ci_%cp_%fi_%fp_%rt_%pid
unique-id-header X-Request-Id
log-format %ci\ %r\ %ST\ %B\ %Tr\ %Tt\ %s\ %ID\ %hr
This works as expected -- X-Request-Id is created as expected, logged and passed to the backend. No problems. However, I'd like to make this request ID generation conditional. No problem -- I should just be able to use http-request set-header instead of unique-id-header:
unique-id-format %{+X}o\ %Ts_%ci_%cp_%fi_%fp_%rt_%pid
http-request set-header X-Request-Id %ID
log-format %ci\ %r\ %ST\ %B\ %Tr\ %Tt\ %s\ %ID\ %hr
(These are all in a front_end section). Maddeningly, however %ID seems to evaluate to empty when used this way. I can use a hardcoded value instead of using %ID and it works. I can also use another log field (like %Ts) and it works. It does not, however, work with %ID. Any clues would be helpful -- thanks in advance.
EDIT: Version is 1.6.11
I had the exact same issue in that I wanted to conditionally set the header if not present and %ID wasn't working as you'd expect. I found solutions suggesting to use %[unique-id] but it turns out that's only in version 1.7+. I have subsequently upgraded to 1.7 and now it works perfectly.
unique-id-format %{+X}o\ %pid%ci%cp%fi%fp%Ts%ms%rt
acl cid_exists req.hdr(X-Correlation-ID) -m found
http-request set-header X-Correlation-ID %[unique-id] unless cid_exists
http-request capture hdr(X-Correlation-ID) len 64
log-format "%ci:%cp [%tr] %ft %b/%s %Th/%Ti/%TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %{+Q}r %[capture.req.hdr(0)]"
The captured X-Correlation-ID header contains either a preexisting CID or one that this loadbalancer has created itself if missing.
The unique-id HTTP sample is referenced here.

How to fix an improper request in HAProxy

We have several (100+) clients in the field with a bug in the HTTP request. The request was previously working when directly routed to our Windows Server, but now with it fails with HAProxy v1.7 in front of it.
Here is an example request:
GET /index.aspx HTTP/1.1 \nHost: host\n\n
There is an extra space after the HTTP version before the \n.
Here is a snapshot of the relevant config.
frontend http_port_80
bind :80
mode http
reqrep (.)\ HTTP/1.1\ (.*) \1\ HTTP/1.1\2
option forwardfor
option accept-invalid-http-request
stats enable
use_backend cert_update if is_updater
use_backend getConsoleHTTP if is_getconsole
default_backend schedule_server
I have tried rewriting the request to remove the extra space and set the option accept-invalid-http-request to address the issue, but we still receive the same error.
{
type: haproxy,
timestamp: 1506545591,
termination_state: PR-,
http_status:400,
http_request:,
http_version:,
remote_addr:192.168.1.1,
bytes_read:187,
upstream_addr:-,
backend_name:http_port_80,
retries:0,
bytes_uploaded:92,
upstream_response_time:-1,
upstream_connect_time:-1,
session_duration:2382,
termination_state:PR
}
Does anyone have any ideas of how to fix the malformed request prior to haproxy rejecting it?