Following HAProxy documentation I added the following to my configuration:
unique-id-format %{+X}o\ %ci:%cp_%fi:%fp_%Ts_%rt:%pid
unique-id-header X-Unique-ID
It works fine but I would also to add the X-Unique-ID to the responses.
I tried with
http-response set-header X-Unique-ID "%ID"
but it doesn't seem to work. Is it possible to add the unique request ID to the response headers and if yes what is the proper way to do it?
PS: It wasn't super clear in my original question but I would like the ID added to the response to be the same than the ID generated for the request that triggered this response. This way I can more easily regroup requests and their responses.
You can try to use http-after-response set-header
http-after-response set-header X-Unique-ID "%ID"
As you have not added the output of haproxy -vv I assume you use the latest haproxy version
Related
I have a mutual-TLS setup with HAProxy terminating incoming SSL connections. I need to perform client certificates validation on the backend, not on haproxy side since we have a dynamic truststore and I cannot just set a single ca-file and delegate all the validation logic to haproxy.
Regular setup works fine (where haproxy validates a client cert against CA cert):
bind *:443 ssl crt /etc/certs/haproxy.pem verify required ca-file /etc/certs/ca.crt
http-request redirect scheme https unless { ssl_fc }
http-request set-header X-SSL-ClientCert %{+Q}[ssl_c_der,base64]
Backend receives X-SSL-ClientCert correctly, but this is not enough.
If verify required ca-file /etc/certs/ca.crt is removed to skip validation on haproxy, X-SSL-ClientCert is empty when read on the backend, that is, HAProxy does not set the header with the client certificate anymore. Any ideas on how to fix this?
Okay, after some tinkering I stumbled upon the following:
HAProxy documentation https://cbonte.github.io/haproxy-dconv/2.3/configuration.html#5.2-verify states two options for verify: [none|required], but it seems there is an undocumented option, namely, optional which seems to do the trick:
bind *:443 ssl crt /etc/certs/haproxy.pem verify optional ca-file /etc/certs/ca.crt
http-request set-header X-SSL-Client-Cert %{+Q}[ssl_c_der,base64]
http-request set-header X-SSL-Client-CN %{+Q}[ssl_c_s_dn(cn)]
http-request set-header X-SSL-Client-Verify %[ssl_c_verify]
You do have the right thought.
On the other hand, I think that there is a more readable writing.
In your /etc/hapee/hapee-lb.cfg
bind :443 ssl crt-list /etc/hapee/certs-file
In your /etc/hapee/certs-file
cert_example1.pem [verify optional ca-file /etc/hapee/certs/ca.crt]
cert_example2.pem
So you can easily list all your certificates and possibly the verify options according to your SNI.
Backend who match
backend
http-request set-header X-SSL-Client-DN %[ssl_c_s_dn]
http-request set-header X-SSL-Client-Cert %{+Q}[ssl_c_der,base64]
http-request set-header X-SSL-Client-CN %{+Q}[ssl_c_s_dn(cn)]
http-request set-header X-SSL-Client-Verify %[ssl_c_verify]
We have deprecated some software which was available under http://localhost/test and is now available under http://localhost/app/testing. We are using HAProxy 1.4
Thus, I want via haproxy, to replace all urls containing /test with /app/testing/. I tried first with a redirect prefix in order to keep the query string, but /test wasn't removed from the url and had something like /app/testing/test/?id=x.
frontend all
bind 0.0.0.0:80
timeout client 86400000
acl is_test path_beg /test
redirect prefix /app/testing code 301 if is_test
Then used a reqrep, which seems to redirect to the new software, but the /test string in the url is never replaced.
frontend all
bind 0.0.0.0:80
timeout client 86400000
reqrep ^([^\ :]*)\ /test[/]?(.*) \1\ /app/testing/\2
Since url rewriting isn't possible with version 1.4 and we didn't want to update HAProxy, we went on using reqrep and keeping the old link as is with
reqrep ^([^\ :]*)\ /test[/]?(.*) \1\ /app/testing/\2
Try this
This works for me with your scenario on HAProxy 1.6
acl test path_beg -i /test
http-request set-header X-Location-Path %[capture.req.uri] if test
http-request replace-header X-Location-Path /test /app/testing if test
http-request redirect location %[hdr(X-Location-Path)] if test
use_backend WHEREVER if test
Using redirect prefix is meant, going by the HAProxy examples, more for changing the hostname, but keeping the path. For example, http://apple.com/my/path could be redirected to http://orange.com/my/path with:
redirect prefix http://orange.com if apple
The HAProxy 1.4 docs say:
With "redirect prefix", the "Location" header is built from the
concatenation of < pfx > and the complete URI path
To me, that suggests that you would expect whatever you put for the "prefix" will be prefixed to what was already in the path. That explains the behavior you were seeing. This is useful for changing to a new domain (e.g. from apple.com to orange.com), but keeping the original path (e.g. /my/path).
You can switch to using redirect location to replace the entire URL. The following would redirect http://apple.com/my/path to http://orange.com/path:
redirect location http://orange.com/path if apple
UPDATE:
In the case where you want to change the URL, but keep the query string, use reqirep or reqrep to rewrite the URL, as you are doing, but also put a redirect prefix into the frontend. The URL will be rewritten and then the user will be redirected to it so they see it.
You might be able to set the "prefix" to "/". The HAProxy docs say:
As a special case, if < pfx > equals exactly "/", then nothing is
inserted before the original URI. It allows one to redirect to the
same URL (for instance, to insert a cookie).
Using your code example, something like this:
reqrep ^([^\ :]*)\ /test[/]?(.*) \1\ /app/testing/\2
redirect prefix / code 301 if is_test
How can I route requests in haproxy using a cookie that was set on the app servers?
Example: SESS=<hash-of-username>
haproxy should not insert cookies by itself in any case.
For testing a specific server behind haproxy I can recommend this approach:
frontend http
acl is_cookie_hack_1 hdr_sub(cookie) server_test_hack=server1
acl is_cookie_hack_2 hdr_sub(cookie) server_test_hack=server2
... insert your normal acl rules here
use_backend bk_server_1 if is_cookie_hack_1
use_backend bk_server_2 if is_cookie_hack_2
... insert your normal use_backend expressions here
backend bk_server_1
...
backend bk_server_2
...
I insert the server_test_hack cookie by javascript in my browser's js console by this script:
document.cookie="server_test_hack=server1";
You can't use your existing cookie for balancing, the way you could use the URI parameter. You can't just take the md5() or build the hash table of the cookie, at least that is not documented. You could use prefix parameter for the cookie to achieve a different result. It might be what you are looking for (if you want to avoid creation of yet another cookie).
So in your case the config would look like this:
backend bk_web
balance roundrobin
cookie SESS prefix indirect nocache
server s1 192.168.10.11:80 check cookie s1
server s2 192.168.10.21:80 check cookie s2
When the request arrives without a cookie, any server is chosen by round-robin and request is redirected to it. When response arrives from the backend, HAProxy checks for the SESS cookie and if it's set, it prepends the server name (sX) to the cookie and sends it to the client. In the browser, the cookie looks like sX~, but when the next request is sent with that cookie, the backend server only sees in the cookie, as HAProxy strips the sX~ part
Source: load balancing, affinity, persistence, sticky sessions: what you need to know
If you just want to read cookies in the request and route accordingly, you can do something like this in your configuration:
frontend http
acl cookie_found hdr_sub(cookie) COOKIENAME
use_backend app_server if cookie_found
backend app_server
balance roundrobin
server channel1 X.X.X.X:PORT #Host1
server channel2 Y.Y.Y.Y:PORT #Host2
Is is possible force HTTPS URLs even when the X-Forwarded-Host header is not present?
Update:
We are using HAProxy in front of the Neo4j server. The configuration is
frontend proxy-ssl
bind 0.0.0.0:1591 ssl crt /etc/haproxy/server.pem
reqadd X-Forwarded-Proto:\ https
default_backend neo-1
This works well when every connection contains only one request. However, for Neo4j drivers which uses keep-alive (like Py2neo), the header is added only to the first request.
Without the X-Forwarded-Proto header, the generated URLs are http://host:1591, instead of https://host:1591.
According to the HAProxy documentation, this is the normal behavior:
since HAProxy's HTTP engine does not support keep-alive, only headers
passed during the first request of a TCP session will be seen. All subsequent
headers will be considered data only and not analyzed. Furthermore, HAProxy
never touches data contents, it stops analysis at the end of headers.
The workaround is to add option http-server-close in the frontend, so it will force that every request is in its own connection, but it will be nicer if we can support keep-alive.
Put something like Apache or Nginx in front of your Neo4j server to perform that task.
In terms of py2neo, I can add some functionality to cater for this situation quite easily. What if I were to include X-Forwarded-Proto: https for all https connections? Would that cause a problem in cases where a proxy isn't used?
here is my config file
listen solr 0.0.0.0:8983
mode http
balance roundrobin
option httpchk GET "/solr/select/?q=id:1234" HTTP/1.1
server solr_slave 1.1.1.1:8983 maxconn 5000 weight 256 check
server solr_master 2.2.2.2:8983 maxconn 5000 weight 1 check
the problem is that my solr server is protected using basic http password authentication and hence the health check fails always
how do i tell haproxy to use those credentials during the health checks?
A little late, but I just came across the same problem, and wanted to share the solution with the world. First, you base64-encode the credentials:
$ echo -n "user:pass" | base64
dXNlcjpwYXNz
(Make sure you use the -n switch so you don't append a newline.)
The option httpchk allows you to add arbitrary HTTP headers to the request; this feature isn't documented very well. (According to this discussion, future versions of Haproxy might get a more user-friendly method.) To use basic authentication:
option httpchk GET /solr/ HTTP/1.0\r\nAuthorization:\ Basic\ dXNlcjpwYXNz
Note that I used HTTP 1.0; for 1.1, you also need a Host header.