I'd like to have a throttling rule in HAProxy that limits rate at which a user can load any particular path, but I don't know of a way to concatenate strings in HAProxy (at least, in the context of generating a key for a stick table). So what I'd like is
tcp-request content track-sc1 concat(req.cook(user), path)
tcp-request content reject if {sc1_http_req_rate gt 10}
HAProxy manipulate string prior to map lookup suggests using regsub to do something somewhat similar, but I think I can only do constant manipulations with that.
The best I've come up with so far is to track path and req.cook(user) separately, and to reject if each of them is too high, but this isn't the actual behavior that I'm looking for.
The link in your question has the answer. You concat in set-header first and then use that.
http-request set-header X-Concat %[req.cook(user)]__%[path]
http-request track-sc0 hdr(X-Concat) table peruser
Related
I am trying to implement rate limiting where specific clients rates can be limited to certain paths using a map file. The relevant configuration is:
frontend fe_svc_a_api
...
log global
log-format "[%t] %{Q}[var(txn.rate_limit)] %{Q}[var(txn.request_rate)] %[sc_http_req_rate(0)]"
acl req_limit var(txn.rate_limit),sub(txn.request_rate) lt 0
http-request set-var(txn.rate_limit) path,map_beg(/opt/haproxy/maps/svc_a_api.map,10)
http-request set-var(txn.request_rate) sc_http_req_rate(0)
http-request set-var(txn.client_ip) req.hdr(X-Real-IP)
http-request track-sc0 var(txn.client_ip) table table_svc_a_rate_limiter
use_backend be_429_5s if req_limit
...
backend table_svc_a_rate_limiter
stick-table type string len 50 size 10K expire 5s store http_req_rate(5s)
backend be_429_5s
mode http
http-request tarpit deny_status 429
timeout tarpit 5s
With this configuration the %{+Q}[var(txn.rate_limit)] and %[sc_http_req_rate(0)] show correct numbers but %{+Q}[var(txn.request_rate)] is “-” and the whole limiting fails as the subtraction never is “less than 0”.
I can’t think of anything other than this line:
http-request set-var(txn.request_rate) sc_http_req_rate(0)
being somehow wrong. What am I doing wrong? What is the proper way of setting a txn variable value to stick-table tracked counter value?
Also - I am actually using a little more complicated stick-table key, but as it is properly populated I don’t think that could have any impact. Another thing is that I have compiled HAProxy (using 2.5.1) with DEFINE="-DMAX_SESS_STKCTR=100"
Trying to do a rewrite and redirect. I've been trying this, it works to some extent but not 100% what I want it to do
acl old url_beg /site/ab
http-request redirect location /new/%[query] if old
the url can be for example
https://host/site/ab/xx
https://host/site/ab/yyyy
https://host/site/ab/zzzzzz
https://host/site/ab/zzzzzz/asdajshdjasd
I am looking to grab the bold marked part and simply redirect the user to https//host/new/boldmarkedpart
Any string that comes after the bold marked part can be trashed. For example "/asdajshdjasd" in the last example.
Any idea how to accomplish this? Thank you!!
If i understand correctly, you want to split the path part of url and get its 4th part.
In string foo1/foo2/foo3/foo4/foo5 you want only foo4.
This should work for you:
acl old path_beg /site/ab
http-request redirect location /new/%[path,field(4,/)] if old
It may be confusing that you want 3rd directory from path and here you take 4th word, but that's because when you split /foo2/foo3/foo4/foo5 by / then the first word is empty.
field converter is documented here: https://cbonte.github.io/haproxy-dconv/2.2/configuration.html#7.3.1-field
Other notes:
%[query] would return the query part of url, which is everything after ? character and you don't have query part at all in your examples.
url in my tests had schema://hostname:port/path, so testing acl old url_beg /site/ab never matched, path is for that
We need to add a header to incoming requests processed by HAproxy. However, each header needs to be composed of uri elements.
If the request is "http://myserver/system/apple/watch"
the header needs to be "Host:applewatch.com" where .com is static.
I am aware of "set-header" command, I just need to extract the second and the third URI elements via REGEX, string them together, and add static field .com.
Is there a way to save a URI element to a variable via regex and then reuse this variable as a header part?
Thanks you.
You can do something like this:
http-request set-header ASDF %[path,word(2,/)]%[path,word(3,/)].com
or
http-request set-header ASDF %[path,regsub(^/system/,''),regsub(/,''),regsub($,'.com')]
What is the neatest way to redirect these
http://example.com/test to http://example.com/test/
(It should work no matter what "test" is). I'm new to HAProxy - I have different apache backends. Have it working for URLs of this format http://example.com/test/ but cannot get the other type to work.
Have tried:
(1)
http-request set-path %[path]/ unless { path_end / }
which doesn't load the html page properly - possibly because it's screwing up e.g. the referenced JS files.
(2)
http-request redirect code 301 prefix / drop-query append-slash if missing_slash
from the documentation. I figure I need some kind of slight adjustment to this but don't know where to start with it. Any hints would be much appreciated.
This would potentially best be done at the back-end, since only the back-end has a way to actually know which paths should be redirected. But, it should be possible from HAProxy.
It should work no matter what "test" is
I am skeptical that this is precisely what you need, because then (e.g.) /css/common.css becomes /css/common.css/, which would be wrong.
I'm inclined to think you want something like this:
http-request redirect code 301 prefix / drop-query append-slash if { path_reg /[^/\.]+$ }
Redirect to the same path, with the query removed and / appended to the end, if the path ends with a / followed by at least 1 (or more) characters that is neither / nor ..
This should redirect /test to /test/ and /hello/world to /hello/world/ but it should leave paths like /js/example.js and /images/cat.png and /favicon.ico unaffected.
Deeper nesting should work correctly, too, because regular expressions like this either find a match or don't, and this expression only considers whatever is after the final slash in the path. If it contains a dot or nothing, then there is no match, which seems correct.
Please bear with me as I am not a coder by nature.
This is what I trying to achieve using HAproxy but after hours of checking, I am unable to make it works somehow.
From
domain.com/alpha
domain.com/beta
To
domain.com/alpha will point to backend1/path/index.cgi
domain.com/beta will point to backend2/path/index.cgi
I have tried, multiple ways but come to no avail, I did read about rewrite/redirect but somehow it makes me very confuse. eg "reqrep"
by using the alpha.domain.com points to backend1/path works as expected but I need the path inline because of certificate limitation.
Thank you in advance and if possible explain abit how it works and whats the correct terms (example: rewrite, redirect) so that I can have clue on that and I will advance from there.
This is what I was able to come up with:
frontend HTTP
mode http
bind *:80
acl alpha url_beg /alpha
acl beta url_beg /beta
use_backend backend_alpha if alpha
use_backend backend_beta if beta
backend backend_alpha
reqrep ^([^\ ]*\ /)alpha[/]?(.*) \1path/index.cgi
server server_alpha localhost:8080
backend backend_beta
reqrep ^([^\ ]*\ /)beta[/]?(.*) \1path/index.cgi
server server_beta localhost:8081
Obviously you would then replace localhost:8080 and localhost:8081 with the correct locations for your case.
Explanation
First, in the frontend named HTTP there are two ACLs (Access Control Lists) which test what is in the beginning of the URL (hence the keyword url_beg). The result of these rules are if the url begins with /alpha then the variable called alpha is set to true and then the same for beta.
Next in the frontend, there are two use_backend commands which will direct the request to backend_alpha if the variable alpha is set to true and the same for backend_beta if beta is set to true.
As a result the frontend does the work of taking the url and deciding which server to use.
The two backends (backend_alpha and backend_beta) are almost identical except for the text alpha and beta and the location of the respective servers. The first command in the backend is the reqrep command which you pointed out. What the reqrep command does is takes a url, searches for a particular part using Regular Expression and then replaces it with something else. In this case we have this url:
http://example.com/alpha
In the first part of the reqrep command:
^([^\ ]*\ /) takes http://example.com/ and stores it in a variable called \1
alpha then matches with the alpha in the given url
[/]?(.*) takes everything after alpha and stores it in a variable called
\2 (in this case \2 would equal nothing as there is nothing after alpha in the url)
Then the second part of the reqrep command says take the contents of \1 (http://example.com/) and add path/index.cgi to the end and make that the new url to send to the server.
As a result for both alpha and beta urls the resulting url sent to the server is http://example.com/path/index.cgi.
Finally the server command sends the request of to the appropriate server.
I would like to point out I am not an expert on the complicated Regular Expression part (I find it a bit confusing as well) but hopefully someone else who knows a bit more can explain it in more detail or correct me if I am wrong.
I hope that helps :)