I am trying to implement rate limiting where specific clients rates can be limited to certain paths using a map file. The relevant configuration is:
frontend fe_svc_a_api
...
log global
log-format "[%t] %{Q}[var(txn.rate_limit)] %{Q}[var(txn.request_rate)] %[sc_http_req_rate(0)]"
acl req_limit var(txn.rate_limit),sub(txn.request_rate) lt 0
http-request set-var(txn.rate_limit) path,map_beg(/opt/haproxy/maps/svc_a_api.map,10)
http-request set-var(txn.request_rate) sc_http_req_rate(0)
http-request set-var(txn.client_ip) req.hdr(X-Real-IP)
http-request track-sc0 var(txn.client_ip) table table_svc_a_rate_limiter
use_backend be_429_5s if req_limit
...
backend table_svc_a_rate_limiter
stick-table type string len 50 size 10K expire 5s store http_req_rate(5s)
backend be_429_5s
mode http
http-request tarpit deny_status 429
timeout tarpit 5s
With this configuration the %{+Q}[var(txn.rate_limit)] and %[sc_http_req_rate(0)] show correct numbers but %{+Q}[var(txn.request_rate)] is “-” and the whole limiting fails as the subtraction never is “less than 0”.
I can’t think of anything other than this line:
http-request set-var(txn.request_rate) sc_http_req_rate(0)
being somehow wrong. What am I doing wrong? What is the proper way of setting a txn variable value to stick-table tracked counter value?
Also - I am actually using a little more complicated stick-table key, but as it is properly populated I don’t think that could have any impact. Another thing is that I have compiled HAProxy (using 2.5.1) with DEFINE="-DMAX_SESS_STKCTR=100"
Related
I need to extra the req.uri from a request in my frontend in haproxy. Here is what my haproxy looks like
frontend fe_ingress
...
http-request set-var(req.uri) path
http-request add-header endpoint %[var(req.uri)]
so I need to extract the req.uri and add it as a header to the subsequent request. Right now, it has / in its beginning but I need to remove the first /. How can I do this?
You can use the regsub filter to modify the value with a regular expression search-and-replace when setting the header. This can look like this:
http-request add-header endpoint %[path,regsub(^/,)]
We need to add a header to incoming requests processed by HAproxy. However, each header needs to be composed of uri elements.
If the request is "http://myserver/system/apple/watch"
the header needs to be "Host:applewatch.com" where .com is static.
I am aware of "set-header" command, I just need to extract the second and the third URI elements via REGEX, string them together, and add static field .com.
Is there a way to save a URI element to a variable via regex and then reuse this variable as a header part?
Thanks you.
You can do something like this:
http-request set-header ASDF %[path,word(2,/)]%[path,word(3,/)].com
or
http-request set-header ASDF %[path,regsub(^/system/,''),regsub(/,''),regsub($,'.com')]
I understand that I can use url_param / urlp to extract the query parameters from the URL that is requested, in HAProxy.
However, I need similar function for extracting parameters from the URL sent as HTTP Header field Referer. I guess url_param is only available for the requested URL, and not possible to use for HTTP Header values? If so, what other options do I have? I need to retrieve the value from query parameter and send it as specific HTTP Header to the backend server.
Sharing my solution (although Im not sure this is the most efficient and accurate way). I solved it with Regex.
# Example HTTP Referer: http://myexample.com/users?user-id=12345
# ACL
acl is_uid_in_hdr_referer hdr_sub(Referer) -i user-id
# Set value from query param "user-id" from Referer header to custom header "user-id"
http-request set-header user-id %[req.hdr(Referer),regsub(.+?user-id=,,g)] if is_uid_in_hdr_referer
I'd like to have a throttling rule in HAProxy that limits rate at which a user can load any particular path, but I don't know of a way to concatenate strings in HAProxy (at least, in the context of generating a key for a stick table). So what I'd like is
tcp-request content track-sc1 concat(req.cook(user), path)
tcp-request content reject if {sc1_http_req_rate gt 10}
HAProxy manipulate string prior to map lookup suggests using regsub to do something somewhat similar, but I think I can only do constant manipulations with that.
The best I've come up with so far is to track path and req.cook(user) separately, and to reject if each of them is too high, but this isn't the actual behavior that I'm looking for.
The link in your question has the answer. You concat in set-header first and then use that.
http-request set-header X-Concat %[req.cook(user)]__%[path]
http-request track-sc0 hdr(X-Concat) table peruser
Please bear with me as I am not a coder by nature.
This is what I trying to achieve using HAproxy but after hours of checking, I am unable to make it works somehow.
From
domain.com/alpha
domain.com/beta
To
domain.com/alpha will point to backend1/path/index.cgi
domain.com/beta will point to backend2/path/index.cgi
I have tried, multiple ways but come to no avail, I did read about rewrite/redirect but somehow it makes me very confuse. eg "reqrep"
by using the alpha.domain.com points to backend1/path works as expected but I need the path inline because of certificate limitation.
Thank you in advance and if possible explain abit how it works and whats the correct terms (example: rewrite, redirect) so that I can have clue on that and I will advance from there.
This is what I was able to come up with:
frontend HTTP
mode http
bind *:80
acl alpha url_beg /alpha
acl beta url_beg /beta
use_backend backend_alpha if alpha
use_backend backend_beta if beta
backend backend_alpha
reqrep ^([^\ ]*\ /)alpha[/]?(.*) \1path/index.cgi
server server_alpha localhost:8080
backend backend_beta
reqrep ^([^\ ]*\ /)beta[/]?(.*) \1path/index.cgi
server server_beta localhost:8081
Obviously you would then replace localhost:8080 and localhost:8081 with the correct locations for your case.
Explanation
First, in the frontend named HTTP there are two ACLs (Access Control Lists) which test what is in the beginning of the URL (hence the keyword url_beg). The result of these rules are if the url begins with /alpha then the variable called alpha is set to true and then the same for beta.
Next in the frontend, there are two use_backend commands which will direct the request to backend_alpha if the variable alpha is set to true and the same for backend_beta if beta is set to true.
As a result the frontend does the work of taking the url and deciding which server to use.
The two backends (backend_alpha and backend_beta) are almost identical except for the text alpha and beta and the location of the respective servers. The first command in the backend is the reqrep command which you pointed out. What the reqrep command does is takes a url, searches for a particular part using Regular Expression and then replaces it with something else. In this case we have this url:
http://example.com/alpha
In the first part of the reqrep command:
^([^\ ]*\ /) takes http://example.com/ and stores it in a variable called \1
alpha then matches with the alpha in the given url
[/]?(.*) takes everything after alpha and stores it in a variable called
\2 (in this case \2 would equal nothing as there is nothing after alpha in the url)
Then the second part of the reqrep command says take the contents of \1 (http://example.com/) and add path/index.cgi to the end and make that the new url to send to the server.
As a result for both alpha and beta urls the resulting url sent to the server is http://example.com/path/index.cgi.
Finally the server command sends the request of to the appropriate server.
I would like to point out I am not an expert on the complicated Regular Expression part (I find it a bit confusing as well) but hopefully someone else who knows a bit more can explain it in more detail or correct me if I am wrong.
I hope that helps :)