This is kind of a weird complicated question.
Context:
I have a bunch of docker containers that need to be routed to from haproxy dynamically. They are each running on different ports on the machine, and are stored in environment variables like this:
a=9873
b=9874
c=9875
These are available to the haproxy server. The request path that comes in will be in the form like this example:
/api/a/action
From that, the taks is as follows:
The /api needs to be removed from the path.
The /a refers to the service, so the environment variable for a needs to be retrieved to get the port of the server
The request needs to be routed to localhost:9873/a/action where the port, 9873, is the environment variable that is the value from the path in the beginning (after removing /api) and then the path is simply appended onto the request (which is the /a/action that remains after removing the /api.
My current config looks like this:
backend api
reqrep ^([^\ ]*\ /)api[/]?(.*) \1\2
server api_server localhost:9871
All this config is doing is removing the /api from the path of the request and sending it to a static port, 9871. *I need this port to be the value held by the environment variable of the same name as the first element in the path (the /a above) and the rest (passing the remaining path) is already working.*
I also would like to be able to get the environment variable of the name prefix_a, where the path will have the name /a, but I need to prepend one common prefix prefix_ to get the environment variable. This can be a separate question or search though, unless it's simple to just put that into the solution.
Please let me know if I can clarify or give more information that might help solve the problem.
(I've done a heck a lot of googling. Here are some related urls but not quite the answer I need:
https://gist.github.com/meineerde/8bea63e64fc47f9a67c0
Dynamic routing to backend based on context path in HAProxy
How can I set up HAProxy to a backend based on a value in the url?
Haproxy route and rewrite based on URI path
haproxy: get the host name
https://serverfault.com/questions/818937/haproxy-is-giving-me-problems-with-regex-replace-is-this-a-bug-or-am-i-doing-so
https://serverfault.com/questions/668025/how-to-use-environment-variable-in-haproxy
http://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.2
How do I set a dynamic variable in HAProxy?
use environment variables in haproxy
Related
I am looking at the stackstorm docker-compose file, and within it almost all containers have a line dns_search: . According to docker-compose documentation, dns_search is for the purpose of configuring search domains.
I am used to seeing this in context of transparently adding a domain to unqualified short domains. For example if I add dns_search: mydomain.com, I would expect "host1" to transparently resolve as "host1.mydomain.com".
I have never seen this set as a single dot . before. What is the effect/purpose of doing this configuration?
I'm posting the answer from the Stackstorm Git project issue see comment/"dns_search: .". Paraphrasing: it was useful in old versions of Docker before 2017, before the ndots configuration was available. Nowadays that configuration has no impact, and in fact has been removed from the stackstorm docker-compose file.
I believe this is because all domain names end in . under the hood, but browsers and other software abstracts this out.
For example. under the hood www.google.com is actually www.google.com.
So, in the docker-compose file, this would essentially be saying "Find me any domain"
A bit more detail on why there's an extra dot, if you're interested:
Domain name resolution is heirachical, reading right to left, with each block, separated by a ., being a step in the process. A DNS resolver will first find a source of ., which will be able to return the address for a resolver for the next block, until it reaches the final block, where it returns the full DNS record.
Extending EdwardTeach's answer:
#ytjohn effectively said they did in the past because putting dns_search: . configures the DNS search domains to be only . instead of inheriting the host ones. I can't confirm that because I didn't test it.
Now, I tested what docker-compose does today, and in a container, cat /etc/resolve.conf returns:
nameserver 127.0.0.11
options ndots:0
Where options ndots:0 is (from resolv.conf docs):
ndots:n
Sets a threshold for the number of dots which must
appear in a name given to res_query(3) (see
resolver(3)) before an initial absolute query will
be made. The default for n is 1, meaning that if
there are any dots in a name, the name will be
tried first as an absolute name before any search
list elements are appended to it. The value for
this option is silently capped to 15.
With ndots:0, all domains will be attempted using the absolute name first, only then using the search list.
How I came to this conclusion
The Github comment:
If you don't set this dns_search: ., then whatever the host has in search in their /etc/resolv.conf will get put into your container's /etc/resolv.conf.
This doesn't happen. My host has search domain[0]: broadband (macOS command: scutil --dns), and in docker containers, it doesn't show broadband (linux command: cat /etc/resolv.conf). Instead, it says options ndots:0
dns_search docs:
dns defines custom DNS search domains to set on container network interface configuration. Can be a single value or a list.
What is a DNS search domain?
It is the DNS service used to resolve hostnames that are not fully qualified, e.g. hostname will try hostname.example.com then hostname.website.com if your search domains list was example.com, website.com. More information on https://superuser.com/a/184366
In another repo (crossdock), their dockerfile had the comment:
`dns_search: . # Ensures unified DNS config.`
I'm using Jmeter for testing APIs and I want to parametrize the project's path from the terminal and then I want to use this parameter in JMeter.
The parameter that I've sent via Command Line :
./jmeter -n -t your_script.jmx -Jurl=abcdef.com
The parameter that I've used in User Defined Variables :
${__P(url)}
But when I run my automation in JMeter, my test scripts are not going to URL that's been defined. When I check the request body, I see POST https://1 as URL.
Please see the attached photos.
https://mylifebox.com/shr/3df5bb35-cf43-4488-b20b-5c2d59656212&language=en
Let's start clean:
In the User Defined Variables configure the variable with the name of url and the value of ${__P(url,)}
In the HTTP Request sampler (or even better HTTP Request Defaults) put ${url} into "Server Name or IP" field:
Run your test in command-line non-GUI mode like:
jmeter -n -t your_script.jmx -Jurl=abcdef.com -f -l result.jtl
mind this -f argument which tells JMeter to overwrite the existing results file (it might be the case you're looking into "old" results where the url property value was starting with 1)
That's it, you should see the HTTP Request sampler making a call to abcdef.com in the .jtl results file. And if you change this url parameter - you will see the impact in the .jtl results file:
Put ${__P(url)} inside Server Name field in HTTP Request.
Domain name or IP address of the web server, e.g. www.example.com. [Do not include the http:// prefix.] Note: If the "Host" header is defined in a Header Manager, then this will be used as the virtual host name.
Don't use User Defined Variables
I am trying to start a google compute engine (GCE) instance with a pre-configured FQDN. We are intending to run an application that is licensed based on the contents of /etc/hosts.
I am starting the instances using the Google Cloud SDK utility - gcloud.
I have tried setting the "hostname" key using the metadata option like so:
gcloud compute instances create mynode (standard opts) --metadata hostname=mynode.example.com
Whenever I log into the developer console, under computer, instances, I can see hostname under "Custom metadata". This appears to be a new, custome key - it has no impact on what:
http://metadata.google.internal/computeMetadata/v1/instance/hostname
returns.
I have also tried setting "instance/hostname" like the below, which causes a parsing error when using gcloud.
--metadata instance/hostname=mynode.example.com
I have successfully used the startup scripts functionality of the metadata server to run a startup script that parses the new, internal IP address of the newly created instance, updated /etc/hosts. This appears to work but doesn't feel "like the google way".
Can I configure the FQDN (specifically, a domain name, as the instance name is always the hostname) of an instance, during instance creation, using the metaserver functionality?
try this:
Go to your GCE >> VM instances panel.
stop your gce instance.
clic on the instance name.
Edit your instance, adding this values on Custom metadata fields:
Key field: hostname / Value field: your.server.hostname
Key field: startup-script / Value field: sudo -s hostnamectl set-hostname your.server.hostname
setup-example-image.png
Finally, start your instance and test with a hostnamectl command.
regards!
According to this article 'hostname' is part of the default metadata entries that provide information about your instance and it is NOT possible to manually edit any of the default metadata pairs. You can also take a look at this video from the Google Team. Within the first few minutes it is mentioned that you cannot modify default metadata pairs. As such, it does not seem like you can specify the hostname upon instance creation other than through the use of a start-up script like you've done already. It is also worth mentioning that the hostname you've specified will get deleted and auto-synced by the metadata server upon reboot unless you're using a start-up script or something that would modify it every time.
If what you're currently doing works for what you're trying to accomplish, it might be the only workaround to your scenario.
Here is a patch for /usr/share/google/set-hostname to set FQDN to GCE instance.
https://gist.github.com/yuki-takeichi/3080521322f0f1d159ea6a343e2323e6
Before you use this patch, you must set your desired FQDN in your instance's metadata by specifying hostname key.
Hostname is set each time instance's IP address is renewed by dhclient. set-hostname is just a hook script which dhclient executes and serves new IP address and internal hostame to, and modifies /etc/hosts. This patch changes the source of hostname by querying instance's metadata from metadata server.
The original set-hostname script is here:
https://github.com/GoogleCloudPlatform/compute-image-packages/blob/master/google_config/bin/set_hostname.
Use this patch at your own risk.
When creating a VM, you can specify a custom FQDN hostname as an optional parameter. This feature is currently in Beta.
$ gcloud beta compute instances create INSTANCE_NAME --hostname example.hostname
This should work across OSes, and eliminate the need for workaround scripts.
More info in the docs.
-- Sirui (Product Manager, Google Compute Engine)
I've looked throughout this site to find answered questions and found a few things that work but with a couple solutions combined. This thread seems the place to answer.
1) echo example.com > /etc/hostname
2) add -- 127.0.1.1 example.com in /etc/hosts
3) add -- hostnamectl set-hostname
example.com -- command to /etc/rc.local script
4) uncomment /etc/dhcp/dhclient.conf line:
supersede domain-name "example.com";
5) profit.... Seems to stick after each reboot
(Note example.com is your domain name: fqdndomain.com - yourfqdndomain.org)
Also note this is for Ubuntu or Debian. Other Unix May slightly vary. I've tested this on Ubuntu 16.04
Always on the wording NOT possible to manually edit any of the default metadata pairs, how about the instant level default metadata "/scheduling"? we could set them manually as mentioned in this article
I'm looking for a way to replace this on my ActiveMQ config:
<transportConnector uri="tcp://localhost:60019"> disableAsyncDispatch="false"/>
with a "not-hardcoded" URI (e.g., replacing "localhost" with a variable that resolves to an instance dependent value). The problem is that as we have many JBoss instances per server, and that URI above resolves to 0.0.0.0:60019, only one instance at a time can be running, unless we configure it in a per-application basis, which is not only frustrating, but there are circumstances where it is not enough (should be per-instance based, which is much more frustrating).
Each JBoss server has its own IP address, so I thought of using ${jboss.bind.address} to circumvent this, but it won't syntax. We also have an environment variable %SERVERIP% which could be used for this calling it from a start up script, but I don't know if ActiveMQ reads an environment variable for assigning its transport connector URI.
Any help would be much appreciated.
Use a PropertyPlaceHolderConfigurer and you should be able to replace the uri with some ${variable} from file or from jvm system variable. This should work since ActiveMQ configuration is really just a Spring context.
I'm running a classic style application in Sinatra and I would like to obtain a URL which the application is bound to. For example, if I start it in a development environment I would expect to get: http://localhost:4567/ while in production environment this would point to: http://example.com/
I know it is possible to retrieve it from the request. However, I need it in configuration block. How to do it?
Use the bind host and bind port method:
set :bind, 'example.com'
set :port, 80
should work.
taken from here. at the beginning of the page you can find how to implement them in you app (just copy it in front of your gets)