When watching kubernetes resources for changes, what exactly is happening under the hood? Does the http suddenly change to a wss connection?
To solve a problem of too many requests to the kube-apiserver I am rewriting some code to what I think is more of an operator pattern.
In our multi-tenant microservice architecture all services use the same library to look up connection details to tenant-specific DBs. The connection details are saved in secrets within the same namespace as the application. Every tenant DB has its own secret.
So on every call all secrets with the correct label are read and parsed for the necessary DB connection details. We have around 400 services/pods...
My idea: instead of reading all secrets on every call, create a cache and update the cache everytime a relevant secret was changed via a watcher.
My concerns: am I just replacing the http requests with equally expensive websockets? As I understand I will now have an open websocket connection for every service/pod, which still is 400 open connections.
Would it be better to have a proxy service to watch the secrets (kube-apiserver requests) and then all services query that service for connection details (intranet requests, kube-apiserver unreleated)?
From the sources:
// ServeHTTP serves a series of encoded events via HTTP with Transfer-Encoding: chunked
// or over a websocket connection.
It pretty much depends on the client which protocol is used (either chunked http or ws), both of them having their cost, which you'll have to compare to your current request frequency.
You may be better of with a proxy cache that either watches or polls in regular intervals, but that depends a lot on your application.
Related
The manual says this about the HTTP URL value of an http listener:
"Displays the generated HTTP URL for the HTTP Listener. This is not an actual
configurable setting, but is instead displayed for copy/paste convenience. Note
that the host in the URL will be the same as the host you used to connect to
the Administrator. The actual host that connecting clients use may be different
due to differing networking environments."
When I have used the feature in the past its value has always begun "http://localhost:" which would be great except this time it is auto-generating " http://'domainName':${Incoming_Pathology_Source_Port}/${Incoming_Pathology_Source_BaseContextPath}/"
For the first time, we are deploying Mirth inside a Kubernetes cluster, 'a different working environment'. (nginx accepts https and we want it pass the messages on as http to Mirth).
Is there any way I can take control of the URL or must I change the configuration of the cluster in some way.
All help/suggestions welcome.
Initial disclosure:
I’m new to nginx and reverse proxy configuration in general.
Background
I have a Swagger-derived, FOSS, https-accessible REST API [written by another party] running on a certain port of an EC2 CentOS 7 instance behind an nginx 1.16.1 reverse proxy (to path https://foo_domain/bar_api/); for my purposes, this API needs to be reachable from a broad variety of services not all of which publish their IP ranges, i.e., the API must be exposed to traffic from any IP.
Access to the API’s data endpoints (e.g., https://foo_domain/bar_api/resource_id) is controlled by a login function located at
https://foo_domain/bar_api/foobar/login
supported by token auth, which is working fine.
Problem
However, the problem is that an anonymous user is able to GET
https://foo_domain/bar_api
without logging in, which results in potentially sensitive data about the API server configuration being returned, such as the API’s true port, server version, some of the available endpoints and parameters, etc. This is not acceptable for the purpose, from a security standpoint.
Question
How do I prevent anonymous GET requests to the /bar_api/ endpoint, while allowing login and authenticated data requests to endpoints beyond /bar_api/ to proceed unhindered? Or, otherwise, how do I prevent any data from being returned upon such requests?
I'am working on a microservice architecture based on Docker, registrator, consul and HAProxy.
I'am also using Consul-template to dynamically generate the HAProxy config file. Everything works fine : When I add multiple instances of the same microservice, the HAProxy configuration is updated immediately and requests are dispatched correctly using a round robin strategy.
My problem occurs when I remove some instances (scale down). If a container is shut down while a request is running I have an error.
I'am new to HAProxy so is there a way to configure HAProxy to tell it to retry a failing request to another endpoint if a container disappears?
Precision : I'am using a layer7 routing mode (mode http) for my frontends and backends. Here is a little sample of my consul-template file :
backend hello-backend
balance roundrobin
mode http
{{range service "HelloWorld" }}server {{.Node}} {{.Address}}:{{.Port}} check
{{end}}
# Path stripping
reqrep ^([^\ ]*)\ /hello/(.*) \1\ /\2
frontend http
bind *:8080
mode http
acl url_hello path_beg /hello
use_backend hello-backend if url_hello
Thank you for your help.
It isn't possible for HAProxy to resend a request that has already been sent to a backend.
Here's a forum post from Willy, the creator.
redispatch only happens when the request is still in haproxy. Once it has been sent, it is cannot be performed. It must not be performed either for non idempotent requests, because there is no way to know whether some processing has begun on the server before it died and returned an RST.
http://haproxy.formilux.narkive.com/nGKXq6WU/problems-with-haproxy-down-servers-and-503-errors
The post is quite old but it's still applicable based on more recent discussions. If a request is larger than tune.bufsize (default is around 16KB iirc) then HAProxy hasn't even retained the entire request in memory at the point an error occurs.
Both fortunately (for the craft) and unfortunately (for purposes of real-world utility), Willy has always insisted on correct behavior by HAProxy, and he is indeed correct that it is inappropriate to retry non-idempotent requests once they have been sent to a back-end server, because there are certainly cases where this would result in duplicate processing.
For GET requests which, by definition, should be idempotent (a GET request must be repeatable without consequence, otherwise it should not have been designed to use GET -- it should have been POST or another verb) there's a viable argument that resending to a different back-end would be a legitimate course of action, but this also is not currently supported.
Varnish, by contrast, does support a do-over, which I have used (behind HAProxy) with success on GET requests where I have on-line and near-line storage for the same object namespace. Old, "unpopular" files are migrated to near-line (slower, cheaper) storage, but all requests are sent to on-line storage, with the retry destination of near-line if on-line returns a 404. But, I've never tried this with requests other than GET.
Ideally, your solution would be for your back-ends to be declared unhealthy, perhaps by deliberately failing their HTTP health checks for a draining time before shutting down. One fairly simple approach is for the health check to require the presence of a static file, which gets deleted from the back-end before shutdown. Or, you can request HAProxy consider the backend to be in maintenance mode through the stats/admin UI or socket, preventing more requests from being initiated while allowing running requests to drain.
So, generally HTTP methods like PUT and DELETE are considered to be insecure.
However, it is recommended to use PUT and DELETE methods for RESTful API's.
Why is that these methods PUT and DELETE are not considered as insecure for RESTful API's
TL;DR
They are considered insecure because a web-server's default behavior would directly impact files on the servers filesystem -- allowing executable code attacks.
A RESTful service doesn't (have to) create files based on the original request.
Internal / firewalled / proxied
An internal API -- is protected by the fact that it's in a private LAN. It is only accessible to other internal (trusted) tools.
Similarly a firewalled internal or external API only accepts requests from certain IPs (trusted servers).
A proxy server can handle encryption and user authentication as well as authorization and then forward the request to the RESTful service.
But still what are the security risks?
If PUT would create executable files on the server that would be very insecure** -- because of the risk of code injection / executable injection...
...but when receiving PUT or DELETE operations we're not talking about file-management per se. We're talking about a specific handler code which analyses the request and does whatever you told it to do with the data (eg.: puts it into a database).
**Especially since after you execute HTTP PUT on a resource (in a RESTful context) one would expect to have access to execute HTTP GET on that same resource (meaning the resource would be directly accessible).
Imagine an application where there are multiple RESTFUL servers exist with different resources.
When a client makes a resource request, currently a blocking call is made such that the resource request is relayed from Server to Server until the resource is found at some Server. Which is very time consuming . Now all the clients are run in constrained environment and Servers are moderately powerful.
Is there a way to do REST resource lookup service to avoid long blocking calls ?
Client should know where to look for a resource without relaying in a happy flow. So build the logic for getting a resource in the client.
Solution 1:
Client A has a list of all the resource servers and has a directory to know which resource is on which server.
Solution 2:
Client A does not know anything so it will query proxy server B which does the look up. This server B has a directory to map a particular resource to a specific server.
Server B will then query Resource server C/D/E/F etc. depending on the resource. They will respond to Server B.
Server B sends the requested resource to Client A
Update 1: Since you do not have control over your clients, go with solution 2 where B acts as a client in relation to your resource servers. As stated before either use a dictionary where each specific resource points to a particular server or use consistent hashing. Since I do not know what language you are using I have no idea whether there is an existing library for you to use.But there are so many already so probably it will fit your needs.