The HAProxy Data Plane API is an executable (details on GitHub here) that runs alongside HAProxy and provides a REST API that can be used to configure HAProxy whilst it is running, therefore providing a control plane for HAProxy.
If multiple instances of HAProxy are being run, how does one ensure consistency of the configuration across all instances? i.e. if the configuration of one instance is modified via a REST call to the API, how can this change be communicated to all instances?
The Data Plane API is a sidecar process that runs next to HAProxy and manages configuration for a single instance. It does not manage configuration of multiple instances. You would have to make changes to the Data Plane API installed on each cluster member. Thankfully, the Data Plane API supports transactions so you may apply changes in parallel and commit all instances at once should your changes work without exception!
Related
I have been learning Kubernetes for a few weeks and now I am trying to figure out the right way to connect a web server to a statefulset correctly.
Let's say I deployed a master-slave Postgres statefulset and now I will connect my web server to it. By using a cluster IP service, the requests will be load balanced across the master and the slaves for both reading (SELECT) and writing (UPDATE, INSERT, DELETE) records, right? But I can't do that because writing requests should be handled by the master. However, when I point my web server to the master using the headless service that will give us a DNS entry for each pod, I won't get any load balancing to the other slave replications and all of the requests will be handled by one instance and that is the master. So how am I supposed to connect them the right way? By obtaining both load balancing to all replications along with the slave in reading records and forwarding writing records requests to the master?
Should I use two endpoints in the web server and configure them in writing and reading records?
Or maybe I am using headless services and statefulsets the wrong way since I am new to Kubernetes?
Well, your thinking is correct - the master should be read-write and replicas should be read only. How to configure it properly? There are different possible approaches.
First approach is what you thinking about, to setup two headless services - one for accessing primary instances, the second one to access to the replica instances - good example is Kubegres:
In this example, Kubegres created 2 Kubernetes Headless services (of default type ClusterIP) using the name defined in YAML (e.g. "mypostgres"):
a Kubernetes service "mypostgres" allowing to access to the Primary PostgreSql instances
a Kubernetes service "mypostgres-replica" allowing to access to the Replica PostgreSql instances
Then you will have two endpoints:
Consequently, a client app running inside a Kubernetes cluster, would use the hostname "mypostgres" to connect to the Primary PostgreSql for read and write requests, and optionally it can also use the hostname "mypostgres-replica" to connect to any of the available Replica PostgreSql for read requests.
Check this starting guide for more details.
It's worth noting that there are many database solutions which are using this approach - another example is MySQL. Here is a good article in Kubernetes documentation about setting MySQL using Stateful set.
Another approach is to use some middleware component which will act as a gatekeeper to the cluster, for example Pg-Pool:
Pg pool is a middleware component that sits in front of the Postgres servers and acts as a gatekeeper to the cluster.
It mainly serves two purposes: Load balancing & Limiting the requests.
Load Balancing: Pg pool takes connection requests and queries. It analyzes the query to decide where the query should be sent.
Read-only queries can be handled by read-replicas. Write operations can only be handled by the primary server. In this way, it loads balances the cluster.
Limits the requests: Like any other system, Postgres has a limit on no. of concurrent connections it can handle gracefully.
Pg-pool limits the no. of connections it takes up and queues up the remaining. Thus, gracefully handling the overload.
Then you will have one endpoint for all operations - the Pg-Pool service. Check this article for more details, including the whole setup process.
I am working on an application that will integrate with an external API that performs access restriction via IP whitelisting.
This application is going to be hosted on a Kubernetes cluster (on AWS if that is of any help).
Would there be a way for my application to have a single source IP that I would just register on the 3rd party API?
Technically, I could go ahead and have the API register all the IPs of the worker nodes. But that feels rather clunky and would need to be amended each time the cluster structure changes as new nodes are added, removed, restarted, etc.
I saw some posts about setting up a sort of reverse proxy/request funnel for the application to only come from one source, but this is just essentially defining a single point of failure, which I would really like to avoid.
I have a REST API with multiple API-versions. Each API backend is composed of several micro-services. It is fair to assume that only the latest REST API resources/code has most churn. The older versions will see churn due to feature backport (rarely) or bug fixes (mostly). I'd like to get recommendations on what DevOps pattern would best fit this scenario - assuming we are using Kubernetes to model our service mesh.
Note that our APIs are mostly async and so it is possible to support several API versions all in the same codebase (packaged in a single container).
Given the above, these configurations are all possible
Service yaml per API version.
Single service yaml with multiple pod templates (one per API version).
Functional Service yamls - one for the front end and each of the other micro-services (POD for message broker, processing worker etc.,).
Additional points to ponder:
Are deployments recommended to separate clusters or to the same cluster (for the API versions). If yes, does this impact updates to specific API versions?
I'm looking for any prescribed patterns or suggestions based on your prior experience.
In general, Deployments manage replicas of Pods, and each Pod runs a specific container image. If your API backend consists of multiple microservices, then each microservice is a Deployment. The microservice that handles API requests is exposed with a (client-facing) Service.
For the multiple API versions, you could just replicate this for each version and you could put an Ingress in front of the Services which routes traffic to one of them based on the requested API version.
If you put all the API versions in the same container, you may have a problem of inconsistent state during updates: 1) inside a Deployment, for a short time both Pod versions exist side by side (if you use the default rolling update); 2) for a short time you might have both updated and not yet updated microservices next to each other.
I have a client that makes asynchronous calls to a gRPC service managed by kubernetes. The function calls are computationally expensive and they each take a while to complete. Therefore many of the calls wait for response in a queue (as shown in this tutorial https://grpc.io/docs/tutorials/async/helloasync-cpp.html or more specific https://github.com/grpc/grpc/blob/v1.4.x/examples/cpp/helloworld/greeter_async_client2.cc). What I notice is that all the calls are served by the same pod and other pods remain unused on my cluster.
If I launch multiple instances of the client it picks up different nodes or pods, but I'm interested in this happening for calls from one async client connection.
Is this possible and if so, does it require some specific configuration?
(I realize that I could open many connections from one script, but this does not seem optimal??)
I should also mention that I'm running a local kubernetes setup with just a few nodes which is setup using kubeadm.
kube-proxy is an L4 load balancer so it's not able to distinguish between separate http requests (L7) in one stream. Depending on what you are trying to achieve an L7 proxy (that supports HTTP/2) could be a solution.
There is a nice overview in this document: https://grpc.io/blog/loadbalancing
I'm looking at consensus-type tools like ZooKeeper, Consul and Eureka and they all seem to market the same set of solutions:
Service discovery
Dynamic, centralized configuration management
Synchronization primitives
Consensus algorithms
However the more I read about these things, the more I struggle to see how service discovery is really any different than a dynamic, centralized configuration management (KV pair) system.
My understanding (so far) of service discovery is that it allows nodes to dynamically search for, find and connnect to remote services. So if an application uses an AuthService for authentication, authorization, it would use service discovery to find an AuthService node at, say, http://auth103.example.org:9103 and use it.
My understanding of dynamic config systems is that they provide a centralized infrastructure for nodes to dynamically receive updates from, as well as publish updates to, config servers. So if an app instance decides it needs to update a configuration for all its other instances, it would contact the config service and update, say, the numPurgerThreads config. The config service would then update all other app instances so that they updated their respective configs properly.
But aren't these exactly the same problem?
In both cases, you:
Connect to a lookup service of some sort
Query it for data; or
Publish data to it, which then ripples out to other nodes
Service discovery is dynamic configuration, right?!?!
What I'm really driving at is: Couldn't I just implement one config service with one of these tools, which coincidentally also solves service discovery? Or is there a reason why I would need to have, say, 1 Consul cluster for config/KV management, and another, say, Consul cluster for service discovery?
Well, if you look at it like that, these are all just flavors of databases, or data stores if you will, as you describe it:
Connect to a lookup service of some sort
Query it for data; or
Publish data to it, which then ripples out to other nodes
All of the use cases you mentioned require some sort of a data store that multiple clients connect to. What makes some of them more optimal then others for different use cases is their interface, data model, consistency guaranties etc.
Specifically, in case of service discovery, a nice feature can be failure detection - e.g. a k/v store that allows removing service data when the service is no longer connected. This way, you can register a service to your service discovery tool and know that when the service goes down or looses connectivity, it will no longer be present in the stored data.
Just to clarify a few terms, (imo) dynamic config systems can update their state at runtime, in contrast to static where everything is defined beforehand (i.e. config files). Centralized CM means there's a single place to store all config data. So centralized CM can both be static or dynamic, right?
IMO, service discovery has a component which CM doesn't and that's a protocol for automatic detection of services.
Guess you'd need a dynamic CM (KV store) to implement service discovery, but you couldn't implement a CM store just using a service discovery (protocol). Take DHCP for example (hope we agree it is a service discovery protocol?) - Dynamic Host Configuration Protocol', so it has a configuration aspect, but also a protocol, which is a bit more than a simple KV store. Btw, it's decentralized, just to illustrate the previous point that CM doesn't have to be centralized..
So service discovery has a configuration aspect, but CM doesn't (necessarily) have a service discovery one. Does this mean SD is a subset of CM? I'd say no.