I am trying to deploy an application with load-balancing within kubernetes
below are my intended deployment diagram
ideally, the application is deployed by a set of pods using k8s deployment with type of "backend"
normally, user instances are stored in the archive. and are restored into one of the pods dynamically upon request, stay there for a TTL time (say 30 minutes), and deleted and backuped into the archive.
ideally, the load balance is deployed by a set of pods using k8s deployment with type of "frontend".
ideally, the frontend is configured as layer7 session sticky with "sticky = host". the host equals the UID of a backend pod
an user requests the service by a SOAP message, which contains parameters "host" and "user" in its body.
when a SOAP message reaches the frontend, the "host" value is extracted from the message body.
if the "host" value is valid, the SOAP message is forwarded to the corresponding backend pod (whose UID equals the host value). otherwise, a random backend pod is assigned.
(processing here upon is application specific)
In a backend pod, the application checks the availability of the user instance by the value of "user".
if already existed, just use it; otherwise, try to restore from the archive; if restoring failed(new user), create a new user instance.
I searched around, and did not find any similar examples.
especially layer7 session sticky configuration, and the implementation of custom acquiring of sticky value from the incoming message body.
This sounds like a use-case where you are doing authentication through the front-end loadbalancer. Have you looked at Istio and Ambassador. Seems like Istio and Envoy could provide the service mesh to route the requests to the pods. Then you would have to write a custom plugin module into Ambassador to create this specific routing and authentication mechanism that you are seeking.
Example of Ambassador custom authentication service: https://www.getambassador.io/user-guide/auth-tutorial
https://www.getambassador.io/user-guide/with-istio
This custom sticky session routing can also be done using other API gateways but still using Istio for routing to the different pods. However it would be best if the pods are defined as separate services in order to have easier segmentation by the API gateway (Ambassador, Kong, Nginx) based on the parameters of the message body.
Related
I am using the Artemis Cloud operator for deploying ActiveMQ Artemis in k8s cluster. Number of replicas configured is 3. Queues are expected to be created by client applications. Operator creates a headless service and service for each pod in the cluster setup.
Whenever client connects to a pod ,it creates a queue in that broker pod.So if client connects to 3 brokers at random time, three queues gets created in pods, one in each pod respectively.So when a producer sends message to the queue, it is sent to the connected pod. It wont be present in each broker pod.(No replication of messages).
My question is what service name should client applications use inorder to connect to artemis pods and also maintain session affinity? In other words, what should be done in order to make a client connect to same broker whenever it tries a connection?(and avoid duplicate queue creation)
What I currently use is a kubernetes clusterip service I created that splits traffics to pods.And queues are created via stomp producer.
Not sure which helm it will be using in the background but you will be using the service name inside the kubernetes app for connection instead of the ClusterIP.
Service name could be starting the with helm chart name. i was checking and referring this chart for ref : https://github.com/deviceinsight/activemq-artemis-helm could be different.
Looks like it's creating the service with Cluster IP:none (Headless service) so connecting to the existing service is not working, i doubt currently it might be returning all the PODs IP.
Option : 2 if above one not work give try to this
In another case, you can create the new service type clusterIP with a different name, everything else will be the same, like port and all.
In service you can notice amqp, mqtt and other port, so you app will connect to new service with port config as per requirement. For example : active-mq-service:61613
If everything working fine for you and you are just looking for session affinity you can add this config to your service and it will start managing the session affinity.
SessionAffinity: ClientIP
If you want to make sure that connections from a particular client are
passed to the same Pod each time, you can select the session affinity
based on the client's IP addresses by setting
service.spec.sessionAffinity to "ClientIP" (the default is "None").
You can also set the maximum session sticky time by setting
service.spec.sessionAffinityConfig.clientIP.timeoutSeconds
appropriately. (the default value is 10800, which works out to be 3
hours).
Ref doc : https://kubernetes.io/docs/concepts/services-networking/service/
I'm trying to figure out which tools from GKE stack I should apply to my use case which is a dynamic deployment of stateful application with dynamic HTTP endpoints.
Stateful in my case means that I don't want any replicas and load-balancing (because the app doesn't scale horizontally at all). I understand though that in k8s/gke nomenclature I'm still going to be using a 'load-balancer' even though it'll act as a reverse proxy and not actually balance any load.
The use case is as follows. I have some web app where I can request for a 'new instance' and in return I get a dynamically generated url (e.g. http://random-uuid-1.acme.io). This domain should point to a newly spawned, single instance of a container (Pod) hosting some web application. Again, if I request another 'new instance', I'll get a http://random-uuid-2.acme.io which will point to another (separate), newly spawned instance of the same app.
So far I figured out following setup. Every time I request a 'new instance' I do the following:
create a new Pod with dynamic name app-${uuid} that exposes HTTP port
create a new Service with NodePort that "exposes" the Pod's HTTP port to the Cluster
create or update (if exists) Ingress by adding a new http rule where I specify that domain X should point at NodePort X
The Ingress mentioned above uses a LoadBalancer as its controller, which is automated process in GKE.
A few issues that I've already encountered which you might be able to help me out with:
While Pod and NodePort are separate resources per each app, Ingress is shared. I am thus not able to just create/delete a resource but I'm also forced to keep track of what has been added to the Ingress to be then able to append/delete from the yaml which is definitely not the way to do that (i.e. editing yamls). Instead I'd probably want to have something like an Ingress to monitor a specific namespace and create rules automatically based on Pod labels. Say I have 3 pods with labels, app-1, app-2 and app-3 and I want Ingress to automatically monitor all Pods in my namespace and create rules based on the labels of these pods (i.e. app-1.acme.io -> reverse proxy to Pod app-1).
Updating Ingress with a new HTTP rule takes around a minute to allow traffic into the Pod, until then I keep getting 404 even though both Ingress and LoadBalancer look as 'ready'. I can't figure out what I should watch/wait for to get a clear message that the Ingress Controller is ready for accepting traffic for newly spawned app.
What would be the good practice of managing such cluster where you can't strictly define Pods/Services manifests because you are creating them dynamically (with different names, endpoints or rules). You surely don't want to create bunch of yaml-s for every application you spawn to maintain. I would imagine something similar to consul templates in case of Consul but for k8s?
I participated in a similar project and our decision was to use Kubernetes Client Library to spawn instances. The instances were managed by a simple web application, which took some customisation parameters, saved them into its database, then created an instance. Because of the database, there was no problem with keeping track of what have been created so far. By querying the database we were able to tell if such deployment was already created or update/delete any associated resources.
Each instance consisted of:
a deployment (single or multi-replica, depending on the instance);
a ClusterIp service (no reason to reserve machine port with NodePort);
an ingress object for shared ingress controller;
and some shared configMaps.
And we also used external DNS and cert manager, one to manage DNS records and another to issue SSL certificates for the ingress. With this setup it took about 10 minutes to deploy a new instance. The pod and ingress controller were ready in seconds but we had to wait for the certificate and it's readiness depended on whether issuer's DNS got our new record. This problem might be avoided by using a wildcard domain but we had to use many different domains so it wasn't an option in our case.
Other than that you might consider writing a Helm chart and make use of helm list command to find existing instances and manage them. Though, this is a rather 'manual' solution. If you want this functionality to be a part of your application - better use a client library for Kubernetes.
I have a cluster with 3 nodes. In each node i have a frontend application running in a Pod and backend application running in a separate Pod.
I send data from the frontend application to the backend application, to do this i utilise the Cluster IP Service and k8 dns resource.
I also have a function in my frontend where i send data to a separate service unrelated to my k8s cluster. I send this data using a standard AJAX request to a url with a payload i.e http://my-seperate-service-unrelated-tok8.com.
All of this works correctly and the cluster operates as i want. - i have this cluster deployed to GKE.
I now want to run this cluster local using minikube, which i have been able to do, however, when i am running locally i do not want to send data to my external service - instead i want to forward it to either a new Pod i will create or just not send it.
The problem here is i need a proxy to intercept outgoing network traffic, check if the outgoing request is the request i am looking for and if it is then redirect it.
I understand each node running in a cluster has a kube-proxy service running within the node - which is used to forward traffic to the relevant services in the cluster.
I would like to either extend this service, or create a new proxy service where i can listen for outgoing traffic to a specific url and redirect it.
Is this possible to do in a k8 cluster? I assume there is a Service i can create to listen for all outgoing requests and redirect specific requests based on rules i set.
I wasn’t sure if k8 clusters have a Service already configured i can simply add to - that’s why i thought of the kube-proxy, would anyone be able to advice on this?
I wanted to add this proxy so i don’t have to change my code when its ran locally in minikube or deployed to GKE.
Any help is greatly appreciated. Thanks!
I did a tool that help you to forward a service to another service,local port, service from other cluster, etc...
This way you can have exactly your same urls, ports and code... but the underlying services gets "replaced", if I understand correctly this is what you are looking for.
Here is a quick example of an stage service being replaced with my local 3000 port
This is the repository with more info and examples: linker-tool
If you are interested let me know if you need help or have any question.
I am currently switching from Service Fabric to Kubernetes and was wondering how to do custom and more complex load balancing.
So far I already read about Kubernetes offering "Services" which do load balancing for pods hidden behind them, but this is only available in more plain ways.
What I want to rewrite right now looks like the following in Service Fabric:
I have this interface:
public interface IEndpointSelector
{
int HashableIdentifier { get; }
}
A context keeping track of the account in my ASP.Net application e.g. inherits this. Then, I wrote some code which would as of now do service discovery through the service fabric cluster API and keep track of all services, updating them when any instances die or are being respawned.
Then, based on the deterministic nature of this identifier (due to the context being cached etc.) and given multiple replicas of the target service of a frontend -> backend call, I can reliably route traffic for a certain account to a certain endpoint instance.
Now, how would I go about doing this in Kubernetes?
As I already mentioned, I found "Services", but it seems like their load balancing does not support custom logic and is rather only useful when working with stateless instances.
Is there also a way to have service discovery in Kubernetes which I could use here to replace my existing code at some points?
StatefulSet
StatefulSet is a building block for stateful workload on Kubernetes with certain guarantees.
Stable and unique network identity
StatefulSet Pods have a unique identity that is comprised of an ordinal, a stable network identity, and stable storage.
As an example, if your StatefulSet has the name sharded-svc
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: sharded-svc
And you have e.g. 3 replicas, those will be named by <name>-<ordinal> where ordinal starts from 0 up to replicas-1.
The name of your pods will be:
sharded-svc-0
sharded-svc-1
sharded-svc-2
and those pods can be reached with a dns-name:
sharded-svc-0.sharded-svc.your-namespace.svc.cluster.local
sharded-svc-1.sharded-svc.your-namespace.svc.cluster.local
sharded-svc-2.sharded-svc.your-namespace.svc.cluster.local
given that your Headless Service is named sharded-svc and you deploy it in namespace your-namespace.
Sharding or Partitioning
given multiple replicas of the target service of a frontend -> backend call, I can reliably route traffic for a certain account to a certain endpoint instance.
What you describe here is that your stateful service is what is called sharded or partitioned. This does not come out of the box from Kubernetes, but you have all the needed building blocks for this kind of service. It may happen that it exists an 3rd party service providing this feature that you can deploy, or it can be developed.
Sharding Proxy
You can create a service sharding-proxy consisting of one of more pods (possibly from Deployment since it can be stateless). This app need to watch the pods/service/endpoints in your sharded-svc to know where it can route traffic. This can be developed using client-go or other alternatives.
This service implements the logic you want in your sharding, e.g. account-nr modulus 3 is routed to the corresponding pod ordinal
Update: There are 3rd party proxies with sharding functionallity, e.g. Weaver Proxy
Sharding request based on headers/path/body fields
Recommended reading: Weaver: Sharding with simplicity
Consuming sharded service
To consume your sharded service, the clients send request to your sharding-proxy that then apply your routing or sharding logic (e.g. request with account-nr modulus 3 is routed to the corresponding pod ordinal) and forward the request to the replica of sharded-svc that match your logic.
Alternative Solutions
Directory Service: It is probably easier to implement sharded-proxy as a directory service but it depends on your requirements. The clients can ask your directory service to what statefulSet replica should I send account-nr X and your serice reply with e.g. sharded-svc-2
Routing logic in client: The probably most easy solution is to have your routing logic in the client, and let this logic calculate to what statefulSet replica to send the request.
Services generally run the proxy in kernel space for performance reasons so writing custom code is difficult. Cillium does allow writing eBPF programs for some network features but I don't think service routing is one of them. So that pretty much means working with a userspace proxy instead. If your service is HTTP-based, you could look at some of the existing Ingress controllers to see if any are close enough or allow you to write your own custom session routing logic. Otherwise you would have to write a daemon yourself to handle it.
Inside of a Kubernetes Cluster I am running 1 node with 2 deployments. React front-end and a .NET Core app. I also have a Load Balancer service for the front end app. (All working: I can port-forward to see the backend deployment working.)
Question: I'm trying to get the front end and API to communicate. I know I can do that with an external facing load balancer but is there a way to do that using the clusterIPs and not have an external IP for the back end?
The reason we are interested in this, it simply adds one more layer of security. Keeping the API to vnet only, we are removing one more entry point.
If it helps, we are deploying in Azure with AKS. I know they have some weird deployment things sometimes.
Pods running on the cluster can talk to each other using a ClusterIP service, which is the default service type. You don't need a LoadBalancer service to make two pods talk to each other. According to the docs on this topic
ClusterIP exposes the service on a cluster-internal IP. Choosing this value makes the service only reachable from within the cluster. This is the default ServiceType.
As explained in the Discovery documentation, if both Pods (frontend and API) are running on the same namespace, the frontend just needs to send requests to the name of the backend service.
If they are running on different namespaces, the frontend API needs to use a fully qualified domain name to be able to talk with the backend.
For example, if you have a Service called "my-service" in Kubernetes Namespace "my-ns" a DNS record for "my-service.my-ns" is created. Pods which exist in the "my-ns" Namespace should be able to find it by simply doing a name lookup for "my-service". Pods which exist in other Namespaces must qualify the name as "my-service.my-ns". The result of these name lookups is the cluster IP.
You can find more info about how DNS works on kubernetes in the docs.
The problem with this configuration is the idea that the Frontend app will be trying to reach out to the API via the internal cluster. But it will not. My app, on the client's browser can not reach services and pods in my Kluster.
My cluster will need something like nginx or another external Load Balancer to allow my client side api calls to reach my API.
You can alternatively used your front end app, as your proxy, but that is highly not advised!
I'm trying to get the front end and api to communicate
By api, if you mean the Kubernetes API server, first setup a service account and token for the front-end pod to communicate with the Kubernetes API server by following the steps here, here and here.
is there a way to do that using the clusterIPs and not have an external IP for the back end
Yes, this is possible and more secure if external access is not needed for the service. Service type ClusterIP will not have an ExternalIP and the pods can talk to each other using ClusterIP:Port within the cluster.