In my current application server which currently runs not in a Kubernetes environment, we use IPS in order to communicate with a specific instance of the same application agent.
the application is stateful and therefore we have to communicate each time with the same instance, with IP as an identifier.
Now we want to deploy this agent application to Kubernetes, however, because it's a stateful application I am a bit confused. How can I communicate each time I start a session with the same instance application? Can I have an IP or some another identifier of the pod? how will it work with the Ingress resource?
I looked at the Statefulset object but also did not understand how can it help me.
I also looked at "sticky session", but I don't want to define an amount of time in order to define one session. I want to be able to access the same pod, and if the pod dies I want to get an exception.
How can I do it in Kubernetes? is it possible?
Related
I have my main application which has its own unique state, let's call it Application A.
This application A starts a few processes which does some parsing work and then it collects it and should send it to a database server outside of the Kuberentes cluster.
I would like to run a few copies of this application A in different pods. however, each instance is unique and cannot be replaced as it has its own state. it means that each client has to talk only with the same instance it started the communication with http requests.
How can it be done in Kubernetes?
do I need to define StatefulSet component?
how do I manage that each client (from outside the cluster) will talk every time with the same instance he started communication on the same object id ? for example to get status on that object.
in case the pod die I don't want to recover. is that possible?
1: yes, sort of
2: not necessarily, but might simplify some things
3: if you use ingress, you can use different methods to maintain backend affinity ie. cookie based, source IP based etc. (nginx example: https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/)
4: you might want to set restartPolicy to Never
With all that said, this really sounds like a bad idea. You should either allow shared state (ie. redis), or statefulset with ability to restart with the same state loaded from local storage. You need to remember that even with the most optimal setup things like this can break (ie. switch to different pod when a backing pod went down, node rescheduling due to cluster scaling etc.)
For the number 4 question. You only need to set up the container-restart-policy. I used this flag to create a pod with this feature: --restart=Never
IMHO, It is not a Kubernetes problem. You could have this scenario in other environments. The idea is to use sticky sessions to have an affinity for all your request. You probably need to search for this setup in your ingress controller documentation. E.g Nginx Ingress
I have a containerized legacy application using VNC as an XServer.
I'd like to run this application in a Kubernetes cluster, and start an application instance on-demand, when a new user logs in the system.
I'd like to
scale the number of PODs on-demand and not automatically with the replicas property
provide unique host/port to the clients to connect to their own VNC server?
How can I achieve this in K8S?
Your question is a bit contradictory, since on-demand scaling is a variant of automatic scaling.
Given your constraints you could create a custom application in which users request their connection data. Upon request your application talks to the kubernetes api and scales up the number of instances.
You would use a headless service to get the list of endpoints and associate a specific endpoint to a user. Your application would reserve a specific port for that user. You then have to create a new nodeport or loadbalancer service to expose this port of your application that forwards to the chosen instance.
After all this sounds like a lot of effort, perhaps your concept is not a good match for kubernetes.
I'm trying to figure out which tools from GKE stack I should apply to my use case which is a dynamic deployment of stateful application with dynamic HTTP endpoints.
Stateful in my case means that I don't want any replicas and load-balancing (because the app doesn't scale horizontally at all). I understand though that in k8s/gke nomenclature I'm still going to be using a 'load-balancer' even though it'll act as a reverse proxy and not actually balance any load.
The use case is as follows. I have some web app where I can request for a 'new instance' and in return I get a dynamically generated url (e.g. http://random-uuid-1.acme.io). This domain should point to a newly spawned, single instance of a container (Pod) hosting some web application. Again, if I request another 'new instance', I'll get a http://random-uuid-2.acme.io which will point to another (separate), newly spawned instance of the same app.
So far I figured out following setup. Every time I request a 'new instance' I do the following:
create a new Pod with dynamic name app-${uuid} that exposes HTTP port
create a new Service with NodePort that "exposes" the Pod's HTTP port to the Cluster
create or update (if exists) Ingress by adding a new http rule where I specify that domain X should point at NodePort X
The Ingress mentioned above uses a LoadBalancer as its controller, which is automated process in GKE.
A few issues that I've already encountered which you might be able to help me out with:
While Pod and NodePort are separate resources per each app, Ingress is shared. I am thus not able to just create/delete a resource but I'm also forced to keep track of what has been added to the Ingress to be then able to append/delete from the yaml which is definitely not the way to do that (i.e. editing yamls). Instead I'd probably want to have something like an Ingress to monitor a specific namespace and create rules automatically based on Pod labels. Say I have 3 pods with labels, app-1, app-2 and app-3 and I want Ingress to automatically monitor all Pods in my namespace and create rules based on the labels of these pods (i.e. app-1.acme.io -> reverse proxy to Pod app-1).
Updating Ingress with a new HTTP rule takes around a minute to allow traffic into the Pod, until then I keep getting 404 even though both Ingress and LoadBalancer look as 'ready'. I can't figure out what I should watch/wait for to get a clear message that the Ingress Controller is ready for accepting traffic for newly spawned app.
What would be the good practice of managing such cluster where you can't strictly define Pods/Services manifests because you are creating them dynamically (with different names, endpoints or rules). You surely don't want to create bunch of yaml-s for every application you spawn to maintain. I would imagine something similar to consul templates in case of Consul but for k8s?
I participated in a similar project and our decision was to use Kubernetes Client Library to spawn instances. The instances were managed by a simple web application, which took some customisation parameters, saved them into its database, then created an instance. Because of the database, there was no problem with keeping track of what have been created so far. By querying the database we were able to tell if such deployment was already created or update/delete any associated resources.
Each instance consisted of:
a deployment (single or multi-replica, depending on the instance);
a ClusterIp service (no reason to reserve machine port with NodePort);
an ingress object for shared ingress controller;
and some shared configMaps.
And we also used external DNS and cert manager, one to manage DNS records and another to issue SSL certificates for the ingress. With this setup it took about 10 minutes to deploy a new instance. The pod and ingress controller were ready in seconds but we had to wait for the certificate and it's readiness depended on whether issuer's DNS got our new record. This problem might be avoided by using a wildcard domain but we had to use many different domains so it wasn't an option in our case.
Other than that you might consider writing a Helm chart and make use of helm list command to find existing instances and manage them. Though, this is a rather 'manual' solution. If you want this functionality to be a part of your application - better use a client library for Kubernetes.
I'm a beginner in Kubernetes and I have a situation as following: I have two differents Pods: PodA and PodB. Firstly, I want to expose PodA to the outside world, so I create a Service (type NodePort or LoadBalancer) for PodA, which is not difficult to understand for me.
Then I want PodA communicate to PodB, and after several hours googling, I found the answer is that I also need to create a Service (type ClusterIP if I want to keep PodB only visible inside the cluster) for PodB, and if I do so, I can let PodA and PodB comminucate to each other. But the problem is I also found this article. According to this webpage, they say that the communication between pods on the same node can be done via cbr0, a Network Bridge, or the communication between pods on different nodes can be done via a route table of the cluster, and they don't mention anything to the Service object (which means we don't need Service object ???).
In fact, I also read the documents of K8s and I found in the Cluster Networking
Cluster Networking
...
2. Pod-to-Pod communications: this is the primary focus of this document.
...
where they also focus on to the Pod-to-Pod communications, but there is no stuff relevant to the Service object.
So, I'm really confusing right now and my question is: Could you please explain to me the connection between these stuff in the article and the Service object? The Service object is a high-level abstract of the cbr0 and route table? And in the end, how can the Pods can communicate to each other?
If I misunderstand something, please, point it out for me, I really appreciate that.
Thank you guys !!!
Motivation behind using a service in a Kubernetes cluster.
Kubernetes Pods are mortal. They are born and when they die, they are not resurrected. If you use a Deployment to run your app, it can create and destroy Pods dynamically.
Each Pod gets its own IP address, however in a Deployment, the set of Pods running in one moment in time could be different from the set of Pods running that application a moment later.
This leads to a problem: if some set of Pods (call them “backends”) provides functionality to other Pods (call them “frontends”) inside your cluster, how do the frontends find out and keep track of which IP address to connect to, so that the frontend can use the backend part of the workload?
That being said, a service is handy when your deployments (podA and podB) are dynamically managed.
Your PodA can always communicate with PodB if it knows the address or the DNS name of PodB. In a cluster environment, there may be multiple replicas of PodB, or an instance of PodB may die and be replaced by another instance with a different address and different name. A Service is an abstraction to deal with this situation. If you use a Service to expose your PodB, then all pods in the cluster can talk to an instance of PodB using that service, which has a fixed name and fixed address no matter how many instances of PodB exists and what their addresses are.
First, I read it as you are dealing with two applications, e.g. ApplicationA and ApplicationB. Don't use the Pod abstraction when you reason about your architecture. On Kubernetes, you are dealing with a distributed system, and it is designed so that you should have multiple instances of your Application, e.g. for High Availability. Each instance of your application is a Pod.
Deploy your applications ApplicationA and ApplicationB as a Deployment resource. Then it is easy do do rolling upgrades without downtime, and Kubernetes will restart any instance of your application if it crash.
For every Deployment or for you, application, create one Service resource, (e.g. ServiceA and ServiceB). When you communicate from ApplicationA to another application, use the Service, e.g. ServiceB. The service will load balance your requests to the instances of the other application, and you can upgrade your Deployment without downtime.
1.Cluster networking : As the name suggests, all the pods deployed in the cluster will be connected by implementing any kubernetes network model like DANM, flannel
Check this link to see how to create a cluster network.
Creating cluster network
With the CNI installed (by implementing cluster network), every pod will get an IP.
2.Service objects created with type ClusterIP, points to the this IPs (via endpoint) created internally to communicate.
Answering your question, Yes, The Service object is a high-level abstract of the cbr0 and route table.
You can use service object to communicate between pods.
You can also implement service mesh like envoy / Istio if the network is complex.
I am currently designing a system where users should be able to start a simulation through a Web Portal and then connect to it with a gRPC client (amongst other things). After the user is finished the simulation then terminates. I want to run the whole system in a kind of microservice architecture in a kubernetes cluster if possible. This is however my first time working with kubernetes and I am unsure if it is possible to achieve this.
As far as I could gather from reading the documentation and googling around it seems like I should be able to launch a pod by calling POST /api/v1/namespaces/{namespace}/pods and making it availble under the Host IP by setting hostPort. However what I dont know is how I would determine a free port on the Node to deploy to or let kubernetes decide that (if hostPort is even the correct choice for this). After that it should be pretty straightforward. Send the user the IP:Port to connect to and he just plugs that into his gRPC client.
Any suggestions on how to best achieve this?
Using hostPort is rather not recommended, so you'd be better off by specifying a service and access your Pod via a service. In your case you can define NodePort service and let Kubernetes decide on the port. Then, fetch the service port using Kubernetes API.