The situation is that I have 2 applications: A and B that are in the same namespace of a cluster on gke. A is on 1 pod and B is on 2 pods.
Everytime a client communicates with our service. It connects first on A with websockets. A then sends http request to B. Since there is 2 pods of B, I would like to have session affinity between the Client from outside and with my application B so that everytime a client connects to A, it will always process his requests through the same pod of B.
Every session affinity option I saw are based on Ingress gateway or services, but since I'm already in the cluster, I don't need an Ingress.
I also saw that there is some services that provides support for http cookies. That would be good but it is always an external service like Nginx or Istio and since I'm working in a highly restricted development environment it is kind of a pain to add those service in the cluster.
Is there anything native to the gke that can provide me with http cookie session affinity or something similar?
In a GKE cluster, when you create a Kubernetes Ingress object, the GKE ingress controller wakes up and creates a Google Cloud Platform HTTP(S) load balancer. The ingress controller configures the load balancer and also configures one or more backend services that are associated with the load balancer.
Beginning with GKE version 1.11.3-gke.18, you can use an Ingress to configure these properties of a backend service:
Timeout
Connection draining timeout
Session affinity
This will be useful for you and it's native of GKE Ingress.
You could have set Client IP based session affinity, which happens at service level, like this one:
apiVersion: v1
kind: Service
metadata:
name: svc-sa
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
selector:
app: nginx
sessionAffinity: ClientIP
sessionAffinityConfig:
clientIP:
timeoutSeconds: 3600
So, this service would route the request to the same backend (Pod), depending on the source IP address if the request.
You need to set a service anyways in front of the application B targeting the two pods. Now, the problem here is that your application A is acting as proxy, so all the requests will be considered from application A.
I know this is now a complete answer, but you might be able to do something in the application A, header wise (X-Forwarded-For), to bypass Pod A IP address, by the the IP address of the original request.
Everytime a client communicates with our service. It connects first on A with websockets. A then sends http request to B. Since there is 2 pods of B, I would like to have session affinity between the Client from outside and with my application B so that everytime a client connects to A, it will always process his requests through the same pod of B.
A communicate with B. A should connect to a specific instance of B depending on what end-user is connected to B.
This is sharding not session affinity but I understand what you mean. This means that your service B need a stable network identity.
Sharding by User identity
Service B need to be deployed as a StatefulSet to get a stable network identity. Then Service A can do sharding e.g. based on username or a range of IP-address or something, so requests for user X is always handled by instance Y.
With Service B deployed as StatefulSet, the instances will be named e.g. app-b-0 and app-b-1 so each instance can be addressed from Service A with a stable identity.
Related
I have 2 node js backend applications, they depend on each other, but I'm confused about how these 2 backends communicate in Kubernetes AWS EKS
In a default k8s setting, creating a service and registering pods into it will create an internal dns address each pod can use to reach another one.
So, if your services are named service-a and service-b, service-a can reach service-b by sending requests to the service-a host.
The FQDN for each service is service-<x>.<namespace>.svc.cluster.local
More information can be found here
I am in the process of learning Kubernetes and creating a sample application on GKE. I am able to create pods, containers, and services on minikube, however, got stuck when exposing it on the internet using my custom domain like hr.mydomain.com.
My application says file-process is running on port 8080, now I want to expose it to the internet. I tried creating the service of load balancer type on GKE. I get the IP of the load balancer and map it to A record of hr.mydomain.com.
My question is - If this service is restarted, does the service IP changes every time and the service becomes inaccessible?
How do I manage it? What are the best practices when mapping domain names to svc?
File service
apiVersion: v1
kind: Service
metadata:
name: file-process-service
labels:
app: file-process-service
spec:
type: LoadBalancer
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
selector:
app: file-process-api
Google Kubrnetes Engine is designed to take as much configuration hassle out of your hands as possible. Even if you restart the service nothing will change in regards to it's availability from the Internet.
Networking (including load balancing) is managed automatically withing the GKE cluster:
...Kubernetes uses Services to provide stable IP addresses for applications running within Pods. By default, Pods do not expose an external IP address, because kube-proxy manages all traffic on each node. Pods and their containers can communicate freely, but connections outside the cluster cannot access the Service. For instance, in the previous illustration, clients outside the cluster cannot access the frontend Service using its ClusterIP.
This means that if you expose the service and it will have external IP it will stay the same until the load balancer is deleted:
The network load balancer is aware of all nodes in your cluster and configures your VPC network's firewall rules to allow connections to the Service from outside the VPC network, using the Service's external IP address. You can assign a static external IP address to the Service.
At this point when you have a load balancer with static public IP in front of your service you can set this IP as an A record for your domain.
Basically I have Following Hdfs Cluster setup using docker-compose:
Node 1 with IP: 192.168.1.1 having service deployed as below:
Namenode1:9000
HMaster1: 8300
ZooKeeper1:1291
Node 2 with IP: 192.168.1.2 having service deployed as below:
Namenode2:9000
ZooKeeper2:1291
How does Traefik / Ngnix - (Ingress Controllers) forwards request to two different services having configured with same port number?
There are several great tutorials on how ingress and load balancing works in kubernetes, e.g. this one by Mark Betz. As a general rule, it helps to think in terms of services and workloads instead of specific nodes where your workloads are running on.
A workload deployed in Kubernetes (a so called Pod) has its own internal IP address, called a ClusterIP. That pod can have one or more ports open, just on that pod-owned ip address.
If you now have several pods to distribute the load, e.g. like 5 web server processes or backend logic, it would be hard for a client (inside the cluster) to keep track of all those pod IPs, because they also change when a pod is updated or just restarted due to a crash. This is why Kubernetes has a so called concept of services. Those provide a stable DNS name and IP which then transparently "forwards" to one of the healthy pods. So your client only needs to know the DNS name and not keep track of the specific pod IPs.
If you now want to expose such a service to the public, there are different ways. Either you set your service to type: LoadBalancer which then sets up some load balancer infrastructure on your cloud provider and routes traffic to the nodes and then to the pods - or - you already have an ingress controller in place and just define the routing based on host names and paths. An ingress controller itself is such a loadbalanced service with an attached cloud load balancer and also has some pods (with e.g. a traefik or nginx container) which then route your packets accordingly.
So coming back to your initial question: If you want to expose a service with several pods of the same kind, then you would first create a Service resource that matches your Pods using the selector and then you create one single ingress resource that provides a hostname/path and references this service. The ingress controller will pick up those ingress resources and configure the traefik or nginx accordingly. The ingress controller doesn't really care about the host IPs and port numbers, because it acts on the internal kubernetes ClusterIPs, so you even don't need (and shouldn't) expose such a service directly when you have an ingress in place.
I hope this answers your question regarding exposing two workloads over an ingress controller. For details, check the Kubernetes docs on Ingresses. Based on the services you named (zookeeper, hdfs) load balancing and ingresses might not be what you need for that case. Zookeeper instances should be internal in most cases and need to be adressed individually, so you might want to check out headless services, for this use case. Also check the Kubernetes docs for a way to run zookeeper.
For a sample microservice based architecture deployed on Google kubernetes engine, I need help to validate my understanding :
We know services are supposed to load balance traffic for pod replicaset.
When we create an nginx ingress controller and ingress definitions to route to each service, a loadbalancer is also setup automatically.
had read somewhere that creating nginx ingress controller means an nginx controller (deployment) and a loadbalancer type service getting created behind the scene. I am not sure if this is true.
It seems loadbalancing is being done by services. URL based routing is
being done by ingress controller.
Why do we need a loadbalancer? It is not meant to load balance across multiple instances. It will just
forward all the traffic to nginx reverse proxy created and it will
route requests based on URL.
Please correct if I am wrong in my understanding.
A Service type LoadBalancer and the Ingress is the way to reach your application externally, although they work in a different way.
Service:
In Kubernetes, a Service is an abstraction which defines a logical set of Pods and a policy by which to access them (sometimes this pattern is called a micro-service). The set of Pods targeted by a Service is usually determined by a selector (see below for why you might want a Service without a selector).
There are some types of Services, and of them is the LoadBalancer type that permit you to expose your application externally assigning a externa IP for your service. For each LoadBalancer service a new external IP will be assign to it.
The load balancing will be handled by kube-proxy.
Ingress:
An API object that manages external access to the services in a cluster, typically HTTP.
Ingress may provide load balancing, SSL termination and name-based virtual hosting.
When you setup an ingress (i.e.: nginx-ingress), a Service type LoadBalancer is created for the ingress-controller pods and a Load Balancer in you cloud provider is automatically created and a public IP will be assigned for the nginx-ingress service.
This load balancer/public ip will be used for incoming connection for all your services, and nginx-ingress will be the responsible to handle the incoming connections.
For example:
Supose you have 10 services of LoadBalancer type: This will result in 10 new publics ips created and you need to use the correspondent ip for the service you want to reach.
But if you use a ingress, only 1 IP will be created and the ingress will be the responsible to handle the incoming connection for the correct service based on PATH/URL you defined in the ingress configuration. With ingress you can:
Use regex in path to define the service to redirect;
Use SSL/TLS
Inject custom headers;
Redirect requests for a default service if one of the service failed (default-backend);
Create whitelists based on IPs
Etc...
A important note about Ingress Load balancing in ingress:
GCE/AWS load balancers do not provide weights for their target pools. This was not an issue with the old LB kube-proxy rules which would correctly balance across all endpoints.
With the new functionality, the external traffic is not equally load balanced across pods, but rather equally balanced at the node level (because GCE/AWS and other external LB implementations do not have the ability for specifying the weight per node, they balance equally across all target nodes, disregarding the number of pods on each node).
An ingress controller(nginx for example) pods needs to be exposed outside the kubernetes cluster as an entry point of all north-south traffic coming into the kubernetes cluster. One way to do that is via a LoadBalancer. You could use NodePort as well but it's not recommended for production or you could just deploy the ingress controller directly on the host network on a host with a public ip. Having a load balancer also gives ability to load balance the traffic across multiple replicas of ingress controller pods.
When you use ingress controller the traffic comes from the loadBalancer to the ingress controller and then gets to backend POD IPs based on the rules defined in ingress resource. This bypasses the kubernetes service and load balancing(by kube-proxy at layer 4) offered by kubernetes service.Internally the ingress controller discovers all the POD IPs from the kubernetes service's endpoints and directly route traffic to the pods.
It seems loadbalancing is being done by services. URL based routing is being done by ingress controller.
Services do balance the traffic between pods. But they aren't accessible outside the kubernetes in Google Kubernetes Engine by default (ClusterIP type). You can create services with LoadBalancer type, but each service will get its own IP address (Network Load Balancer) so it can get expensive. Also if you have one application that has different services it's much better to use Ingress objects that provides single entry point. When you create an Ingress object, the Ingress controller (e.g. nginx one) creates a Google Cloud HTTP(S) load balancer. An Ingress object, in turn, can be associated with one or more Service objects.
Then you can get the assigned load balancer IP from ingress object:
kubectl get ingress ingress-name --output yaml
As a result your application in pods become accessible outside the kubernetes cluster:
LoadBalancerIP/url1 -> service1 -> pods
LoadBalancerIP/url2 -> service2 -> pods
I have a docker image that serves a simple static web page.
I have a working Kubernetes cluster of 4 nodes (physical servers not in the cloud anywhere).
I want to run that docker image on 2 of the 4 Kubernetes nodes and have it be accessible to the world outside the cluster and load balanced and have it move it to another node if one dies.
Do I need to make a pod then a replication controller then a kube proxy something?
Or do I need to just make a replication controller and expose it somehow?
Do I need to make service?
I don't need help with how to make any of those things, that seems well documented, but what I can't tell what I need to make.
What you need is to expose your service (that consists of pods which are run/scaled/restarted by your replication controller). Using deployment instead of replication controller has additional benefits (mainly for updating the app).
If you are on bare metal then you probably wish to expose your service via type: NodePort - so every node in your cluster will open a static port that routes traffic to pods.
You can then either point your load balancer to that nodes on that port, or make a DNS entry with all Kubernetes nodes.
Docs: http://kubernetes.io/docs/user-guide/quick-start/
You'll need:
1) A load balancer on one of your nodes in your cluster, that is a reverse proxy Pod like nginx to proxy the traffic to an upstream.
This Pod will need to be exposed to the outside using hostPort like
ports:
- containerPort: 80
hostPort: 80
name: http
- containerPort: 443
hostPort: 443
name: https
2) A Service that will use the web server selector as target.
3) Set the Service name (which will resolve to the Service IP) as the upstream in nginx config
4) Deploy your web server Pods, which will have the selector to be targeted by the Service.
You might also want to look at External IP for the Service
http://kubernetes.io/docs/user-guide/services/#external-ips
but I personally never managed to get that working on my bare metal cluster.