I'm kinda new to Kubernetes, and I would like to understand what is the purpose of Kube-proxy in Azure AKS/regular cluster.
from what I understand, Kube-proxy is updated by the API cluster from the various deployments configurations, which then updates the IP-table stack in the Linux kernel that responsible for the traffic routes between pods and services.
Am I missing something important?
Thanks!!
Basically kube-proxy component runs on each node to provide network features. It is run as a Kubernetes DaemonSet and its configuration is stored on a Kubernetes ConfigMap. You can edit the kube-proxy DaemonSet or ConfigMap on the kube-system namespace using commands:
$ kubectl -n kube-system edit daemonset kube-proxy
or
$ kubectl -n kube-system edit configmap kube-proxy
kube-proxy currently supports three different operation modes:
User space: This mode gets its name because the service routing takes place in kube-proxy in the user process space
instead of in the kernel network stack. It is not commonly used as it is slow and outdated.
IPVS (IP Virtual Server): Built on the Netfilter framework, IPVS implements Layer-4 load balancing in the Linux kernel, supporting multiple load-balancing algorithms, including least connections and shortest expected delay. This kube-proxy mode became generally available in Kubernetes 1.11, but it requires the Linux kernel to have the IPVS modules loaded. It is also not as widely supported by various Kubernetes networking projects as the iptables mode.
iptables: This mode uses Linux kernel-level Netfilter rules to configure all routing for Kubernetes Services. This mode is the default for kube-proxy on most platforms. When load balancing for multiple backend pods, it uses unweighted round-robin scheduling.
IPVS (IP Virtual Server): Built on the Netfilter framework, IPVS implements Layer-4 load balancing in the Linux kernel, supporting multiple load-balancing algorithms, including least connections and shortest expected delay. This kube-proxy mode became generally available in Kubernetes 1.11, but it requires the Linux kernel to have the IPVS modules loaded. It is also not as widely supported by various Kubernetes networking projects as the iptables mode.
Take a look: kube-proxy, kube-proxy-article, aks-kube-proxy.
Read also: proxies-in-kubernetes.
Related
I was making some research about how K8s resolves the services using the clusterIP services and how CNIs like WeaveNet or how service meshes like Istio provide additional features to this functionality. However, I'm new on the topic and I'd like to share here what I've found to see if somebody can expand and correct my points:
Istiod has a service registry. This service registry is filled with the entries coming from K8s services clusterIPs (which in turn is the service registry of K8s) and other possible external services defined with Kind: ServiceEntry
(see seciton 5.5 of book istio in action)
This service registry is then mixed with more information about virtualservices and destination rules. These new/added K8s kinds are CRDs from Istio. They are what give the features of L7 load balancing that allow to distribute traffic by HTTP headers or URI path.
Without Istio, K8s has different (3) ways to implement the clusterIPs services concept. This services provide load balancing at L4.
https://kubernetes.io/docs/concepts/services-networking/service/
The most extended one nowadays is the iptables proxy mode. The iptables of the Linux machine are populated in bases of what theh kube-proxy provides. Kube-proxy gets those data from the kube-apiserver and (problably the core-dns). The kube-apisever will in turn consult the etcd database to know about the k8s clusterIP services. The entry of the iptables is populated with a the clusterIP->pod IP with only one pod IP out of the many pod that a deployment behind the clusterIP could be.
Any piece of code/application inside of the container could make calls directy to the kube-apiserver if using the correct authentication and get the pod address but that would be not practic
K8s can use CNIs (container network interfaces). One example of this would be Weavenet.
https://www.weave.works/docs/net/latest/overview/
Wevenet creates a new layer 2 network using Linux kernel features. One daemon sets up this L2 network and manages the routing between machines and there are various ways to attach machines to the network.
In this network the containers can be exposed to the outside world.
Weavenet implements a micro DNS server at each node. You simply name containers and the routing just can work without the use of services, including the load balancing across multiple continers with the same name.
I have this mixed cluster which shows all the nodes as Ready (both Windows & Linux ones). However, only the Linux nodes have aws-node & kube-proxy pods. I RDPed into a Windows node and can see a kube-proxy service.
My question remains: do the Windows nodes need aws-node & kube-proxy pods in the kube-system namespace or do they work differently than Linux ones?
kube-proxy pods are part of default installation of Kubernetes. They are automaticaly created, and are needed on both Linux and Windows.
kube-proxy is a network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service concept.
kube-proxy maintains network rules on nodes. These network rules allow network communication to your Pods from network sessions inside or outside of your cluster.
[source]
aws-node pod is a part of AWS CNI plugin for Kubernetes
The Amazon VPC Container Network Interface (CNI) plugin for Kubernetes is deployed with each of your Amazon EC2 nodes in a Daemonset with the name aws-node. The plugin consists of two primary components:
[...]
[source]
It is currently only supported on Linux. Windows nodes are using a different CNI plugin - vpc-shared-eni
In the documentation about affinity and anti-affinity rules for kubernetes there is a pratical use case arround a web application and a local redis cache.
The redis deployment has PodAntiAffinity configured to ensure the scheduler does not co-locate replicas on a single node.
The webapplication deployment has a pod affinity to ensure the app is scheduled with the pod that has label store (Redis).
To connect to the redis from the webapp we would have to define a service.
Question: How are we sure that the webapp will always use the redis that is co-located on the same node and not another one? If I read the version compatibility from Kubernetes v1.2 the iptables mode for kube-proxy became the default.
Reading the docs about iptable mode for kube-proxy it says by default, kube-proxy in iptables mode chooses a backend at random.
So my answer to the question would be:
No we can't be sure. If you want to be sure then put the redis and webapp in one pod?
This can be configured in the (redis) Service, but in general it is not recommended:
Setting spec.externalTrafficPolicy to the value Local will only proxy requests to local endpoints, never forwarding traffic to other nodes
This is a complex topic, read more here:
https://kubernetes.io/docs/tutorials/services/source-ip/
https://kubernetes.io/docs/concepts/services-networking/service/
Is there an easy way to enable Network Policies in single-node k8s cluster managed by Docker Desktop for Mac?
A single-node k8s cluster managed by Docker Desktop for Mac is imply a VM provisioned by the Docker for Mac Daemon that is then bootstrapped with a Kubernetes cluster. Docker has extended this solution in some ways to make it easier for developers to use but it is effectively similar to using Minikube.
A NetworkPolicy is a Kubernetes resource and as you have discovered, it is not enabled in your environment by default. This is because the NetworkPolicy resource requires a controller to be installed to enabled the enforcement of NetworkPolicy rules after they have been declared. Many applications can be installed to provide this functionality. The most common way is by installing a CNI like Calico.
After you do this, Calico will be able to enforce your NetworkPolicy rules that you have defined. They will automatically move from the Pending to Ready state in the cluster.
I've had the opportunity to install k8s clusters on CentOS VMs. In most cases, i used flanneld as overlay. On some other cases, though, i noticed flannel pods in kube-system namespace. IMHO, we need not have both flanneld and flannel pods for underlying CNI to function properly with kubernetes.
Have read plenty of documentation on how flannel overlay fits into kubernetes ecosystem. However, i haven't found the answers to some questions. Hope somebody can provide pointers.
What is the basis for choosing flanneld or flannel pod?
Are there any differences in functionality between flanneld and flannel pod?
How does the flannel pod provide CNI functionality? My understanding is the pod populates etcd with IP address k/v pairs but how is this info really used?
Do most CNI plugins have a choice between running as daemon or pod?
You are right, you don't need both of them because they do the same job. There is no differences between them just where the daemon run in system, in isolated container or in system as regular daemon. All CNI plugins bases on CNI library and route the traffic. Flannel use system ETCD as key-value storage. if you have ETCD inside kubernetes cluster it will use this if external it will use external ETCD. it is only you choose what prefer to you, For example If you are running external ETCD usually people running flannel as daemon in system.