I have a multi-node cluster setup. There are Kubernetes network policies defined for the pods in the cluster. I can access the services or pods using their clusterIP/podIP only from the node where the pod resides. For services with multiple pods, I cannot access the service from the node at all (I guess when the service directs the traffic to the pod with the resident node same as from where I am calling then the service will work).
Is this the expected behavior?
Is it a Kubernetes limitation or a security feature?
For debugging etc., we might need to access the services from the node. How can I achieve it?
No, it is not the expected behavior for Kubernetes. Pods should be accessible for all the nodes inside the same cluster through their internal IPs. ClusterIP service exposes the service on a cluster-internal IP and making it reachable from within the cluster - it is basically set by default for all the service types, as stated in Kubernetes documentation.
Services are not node-specific and they can point to a pod regardless of where it runs in the cluster at any given moment in time. Also make sure that you are using the cluster-internal port: while trying to reach the services. If you still can connect to the pod only from node where it is running, you might need to check if something is wrong with your networking - e.g, check if UDP ports are blocked.
EDIT: Concerning network policies - by default, a pod is non-isolated either for egress or ingress, i.e. if no NetworkPolicy resource is defined for the pod in Kubernetes, all traffic is allowed to/from this pod - so-called default-allow behavior. Basically, without network policies all pods are allowed to communicate with all other pods/services in the same cluster, as described above.
If one or more NetworkPolicy is applied to a particular pod, it will reject all traffic that is not explicitly allowed by that policies (meaning, NetworkPolicythat both selects the pod and has "Ingress"/"Egress" in its policyTypes) - default-deny behavior.
What is more:
Network policies do not conflict; they are additive. If any policy or policies apply to a given pod for a given direction, the connections allowed in that direction from that pod is the union of what the applicable policies allow.
So yes, it is expected behavior for Kubernetes NetworkPolicy - when a pod is isolated for ingress/egress, the only allowed connections into/from the pod are those from the pod's node and those allowed by the connection list of NetworkPolicy defined.
To be compatible with it, Calico network policy follows the same behavior for Kubernetes pods.
NetworkPolicy is applied to pods within a particular namespace - either the same or different with the help of the selectors.
As for node specific policies - nodes can't be targeted by their Kubernetes identities, instead CIDR notation should be used in form of ipBlock in pod/service NetworkPolicy - particular IP ranges are selected to allow as ingress sources or egress destinations for pod/service.
Whitelisting Calico IP addresses for each node might seem to be a valid option in this case, please have a look at the similar issue described here.
Related
Could you please explain use of each "Kind" of OpenShift in a short sentences?
It is okay, that deployment contains data about, image source, pod counts, limits etc.
With the route we can determine the URL for each deployment as well as Ingress, but what is the difference and when should use route and when ingress?
And what is the exact use of service?
Thanks for your help in advance!
Your question cannot be answered simply in short words or one line answers, go through the links and explore more,
Deployment: It is used to change or modify the state of the pod. A pod can be one or more running containers or a group of duplicate pods called ReplicaSets.
Service: Each pod is given an IP address when using a Kubernetes service. The service provides accessibility, connects the appropriate pod automatically, and this address may not be directly identifiable.
Route:Similar to the Kubernetes Ingress resource, OpenShift's Route was developed with a few additional features, including the ability to split traffic between multiple backends.
Ingress: It offers routing rules for controlling who can access the services in a Kubernetes cluster.
Difference between route and ingress?
OpenShift uses HAProxy to get (HTTP) traffic into the cluster. Other Kubernetes distributions use the NGINX Ingress Controller or something similar. You can find more in this doc.
when to use route and ingress: It depends on your requirements. From the image below you can find the feature of the ingress and route and you select according to your requirements.
Exact use of service:
Each pod in a Kubernetes cluster has its own unique IP address. However, the IP addresses of the Pods in a Deployment change as they move around. Therefore, using Pod IP addresses directly is illogical. Even if the IP addresses of the member Pods change, you will always have a consistent IP address with a Service.
A Service also provides load balancing. Clients call a single, dependable IP address, and the Service's Pods distribute their requests evenly.
I want to create a globalnetworkpolicy for an interface. I am using Calico HostendPoint for the interface and defining globalnetworkpolicy for the hostendpoint. I would like to create a globalnetworkpolicy that allows only ingress from within the cluster. A sample is given here.
In-cluster traffic is the traffic from pods and from nodes.
I have the podCIDR, so I can use that to ensure that traffic from pods are allowed.
How do I allow traffic from nodes' own IPAddresses as per the link above?
What is the nodes' own IPaddresses mentioned in the link?
It is basically referring to Kubernetes node - more precisely to node resource, which is created when a calico/node instance is started. Calico automatically detects each node’s IP address and subnet, and alongside with AS association and tunnel address (IP-in-IP or VXLAN), they are listed in node resource configuration.
The external IP is perfectly reachable from outside the cluster. It's perfectly reachable from all nodes within the cluster. However, when I try to telnet to the URL from a pod within the cluster that is not on the same node as a pod that is part of the service backend, the connection always times out.
The external IP is reachable by pods that run on the same node as a pod that is part of the service backend.
All pods can perfectly reach the cluster IP of the service.
When I set externalTrafficPolicy to Cluster, the pods are able to reach the external URL regardless of what node they're on.
I am using iptables proxying and kubernetes 1.16
I'm completely at a loss here as to why this is happening. Is someone able to shed some light on this?
From the official doc here,
service.spec.externalTrafficPolicy - denotes if this Service desires to route external traffic to node-local or cluster-wide endpoints. There are two available options: Cluster (default) and Local. Cluster obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. Local preserves the client source IP and avoids a second hop for LoadBalancer and NodePort type services, but risks potentially imbalanced traffic spreading.
The service could be either node-local or cluster-wide endpoints. When you define the externalTrafficPolicy as Local, it means node-local. So, other nodes are not able to reach it.
So, you will need to set the externalTrafficPolicy as Cluster instead.
I want to write a minimal egress NetworkPolicy for a workload running on GKE that needs to talk to the apiserver.
Is there any other way of doing this other than creating the cluster, getting the address of the LB fronting the apiserver(s) and then templating this into a NetworkPolicy? Is there any way of picking an external apiserver IP beforehand that can remain static over cluster recreates? Or, as a last resort, getting some list of CIDRs that GKE will always choose the apiserver LB IP from?
best choice is to use calico by enabling network policy on GKE, check this documentation, and here are some tips to Allow or Denie ingress or engress
We are looking at setting up network policies for our Kubernetes cluster. However in at least one of our namespaces we have an ExternalName service (kubernetes reference - service types) for an AWS RDS intance. We would like to restrict traffic to this ExternalName service to be from a particular set of pods, or if that is not possible, from a particular namespace. Neither the namespace isolation policy or the NetworkPolicy resoure seem to apply to ExternalName services. After searching the documentation for both Weave and Project Calico, there doesn't seem to be any mention of such functionality.
Is it possible to restrict network traffic to an ExternalName service to be from a specific set of pods or from a particular namespace?
You can't really do that. ExternalName services are a DNS construct. A client performs a DNS lookup for the service and kube-dns returns the CNAME record for, in your case, the RDS instance. Then the client connects to RDS directly.
There are two possible ways to tackle this:
Block just DNS lookups (pods can still connect to the DB if they know the IP or fully qualified RDS hostname):
change namespace isolation to support ExternalName services
make kube-dns figure the client pod behind each request it gets
make kube-dns aware of namespace isolation settings and apply them, so it only returns CNAME records to authorized pods
Return DNS lookups, but block RDS connections:
extend NetworkPolicy somehow to also control egress traffic
blacklist/whitelist RDS IPs wholesale (easier said than done, since they are dynamic) or make the network controllers track the results from DNS lookups and block/allow connections accordingly.
In either case, you'll have to file a number of feature requests in Kubernetes and downstream.
Source: I wrote the EN support code.