You can find mentions of that resource in the following Questions: 1, 2. But I am not able to figure out what is the use of this resource.
Yes, it's true, the provided (in comments) link to the documentation might be confusing so let me try to clarify you this.
As per the official documentation the apiserver proxy:
is a bastion built into the apiserver
connects a user outside of the cluster to cluster IPs which otherwise might not be reachable
runs in the apiserver processes
client to proxy uses HTTPS (or http if apiserver so configured)
proxy to target may use HTTP or HTTPS as chosen by proxy using available information
can be used to reach a Node, Pod, or Service
does load balancing when used to reach a Service
So answering your question - setting node/proxyresource in clusterRole allows k8s services access kubelet endpoints on specific node and path.
As per the official documentation:
There are two primary communication paths from the control plane
(apiserver) to the nodes. The first is from the apiserver to the
kubelet process which runs on each node in the cluster. The second is
from the apiserver to any node, pod, or service through the
apiserver's proxy functionality.
The connections from the apiserver to the kubelet are used for:
fetching logs for pods
attaching (through kubectl) to running pods
providing the kubelet's port-forwarding functionality
Here are also few running examples of using node/proxy resource in clusterRole:
How to Setup Prometheus Monitoring On Kubernetes Cluster
Running Prometheus on Kubernetes
It is hard to find in Kubernetes official documents any information about this sub-resource.
In context of RBAC, the format node/proxy can be used to grant access to the sub-resource named proxy for node resource. Also the same access can be granted for pods and services.
We can see it from the output of available resourses from the Kubernetes API server (API Version: v1.21.0):
===/api/v1===
...
nodes/proxy
...
pods/proxy
...
services/proxy
...
Detailed information about usage of proxy sub-resource can be found in The Kubernetes API (depends on the version you use) - section Proxy Operations for every mentioned resource: pods, nodes, services.
Related
I was making some research about how K8s resolves the services using the clusterIP services and how CNIs like WeaveNet or how service meshes like Istio provide additional features to this functionality. However, I'm new on the topic and I'd like to share here what I've found to see if somebody can expand and correct my points:
Istiod has a service registry. This service registry is filled with the entries coming from K8s services clusterIPs (which in turn is the service registry of K8s) and other possible external services defined with Kind: ServiceEntry
(see seciton 5.5 of book istio in action)
This service registry is then mixed with more information about virtualservices and destination rules. These new/added K8s kinds are CRDs from Istio. They are what give the features of L7 load balancing that allow to distribute traffic by HTTP headers or URI path.
Without Istio, K8s has different (3) ways to implement the clusterIPs services concept. This services provide load balancing at L4.
https://kubernetes.io/docs/concepts/services-networking/service/
The most extended one nowadays is the iptables proxy mode. The iptables of the Linux machine are populated in bases of what theh kube-proxy provides. Kube-proxy gets those data from the kube-apiserver and (problably the core-dns). The kube-apisever will in turn consult the etcd database to know about the k8s clusterIP services. The entry of the iptables is populated with a the clusterIP->pod IP with only one pod IP out of the many pod that a deployment behind the clusterIP could be.
Any piece of code/application inside of the container could make calls directy to the kube-apiserver if using the correct authentication and get the pod address but that would be not practic
K8s can use CNIs (container network interfaces). One example of this would be Weavenet.
https://www.weave.works/docs/net/latest/overview/
Wevenet creates a new layer 2 network using Linux kernel features. One daemon sets up this L2 network and manages the routing between machines and there are various ways to attach machines to the network.
In this network the containers can be exposed to the outside world.
Weavenet implements a micro DNS server at each node. You simply name containers and the routing just can work without the use of services, including the load balancing across multiple continers with the same name.
I see in an article that I can access to pods from kubeproxy, so what is the role of kubernetes service here? and what is the difference between Kube Proxy and service? finally,
is kube proxy part of service?
As far as I understand:
Service is a Kubernetes object that has a stable name and stable IP and sits in front of a set of pods. All requests sent to the pods should go to the service.
Kube-proxy is a networking component running on every cluster node(basically its a Daemonset). It implements the low-level rules to allow communication to pods from inside as well as outside the Kubernetes Cluster. We can say that kube-proxy is a part of service.
So when a user tries to reach an application deployed on Kubernetes first it reaches the service and then forwards the request one of the underlying pods. This is done by using the rules that Kube proxy created.
For more understanding refer this video : Kube proxy & blog
Closer look at Kube proxy
From my understanding
If you are only accessing the pod ports inside the cluster, then there are no Service involved, as you need Service objects to expose your pods outside of your Cluster
Service exposes your pods outside of your Cluster. Service provides a stable virtual IP address. A controller keeps track of the pods that are associated with the Service. While kube-proxy is a daemon running on each node and watches the service resources defined in the cluster and manages the rules for the requests on a Service’s backend pods
kube-proxy interacts with the Service so kube-proxy can change the iptable rules when there are changes on Service objects. Hence they are separate entities.
We can discuss this for a while, but let's short a long story.
Requests come to Service
Then Service passes it to Kube-Proxy
Kube-Proxy decides to which Pod this request go
How requests are forwarded from Service to Pod
Kube Proxy forwards the request
Responsible for maintaining a list of Service IPs and corresponding Pod IPs
Check this section for more details...
I have a multi-node cluster setup. There are Kubernetes network policies defined for the pods in the cluster. I can access the services or pods using their clusterIP/podIP only from the node where the pod resides. For services with multiple pods, I cannot access the service from the node at all (I guess when the service directs the traffic to the pod with the resident node same as from where I am calling then the service will work).
Is this the expected behavior?
Is it a Kubernetes limitation or a security feature?
For debugging etc., we might need to access the services from the node. How can I achieve it?
No, it is not the expected behavior for Kubernetes. Pods should be accessible for all the nodes inside the same cluster through their internal IPs. ClusterIP service exposes the service on a cluster-internal IP and making it reachable from within the cluster - it is basically set by default for all the service types, as stated in Kubernetes documentation.
Services are not node-specific and they can point to a pod regardless of where it runs in the cluster at any given moment in time. Also make sure that you are using the cluster-internal port: while trying to reach the services. If you still can connect to the pod only from node where it is running, you might need to check if something is wrong with your networking - e.g, check if UDP ports are blocked.
EDIT: Concerning network policies - by default, a pod is non-isolated either for egress or ingress, i.e. if no NetworkPolicy resource is defined for the pod in Kubernetes, all traffic is allowed to/from this pod - so-called default-allow behavior. Basically, without network policies all pods are allowed to communicate with all other pods/services in the same cluster, as described above.
If one or more NetworkPolicy is applied to a particular pod, it will reject all traffic that is not explicitly allowed by that policies (meaning, NetworkPolicythat both selects the pod and has "Ingress"/"Egress" in its policyTypes) - default-deny behavior.
What is more:
Network policies do not conflict; they are additive. If any policy or policies apply to a given pod for a given direction, the connections allowed in that direction from that pod is the union of what the applicable policies allow.
So yes, it is expected behavior for Kubernetes NetworkPolicy - when a pod is isolated for ingress/egress, the only allowed connections into/from the pod are those from the pod's node and those allowed by the connection list of NetworkPolicy defined.
To be compatible with it, Calico network policy follows the same behavior for Kubernetes pods.
NetworkPolicy is applied to pods within a particular namespace - either the same or different with the help of the selectors.
As for node specific policies - nodes can't be targeted by their Kubernetes identities, instead CIDR notation should be used in form of ipBlock in pod/service NetworkPolicy - particular IP ranges are selected to allow as ingress sources or egress destinations for pod/service.
Whitelisting Calico IP addresses for each node might seem to be a valid option in this case, please have a look at the similar issue described here.
So the idea is Kubernetes dashboard accesses Kubernetes API to give us beautiful visualizations of different 'kinds' running in the Kubernetes cluster and the method by which we access the Kubernetes dashboard is by the proxy mechanism of the Kubernetes API which can then be exposed to a public host for public access.
My question would be is there any possibility that we can access Kubernetes API proxy mechanism for some other service inside a Kubernetes cluster via that publically exposed address of Kubernetes Dashboard?
Sure you can. So after you set up your proxy with kubectl proxy, you can access the services with this format:
http://localhost:8001/api/v1/namespaces/kube-system/services/<service-name>:<port-name>/proxy/
For example for http-svc and port name http:
http://localhost:8001/api/v1/namespaces/default/services/http-svc:http/proxy/
Note: it's not necessarily for public access, but rather a proxy for you to connect from your public machine (say your laptop) to a private Kubernetes cluster.
You can do it by changing your service to NodePort:
$ kubectl -n kube-system edit service kubernetes-dashboard
You should see yaml representation of the service. Change type: ClusterIP to type: NodePort and save file.
Note: This way of accessing Dashboard is only possible if you choose to install your user certificates in the browser. Certificates used by kubeconfig file to contact API Server can be used.
Please check the following articles and URLs for better understanding:
Stackoverflow thread
Accessing Dashboard 1.7.X and above
Deploying a publicly accessible Kubernetes Dashboard
How to access kubernetes dashboard from outside cluster
Hope it will help you!
Exposing Kubernetes Dashboard not secure at all , but your answer is about K8s API Server that need to be accessible by external services.
The right answer differs according your platform and infrastructure , but as general points
[Network Security] Limit IP public reachability to K8s API Servers(s) / Load balancer if exist as a white list mechanism
[Network Security] Private-to-Private reachability is better like vpn or AWS PrivateLink
[ API Security ] Limit Privileges by clusterrole/role to enforce RBAC , better to keep it ReadOnly verbs { Get , List }
[ API Security ] enable audit logging for k8s components to keep track of events and actions
I have deployment my kubernetes cluster using kubeadm.
Now I want to gather cluster based information like master node IP, port on which apiserver is listening and name of the cluster.
With kubectl cluster-info gives me some data but I am looking to fetch cluster level information with the help of K8s rest API.
One way which i have tried is look for apiserver pod and get the data. It's giving me cluster level data but I need some other cleaner way of doing it.
Thanks in advance!
If you have ran the apiserver, you can access the kubernetes REST API on port 8001.
One way to expose it is like this :
sudo kubectl proxy --address='0.0.0.0' --port=8001 --accept-hosts='^*$'&
then you can visit http://YOUR_VM_IP:8001/api
there you can see all the list of APIs and all the information you want.