We have a kubernetes cluster for the entire organisation. And there are namespaces for individual teams to host their projects in that cluster. The problem seems to be monitoring individual namespace. Since we have only access to our namespace we can't setup any monitoring for the PODs, containers, or the nodes on which our pods are residing. Is there any way to accomplish monitoring of only the things within the namespace without bothering the kubernetes cluster?
Monitoring should be provided on central level for all the basics of the cluster and more, but sure, you just need limited scope. If you deploy prometheus and configure only targets in your namespace you should be ok. Same goes for any other solutions.
For reference see https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Ckubernetes_sd_config%3E
# Optional namespace discovery. If omitted, all namespaces are used.
namespaces:
names:
[ - <string> ]
Related
Are there any limitations in Kubernetes when it comes to deploying multiple deployments simultaneously or concurrently under the same namespace ?
This is assuming the resources are available and sufficient.
So I've installed this https://github.com/prometheus-operator/kube-prometheus to my K8s cluster. The problem with monitoring it configured to monitor only kube-system and default namespaces.
How can I modify this to monitor any namespace. For example, I have created 100 namespaces. So how to not write RBAC for each.
References:
https://github.com/prometheus-operator/kube-prometheus/blob/main/manifests/prometheus-roleBindingSpecificNamespaces.yaml
https://github.com/prometheus-operator/kube-prometheus/blob/main/manifests/prometheus-roleSpecificNamespaces.yaml
https://prometheus-operator.dev/docs/kube/monitoring-other-namespaces/
Manifest(yml) with kubernetes resource type(kind: Namespace) can be applied through kubectl to create a virtual cluster
In our environment, manifest yaml's are applied using kubectl to create kubernetes resource types(deployment, service, autoscaling, ingress) under the given namespace
But, rancher is used to create kubernetes resource type(kind: Namespace virtual cluster).
What is the advantage of creating kubernetes resource type(Namespace) using rancher? instead of a manifest yaml applied through kubectl
Rancher uses concept of "Project" which is not present in "vanilla" kubernetes, which allows you to assign RBAC roles, PodSecurityPolicy etc to a group of namespaces in easy way.
If you are not using rancher to create projects and namespaces - you have to assign all these Roles and PSPs by yourself. For example, if you have default restricted policy on your cluster, namespace created by kubectl create namespace foo won't be able to run any pods by default, see https://rancher.com/docs/rancher/v2.5/en/admin-settings/pod-security-policies/
Namespaces that are not assigned to projects do not inherit PSPs, regardless of whether the PSP is assigned to a cluster or project. Because these namespaces have no PSPs, workload deployments to these namespaces will fail, which is the default Kubernetes behavior.
To sum it up, namespaces can be created using kubectl create namespace or manifests, but it might be cumbersome to make it all work well. Using rancher to provision namespaces is easier to maintain and troubleshoot.
As for advantages, having ability to group namespaces under "project" and assign resources, PSP and roles to a group of namespaces with rancher UI support is one of the main selling points of having rancher in a first place. Namespace objects themselves are basically the same as anywhere else.
I have a pod running in a statefulset but it needs to know the hostname or address of all pods running in another statefulset to communicate with them. The second statefulset is being created by a separate helm chart. Can the pod work this out dynamically? Can I inject this information into the pod through an env similar to setting .Status.ip?
Edit: Each statefulSet has its own headless service
As discussed in the comments, the way to go here is to use a service-resource as this will give you a static DNS within the cluster to reach all the pods that a targeted by that service.
The DNS for the service is:
the services name if you access it from within the same namespace
<my-service-name>.<namespace-name>.svc.cluster.local if you access it from another namespace, and where cluster.local is the clusters domain that might differ from cluster to cluster depending on the clusters configuration
If you further need more configuration options, e.g. when you want to deploy your chart into different cloud environments where the clusters-domain might actually differ, you can use kustomize.io to adjust your configuration at apply time.
I am trying to setup Prometheus for my Kubernetes cluster. I am using https://github.com/coreos/kube-prometheus . I am using default namespace only. When I apply resources from manifests (https://github.com/coreos/kube-prometheus/tree/master/manifests) folder, It creates resources but in targets, it doesn't show my pod service.
I am new to Kubernetes so I need help with this. How do I configure my pod to show up in Prometheus targets?
I tried https://github.com/coreos/kube-prometheus#quickstart
For better manageability use Prometheus Operator.
You need to define ServiceMonitor or PodMonitor
ServiceMonitor which declaratively specifies how groups of services should be monitored. The Operator automatically generates Prometheus scrape configuration based on the definition and the targets will have the IPs of all the pods behind the service.
PodMonitor, which declaratively specifies how groups of pods should be monitored. The Operator automatically generates Prometheus scrape configuration based on the definition and targets will have the Pod IP
Refer to the examples here