I'm trying to understand what the memory capacity means when running: kubectl describe node <mynode>
Snippet of the Result:
status:
capacity:
cpu: '16'
ephemeral-storage: 129886128Ki
hugepages-1Gi: '0'
hugepages-2Mi: '0'
memory: 65861880Ki
pods: '110'
allocatable:
cpu: 15820m
ephemeral-storage: '119703055367'
hugepages-1Gi: '0'
hugepages-2Mi: '0'
memory: 61361400Ki
pods: '110'
I know the memory under allocatable is a calculation based on the information we can find on resource reservations. My question is not about this.
Why is the capacity 65861880Ki while this is a VM with 64G of memory. I assume 64GiB and not GB but even then 65861880Ki is lower than 64GiB (67108864KiB) which is a difference of 1246984KiB (=1,19GiB or 1.3GB).
What am I missing?
Related
I am managing a Kubernetes cluster with 10 nodes(On-prem) and the node's configuration is not identical, 5 nodes are of 64 cores and 125G ram, and 5 nodes are of 64 cores and 256G ram.
Most of the time I keep getting alerts saying the node CPU/MEMORY is high and I see the pods are getting restarted, as it is consuming 92-95% of CPU and memory on certain nodes, I want to apply CPU and Memory Allocation on nodes so that the CPU utilization doesn't go very high.
I tried manually editing the node configuration but that did not work.
Any leads for this will be helpful!
In K8s, you can limit the resources usage for the pod containers and reserve a some cpus/memory for the container to avoid this problem:
---
apiVersion: v1
kind: Pod
metadata:
name: <pod name>
spec:
containers:
- name: c1
image: ...
resources:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
- name: c2
image: ...
resources:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
Found the Kubernetes document for setting the node level allocatable resource.
Fixed using the below document
https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable
https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/
I'm running a 4vCPU 8GB Node Pool, but all of my nodes report this for Capacity:
Capacity:
cpu: 4
ephemeral-storage: 165103360Ki
hugepages-2Mi: 0
memory: 8172516Ki
pods: 110
I'd expect it to show 8388608Ki (the equivalent of 8192Mi/8Gi).
How come?
Memory can be reserved for both system services (system-reserved) and the Kubelet itself (kube-reserved). https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ has details but DO is probably setting it up for you.
Suppose for a pod i define the resources as below
resources:
requests:
memory: 500Mi
cpu: 500m
limits:
memory: 1000Mi
cpu: 1000m
This means i would be requiring minimum of 1/2 cpu core (or 1/2 vCPU). In cloud (AWS) we have different ec2 families. if we create a cluster using C4 or R4 types of instances does the performance change. Do we need to baseline the CPU usage based on the instance family on which we are going to run the pod.
I am running filebeat as deamon set with 1Gi memory setting. my pods getting crashed with OOMKilled status.
Here is my limit setting
resources:
limits:
memory: 1Gi
requests:
cpu: 100m
memory: 1Gi
What is the recommended memory setting to run the filebeat.
Thanks
The RAM usage of Filebeat is relative to how much it is doing, in general. You can limit the number of harvesters to try and reduce things, but overall you just need to run it uncapped and measure what the normal usage is for your use case and scenario.
I would like to know, what are the actual memory and CPU capacity in mi and m in the following results:
Capacity:
alpha.kubernetes.io/nvidia-gpu: 0
cpu: 2
memory: 2050168Ki
pods: 20
Allocatable:
alpha.kubernetes.io/nvidia-gpu: 0
cpu: 2
memory: 2050168Ki
pods: 20
2 CPUs (2 cores) and 2050168Kb of RAM (more simply, 2GB). Which also happens to be the Minikube defaults.