Kubernetes install in offline mode error : timeout at wait-control-plane - kubernetes

Question

I am new to Kubernetes and I am trying to install Kubernetes with kubeadm in offline mode. I am using Kubernetes v1.24.1. After bringing all the images into my Oracle linux 8 server and doing the neccessary changes as suggested by the official guide my Kubeadm init command is running into a timeout with the following error:
[root#hhpsoscn0001 ~]# kubeadm init --ignore-preflight-errors all --config Configfile.yaml
[init] Using Kubernetes version: v1.24.1
[preflight] Running pre-flight checks
[WARNING Firewalld]: firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly
[WARNING FileAvailable--etc-kubernetes-manifests-kube-apiserver.yaml]: /etc/kubernetes/manifests/kube-apiserver.yaml already exists
[WARNING FileAvailable--etc-kubernetes-manifests-kube-controller-manager.yaml]: /etc/kubernetes/manifests/kube-controller-manager.yaml already exists
[WARNING FileAvailable--etc-kubernetes-manifests-kube-scheduler.yaml]: /etc/kubernetes/manifests/kube-scheduler.yaml already exists
[WARNING FileAvailable--etc-kubernetes-manifests-etcd.yaml]: /etc/kubernetes/manifests/etcd.yaml already exists
[WARNING Port-10250]: Port 10250 is in use
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Using existing ca certificate authority
[certs] Using existing apiserver certificate and key on disk
[certs] Using existing apiserver-kubelet-client certificate and key on disk
[certs] Using existing front-proxy-ca certificate authority
[certs] Using existing front-proxy-client certificate and key on disk
[certs] Using existing etcd/ca certificate authority
[certs] Using existing etcd/server certificate and key on disk
[certs] Using existing etcd/peer certificate and key on disk
[certs] Using existing etcd/healthcheck-client certificate and key on disk
[certs] Using existing apiserver-etcd-client certificate and key on disk
[certs] Using the existing "sa" key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/admin.conf"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Using existing kubeconfig file: "/etc/kubernetes/scheduler.conf"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all running Kubernetes containers by using crictl:
- 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint unix:///var/run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
Running Kubelet status command is showing me this
[root#hhpsoscn0001 ~]# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Fri 2022-06-24 08:32:31 UTC; 7min ago
Docs: https://kubernetes.io/docs/
Main PID: 155445 (kubelet)
Tasks: 20 (limit: 202900)
Memory: 47.3M
CGroup: /system.slice/kubelet.service
└─155445 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.141148 155445 kubelet.go:2419] "Error getting node" err="node \"hhpsoscn0001\" not foun>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.242350 155445 kubelet.go:2419] "Error getting node" err="node \"hhpsoscn0001\" not foun>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.342806 155445 kubelet.go:2419] "Error getting node" err="node \"hhpsoscn0001\" not foun>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.381220 155445 controller.go:144] failed to ensure lease exists, will retry in 7s, error>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.443850 155445 kubelet.go:2419] "Error getting node" err="node \"hhpsoscn0001\" not foun>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: I0624 08:39:51.539027 155445 kubelet_node_status.go:70] "Attempting to register node" node="hhpsoscn00>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.539440 155445 kubelet_node_status.go:92] "Unable to register node with API server" err=>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.544727 155445 kubelet.go:2419] "Error getting node" err="node \"hhpsoscn0001\" not foun>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.645092 155445 kubelet.go:2419] "Error getting node" err="node \"hhpsoscn0001\" not foun>
Jun 24 08:39:51 hhpsoscn0001 kubelet[155445]: E0624 08:39:51.745630 155445 kubelet.go:2419] "Error getting node" err="node \"hhpsoscn0001\" not foun>
lines 1-22/22 (END)
and ps x | grep kubelet command shows me this :
[root#hhpsoscn0001 ~]# ps x | grep kubelet
156080 ? Ssl 0:10 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/containerd/containerd.sock --pod-infra-container-image=docker-hub.m-net.vv/k8s/pause:3.7
156734 pts/4 S+ 0:00 grep --color=auto kubelet
Can somebody please tell me what is wrong?

Answer 1

Looks like you have tried to run kubeadm init multiple times.
Try this out
kubeadm reset --force
This will reset your bootstrap server, once done this should come up. Right now from the provided logs ... it looks like you must have changed the hostname post init or could be kubelet bootup was form previous init.