Having started the build of a new K8s 1.21 cluster: -
kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=$private_ip --apiserver-cert-extra-sans=$public_ip --kubernetes-version ${KUBE_VERSION}
I saw: -
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI.
Here is one example how you may list all Kubernetes containers running in cri-o/containerd using crictl:
- 'crictl --runtime-endpoint /run/containerd/containerd.sock ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'crictl --runtime-endpoint /run/containerd/containerd.sock logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
To see the stack trace of this error execute with --v=5 or higher
I did a spot of digging in the system log ( /var/log/syslog ) and found this: -
Jun 6 12:17:53 hurlinux2 containerd[39485]: time="2021-06-06T12:17:53.210883120-07:00" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-apiserver-telicity1.fyre.ibm.com,Uid:599ab88dc99dd5fbdb7c6a92e4e965ba,Namespace:kube-system,Attempt:0,} failed, error" error="failed to create containerd task: failed to create shim: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v2.task/k8s.io/177b079efbe1b71b65586a467313d1a15e802a2b2d323a7ed974d5d2e99e33f5/log.json: no such file or directory): exec: \"runc\": executable file not found in $PATH: unknown"
Jun 6 12:17:53 hurlinux2 kubelet[41466]: E0606 12:17:53.211759 41466 remote_runtime.go:116] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create containerd task: failed to create shim: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v2.task/k8s.io/177b079efbe1b71b65586a467313d1a15e802a2b2d323a7ed974d5d2e99e33f5/log.json: no such file or directory): exec: \"runc\": executable file not found in $PATH: unknown"
which made me think "Hmmmm, wonder what I forgot"
A quick trip to Aptitude ...
apt-get install runc
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
runc
0 upgraded, 1 newly installed, 0 to remove and 8 not upgraded.
Need to get 4,018 kB of archives.
After this operation, 15.7 MB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu focal-updates/main amd64 runc amd64 1.0.0~rc93-0ubuntu1~20.04.2 [4,018 kB]
Fetched 4,018 kB in 1s (4,037 kB/s)
Selecting previously unselected package runc.
(Reading database ... 107797 files and directories currently installed.)
Preparing to unpack .../runc_1.0.0~rc93-0ubuntu1~20.04.2_amd64.deb ...
Unpacking runc (1.0.0~rc93-0ubuntu1~20.04.2) ...
Setting up runc (1.0.0~rc93-0ubuntu1~20.04.2) ...
Processing triggers for man-db (2.9.1-1) ...
and a kubeadm reset ( to clear down the borked cluster creation ) and I was then able to re-run kubeadm init as before, and we're good to go .....
No comments:
Post a Comment