I am trying to enter to a running pod using command -
kubectl exec pod1-temp-6b6b4b75c6-c9hn5 -- /bin/bash
but its throwing me Error saying-
error: Internal error occurred: error executing command in container:
failed to exec in container: failed to start exec
"ee928c2cbe5699f60603abd7a0d94bae57ee2eb4bb3d3a74bc90aef7dbd1a4d8":
OCI runtime exec failed: exec failed: container_linux.go:380: starting
container process caused: exec: "C:/Program Files/Git/usr/bin/bash":
stat C:/Program Files/Git/usr/bin/bash: no such file or directory:
unknown
Its working for others but in my system its causing issue that means its local issue.
I have also checked the path and bash exec. file is also there as shown below -
enter image description here
[WARNING Firewalld]: firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: time="2023-02-19T02:25:52Z" level=fatal msg="validate service connection: CRI v1 runtime API is not implemented for endpoint "unix:///var/run/containerd/containerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher
I Enabled the port 6443 and 10250 in Security group.
Also tried the commands-
rm /etc/containerd/config.toml
systemctl restart containerd
kubeadm init
The containerd is not available in etc directory.
I am creating a egress-operator. I have a pod egress-operator-controller-manager created from makefile by command make deploy IMG=my_azure_repo/egress-operator:v0.1. The pod is failing , its showing error in pod description as:
State: Waiting
Reason: RunContainerError
Last State: Terminated
Reason: StartError
Message: failed to create containerd task: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "/manager": stat /manager: no such file or directory: unknown
Failed 44s (x3 over 59s) kubelet Error: failed to create containerd task:
OCI runtime create failed: container_linux.go:380: starting container process caused:
exec: "/manager": stat /manager: no such file or directory: unknown
I suspect that in manager.yaml, under command: /manager is executed.Can someone let me know what is going wrong in this manager.yaml and whether /manager is valid under command: in deployment.yaml
Debugging UPDATE
Instead of running Dockerfile, now I just build and run image_azure_repo/egress-operator:v0.1 locally (on same ubuntu 18 VM), When I try to login with: docker exec -it <container_id> /bin/bash, I am getting error :
OCI runtime exec failed: exec failed: container_linux.go:380:
starting container process caused: exec: "/bin/bash": stat /bin/bash:
no such file or directory: unknown
This is similar error what I see in pod description. Instead of /bin/bash , I also tried docker exec with /bin/sh and only sh ; Its giving same error
This should be a mismatch for libc.
Golang uses glibc by default while alpine uses musl.
You could try those to fix it:
Use a base image using glibc
Install glibc in your alpine image
In Dockerfile , the Entrypoint you have mentioned is ENTRYPOINT ["/manager"] but shouldnt it be ENTRYPOINT ["./manager"]
regarding "starting container process caused: exec: "/bin/bash": stat /bin/bash: " , it seems base image does not have /bin/bash it might have just /bin/sh
Env:amazon linux2
kernel version: 4.14.198-152.320.amzn2.x86_64
concourse version:v6.6.0
Issue:
After I setup concourse on amazon EC2, I can login concourse web and create pipeline. But when I trigger the pipeline build, I got below error message:
runc run: exit status 1: container_linux.go:367: starting container process caused: process_linux.go:338: getting the final child's pid from pipe caused: EOF
Checking the backend log:
{"timestamp":"2020-10-15T03:14:44.622775204Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.create.completing","data":{"bundle":"/home/concourse/worker/depot/2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","session":"99.3.2"}}
{"timestamp":"2020-10-15T03:14:44.624243772Z","level":"debug","source":"guardian","message":"guardian.create.containerizer-create.create.runc","data":{"bundle":"/home/concourse/worker/depot/2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","message":"nsexec:601 nsexec started","session":"99.3.2"}}
{"timestamp":"2020-10-15T03:14:44.624281598Z","level":"debug","source":"guardian","message":"guardian.create.containerizer-create.create.runc","data":{"bundle":"/home/concourse/worker/depot/2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","message":"try_mapping_tool:240 nsenter: mapping tool not present: Operation not permitted","session":"99.3.2"}}
{"timestamp":"2020-10-15T03:14:44.624308669Z","level":"debug","source":"guardian","message":"guardian.create.containerizer-create.create.runc","data":{"bundle":"/home/concourse/worker/depot/2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","message":"nsexec:889 nsenter: failed to sync with parent: read(SYNC_USERMAP_ACK): Invalid argument","session":"99.3.2"}}
{"timestamp":"2020-10-15T03:14:44.624328533Z","level":"debug","source":"guardian","message":"guardian.create.containerizer-create.create.runc","data":{"bundle":"/home/concourse/worker/depot/2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","message":"log pipe has been closed: EOF","session":"99.3.2"}}
{"timestamp":"2020-10-15T03:14:44.624349468Z","level":"debug","source":"guardian","message":"guardian.create.containerizer-create.create.runc","data":{"bundle":"/home/concourse/worker/depot/2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","message":"container_linux.go:367: starting container process caused: process_linux.go:338: getting the final child's pid from pipe caused: EOF","session":"99.3.2"}}
{"timestamp":"2020-10-15T03:14:44.624379918Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.create.finished","data":{"bundle":"/home/concourse/worker/depot/2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","session":"99.3.2"}}
{"timestamp":"2020-10-15T03:14:44.624412285Z","level":"error","source":"guardian","message":"guardian.create.containerizer-create.runtime-create-failed","data":{"error":"runc run: exit status 1: container_linux.go:367: starting container process caused: process_linux.go:338: getting the final child's pid from pipe caused: EOF","handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","session":"99.3"}}
{"timestamp":"2020-10-15T03:14:44.624429680Z","level":"info","source":"guardian","message":"guardian.create.containerizer-create.finished","data":{"handle":"2b4ce51a-c28d-4aac-7122-b82cd1cbb99e","session":"99.3"}}
I already highlight the message which may cause this issue. Thank you
to help me to find the root cause.
I found the root cause is system missing some tools. I replace it using a new system image, then working fine.
I am using docker to install postgresql 9.6 from centos7 image. I got an error while initialize the postgresql and also problem with this service.
sh-4.2# usr/pgsql-9.6/bin/postgresql96-setup initdb
Failed to get D-Bus connection: Operation not permitted
failed to find PGDATA setting in postgresql-9.6.service
sh-4.2# systemctl start postgresql-9.6.service
Failed to get D-Bus connection: Operation not permitted