I am trying to deploy a Springboot app from my Docker hub account (https://hub.docker.com/r/sonamsamdupkhangsar/springboot-docker/~/dockerfile/) on OpenShift. I selected the Yaml section to paste my YAML config from my github (https://github.com/sonamsamdupkhangsar/springboot-docker/blob/master/springboot-hello-deployment.yaml)
After waiting a while I used to get a error saying the "mkdir" error failed. Now this morning I am seeing a another error: warningThe pod has been stuck in the pending state for more than five minutes.
Any ideas?
thanks
Related
I have setup pipelines as per the developer docs, and I see this error when trying to run a build
docker pull myInstance.cloud:8082/docker-local/pipelines-u18java:17 Error response from daemon: Get "https://myInstance.cloud:8082/v2/": remote error: tls: unrecognized name `retrying 3 of 3 times...
Adding insecure registries in nodePool settings and then re-initialising the node fixed the issue.
Found about the fix here: https://www.jfrog.com/confluence/display/JFROG/Managing+Pipelines+Node+Pools
Summary in one sentence
I want to deploy Mattermost locally on a Kubernetes cluster using Minikube
Steps to reproduce
I used this tutorial and the Github documentation:
https://mattermost.com/blog/how-to-get-started-with-mattermost-on-kubernetes-in-just-a-few-minutes/
https://github.com/mattermost/mattermost-operator/tree/v1.15.0
To start minikube: minikube start --kubernetes-version=v1.21.5
To start ingress; minikube addons enable ingress
I cloned the Github repo with tag v1.15.0 (second link)
In the Github documentation (second link) they state that you need to install Custom Resources by running: kubectl apply -f ./config/crd/bases
Afterwards I installed MinIO and MySQL operators by running: make mysql-minio-operators
Started the Mattermost-operator locally by running: go run .
In the end I deployed Mattermost (I followed step 2, 7 and 9 from the first link)
Observed behavior
Unfortunately I keep getting the following error in the mattermost-operator:
INFO[1419] [opr.controllers.Mattermost] Reconciling Mattermost Request.Name=mm-demo Request.Namespace=mattermost
INFO[1419] [opr.controllers.Mattermost] Updating resource Reconcile=fileStore Request.Name=mm-demo Request.Namespace=mattermost kind="&TypeMeta{Kind:,APIVersion:,}" name=mm-demo-minio namespace=mattermost patch="{\"status\":{\"availableReplicas\":0}}"
INFO[1419] [opr.controllers.Mattermost.health-check] mattermost pod not ready: pod mm-demo-ccbd46b9c-9nq8k is in state 'Pending' Request.Name=mm-demo Request.Namespace=mattermost
INFO[1419] [opr.controllers.Mattermost.health-check] mattermost pod not ready: pod mm-demo-ccbd46b9c-tp567 is in state 'Pending' Request.Name=mm-demo Request.Namespace=mattermost
ERRO[1419] [opr.controllers.Mattermost] Error checking Mattermost health Request.Name=mm-demo Request.Namespace=mattermost error="found 0 updated replicas, but wanted 2"
By using k9s I can see that mm-demo won't start. See below for photo.
Another variation of deployment
Also tried another variation by following all the steps from the first link (without the licences secret step). At this point the mattermost-operator is visible using k9s and won't getting any errors. But unfortunately the mm-demo pod keeps crashing (empty logs, so seeing no errors or something).
Anybody an idea?
As #Ashish faced the same issue, he fixed it by upgrading the resources.
Minikube will be able to run all the pods by running minikube start --kubernetes-version=v1.21.5 --memory 4000 --cpus 4
I am trying to run the Fabric 2.0 test-network in Kubernetes (locally, in minikube) and am facing an issue with the installing or running of the chaincode by the peers (in a docker container, it seems).
I created kubernetes files based on the docker-compose-test-net.yaml and successfully deployed the network, generated the crypto material, created and joined the channel, installed the chaincode on the peers, commited its definition. But when I try to invoke it, I have the following error:
Error: endorsement failure during invoke. response: status:500 message:"error in simulation:
failed to execute transaction 68e996b0d17c210af9837a78c0480bc7ba0c7c0f84eec7da359a47cd1f5c704a:
could not launch chaincode fabcar_01:bb76beb676a23a9be9eb377a452baa4b756cb1dc3a27acf02ecb265e1a7fd3df:
chaincode registration failed: container exited with 0"
I included in that pastebin the logs of the peer. We can see in there that it starts the container, but then I don't understand what happens with it: https://pastebin.com/yrMwG8Nd
I then tried as explained here: https://github.com/IBM/blockchain-network-on-kubernetes/issues/27. Where they say that
IKS v1.11 and onwards now use containerd as its container runtime
instead of the docker engine therefore using docker.sock is no longer
possible.
And they propose to deploy a docker pod (dind) with that file and that file and change the occurences of unix:///host/var/run/docker.sock to tcp://docker:2375.
But then I have the following error when I try to install the chaincode:
Error: chaincode install failed with status:
500 - failed to invoke backing implementation of 'InstallChaincode':
could not build chaincode:
docker build failed:
docker image inspection failed:
cannot connect to Docker endpoint
So it seems it cannot connect to the Docker endpoint. But I cannot find how to fix this.
If you have an idea, it would help a lot!
I found my issue:
For the peers, I was setting:
- name: CORE_PEER_CHAINCODEADDRESS
value: peer0-org1-example-com:7052
- name: CORE_PEER_CHAINCODELISTENADDRESS
value: 0.0.0.0:7052
like they do for the test-network with docker-compose.
Removing those made it work. I guess there were important for the docker-compose setup, but not adequate for kubernetes.
I'm following the Kubernetes install instructions for Helm: https://docs.cert-manager.io/en/latest/getting-started/install/kubernetes.html
With Cert-manager v0.81 on K8 v1.15, Ubuntu 18.04 on-premise.
When I get to testing the installation, I get these errors:
error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "issuers.admission.certmanager.k8s.io": the server is currently unable to handle the request
Error from server (InternalError): error when creating "test-resources.yaml": Internal error occurred: failed calling webhook "certificates.admission.certmanager.k8s.io": the server is currently unable to handle the request
If I apply the test-resources.yaml before installing with Helm, I'm not getting the errors but it is still not working.
These errors are new to me, as Cert-manager used to work for me on my previous install about a month ago, following the same installation instructions.
I've tried with Cert-Manager 0.72(CRD 0.7) as well as I think that was the last version I managed to get installed but its not working either.
What does these errors mean?
Update: It turned out to be an internal CoreDNS issue on my cluster. Somehow not being configured correctly. Possible related to wrong POD_CIDR configuration.
If you experience this problem, check the logs of CoreDNS(Or KubeDNS) and you may see lots of errors related to contacting services. Unfortunately, I no longer have the errors.
But this is how I figured out that my network setup was invalid.
I'm using Calico(Will apply for other networks as well) and its network was not set to the same as the POD_CIDR network that I initialized my Kubernetes with.
Example
1. Set up K8:
kubeadm init --pod-network-cidr=10.244.0.0/16
Configure Calico.yaml:
- name: CALICO_IPV4POOL_CIDR
value: "10.244.0.0/16"
I also tried cert-manager v0.8.0 a very similar setup on Ubuntu 18.04 and k8s v1.14.1 and I began to get the same error when i tore down cert-manager using kubectl delete and reinstalled it, after experiencing some network issues on the cluster.
I stumbled on a solution that worked. On the master node, simply restart the apiserver container:
$ sudo docker ps -a | grep apiserver
af99f816c7ec gcr.io/google_containers/kube-apiserver#sha256:53b987e5a2932bdaff88497081b488e3b56af5b6a14891895b08703129477d85 "/bin/sh -c '/usr/loc" 15 months ago Up 19 hours k8s_kube-apiserver_kube-apiserver-ip-xxxxxc_0
40f3a18050c3 gcr.io/google_containers/pause-amd64:3.0 "/pause" 15 months ago Up 15 months k8s_POD_kube-apiserver-ip-xxxc_0
$ sudo docker restart af99f816c7ec
af99f816c7ec
$
Then try applying the test-resources.yaml again:
$ kubectl apply -f test-resources.yaml
namespace/cert-manager-test unchanged
issuer.certmanager.k8s.io/test-selfsigned created
certificate.certmanager.k8s.io/selfsigned-cert created
If that does not work, this github issue mentions that the master node might need firewall rules to be able to reach the cert-manager-webhook pod. The exact steps to do so will depend on which cloud platform you are on.
An invalid command parameter got into the deployment for a worker process in a Deis app. Now whenever I run a deis pull for a new image this broken parameter gets passed to the deployment so the worker doesn't start up successfully.
If I go into kubectl I can see the following parameter being set in the deployment for the worker (path /spec/template/spec/containers/0)
"command": [
"/bin/bash",
"-c"
],
Which results in the pod not starting up properly:
Error: failed to start container "worker": Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"/bin/bash\": stat /bin/bash: no such file or directory"
Error syncing pod
Back-off restarting failed container
This means that for every release/pull I've been going in and manually removing that parameter from the worker deployment setup. I've run kubectl delete deployment and recreated it with valid json (kubectl create -f deployment.json). This fixes things until I run deis pull again, at which point the broken parameter is back.
My thinking is that that broken command parameter is persisted somewhere in the deis database or the like and that it's being reset when I run deis pull.
I've tried the troubleshooting guide and dug around in the deis-database but I can't find where the deployment for the worker process is being created or where the deployment parameters that get passed to kubernetes when you run a deis pull come from.
Running deis v2.10.0 on Google Cloud