I have setup pipelines as per the developer docs, and I see this error when trying to run a build
docker pull myInstance.cloud:8082/docker-local/pipelines-u18java:17 Error response from daemon: Get "https://myInstance.cloud:8082/v2/": remote error: tls: unrecognized name `retrying 3 of 3 times...
Adding insecure registries in nodePool settings and then re-initialising the node fixed the issue.
Found about the fix here: https://www.jfrog.com/confluence/display/JFROG/Managing+Pipelines+Node+Pools
Related
I use podman with a local registry. I am able to pull the images from the command line and also see the manifest. When I deploy my k8s it fails to pull the image with error access denied. Any idea's? I Googled for days now but do not get an answer that works.
I run Ubuntu 22.04 on VMWARE if that maybe makes a difference. Thank you.
kubelet Failed to pull image "localhost:5001/datawire/aes:3.1.0": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5001/datawire/aes:3.1.0": failed to resolve reference "localhost:5001/datawire/aes:3.1.0": failed to do request: Head "http://localhost:5001/v2/datawire/aes/manifests/3.1.0": dial tcp 127.0.0.1:5001: connect: connection refused
When Kubernetes attempts to create a new pod but is unable to pull the required container image, an error message stating "Failed to pull image" will be displayed. When you try to add a new resource to your cluster with a command like "kubectl apply," you should see this right away. When you inspect the pod with "kubectl describe pod/my-pod," the error will appear in the Events.
Pull errors are caused by the nodes in your cluster. Each node's Kubelet worker process is in charge of obtaining the images required to process a pod scheduling request. When a node is unable to download an image, the status is reported to the cluster control plane.
Images may not pull for a variety of reasons. It's possible that your nodes' networking failed, or that the cluster as a whole is experiencing connectivity issues. If you are online, the registry is up, and pull errors continue to occur, your firewall or traffic filtering may be malfunctioning.
Refer this doc1 and doc2 for more information.
I am trying to run the Fabric 2.0 test-network in Kubernetes (locally, in minikube) and am facing an issue with the installing or running of the chaincode by the peers (in a docker container, it seems).
I created kubernetes files based on the docker-compose-test-net.yaml and successfully deployed the network, generated the crypto material, created and joined the channel, installed the chaincode on the peers, commited its definition. But when I try to invoke it, I have the following error:
Error: endorsement failure during invoke. response: status:500 message:"error in simulation:
failed to execute transaction 68e996b0d17c210af9837a78c0480bc7ba0c7c0f84eec7da359a47cd1f5c704a:
could not launch chaincode fabcar_01:bb76beb676a23a9be9eb377a452baa4b756cb1dc3a27acf02ecb265e1a7fd3df:
chaincode registration failed: container exited with 0"
I included in that pastebin the logs of the peer. We can see in there that it starts the container, but then I don't understand what happens with it: https://pastebin.com/yrMwG8Nd
I then tried as explained here: https://github.com/IBM/blockchain-network-on-kubernetes/issues/27. Where they say that
IKS v1.11 and onwards now use containerd as its container runtime
instead of the docker engine therefore using docker.sock is no longer
possible.
And they propose to deploy a docker pod (dind) with that file and that file and change the occurences of unix:///host/var/run/docker.sock to tcp://docker:2375.
But then I have the following error when I try to install the chaincode:
Error: chaincode install failed with status:
500 - failed to invoke backing implementation of 'InstallChaincode':
could not build chaincode:
docker build failed:
docker image inspection failed:
cannot connect to Docker endpoint
So it seems it cannot connect to the Docker endpoint. But I cannot find how to fix this.
If you have an idea, it would help a lot!
I found my issue:
For the peers, I was setting:
- name: CORE_PEER_CHAINCODEADDRESS
value: peer0-org1-example-com:7052
- name: CORE_PEER_CHAINCODELISTENADDRESS
value: 0.0.0.0:7052
like they do for the test-network with docker-compose.
Removing those made it work. I guess there were important for the docker-compose setup, but not adequate for kubernetes.
I'm facing a pretty strange Problem.
First of all my setup:
I got a private Gitlab server which uses Gitlab CI Runners on Kubernetes to build Docker Images. For that purpose I use the Kaniko Image.
The Runners are provisioned by Gitlab itself with the built-in Kubernetes management. All that is running behind a PFSense server.
Now to my problem:
Sometimes the Kaniko Pods can't resolve the Hostname of the GitLab server.
This leads to failed git pull and so to a failed build.
I would rate the chance to fail by 60%, which is way too high for us.
After retrying the build a few times, it will run without any problem.
The Kubernetes Cluster running the Gitlab CI is setup on CentOS 7.
SELinux and FirewallD are disabled. All of the Hosts can resolve the GitLab Server. It is also not related to a specific Host Server, which is causing the problem. I have seen it fail on all of the 5 Servers including the Manager Server. Also I haven't seen this problem appear in other Pods. But the other Deployments in the cluster dont really do connections via DNS. I am sure that the Runner is able to access DNS at all, because it is pulling the Kaniko Image from gcr.io.
Has anyone ever seen this problem or knows a workaround?
I have already tried spawning Pods that only do DNS requests to the Domain. I didn't see a single fail.
Also I tried to Reboot the whole Cluster and Gitlab instance.
I tried to do a static overwrite of the DNS route in PFSense. Still same problem.
Here is my CI config:
build:
stage: build
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- echo $REGISTRY_AUTH > /kaniko/.docker/config.json
- /kaniko/executor --context $CI_PROJECT_DIR --dockerfile $CI_PROJECT_DIR/Dockerfile --destination $REGISTRY_URL/$REGISTRY_IMAGE:$CI_JOB_ID
only:
- master
The following error happens:
Initialized empty Git repository in /builds/MYPROJECT/.git/
Fetching changes...
Created fresh repository.
fatal: unable to access 'https://gitlab-ci-token:[MASKED]#git.mydomain.com/MYPROJECT.git/': Could not resolve host: git.mydomain.com
We had same issue for couple of days. We tried change CoreDNS config, move runners to different k8s cluster and so on. Finally today i checked my personal runner and found that i'm using different version. Runners in cluster had gitlab/gitlab-runner:alpine-v12.3.0, when mine had gitlab/gitlab-runner:alpine-v12.0.1. We added line
image: gitlab/gitlab-runner:alpine-v12.1.0
in values.yaml and this solved problem for us
There are a env for gitlab-runner that can solve this problem
- name: RUNNER_PRE_CLONE_SCRIPT
value: "exec command before git fetch ..."
for example:
edit /etc/hosts
echo '127.0.0.1 git.demo.xxxx' >> /etc/hosts
or edit /etc/resolv.conf
echo 'nameserver 8.8.8.8' > /etc/resolv.conf
hope it works for you
I'm trying to install Ceph using Helm on Kunbernetes following this tutorial
install ceph
Probably the problem is that I installed trow registry before because as soon as I run the helm step
helm install --name=ceph local/ceph --namespace=ceph -f ~/ceph-overrides.yaml
I get this error in ceph namespace
Error creating: Internal error occurred: failed calling webhook "validator.trow.io": Post https://trow.kube-public.svc:443/validate-image?timeout=30s: dial tcp 10.102.137.73:443: connect: connection refused
How can I solve this?
Apparently you are right with the presumption, I have a few concerns about this issue.
Trow registry manager controls the images that run in the cluster via implementing Admission webhooks that validate every request before pulling image, and as far as I can see Docker Hub images are not accepted by default.
The default policy will allow all images local to the Trow registry to
be used, plus Kubernetes system images and the Trow images themselves.
All other images are denied by default, including Docker Hub images.
Due to the fact that during Trow installation procedure you might require to distribute and approve certificate in order to establish secure HTTPS connection from target node to Trow server, I would suggest to check certificate presence on the node where you run ceph-helm chart as described in Trow documentation.
The other option you can run Trow registry manager with disabled TLS over HTTP, as was guided in the installation instruction.
This command should help to get it cleaned.
kubectl delete ValidatingWebhookConfiguration -n rook-ceph rook-ceph-webhook
I am trying to deploy a Springboot app from my Docker hub account (https://hub.docker.com/r/sonamsamdupkhangsar/springboot-docker/~/dockerfile/) on OpenShift. I selected the Yaml section to paste my YAML config from my github (https://github.com/sonamsamdupkhangsar/springboot-docker/blob/master/springboot-hello-deployment.yaml)
After waiting a while I used to get a error saying the "mkdir" error failed. Now this morning I am seeing a another error: warningThe pod has been stuck in the pending state for more than five minutes.
Any ideas?
thanks