I'm a newer to Docker, when i follow https://docs.confluent.io/current/installation/docker/development.html#create-a-docker-image-containing-local-connectors to build a Docker image with local connector, i got some issue.
below is my Dockerfile:
FROM confluentinc/cp-kafka-connect-base:5.3.0
COPY kafka-connect-solr-sink-0.0.1.zip /tmp/kafka-connect-solr-sink-0.0.1.zip
RUN confluent-hub install --no-prompt /tmp/kafka-connect-solr-sink-0.0.1.zip
when I run "docker build . -t my-custom-image:1.0.0", got below error:
Sending build context to Docker daemon 6.876MB Step 1/3 : FROM
confluentinc/cp-kafka-connect-base:5.3.0 ---> b1491496b48d Step 2/3 :
COPY kafka-connect-solr-sink-0.0.1.zip
/tmp/kafka-connect-solr-sink-0.0.1.zip ---> Using cache --->
85689e66c40a Step 3/3 : RUN confluent-hub install --no-prompt
/tmp/kafka-connect-solr-sink-0.0.1.zip ---> Running in 1d3486732517
Running in a "--no-prompt" mode Failed to unzip
'/tmp/kafka-connect-solr-sink-0.0.1.zip' into
'/tmp/confluent-hub-tmp7724578044817786588' String index out of range:
-1
Error: Unknown error The command '/bin/sh -c confluent-hub install
--no-prompt /tmp/kafka-connect-solr-sink-0.0.1.zip' returned a non-zero code: 7
Anyone can kindly help on this? Thanks.
From the Docker file ,Ensure you set the required proxy settings so that you do not get this error.
In Dockerfile....
ENV http_proxy <proxy_host>:
ENV https_proxy <proxy_host>:
After adding this before running other commands it
Related
I am trying to use a plugin in my eks based k8s cluster,
I am using a Github Action controller that spawns on demand Container as Self Hosted runner
When the Github action start this plugin or any other that needs to build itself as a docker image fails with below error, any thoughts or ideas ?
This is my self hosted runner image Link
FYI : If i run a standalone alpine container in the cluster all typical cmd works, and this also works with default ubuntu based self hosted runner, so i dont think its the cluster
/usr/local/bin/docker build -t 60e226:1b6fc15462134e6fb8520b7df48cf7fd -f "/runner/_work/_actions/aquasecurity/trivy-action/master/Dockerfile" "/runner/_work/_actions/aquasecurity/trivy-action/master"
Sending build context to Docker daemon 644.6kB
Step 1/5 : FROM ghcr.io/aquasecurity/trivy:0.[3](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:3)7.1
0.37.1: Pulling from aquasecurity/trivy
c158987b0551: Pulling fs layer
67a7d067ef7d: Pulling fs layer[6]Download complete
67a7d067ef7d: Pull complete
2ec1cdd48f38: Verifying Checksum
2ec1cdd48f38: Download complete
2ec1cdd48f38: Pull complete
fe56e6aa700e: Pull complete
Digest: sha256:7c[16](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:16)7f7f3002948f1ec099555aa968bd8b8b097780603a38cc801fe965da0a69
Status: Downloaded newer image for ghcr.io/aquasecurity/trivy:0.37.1
---> c3e68408cd24
Step 2/5 : COPY entrypoint.sh /
---> 1f1da443ea86
Step 3/5 : RUN apk --no-cache add bash curl npm
---> Running in 647f7f479cac
fetch https://dl-cdn.alpinelinux.org/alpine/v3.[17](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:17)/main/x86_64/APKINDEX.tar.gz
48ABC73BEB7F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:[18](https://github.com//docker-images/actions/runs/4134005760/jobs/7147011143#step:3:18)89:
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/main: Permission denied
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz
48ABC73BEB7F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1889:
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.17/community: Permission denied
ERROR: unable to select packages:
bash (no such package):
required by: world[bash]
curl (no such package):
required by: world[curl]
npm (no such package):
required by: world[npm]
The command '/bin/sh -c apk --no-cache add bash curl npm' returned a non-zero code: 3
Warning: Docker build failed with exit code 3, back off 6.807 seconds before retry.
It was expected to build the docker image and proceed with the github action workflow
Tried different flavors of image and nothing worked except for ubunut-latest
the plugin in question
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action#master
with:
image-ref: 'test:latest'
format: 'table'
exit-code: '1'
ignore-unfixed: true
vuln-type: 'os,library'
severity: 'CRITICAL,HIGH'
Until now I worked a lot with github/bitbucket and jenkins/bamboo. Right now I'm trying to setup a Gitlab CE server with a private kubernetes cluster.
I want to run a hello world project in java with gitlabs AutoDevOps in kubernetes, this is the repo I'm using:
https://github.com/dstar55/docker-hello-world-spring-boot
Everything works fine until runner gets created in kubernetes, downloads the image but gets stuck on downloading maven resources.
Running on runner-h6cwaztm-project-8-concurrent-0jvd9f via runner-gitlab-runner-6dcf7dd458-jl69h...
Fetching changes with git depth set to 50...
00:02
Initialized empty Git repository in /builds/.../hello-world-spring/.git/
Created fresh repository.
From https://.../hello-world-spring
* [new ref] refs/pipelines/14 -> refs/pipelines/14
* [new branch] master -> origin/master
Checking out ad24ac6b as master...
Skipping Git submodules setup
$ if [[ -z "$CI_COMMIT_TAG" ]]; then # collapsed multi-line command
$ /build/build.sh
Logging to GitLab Container Registry with CI credentials...
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
Building Dockerfile-based application...
Step 1/10 : FROM maven:3.5.2-jdk-8-alpine AS maven_build
3.5.2-jdk-8-alpine: Pulling from library/maven
22bc7fb81913: Pull complete
Digest: sha256:7cebda60f8a541e1bf2330306d22f9786f989187f4ec96539d398a0d4dbfdadb
Status: Downloaded newer image for maven:3.5.2-jdk-8-alpine
---> 293423a981a7
Step 2/10 : COPY pom.xml /tmp/
---> c0e609a509a8
Step 3/10 : COPY src /tmp/src/
---> e735a08f2b39
Step 4/10 : WORKDIR /tmp/
---> Running in 90620c0ca3ad
Removing intermediate container 90620c0ca3ad
---> a5d9fdc62aa9
Step 5/10 : RUN mvn package
---> Running in dc90f43fc83b
[INFO] Scanning for projects...
Downloading from central: https://repo.maven.apache.org/maven2/org/springframework/boot/spring-boot-starter-parent/2.2.1.RELEASE/spring-boot-starter-parent-2.2.1.RELEASE.pom
It never throws an error (until it timesout) and it never goes past this point.
Kubernetes has 4 nodes 1 master and 3 slaves, using flannel and MetalLB
Edit:
I added a curl command instead of mvn package and it seems the download speed is 0, how is that possible?
Step 5/11 : RUN curl https://repo.maven.apache.org/maven2/org/springframework/boot/spring-boot-starter-parent/2.2.1.RELEASE/spring-boot-starter-parent-2.2.1.RELEASE.pom --output test.pom
---> Running in db2bc24c6a4f
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:05:00 --:--:-- 0
curl: (28) Operation timed out after 300689 milliseconds with 0 out of 0 bytes received
The command '/bin/sh -c curl https://repo.maven.apache.org/maven2/org/springframework/boot/spring-boot-starter-parent/2.2.1.RELEASE/spring-boot-starter-parent-2.2.1.RELEASE.pom --output test.pom' returned a non-zero code: 28
ERROR: Job failed: command terminated with exit code 1
According to place where CI hangs, your pipeline stuck at mvn package:
Step 5/10 : RUN mvn package
---> Running in dc90f43fc83b
[INFO] Scanning for projects...
Downloading from central: https://repo.maven.apache.org/maven2/org/springframework/boot/spring-boot-starter-parent/2.2.1.RELEASE/spring-boot-starter-parent-2.2.1.RELEASE.pom
So, you can try to restart Artifactory.
Also, you can debug mvn packages with mvn clean package -X -e
See: this answer :
java - Maven hanging indefinitely while checking for updates - Stack Overflow
mvn clean package -X -e
I've been trying to setup Cloud Code with VSCode and I've been running in to problems when starting the deploy process with Cloud Code: Deploy.
I've tried deploying the samples, python-hello-world-1 as well as the go-hello-world-1, to my kubernetes cluster on GKE but always end up getting errors when the deploy process starts package downloading:
Go Output
Running: skaffold run --enable-rpc -v info --rpc-http-port 49869 --filename skaffold.yaml --default-repo gcr.io/abx-lernende
starting gRPC server on port 50051
starting gRPC HTTP server on port 49869
Using kubectl context: gke_abx-lernende_europe-west4-a_joshu-test-cluster
Generating tags...
- go-hello-world -> gcr.io/abx-lernende/go-hello-world:latest
Checking cache...
- go-hello-world: Not found. Building
Building [go-hello-world]...
Sending build context to Docker daemon 57.86kB
Step 1/8 : FROM golang:1.13
---> 6586e3d10e96
Step 2/8 : RUN go get -u -v github.com/go-delve/delve/cmd/dlv
---> Running in b75ce8e5dae9
[91mgithub.com/go-delve/delve (download)
[0m[91m# cd .; git clone -- https://github.com/go-delve/delve /go/src/github.com/go-delve/delve
Cloning into '/go/src/github.com/go-delve/delve'...
fatal: unable to access 'https://github.com/go-delve/delve/': Failed to connect to github.com port 443: Connection refused
package github.com/go-delve/delve/cmd/dlv: exit status 128
[0mfailed to build: build failed: building [go-hello-world]: build artifact: unable to stream build output: The command '/bin/sh -c go get -u -v github.com/go-delve/delve/cmd/dlv' returned a non-zero code: 1
Exited with code 1.
Python Output
Running: skaffold run --enable-rpc -v info --rpc-http-port 50185 --filename
skaffold.yaml --default-repo gcr.io/abx-lernende
starting gRPC server on port 50051
starting gRPC HTTP server on port 50185
Skaffold &{Version:v1.3.1 ConfigVersion:skaffold/v2alpha3 GitVersion: GitCommit:6ba887a42438d1da578a005cf550e618fee6dfb8 GitTreeState:clean BuildDate:2020-01-31T19:55:18Z GoVersion:go1.13.4 Compiler:gc Platform:windows/amd64}
Using kubectl context: gke_abx-lernende_europe-west4-a_joshu-test-cluster
Generating tags...
- python-hello-world -> Tags generated in 0s
gcr.io/abx-lernende/python-hello-world:latest
Checking cache...
- python-hello-world: Cache check complete in 6.0001ms
Not found. Building
Building [python-hello-world]...
Sending build context to Docker daemon 4.608kB
Step 1/7 : FROM python:3.8
---> efdecc2e377a
Step 2/7 : WORKDIR /app
---> Using cache
---> a131b81cad66
Step 3/7 : COPY requirements.txt .
---> Using cache
---> 4625ef1862bd
Step 4/7 : RUN pip install --trusted-host pypi.python.org -r requirements.txt
---> Running in 4da23a158ae3
[91mWARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f17ba9c9d60>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/flask/
Im assuming this is due to me being behind a corporate proxy. As counter measures I have explicitly configured VSCode, Git, pip, go and google cloud sdk all to use said proxy. On top of that I set the Windows ENV variables for the proxy. sadly without success.
Thanks!
You can configure docker to pass through proxy information into the containers by adding something like the following to your ~/.docker/config.json:
{
"proxies": {
"default": {
"httpProxy": "http://192.168.1.12:3128",
"httpsProxy": "http://192.168.1.12:3128"
}
}
}
Docker will set the HTTP_PROXY/HTTPS_PROXY environment variables within the container which is picked up by many tools.
I just want to know whether I can run Karate test in a pod. Or is there any good suggestion on how to run it?
I tried to run the Karate test in terminal and it works. Just want to know if I can run it from Kubernetes pod. Nginx also running in the pod.
You can everything in pod whatever you are running outside environment. Pod run the container inside it.
So create the docker file and generate the docker image using docker file. Using that docker image and start the karate pod.
You can write the docker file like this
FROM maven:3-jdk-8-alpine
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY settings.xml /usr/share/maven/ref/
COPY pom.xml /tmp/pom.xml
COPY . /usr/src/app
RUN mvn -B -f /tmp/pom.xml -s /usr/share/maven/ref/settings-docker.xml prepare-package -DskipTests
CMD ["/usr/src/app/maven_runner.sh"]
I found here one example : https://github.com/neillfontes/karate-sample
Posting as Community Wiki for future use.
#Harsh Manvar provided good example, however if you will just build it from Dockerfile, you will recieved errors. You have to download all files mentioned in Github. Correct oreder will be:
$ git clone https://github.com/neillfontes/karate-sample.git
$ cd karate-sample
$ docker build -t karate_docker .
After image was built you can check it:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
karate_docker latest 9dc6d7a5278a About a minute ago 136MB
Later you can start testing using:
$ docker run karate_docker
START: Running tests...
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running demo.DemoTest
11:57:49.684 [main] DEBUG c.i.karate.cucumber.CucumberRunner - init test class: class demo.DemoTest
11:57:50.412 [main] DEBUG c.i.karate.cucumber.CucumberRunner - loading feature: /usr/src/app/target/test-classes/demo/features/get-token.feature
11:57:50.663 [main] DEBUG c.i.karate.cucumber.CucumberRunner - loading feature: /usr/src/app/target/test-classes/demo/features/make-request.feature
11:57:53.898 [main] INFO com.intuit.karate.ScriptBridge - karate.env system property was: null
11:57:54.867 [main] DEBUG c.i.k.h.a.RequestLoggingInterceptor -
1 > POST http://brentertainment.com/oauth2/lockdin/token
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Content-Length: 96
I am trying to install Hyperledger sample on Ubuntu 14.04.
Followed all the steps in the tutorial. I am stuck at Build your first network script. When I run ./bynf.sh -m -up I get the following error:
boss#ubuntu:/tmp/fabric-samples/first-network$ ./byfn.sh -m up
Starting with channel 'mychannel' and CLI timeout of '10000'
Continue (y/n)? y
proceeding ...
./byfn.sh: line 116: docker-compose: command not found
ERROR !!!! Unable to start network
Error response from daemon: No such container: cli
boss#ubuntu:/tmp/fabric-samples/first-network$
Did not have Docker Compose Installed on my Ubuntu.
Followed the steps in the link below and got it working:
https://docs.docker.com/compose/install/#install-compose