Getting error in configuring s3 Sink conector - apache-kafka

I have cloned landoop fast-data-dev docker repo from this GitHub repo.
and built the image using command docker build --tag=landoop .
After building the image, I ran it using:
docker run --rm -p 2181:2181 -p 3030:3030 -p 8081-8083:8081-8083 -p 9581-9585:9581-9585 -p 9092:9092 -e ADV_HOST=10.10.X.X -e DEBUG=1 -e AWS_ACCESS_KEY_ID=XXX -e AWS_SECRET_ACCESS_KEY=XXX landoop
Once the UI was up, I tried to create a s3 sink connection but it failed saying:
Caused by: java.io.FileNotFoundException: /usr/lib/libnss3.so
Also I don't see the libnss3.so file in the location. However if I run the docker container directly using the command below, I can see the file in the location and there is no error when creating the s3 sink connector.
docker run --rm --net=host landoop/fast-data-dev
Has anyone faced this error?

Answering my own question so that others can benefit,if it's not appropriate please leave a comment and I will make it a comment. I figured out that the libnss3 library was missing from debian image and had to install while building the image. For this I edited the setp-and-run.sh and added the libnss3, the script looks like :
FROM debian as compile-lkd
RUN apt-get update \
&& apt-get install -y \
unzip \
wget \
libnss3 \

Related

Use local mirror of Alpine when build images of docker-compose instead legacy repositories

Is it possible to use a local repository (mirror) that is used by docker when building a container with docker-compose?
I have a local mirror, and my interest is that when I build a container with docker-compose build, the apk is downloaded from the local mirror, instead of the Internet
Example.
If a local deploy with Dockerfile
FROM alpine:3.13
# Install packages and remove default server definition
RUN apk --no-cache add php8=8.0.2-r0 php8-fpm php8-opcache php8-mysqli php8-json \
php8-openssl php8-curl php8-soap php8-zlib php8-xml php8-phar php8-intl php8-dom php8-xmlreader php8-ctype \
php8-session php8-simplexml php8-mbstring php8-gd nginx supervisor curl php8-exif php8-zip php8-fileinfo \
php8-iconv php8-soap tzdata htop mysql-client php8-pecl-imagick php8-pecl-redis php8-tokenizer php8-xmlwriter \
nano && rm /etc/nginx/conf.d/default.conf
When deploy container with docker-compose build I like that use my local repository of alpine instead any internet mirrors of Alpine such:
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.13/community/x86_64/APKINDEX.tar.gz
(1/124) Installing ca-certificates (20191127-r5)
(2/124) Installing brotli-libs (1.0.9-r3)
(3/124) Installing nghttp2-libs (1.42.0-r1)
...
After read alpine docs Alpine Linux package management I get solution using static IP of local mirror. I tried to use the name of the hostname (repoalpine.test) but I couldn't find how to make the hostname public on the docker network.
RUN apk --no-cache -X http://172.20.0.254/v3.13/main -X http://172.20.0.254/v3.13/community \
add php8=8.0.2-r0 php8-fpm php8-opcache php8-mysqli php8-json \
php8-openssl php8-curl php8-soap php8-zlib php8-xml php8-phar php8-intl php8-dom php8-xmlreader php8-ctype \
php8-session php8-simplexml php8-mbstring php8-gd nginx supervisor curl php8-exif php8-zip php8-fileinfo \
php8-iconv php8-soap tzdata htop mysql-client php8-pecl-imagick php8-pecl-redis php8-tokenizer php8-xmlwriter \
nano && rm /etc/nginx/conf.d/default.conf
Now work
Step 7/22 : RUN apk --no-cache -X http://172.20.0.254/v3.13/main -X http://172.20.0.254/v3.13/community add php8=8.0.2-r0 php8-fpm php8-opcache php8-mysqli php8-json php8-openssl php8-curl php8-soap php8-zlib php8-xml php8-phar php8-intl php8-dom php8-xmlreader php8-ctype php8-session php8-simplexml php8-mbstring php8-gd nginx supervisor curl php8-exif php8-zip php8-fileinfo php8-iconv php8-soap tzdata htop mysql-client php8-pecl-imagick php8-pecl-redis php8-tokenizer php8-xmlwriter nano && rm /etc/nginx/conf.d/default.conf
---> Running in 4f2c6521e6e6
fetch http://172.20.0.254/v3.13/community/x86_64/APKINDEX.tar.gz
...
You need a local Docker Registry

Don't find /var/lib/postgresql/data/ directory on ubuntu when created docker image

I found the following mentioned at many places -
docker run -d \
--name some-postgres \
-e POSTGRES_PASSWORD=mysecretpassword \
-e PGDATA=/var/lib/postgresql/data/pgdata \
-v /custom/mount:/var/lib/postgresql/data \
postgres
My only question is that I am unable to find /var/lib/postgresql/data/pgdata directory itself. I don't see any postgresql directory under /var/lib. Why is it? And just wonder how does it work if there is no directory?
The -v in your command mounts /custom/mount on your host (the machine where you run docker command) to container's /var/lib/postgresql/data. So the pgdata you are looking for is on host's /custom/mount/pgdata.
Of course, /custom/data is only an example name, you have to replace it with your real directory.

Running new meteorhacks/meteord application, don't have meteor app

I try to create a new project based on the meteor with docker.
I found the repository for this:
https://github.com/meteorhacks/meteord
I created Dockerfile and put there
FROM meteorhacks/meteord:onbuild
And then run:
docker run meteorhacks/meteord
docker run mongo
After downloading all packages so finally I run
docker run -i -t 807754a01782 -d
-e ROOT_URL=http://localhost:3000
-e MONGO_URL=mongodb://127.0.0.1:27017/
-e MONGO_OPLOG_URL=mongodb://127.0.0.1:27017/
-p 8080:80 myapp
Based on this example:
docker run -d \
-e ROOT_URL=http://yourapp.com \
-e MONGO_URL=mongodb://url \
-e MONGO_OPLOG_URL=mongodb://oplog_url \
-p 8080:80 \
yourname/app
Inside myapp folder, I have fresh meteor project.
But as a result, I received
> You don't have an meteor app to run in this image.
Can anyone help me and give me some clues what I'm doing wrong? Or I misunderstanding how Docker with this repository works?
EDIT:
The problem was in command correct command is:
docker run -d
-e ROOT_URL=http://localhost:3000
-e MONGO_URL=mongodb://127.0.0.1:27017/
-e MONGO_OPLOG_URL=mongodb://127.0.0.1:27017/ -p 8080:80
meteorhacks/meteord:base
But now when I check the status of this container I see it is excited. How Can I check what causing the problem?

eclipse che docker - can not create workspace - <no value>/lib/linux_amd64/terminal

I tried so set up eclipse che as described in https://eclipse.org/che/docs/setup/docker/
with the following command:
docker run -p 8080:8080 \
--name che \
--rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /myimage \
eclipse/che-server:5.0.0-latest
che runs successfully, but during the creation of the workspace the following error message appears:
Caused by: org.eclipse.che.api.core.ServerException: Error response from docker API, status: 500, message: create <no va
lue>/lib/linux_amd64/terminal: "<no value>/lib/linux_amd64/terminal" includes invalid characters for a local volume name
, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intented to pass a host directory, use absolute path
I experience this both on a debian and on a windows instance. I'm quite new to docker. What could be the cause?
I did face the same problem all through and realized that I had done my installations wrong. Following the instructions on the eclipse che website can be quite challenging if you are a newbie to it.
Kindly follow the following steps to install eclipse che using docker image and the issue you are currently facing will be gone.
(The following commands are with the assumption that you are on Debian Distribution. I am on Ubuntu 16.04)
1. Install JDK on the machine.
$ sudo apt-get update
$ sudo apt-get install default-jdk
2. Install Docker
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
$ sudo apt-get update
$ sudo apt-get install -y docker-ce
3. Verify if docker has successfully installed
$ systemctl status docker
$ docker -v
$ sudo docker info
$ sudo docker hello-world
4. Install Eclipse CHE
$ cd ~
$ sudo mkdir eclipseche
$ sudo docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v ~/eclipseche:/data eclipse/che start
Docker is complaining about the syntax of this:
-v /myimage
As it's documented, they recommend to put the /data of che to a volume, in order to persist data between docker runs.
So, put this volume:
-v $(pwd)/che-data:/data
Resulting in this command:
docker run -p 8080:8080 \
--name che \
--rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $(pwd)/che-data:/data \
eclipse/che-server:5.0.0-latest
$(pwd)/che-data is the directory in your host machine where the che's data will be saved.
/data is the directory inside che container, linked to ./che-data outside container.
-v /myimage \ is wrong :/data is correct

"docker run" fails with "-v" options on CentOS6.5

Summary
I'm trying to set up Mesos cluster using Docker containers.
But docker run command fails with -v volume options, and succeeds without volume options.
Message is as follows,
FATA[0008] Error response from daemon: No such image: redjack/mesos-master:latest (tag: latest)
despite the image redjack/mesos-master:latest exists.
It's strange that adding volume options causes No such image error.
Does anyone have solutions or hints for this problem?
Thank you in advance!
Details
I'm using redjack/mesos-master for master and redjack/mesos-slave for slave.
Failed Commands
Following command is for master.
$ docker run \
-e MESOS_LOG_DIR=/var/log \
-e MESOS_HOSTNAME=<hostname> \
-e MESOS_PORT=5050 \
-e MESOS_REGISTRY=in_memory \
-e MESOS_WORK_DIR=/tmp/mesosphere \
-e MESOS_CONTAINERIZERS=docker,mesos \
-v /var/log:/var/log \
-v /sys/cgroup:/cgroup \
-v /proc:/proc \
redjack/mesos-master:latest
This returns following messages.
Unable to find image 'redjack/mesos-master:latest' locally
Pulling repository redjack/mesos-master
e43e645e4613: Download complete
e9e06b06e14c: Download complete
a82efea989f9: Download complete
37bea4ee0c81: Download complete
07f8e8c5e660: Download complete
a170eebbd2bf: Download complete
104ab0553e3a: Download complete
d6f04b0daf32: Download complete
bdb0fee63b5c: Download complete
943ba7734c82: Download complete
e8b0687de36f: Download complete
9ae9def4d95e: Download complete
776c4db2701b: Download complete
63180ef60d78: Download complete
Status: Image is up to date for redjack/mesos-master:latest
FATA[0008] Error response from daemon: No such image: redjack/mesos-master:latest (tag: latest)
Despite the message above, image redjack/mesos-master exists
$ docker images | grep mesos-master
redjack/mesos-master latest e43e645e4613 2 weeks ago 1.092 GB
Succeeded Commands
As mentioned above it succeeds without -v options
$ docker run \
-e MESOS_LOG_DIR=/var/log \
-e MESOS_HOSTNAME=<hostname> \
-e MESOS_PORT=5050 \
-e MESOS_REGISTRY=in_memory \
-e MESOS_WORK_DIR=/tmp/mesosphere \
-e MESOS_CONTAINERIZERS=docker,mesos \
redjack/mesos-master:latest
Environments
OS:
CentOS 6.5
Docker:
$ docker version
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.3.3
Git commit (client): a8a31ef/1.5.0
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.3.3
Git commit (server): a8a31ef/1.5.0
Thanks for user2915097, the problem was solved by changing -v /sys/cgroup:/cgroup to -v /cgroup:/cgroup.