why can't the first container "see" the later containers - apache-zookeeper

I have a series of containers that are started by docker-compose. Specifically they are multiple zookeeper containers:
zk1:
image: seven10/zookeeper:3.4.6
container_name: zk1
hostname: zk1
restart: always
ports:
- "2181:2181"
- "2888:2888"
- "3888:3888"
environment:
- ZOOKEEPER_ID=1
net: ${MY_NETWORK_NAME}
volumes:
- /seven10/zk/zk1/data:/opt/zookeeper-3.4.6/data
zk2:
image: seven10/zookeeper:3.4.6
container_name: zk2
restart: always
hostname: zk2
ports:
- "2182:2181"
- "2889:2888"
- "3889:3888"
environment:
- ZOOKEEPER_ID=2
net: ${MY_NETWORK_NAME}
volumes:
- /seven10/zk/zk2/data:/opt/zookeeper-3.4.6/data
zk3:
image: seven10/zookeeper:3.4.6
container_name: zk3
hostname: zk3
restart: always
ports:
- "2183:2181"
- "2890:2888"
- "3890:3888"
environment:
- ZOOKEEPER_ID=3
net: ${MY_NETWORK_NAME}
volumes:
- /seven10/zk/zk3/data:/opt/zookeeper-3.4.6/data
So when I go to start the containers, zk1 gives me this warning at the start:
WARN [WorkerSender[myid=1]:QuorumCnxManager#382] - Cannot open channel to 3 at
election address zk3:3888
java.net.UnknownHostException: zk3
but then doesn't say anything else about zk3 after a couple of seconds.
However, zk1 gives the following error for zk2 continuously:
2016-02-18 15:28:57,384 [myid:1] - WARN [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2181:Learner#233] - Unexpected exception, tries=0, connecting to zk2:2888
java.net.UnknownHostException: zk2
zk2 doesn't say ANYTHING ever about zk1, but briefly complains with the "cannot open channel" error for zk3.
zk3 doesn't every mention zk1 or zk2.
So the big problem is that zk1 can't find zk2 ever. It just spams the logs and refuses connections from kafka. Why is this so and how should I go about solving this problem?
My dev box is using docker version 1.9.1 and docker-compose version 1.5.1 on ubuntu 14.04 (Mint Rafello I think?), although the target environment will be ubuntu 15.10.

Does your host system know how to link zk1/2/3 to an IP address? If you're launching all three servers on the same node, you should use localhost as the host name (the server name should still be unique)

Related

How to setup kafka-kinesis-connector if I use a Kafka container?

I'm a little bit confused. I've been following this to get started and the installation method.
https://aws.amazon.com/premiumsupport/knowledge-center/kinesis-kafka-connector-msk/
Setup AWS cli and configure it
Install maven
Compile connector
Set classpath with the jar generated
Set up the properties file of the connector
FYI: I have a docker-compose that creates all my containers (kafka, mqtt, etc.)
(all of the above is setup on-premise)
And then, I executed all this on my machine itself and not on the Kafka container, so for the last step how would that work when I try to run it standalone?
version: '3'
services:
nodered:
container_name: nodered
image: nodered/node-red
ports:
- "1880:1880"
volumes:
- ./nodered:/data
depends_on:
- mosquitto
environment:
- TZ=America/Toronto
- NODE_RED_ENABLE_PROJECTS=true
restart: always
mosquitto:
image: eclipse-mosquitto
container_name: mqtt
restart: always
ports:
- "1883:1883"
volumes:
- "./mosquitto/config:/mosquitto/config"
- "./mosquitto/data:/mosquitto/data"
- "./mosquitto/log:/mosquitto/log"
environment:
- TZ=America/Toronto
user: "${PUID}:${PGID}"
portainer:
ports:
- "9000:9000"
container_name: portainer
restart: always
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
- "./portainer/portainer_data:/data"
image: portainer/portainer-ce
zookeeper:
image: zookeeper:3.4
container_name: zookeeper
ports:
- "2181:2181"
volumes:
- "zookeeper_data:/data"
kafka:
image: wurstmeister/kafka:1.0.0
container_name: kafka
ports:
- "9092:9092"
- "9093:9093"
volumes:
- "kafka_data:/data"
environment:
- KAFKA_ZOOKEEPER_CONNECT=10.0.0.129:2181
- KAFKA_ADVERTISED_HOST_NAME=10.0.0.129
- JMX_PORT=9093
- KAFKA_ADVERTISED_PORT=9092
- KAFKA_LOG_RETENTION_HOURS=1
- KAFKA_MESSAGE_MAX_BYTES=10000000
- KAFKA_REPLICA_FETCH_MAX_BYTES=10000000
- KAFKA_GROUP_MAX_SESSION_TIMEOUT_MS=60000
- KAFKA_NUM_PARTITIONS=2
- KAFKA_DELETE_RETENTION_MS=1000
depends_on:
- zookeeper
restart: on-failure
cmak:
image: hlebalbau/kafka-manager:1.3.3.16
container_name: kafka-manager
restart: always
depends_on:
- kafka
- zookeeper
ports:
- "9080:9080"
environment:
- ZK_HOSTS=10.0.0.129
- APPLICATION_SECRET=letmein
command: -Dconfig.file=/kafka-manager/conf/application.conf -Dapplication.home=/kafkamanager -Dhttp.port=9080
volumes:
zookeeper_data:
driver: local
kafka_data:
driver: local
I guess I have to go into my Kafka container and run the below code, but how can I reference by machine path... I'm stuck here or perhaps I'm missing something:
./bin/connect-standalone.sh {{path_from_machine_where_jar_is}}/kinesis-kafka-connector/config/worker.properties {{path_from_machine_where_jar_is}}/kinesis-kafka-connector/config/kinesis-streams-kafka-
connector.properties
Or I have to run all the previous steps in my Kafka container directly...
I was thinking of just doing this, copy my jar file and just moving it in my kafka container.
docker cp /hostfile (container_id):/(to_the_place_you_want_the_file_to_be)
Thank you!
guess I have to go into my Kafka container and run the below code
No. Containers should only run one process, which is kafka server.
So, either you download Kafka locally and run the Connect scripts from your host.
Or you simply add a new container for Kafka Connect, which will run Connect distributed mode, rather than standalone.
In either case, yes, you need to copy (or mount) the jar into Connect's plugin path
Alternatively, run MSK rather than Kinesis and produce your Kafka data there rather than locally.

What can be the reason why I can't connect to CateDB with grafana and locally with Dbeaver?

Cheers
I am currently trying to reproduce the tutorial to develop an application / solution with Fiware seen in: Desarrolla tu primer aplicaciĆ³n en Fiware and I am having difficulty connecting Grafana with the Crate database (recommended by Fiware).
My docker-compose file configuration is:
version: "3.5"
services:
orion:
image: fiware/orion
hostname: orion
container_name: fiware-orion
networks:
- fiware_network
depends_on:
- mongo-db
ports:
- "1026:1026"
command: -dbhost mongo-db -logLevel DEBUG -noCache
mongo-db:
image: mongo:3.6
hostname: mongo-db
container_name: db-mongo
networks:
- fiware_network
ports:
- "27017:27017"
command: --bind_ip_all --smallfiles
volumes:
- mongo-db:/data
grafana:
image: grafana/grafana:8.2.6
container_name: fiware-grafana
networks:
- fiware_network
ports:
- "3000:3000"
depends_on:
- crate
quantumleap:
image: fiware/quantum-leap
container_name: fiware-quantumleap
networks:
- fiware_network
ports:
- "8668:8668"
depends_on:
- mongo-db
- orion
- crate
environment:
- CRATE_HOST=crate
crate:
image: crate:1.0.5
networks:
- fiware_network
ports:
# Admin UI
- "4200:4200"
# Transport protocol
- "4300:4300"
command: -Ccluster.name=democluster -Chttp.cors.enabled=true -Chttp.cors.allow-origin="*"
volumes:
- cratedata:/data
volumes:
mongo-db:
cratedata:
networks:
fiware_network:
driver: bridge
After starting the containers, I have a positive response from OCB (Orion), from Quantumleap and even after creating the subscription between Orion and quantumleap, in the Crate database, the data is stored and updated correctly.
Unfortunately I am not able to get the visualization of the data in grafana.
I thought that the reason was the fact that crateDB was removed as a grafana plugin versions ago, but after researching how to connect crateDB with grafana, through a postgres data source (I read on: Visualizing time series data with Grafana and CrateDB), I'm still having difficulty getting the connection, getting in grafana "Query error
dbquery error: EOF"
grafana settings img
The difference with respect to the guide is the listening port, since with input port 5432 I get a response indicating that it is not possible to connect to the server. I'm using port 4300.
After configuring, and trying to query from grafana, I get the mentioned EOF error
EOF error in grafana img
I tried to connect from a database IDE (Dbeaver) and I get exactly the same problem.
EOF error in Dbeaver img
Is there something I am missing?
What should I change in my docker configuration, or ports, or anything else to fix this?
I think it is worth mentioning that I am studying this because I am being asked in a project to visualize context switches in real time with Fiware and grafana.
Thanks in advance
The PostgreSQL port 5432 must be exposed as well by the CrateDB docker image.
Additionally, I highly recommend to use a recent (or just the latest) CrateDB version, current stable is 5.0.0, your used version 1.0.5 is very old and not maintained anymore.
Full example entry:
crate:
image: crate:latest
networks:
- fiware_network
ports:
# Admin UI
- "4200:4200"
# Transport protocol
- "4300:4300"
# PostgreSQL protocol
- "5432:5432"
command: -Ccluster.name=democluster -Chttp.cors.enabled=true -Chttp.cors.allow-origin="*"
volumes:
- cratedata:/data

Failed to collect cluster Default info java.lang.IllegalStateException: Error while creating AdminClient for Cluster Default

I would like to use network_mode: bridge for kafka for being able to reach kafka through localhost:9092 from another service
I'm trying to use the provectus/kafka-ui but when I open the consumers menu I get the following error
my docker-compose.yml file :
kafka-ui:
container_name: kafka-ui
image: provectuslabs/kafka-ui:latest
ports:
- 8080:8080
depends_on:
- kafka
environment:
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092
KAFKA_CLUSTERS_0_JMXPORT: 9997
kafka:
image: johnnypark/kafka-zookeeper
ports:
- "2181:2181"
- "9092:9092"
network_mode: bridge
environment:
ADVERTISED_HOST: 127.0.0.1
NUM_PARTITIONS: 1
volumes:
- /var/run/docker.sock:/var/run/docker.sock
log error:
2022-01-13 09:16:50,014 ERROR [parallel-5] c.p.k.u.s.MetricsService: Failed to collect cluster Default info
java.lang.IllegalStateException: Error while creating AdminClient for Cluster Default
provectus/kafka-ui
I was using the johnnypark/kafka-zookeeper library for both kafka and zookeeper. I was able to solve this problem by using two separate libraries as in the example below
zookeeper1:
image: confluentinc/cp-zookeeper:5.2.4
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
kafka1:
image: confluentinc/cp-kafka:5.3.1
depends_on:
- zookeeper1
ports:
- 9093:9093
- 9998:9998
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper1:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:29092,PLAINTEXT_HOST://localhost:9093
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
JMX_PORT: 9998
KAFKA_JMX_OPTS: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=kafka1 -Dcom.sun.management.jmxremote.rmi.port=9998
being able to reach kafka through localhost:9092 from another service
You can't use localhost to reach Kafka since that would be the Kafka UI container itself.
Changing ADVERTISED_HOST to kafka and using kafka:9092 from other containers is correct for a bridge network. However, this have the side effect of preventing any access to Kafka outside the Docker network, such as clients directly on the host machine.
Internal and External clients can be configured separately. bitnami/bitnami-docker-kafka
Here's an example using Bitnami's Kafka Image - this allows host clients to connect on port 9093 while allowing kafka-ui to connect with the default port.
version: "3"
services:
zookeeper:
image: 'bitnami/zookeeper:latest'
ports:
- '2181:2181'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
kafka:
image: 'bitnami/kafka:latest'
ports:
- '9092:9092'
- '9093:9093'
environment:
- KAFKA_BROKER_ID=1
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
- KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9093
- KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka:9092,EXTERNAL://localhost:9093
- KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
- ALLOW_PLAINTEXT_LISTENER=yes
depends_on:
- zookeeper
kafka-ui:
image: provectuslabs/kafka-ui
container_name: kafka-ui
ports:
- "8081:8081"
restart: always
environment:
- KAFKA_CLUSTERS_0_NAME=local
- KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:9092
- SERVER_PORT=8081

How to use fluent-bit with Docker-compose

I want to use the fluent-bit docker image to help me persist the ephemeral docker container logs to a location on my host (and later use it to ship logs elsewhere).
I am facing issues such as:
Cannot start service clamav: failed to initialize logging driver: dial tcp 127.0.0.1:24224: connect: connection refused
I have read a number of post including
configuring fluentbit with docker but I'm still at a lost.
My docker-compose is made up of nginx, our app, keycloak, elasticsearch and clamav. I have added fluent-bit, made it first to starts via depends on. I changed the other services to use the fluentd logging driver.
Part of config:
clamav:
container_name: clamav-app
image: tiredofit/clamav:latest
restart: always
volumes:
- ./clamav/data:/data
- ./clamav/logs:/logs
environment:
- ZABBIX_HOSTNAME=clamav-app
- DEFINITIONS_UPDATE_FREQUENCY=60
networks:
- iris-network
expose:
- "3310"
depends_on:
- fluentbit
logging:
driver: fluentd
fluentbit:
container_name: iris-fluent
image: fluent/fluent-bit:latest
restart: always
networks:
- iris-network
volumes:
- ./fluent-bit/etc:/fluent-bit/etc
ports:
- "24224:24224"
- "24224:24224/udp"
I have tried to proxy_pass 24224 to fluentbit in nginx and start nginx first, and that avoided the error on clamav and es, but same error with keycloak.
So how can I configure the service to use the host or is it that localhost is not the "external" host?

Sample Mesos marathon setup

I am trying to setup one single mesos master, mesos slave and a marathon instance using docker compose. Mesos master and slave starts up without any issues but it's throwing error at marathon start up:
marathon:
image: mesosphere/marathon:v1.1.2
network_mode: host
environment:
MESOS_MASTER: zk://127.0.0.1:2181/mesos
depends_on:
- zookeeper
In startup I am seeing the below error:
mesos_mesos_slave_one_1 exited with code 1 marathon_1
[2017-08-25 01:50:28,344] INFO Starting Marathon 1.1.2 with
(mesosphere.marathon.Main$:main) marathon_1 |
[scallop] Error:
Required option 'master' not found mesos_marathon_1.
exited with code 1
Thanks in advance.
docker compose file:
version: '2'
services:
zookeeper:
image: zookeeper
network_mode: host
mesos_master:
image: mesosphere/mesos-master:1.0.1-2.0.93.ubuntu1404
network_mode: host
environment:
- MESOS_ZK=zk://127.0.0.1:2181/mesos
- MESOS_HOSTNAME=127.17.0.4
depends_on:
- zookeeper
mesos_slave_one:
image: mesosphere/mesos-slave:1.0.1-2.0.93.ubuntu1404
network_mode: host
environment:
- MESOS_MASTER=zk://127.0.0.1:2181/mesos
- MESOS_WORK_DIR= /tmp
- MESOS_CONTAINERIZERS=docker
- MESOS_HOSTNAME=127.17.0.4
- MESOS_PORT=5051
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup
- /var/run/docker.sock:/var/run/docker.sock
- /usr/bin/docker:/usr/bin/docker:ro
depends_on:
- zookeeper
marathon:
image: mesosphere/marathon:v1.1.2
network_mode: host
environment:
- MESOS_MASTER=zk://127.0.0.1:2181/mesos
- master=local
depends_on:
- zookeeper
My bad I defined MESOS_MASTER instead of marathon master in my docker compose file.
MARATHON_MASTER: zk://127.0.0.1:2181/mesos