I have started Confluent Platform on my windows 10 using docker with the help of https://docs.confluent.io/current/quickstart/ce-docker-quickstart.html. Now I want to try using Confluent CLI. But I don't see any documentation on how to use confluent cli on docker. Can you please suggest me how can I do this !
Confluent does not provide a docker image for the CLIs at this time (that I'm aware of). Until that time, you could build a simple image locally to package up the CLI for experimenting w/ the command.
Create Dockerfile:
FROM ubuntu:latest
RUN apt update && apt upgrade
RUN apt install -y curl
RUN curl -L --http1.1 https://cnfl.io/cli | sh -s -- -b /usr/local/bin
Then build with:
docker build -t confluent-cli:latest .
Then run on the cp-all-in-one network with:
$ docker run -it --rm --network="cp-all-in-one_default" confluent-cli:latest bash
Then from the containers shell, experiement w/ the command:
root#421e53d4a04a:/# confluent
Manage your Confluent Platform.
Usage:
confluent [command]
Available Commands:
cluster Retrieve metadata about Confluent clusters.
completion Print shell completion code.
help Help about any command
iam Manage RBAC, ACL and IAM permissions.
local Manage a local Confluent Platform development environment.
login Log in to Confluent Platform (required for RBAC).
logout Logout of Confluent Platform.
secret Manage secrets for Confluent Platform.
update Update the confluent CLI.
version Print the confluent CLI version.
Flags:
-h, --help help for confluent
-v, --verbose count Increase verbosity (-v for warn, -vv for info, -vvv for debug, -vvvv for trace).
--version version for confluent
Use "confluent [command] --help" for more information about a command.
Here is the image:
https://hub.docker.com/r/confluentinc/confluent-cli
Basically run the following commands:
devbox1#devbox1:~/onibex/wa$ docker pull confluentinc/confluent-cli
devbox1#devbox1:~/onibex/wa$ docker run confluentinc/confluent-cli
To check if the image was added:
devbox1#devbox1:~/onibex/wa$ docker ps -a | grep confluent-cli
a5ecf9223d35 confluentinc/confluent-cli
Add "sudo" if it is needed.
Related
----- i use kafka, kafka-connect(image: confluentinc/cp-kafka-connect)
when you use kafka in docker container if you wanna operate kafka, you have to go into the container(like 'docker exec -it kafka' or 'docker exec -it kafka-connect' ----> this is another question what i wanna ask) , right..??
i tried putting some connector (jdbc connector, mysql connector) into kafka-connect container, but it didn't work.
so.. my question is
after docker-compose up(put in container), if i wanna connect with some connectors('./bin/connect-distributed.sh ./etc/kafka/connect-distributed.properties'),
what container i have to go into???
if i type plugin path, where should i write?( kafka? kafka-connect?)
I wouldn't mind if it was difficult to read... sorry for that
No, you don't need to exec anywhere unless you cannot download Kafka on your host machine to get the CLI scripts. But you'd only exec for kafka-topics, console producer/consumer, kafka-consumer-groups, etc, not any of the Connect scripts.
The Connect container automatically runs the Distributed script and you simply provide CONNECT_PLUGIN_PATH as an environment variable to any directory in the container you want to use for the plugins (I like /opt/connectors if I mount volume, but that's not where confluent-hub installs to for that image). That variable doesn't do anything for the broker image, only Connect.
Related How to install connectors to the docker image of apache kafka connect
If your requirement is startup a Kafka Connect.
You can use the basic guide published by Confluent "Build Your Own Apache Kafka® Demos"
Basically you need execute the following instructions:
git clone https://github.com/confluentinc/cp-all-in-one.git
cd cp-all-in-one/cp-all-in-one
git checkout 7.1.1-post
docker-compose up -d
This has Control Center at http://localhost:8088
If you need install a Connector, you can go to the https://www.confluent.io/hub select your specific connector.
Then, you can create your DockerImage of specific Kafka Connect server.
1.- Write a Dockerfile.
vim Dockerfile
2.- Add connector "example JDBC" from Confluent Hub.
FROM confluentinc/cp-kafka-connect
ENV MYSQL_DRIVER_VERSION 5.1.39
RUN confluent-hub install --no-prompt confluentinc/kafka-connect-jdbc:10.5.0
RUN curl -k -SL "https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-${MYSQL_DRIVER_VERSION}.tar.gz" \
| tar -xzf - -C /usr/share/confluent-hub-components/confluentinc-kafka-connect-jdbc/lib \
--strip-components=1 mysql-connector-java-5.1.39/mysql-connector-java-${MYSQL_DRIVER_VERSION}-bin.jar
3.- Build the docker image.
docker build . -t my-kafka-connect-jdbc:1.0.0
4.- Then, you can go to edit your docker-compose.yml, change the line 57
from:
image: cnfldemos/cp-server-connect-datagen:0.5.3-7.1.0
to:
image: my-kafka-connect-jdbc:1.0.0
5.- Finally, stop and start your Confluent Platform local environment:
docker-compose down
docker-compose up
Verify your docker
docker ps
Test your Connect server:
curl --location --request GET 'http://localhost:8083/connectors'
I am trying to install a K8S cluster with Rancher, after installing docker successfully I ran the following command to install Rancher containers:
$ sudo docker run -d --restart=unless-stopped -p 8088:8088 rancher/server:stable
The console I got was:
As you can see I am not being able to download the Rancher containers, what could I do to make it work?
Well, starting with the point this Rancher installation is based on what is proposed in this link https://phoenixnap.com/kb/install-rancher-on-ubuntu. The Rancher version to be installed following the script in the link, according to Docker Hub, is 1.x.
So, I don't recommend the command proposed in the script:
sudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server:stable
See, the rancher/server must be replaced by: rancher/rancher:stable, then you will install latest Rancher version 2.x.
Also, to avoid the timeout problem you instead of using the command "docker run" go for "docker pull" first, then "docker run". In other words, after finishing the "pull" then you go with the "run" option. I recommend you run the following commands which will be faster and effective:
sudo docker pull rancher/rancher:latest
sudo docker run -d --restart=unless-stopped -p 8088:8088 -p 8443:8443 --privileged rancher/rancher:latest
After running that everything is in good shape, you can run your Rancher. I hope that can help someone, I lost one day to figure that out, since the download time takes hours each time you run to timeout and fail.
I have been trying to install Entando 6 on my Mac following the instructions on http://docs.entando.com, however when deploying to Kubernetes I get an error with quickstart-kc-deployer. Has anyone managed to successfully go through with the installation?
deployment failure
Also I am new to Kubernetes and trying to access any logs, however as of now I have not been able to access logs and understand a bit more what the root cause of the failure is. Help on that is also more than welcome as well.
Thanks.
If you're in a local development environment the best bet would be to try the new instructions at dev.entando.org. If you're installing on a cloud Kubernetes provider try the updated instructions here.
I've reproduced them here for completeness:
Install Multipass (https://multipass.run/#install
Launch VM
multipass launch --name ubuntu-lts --cpus 4 --mem 8G --disk 20G
Open a shell multipass shell ubuntu-lts
Install k3s curl -sfL https://get.k3s.io | sh -
Download Entando custom resource definitions
curl -L -C - https://raw.githubusercontent.com/entando/entando-releases/v6.2.0/dist/qs/custom-resources.tar.gz | tar -xz
Create custom resources
sudo kubectl create -f dist/crd
Create namespace
sudo kubectl create namespace entando
Download Helm chart
curl -L -C - -O https://raw.githubusercontent.com/entando/entando-releases/v6.2.0/dist/qs/entando.yaml
Configure access to your cluster
IP=$(hostname -I | awk '{print $1}')
sed -i "s/192.168.64.25/$IP/" entando.yaml
If you want to deploy on a cloud provider (EKS, AKS, GKE) then there are new instructions under the Configuration and Operations section at
https://dev.entando.org/next/tutorials
I installed confluent community platform Manual Install using Systemd on Ubuntu and Debian
I can start:
sudo systemctl start confluent-zookeeper
sudo systemctl start confluent-kafka
...
but where is my confluent path?
I do not have confluent cli
please help
I encountered same problem when setting up elastisearch for confluent. In my system /usr/bin acted as /bin and remaining set up was successfull . So try using /usr as path to confluent if following Manual Install using Systemd on Ubuntu and Debian.
You can manually install Confluent CLI:
curl -L https://cnfl.io/cli | sh -s -- -b /<path-to-directory>/bin
Run this script to install the Confluent CLI. This command creates a
bin directory in your designated location (<path-to-directory>/bin).
The location must be in your PATH (e.g. /usr/local/bin).
I am trying to install Kubernetes locally on my CentOS. I am following this blog http://containertutorials.com/get_started_kubernetes/index.html, with appropriate changes to match CentOS and latest Kubernetes version.
./kube-up.sh script runs and exists with no errors and I don't see the server started on port 8080. Is there a way to know what was the error and if there is any other procedure to follow on CentOS 6.3
The easiest way to install the kubernetes cluster is using kubeadm. The initial post which details the steps of setup is here. And the detailed documentation for the kubeadm can be found here. With this you will get the latest released kubernetes.
If you really want to use the script to bring up the cluster, I did following:
Install the required packages
yum install -y git docker etcd
Start docker process
systemctl enable --now docker
Install golang
Latest go version because default centos golang is old and for kubernetes to compile we need at least go1.7
curl -O https://storage.googleapis.com/golang/go1.8.1.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.8.1.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin
Setup GOPATH
export GOPATH=~/go
export GOBIN=$GOPATH/bin
export PATH=$PATH:$GOBIN
Download k8s source and other golang dependencies
Note: this might take sometime depending on your internet speed
go get -d k8s.io/kubernetes
go get -u github.com/cloudflare/cfssl/cmd/...
Start cluster
cd $GOPATH/src/k8s.io/kubernetes
./hack/local-up-cluster.sh
In new terminal
alias kubectl=$GOPATH/src/k8s.io/kubernetes/cluster/kubectl.sh
kubectl get nodes