Creating pod and service for custom kafka connect image with kubernetes - kubernetes

Question

I had successfully created a custom kafka connector image containing confluent hub connectors.
I am trying to create pod and service to launch it in GCP with kubernetes.
How should I configure yaml file ? The next part of code I took from quick-start guide. This is what I've tried:
Dockerfile:
FROM confluentinc/cp-kafka-connect-base:latest
ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components,/usr/share/java/kafka-connect-jdbc"
RUN confluent-hub install --no-prompt confluentinc/kafka-connect-jdbc:10.2.6
RUN confluent-hub install --no-prompt debezium/debezium-connector-mysql:1.7.1
RUN confluent-hub install --no-prompt debezium/debezium-connector-postgresql:1.7.1
RUN confluent-hub install --no-prompt confluentinc/kafka-connect-oracle-cdc:1.5.0
RUN wget -O /usr/share/confluent-hub-components/confluentinc-kafka-connect-jdbc/lib/mysql-connector-java-8.0.26.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.26/mysql-connector-java-8.0.26.jar
Modifield part of confluent-platform.yaml
apiVersion: platform.confluent.io/v1beta1
kind: Connect
metadata:
name: connect
namespace: confluent
spec:
replicas: 1
image:
application: maxprimeaery/kafka-connect-jdbc:latest #confluentinc/cp-server-connect:7.0.1
init: confluentinc/confluent-init-container:2.2.0-1
configOverrides:
server:
- config.storage.replication.factor=1
- offset.storage.replication.factor=1
- status.storage.replication.factor=1
podTemplate:
resources:
requests:
cpu: 200m
memory: 512Mi
probe:
liveness:
periodSeconds: 10
failureThreshold: 5
timeoutSeconds: 500
podSecurityContext:
fsGroup: 1000
runAsUser: 1000
runAsNonRoot: true
And that's the error I get in console for connect-0 pod:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 45m default-scheduler Successfully assigned confluent/connect-0 to gke-my-kafka-cluster-default-pool-6ee97fb9-fh9w
Normal Pulling 45m kubelet Pulling image "confluentinc/confluent-init-container:2.2.0-1"
Normal Pulled 45m kubelet Successfully pulled image "confluentinc/confluent-init-container:2.2.0-1" in 17.447881861s
Normal Created 45m kubelet Created container config-init-container
Normal Started 45m kubelet Started container config-init-container
Normal Pulling 45m kubelet Pulling image "maxprimeaery/kafka-connect-jdbc:latest"
Normal Pulled 44m kubelet Successfully pulled image "maxprimeaery/kafka-connect-jdbc:latest" in 23.387676944s
Normal Created 44m kubelet Created container connect
Normal Started 44m kubelet Started container connect
Warning Unhealthy 41m (x5 over 42m) kubelet Liveness probe failed: HTTP probe failed with statuscode: 404
Normal Killing 41m kubelet Container connect failed liveness probe, will be restarted
Warning Unhealthy 5m (x111 over 43m) kubelet Readiness probe failed: HTTP probe failed with statuscode: 404
Warning BackOff 17s (x53 over 22m) kubelet Back-off restarting failed container
Should I create separate pod and service for custom kafka connector or I have to configure the code above ?
UPDATE to my question
I've found out how to configure it in kubernetes adding this to connect pod
apiVersion: platform.confluent.io/v1beta1
kind: Connect
metadata:
name: connect
namespace: confluent
spec:
replicas: 1
image:
application: confluentinc/cp-server-connect:7.0.1
init: confluentinc/confluent-init-container:2.2.0-1
configOverrides:
server:
- config.storage.replication.factor=1
- offset.storage.replication.factor=1
- status.storage.replication.factor=1
build:
type: onDemand
onDemand:
plugins:
locationType: confluentHub
confluentHub:
- name: kafka-connect-jdbc
owner: confluentinc
version: 10.2.6
- name: kafka-connect-oracle-cdc
owner: confluentinc
version: 1.5.0
- name: debezium-connector-mysql
owner: debezium
version: 1.7.1
- name: debezium-connector-postgresql
owner: debezium
version: 1.7.1
storageLimit: 4Gi
podTemplate:
resources:
requests:
cpu: 200m
memory: 1024Mi
probe:
liveness:
periodSeconds: 180 #DONT CHANGE THIS
failureThreshold: 5
timeoutSeconds: 500
podSecurityContext:
fsGroup: 1000
runAsUser: 1000
runAsNonRoot: true
But I still can't add mysql-connector from Maven repo
I tried also making new docker image but it doesn't work. Also I tried new part of code:
locationType: url #NOT WORKING. NO IDEA HOW TO CONFIGURE THAT
url:
- name: mysql-connector-java
archivePath: https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.26/mysql-connector-java-8.0.26.jar
checksum: sha512sum #definitely wrong

Answer 1

After some retries I found out that I just had to wait a little bit longer.
probe:
liveness:
periodSeconds: 180 #DONT CHANGE THIS
failureThreshold: 5
timeoutSeconds: 500
This part periodSeconds: 180 will add more time to make the pod Running and I can just use my own image.
image:
application: maxprimeaery/kafka-connect-jdbc:5.0
init: confluentinc/confluent-init-container:2.2.0-1
And build part can be removed after those changes.