Running a Keycloak Cluster in GKE - keycloak

Question

Im trying to get KeyCloak 20.0.1 running as a cluster in GKE.
The deployment is not a problem, but figuring out how the cluster cache is working is a pain.
The deployment is running using a Cloud SQL (Mysql 5.7) instance.
The question is, should I use
cache-stack=kubernetes or cache-stack=google, or can I use UDP og TCP?
If I should use the kubernets cache stack, how do I configure the headless-service thing needed?
Hopefully someone is already running Keycloak as a cluster in GKE and are willing to share some knowledge and maybe the yaml files for the deployment.
I have tried to configure all different cache-stack options, the result is the following in the logs:
[org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) keycloak: no members discovered after 2008 ms: creating cluster as coordinator
"

Answer 1

I found the answer.
The parameter for the jGroup described in https://www.keycloak.org/server/caching, -Djgroups.dns.query=, need to be set to to the service name for the clusterIP in the deployment yml file. In my case ${GKE_NAME}-jgroups-ping
Here is the yml files and Docerfile for doing cluster deployment of Keycloak in GKE
Im using bitbucket pipeline.
deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
labels:
io.kompose.service: ${GKE_NAME}
name: ${GKE_NAME}
namespace: default
spec:
replicas: 4
selector:
matchLabels:
app: ${GKE_NAME}
template:
metadata:
labels:
app: ${GKE_NAME}
spec:
containers:
- image: $IMAGE_NAME
imagePullPolicy: Always
args: ["start", "--optimized","-Djgroups.dns.query=${GKE_NAME}-jgroups-ping"]
name: ${GKE_NAME}
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 4444
- containerPort: 8888
envFrom:
- configMapRef:
name: ${GKE_NAME}
readinessProbe:
httpGet:
path: /realms/master
port: 8080
---
apiVersion: v1
kind: Service
metadata:
labels:
service: ${GKE_NAME}
name: ${GKE_NAME}-service
namespace: default
spec:
selector:
app: ${GKE_NAME}
ports:
- name: http
protocol: TCP
port: 8080
targetPort: 8080
type: LoadBalancer
externalTrafficPolicy: Cluster
---
apiVersion: v1
kind: Service
metadata:
labels:
service: ${GKE_NAME}
name: ${GKE_NAME}-jgroups-ping
spec:
clusterIP: None
ports:
- port: 4444
name: ping
protocol: TCP
targetPort: 4444
selector:
app: ${GKE_NAME}
sessionAffinity: None
type: ClusterIP
---
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
name: ${GKE_NAME}
namespace: default
spec:
domains:
- $DOMAIN
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
networking.gke.io/managed-certificates: "${GKE_NAME}"
kubernetes.io/ingress.allow-http: "false"
kubernetes.io/ingress.global-static-ip-name: "${GKE_STATIC_IP_NAME}"
name: ${GKE_NAME}-ingress
namespace: default
spec:
defaultBackend:
service:
name: ${GKE_NAME}-service
port:
number: 8080
Dockerfile
# Dockerfile
FROM quay.io/keycloak/keycloak:latest
ENV KC_HEALTH_ENABLED=true
ENV KC_METRICS_ENABLED=true
ENV KC_DB=mysql
ENV KC_CACHE=ispn
ENV KC_CACHE_STACK=kubernetes
WORKDIR /opt/keycloak
# Replace with your own certificate if needed
RUN keytool -genkeypair -storepass password -storetype PKCS12 -keyalg RSA -keysize 2048 -dname "CN=server" -alias server -ext "SAN:c=DNS:localhost,IP:127.0.0.1" -keystore conf/server.keystore
RUN bin/kc.sh build
EXPOSE 8080
ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]