Keycloak does not use postgres database and tries to connect h2 database - postgresql

Question

I am trying to configure Keycloak to use postgres using docker-compose.
Docker compose file for reference:
version: "3.9"
services:
keycloak-postgres:
image: postgres:latest
restart: unless-stopped
ports:
- 5432:5432
environment:
POSTGRES_DB: ${POSTGRESQL_DB}
POSTGRES_USER: ${POSTGRESQL_USER}
POSTGRES_PASSWORD: ${POSTGRESQL_PASS}
volumes:
- postgres_data:/var/lib/postgresql/data
keycloak:
depends_on:
- keycloak-postgres
image: quay.io/keycloak/keycloak
container_name: keycloak
ports:
- 8030:8080
environment:
KC_DB: postgres
KC_DB_URL_HOST: keycloak-postgres
KC_DB_URL_DATABASE: ${POSTGRESQL_DB}
KC_DB_USERNAME: ${POSTGRESQL_USER}
KC_DB_PASSWORD: ${POSTGRESQL_PASS}
KEYCLOAK_ADMIN: ${KEYCLOAK_ADMIN}
KEYCLOAK_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD}
KC_HOSTNAME: ${KEYCLOAK_HOSTNAME}
KC_PROXY: edge
KC_HTTP_ENABLED: true
restart: unless-stopped
command:
- start --optimized
volumes:
postgres_data:
driver: local
I have found that if I run start without the optimized flag, keycloak starts without any issues, but also does not use the postgres database - as there are no tables or anything created by Keycloak when I connect to the DB.
When I run with the optimized flag, I get the following error:
URL format error; must be "jdbc:h2:{ {.|mem:}[name] | [file:]fileName | {tcp|ssl}:[//]server[:port][,server2[:port]]/name }[;key=value...]" but is "jdbc:postgresql://keycloak-postgres:5432/keycloak" [90046-214]
From what I can make out the postgres connection string which Keycloak has generated is correct. However it is trying to connect to a h2 database, which is clearly incorrect.
I have looked through all the configuration options and just can't make out why:
a) Keycloak isn't storing any data in postgres in start mode.
b) Keycloak is trying to access a H2 database in --optimized mode.
Update
Following advice from sonOfRa and to try and simplify the problem I have now tried the following:
Run postgres as a seperate docker.
Created the below Dockerfile as per the documentation (have also tried with sonOfRa's cut down Dockerfile):
FROM quay.io/keycloak/keycloak:latest as builder
# Enable health and metrics support
ENV KC_HEALTH_ENABLED=true
ENV KC_METRICS_ENABLED=true
# Configure a database vendor
ENV KC_DB=postgres
RUN /opt/keycloak/bin/kc.sh build
FROM quay.io/keycloak/keycloak:latest
COPY --from=builder /opt/keycloak/ /opt/keycloak/
ENV KC_DB_URL_HOST=192.168.1.25
ENV KC_DB_USERNAME=keycloak
ENV KC_DB_PASSWORD=keycloak_db_password
ENV KC_HOSTNAME=localhost
ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]
Run the following command to build the new Dockerfile:
docker build . -t mykeycloak
Run the following command to start Keycloak:
docker run --name mykeycloak \
-p 8030:8080 \
-e KEYCLOAK_ADMIN=admin \
-e KEYCLOAK_ADMIN_PASSWORD=change_me \
-e KC_HOSTNAME=auth.url.com \
-e KC_PROXY=edge \
-e KC_HTTP_ENABLED=true \
mykeycloak start
Output from console:
2023-01-11 14:06:19,961 INFO [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: <unset>, Hostname: auth.url.com, Strict HTTPS: true, Path: <request>, Strict BackChannel: false, Admin URL: <unset>, Admin: <request>, Port: -1, Proxied: true
2023-01-11 14:06:25,844 WARN [io.quarkus.agroal.runtime.DataSources] (main) Datasource <default> enables XA but transaction recovery is not enabled. Please enable transaction recovery by setting quarkus.transaction-manager.enable-recovery=true, otherwise data may be lost if the application is terminated abruptly
2023-01-11 14:06:28,797 INFO [org.infinispan.server.core.transport.EPollAvailable] (keycloak-cache-init) ISPN005028: Native Epoll transport not available, using NIO instead: java.lang.UnsatisfiedLinkError: could not load a native library: netty_transport_native_epoll_aarch_64
2023-01-11 14:06:29,311 WARN [org.infinispan.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
2023-01-11 14:06:29,436 WARN [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
2023-01-11 14:06:29,541 INFO [org.keycloak.broker.provider.AbstractIdentityProviderMapper] (main) Registering class org.keycloak.broker.provider.mappersync.ConfigSyncEventListener
2023-01-11 14:06:29,581 INFO [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
2023-01-11 14:06:30,440 INFO [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000128: Infinispan version: Infinispan 'Triskaidekaphobia' 13.0.10.Final
2023-01-11 14:06:30,819 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000078: Starting JGroups channel `ISPN`
2023-01-11 14:06:30,820 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000088: Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
2023-01-11 14:06:31,143 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
2023-01-11 14:06:31,144 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 20.00MB, but the OS only allocated 212.99KB
2023-01-11 14:06:31,146 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
2023-01-11 14:06:31,147 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 25.00MB, but the OS only allocated 212.99KB
2023-01-11 14:06:33,179 INFO [org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) cb354516ab9d-30183: no members discovered after 2009 ms: creating cluster as coordinator
2023-01-11 14:06:33,213 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000094: Received new cluster view for channel ISPN: [cb354516ab9d-30183|0] (1) [cb354516ab9d-30183]
2023-01-11 14:06:33,228 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000079: Channel `ISPN` local address is `cb354516ab9d-30183`, physical addresses are `[172.17.0.2:52593]`
2023-01-11 14:06:35,021 INFO [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: cb354516ab9d-30183, Site name: null
2023-01-11 14:06:41,372 INFO [org.keycloak.quarkus.runtime.storage.legacy.liquibase.QuarkusJpaUpdaterProvider] (main) Initializing database schema. Using changelog META-INF/jpa-changelog-master.xml
2023-01-11 14:06:53,286 INFO [org.keycloak.services] (main) KC-SERVICES0050: Initializing master realm
2023-01-11 14:07:00,559 INFO [io.quarkus] (main) Keycloak 20.0.2 on JVM (powered by Quarkus 2.13.3.Final) started in 45.755s. Listening on: http://0.0.0.0:8080
2023-01-11 14:07:00,561 INFO [io.quarkus] (main) Profile prod activated.
2023-01-11 14:07:00,562 INFO [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-h2, jdbc-mariadb, jdbc-mssql, jdbc-mysql, jdbc-oracle, jdbc-postgresql, keycloak, logging-gelf, narayana-jta, reactive-routes, resteasy, resteasy-jackson, smallrye-context-propagation, smallrye-health, smallrye-metrics, vault, vertx]
2023-01-11 14:07:02,212 INFO [org.keycloak.services] (main) KC-SERVICES0009: Added user 'admin' to realm 'master'
Unforuntely the result is the same.
I can access keycloak from the set URL and login using the admin user created on run. Everything seemingly works on the UI, except it does not store any data in the postgres database.

Answer 1

This is due to your use of the --optimized parameter. If you use it, it is assumed that you have already ran "build", which you did not do. It is recommended to create your own docker image which uses the upstream docker image as a base. This is described in the documentation here.
Essentially, you need to run the build command with --db=postgres (or the KC_DB=postgres environment variable), in order to tell Quarkus to build an optimized image that will later use postgres. That image can then be started with --optimized and it will correctly use postgres instead of H2.
Step 1 is to create a Dockerfile (not a docker-compose.yml!)
FROM quay.io/keycloak/keycloak
# Configure a database vendor
ENV KC_DB=postgres
WORKDIR /opt/keycloak
RUN /opt/keycloak/bin/kc.sh build
ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]
You can also include additional things at this point, like custom providers, but this is the minimal data that you need in order to make it work.
Now you have 2 options: You can build this image with docker build and push it to your own docker registry with docker push, or you can use it directly from your docker-compose.yaml. If you build and push, replace the image: quay.io/keycloak/keycloak line with image: your.registry/wherever/you/pushed. If you want to use it directly in your compose-file, you can remove the image: line completely, and replace it with
build: .
When doing this, you must ensure that the Dockerfile is in the same directory as the docker-compose.yaml