Running SonarQube with Docker in CI/CD pipeline - postgresql

I'm trying to get SonarQube stood up and scanning applications via Docker containers on an EC2 instance. I've spent the past day poring over SonarQube and Postgres documentation and am having very little luck.
The most sensible guide I've found is the docker-sonarqube project maintained by SonarSource. More specifically, I am following the SonarQube/Postgres guide using docker-compose.
My docker-compose.yml file looks identical to the one provided by SonarSource:
sonarqube:
build: "5.2"
ports:
- "9000:9000"
links:
- db
environment:
- SONARQUBE_JDBC_URL=jdbc:postgresql://db:5432/sonar
volumes_from:
- plugins
db:
image: postgres
volumes_from:
- datadb
environment:
- POSTGRES_USER=sonar
- POSTGRES_PASSWORD=sonar
datadb:
image: postgres
volumes:
- /var/lib/postgresql
command: /bin/true
plugins:
build: "5.2"
volumes:
- /opt/sonarqube/extensions
- /opt/sonarqube/lib/bundled-plugins
command: /bin/true
docker ps -a yields:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2d003aef18f2 dockersonarqube_sonarqube "./bin/run.sh" 47 seconds ago Up 46 seconds 0.0.0.0:9000->9000/tcp dockersonarqube_sonarqube_1
c7d5043f4381 dockersonarqube_plugins "./bin/run.sh /bin/tr" 48 seconds ago Exited (0) 46 seconds ago dockersonarqube_plugins_1
590c72b4a723 postgres "/docker-entrypoint.s" 48 seconds ago Up 47 seconds 5432/tcp dockersonarqube_db_1
c105e6aebe09 postgres "/docker-entrypoint.s" 49 seconds ago Exited (0) 48 seconds ago dockersonarqube_datadb_1
Latest output from the sonarqube_1 container is:
sonarqube_1 | 2016.01.20 17:49:09 INFO web[o.s.s.a.TomcatAccessLog] Web server is started
sonarqube_1 | 2016.01.20 17:49:09 INFO web[o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
sonarqube_1 | 2016.01.20 17:49:09 INFO app[o.s.p.m.Monitor] Process[web] is up
What does concern me is the latest output from the db_1 container:
PostgreSQL init process complete; ready for start up.
LOG: database system was shut down at 2016-01-20 17:48:40 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
ERROR: relation "schema_migrations" does not exist at character 21
STATEMENT: select version from schema_migrations
ERROR: relation "schema_migrations" does not exist at character 21
STATEMENT: select version from schema_migrations
ERROR: relation "schema_migrations" does not exist at character 21
STATEMENT: select version from schema_migrations
ERROR: relation "schema_info" does not exist at character 15
STATEMENT: SELECT * FROM "schema_info" LIMIT 1
Navigating to http://my.instance.ip:9000 is unsuccessful. I am able to hit the respective ports of other running containers from the same machine.
Could anyone help to point me in the right direction? Any other guides or documentation that may serve me better? I also see issues with the documentation stating that analyzing a project begins with mvn sonar:sonar, but I'll defer that for now. Thank you very much in advance!

Use this image
I modified this image to talk to a RDS instance.
EC2(docker-sonar)<==> RDS postgres

Related

Postgresql Docker container not receiving TCP requests in swarm mode

I am not quite sure where my problem is, I can only describe some symtoms, so please be patient with error logs/configurations.
I want to install a HA postgresql database. The easiest ways to me seems to do it via preconfigured docker images.
I am using the bitnami postgresql image for this with the following configuration in swarm mode on two separate nodes.
version: '3.8'
services:
postgresql-master:
image: 'docker.io/bitnami/postgresql:15'
ports:
- '5432:5432'
networks:
- postgres_network
volumes:
- '/localVol:/bitnami/postgresql'
environment:
- POSTGRESQL_REPLICATION_MODE=master
- POSTGRESQL_REPLICATION_USER=repmgr_username
- POSTGRESQL_REPLICATION_PASSWORD=repmgr_password
- POSTGRESQL_USERNAME=username
- POSTGRESQL_PASSWORD=password
- POSTGRESQL_DATABASE=dbname
- POSTGRESQL_SYNCHRONOUS_COMMIT_MODE=on
- POSTGRESQL_NUM_SYNCHRONOUS_REPLICAS=1
deploy:
placement:
constraints:
- node.labels.type == primary
postgresql-slave:
image: 'docker.io/bitnami/postgresql:15'
ports:
- '5432'
networks:
- postgres_network
depends_on:
- postgresql-master
environment:
- POSTGRESQL_USERNAME=username
- POSTGRESQL_PASSWORD=password
- POSTGRESQL_REPLICATION_MODE=slave
- POSTGRESQL_REPLICATION_USER=repmgr_username
- POSTGRESQL_REPLICATION_PASSWORD=repmgr_password
- POSTGRESQL_MASTER_HOST=postgresql-master
- POSTGRESQL_MASTER_PORT_NUMBER=5432
volumes:
- '/localVol:/bitnami/postgresql'
deploy:
placement:
constraints:
- node.labels.type != primary
networks:
postgres_network:
driver: overlay
external: false
internal: true
ipam:
config:
- subnet: 10.70.1.0/24
The swarm is created via a simple init command and the node is joined via the join command. No extra config.
When running this file with docker compose up (without the deploy constraints) on one host, the two containers are up and running, replicating the database and so on. Working as desired.
When running this file as is with docker stack up, the primary is running and stable, the secondary is not; see logs
Primary
postgresql 14:07:57.00 INFO ==> ** Starting PostgreSQL setup **
postgresql 14:07:57.06 INFO ==> Validating settings in POSTGRESQL_* env vars..
postgresql 14:07:57.07 INFO ==> Loading custom pre-init scripts...
postgresql 14:07:57.09 INFO ==> Initializing PostgreSQL database...
postgresql 14:07:57.10 INFO ==> Custom configuration /opt/bitnami/postgresql/conf/pg_hba.conf detected
postgresql 14:07:57.17 INFO ==> Deploying PostgreSQL with persisted data...
postgresql 14:07:57.21 INFO ==> Configuring replication parameters
postgresql 14:07:57.28 INFO ==> Configuring fsync
postgresql 14:07:57.31 INFO ==> Loading custom scripts...
postgresql 14:07:57.31 INFO ==> Enabling remote connections
postgresql 14:07:57.33 INFO ==> ** PostgreSQL setup finished! **
postgresql 14:07:57.34 INFO ==> ** Starting PostgreSQL **
2022-11-16 14:07:57.363 GMT [1] LOG: pgaudit extension initialized
2022-11-16 14:07:57.374 GMT [1] LOG: starting PostgreSQL 15.0 on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-11-16 14:07:57.377 GMT [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2022-11-16 14:07:57.378 GMT [1] LOG: could not create IPv6 socket for address "::": Address family not supported by protocol
2022-11-16 14:07:57.380 GMT [1] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2022-11-16 14:07:57.384 GMT [83] LOG: database system was shut down at 2022-11-16 14:07:36 GMT
2022-11-16 14:07:57.392 GMT [1] LOG: database system is ready to accept connections
secondary
postgresql 14:40:51.58 INFO ==> ** Starting PostgreSQL setup **
postgresql 14:40:51.64 INFO ==> Validating settings in POSTGRESQL_* env vars..
postgresql 14:40:51.65 INFO ==> Loading custom pre-init scripts...
postgresql 14:40:51.66 INFO ==> Initializing PostgreSQL database...
postgresql 14:40:51.70 INFO ==> pg_hba.conf file not detected. Generating it...
postgresql 14:40:51.70 INFO ==> Generating local authentication configuration
postgresql 14:40:51.74 INFO ==> Waiting for replication master to accept connections (60 timeout)...
postgresql-master:5432 - no response
The secondary restarts itself after a time of constantly logging no response.
I have tried pinging the containers which works. Also when exposing the port of the primary to the host, it is possible to access the database from the host BUT it is not possible to send any TCP traffic to both container as tried with netcat and tcpdump. Netcat is able to send packets, but tcpdump on the primary and secondary does not show requests.
Anybody got a tip for me?
I just found the error.
As someone states in his blog, a specific port (4789) is blocked when virtualising with an ESXi stack. This is the default port for overlay network traffic.
Simply changing that port when initialising a swarm solves the problem.
docker swarm init --data-path-port 4788

Micronaut + PostgreSQL + Docker-compose issue

I'm trying to run a Micronaut service that uses PostgreSQL on a docker-compose file. But I'm having the following issue:
21:12:32.741 [main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
21:12:33.744 [main] ERROR com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Exception during pool initialization.
org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:315)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:225)
at org.postgresql.Driver.makeConnection(Driver.java:465)
at org.postgresql.Driver.connect(Driver.java:264)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:358)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:477)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:560)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81)
at io.micronaut.configuration.jdbc.hikari.HikariUrlDataSource.<init>(HikariUrlDataSource.java:35)
at io.micronaut.configuration.jdbc.hikari.DatasourceFactory.dataSource(DatasourceFactory.java:66)
at io.micronaut.configuration.jdbc.hikari.$DatasourceFactory$DataSource0Definition.build(Unknown Source)
at io.micronaut.context.BeanDefinitionDelegate.build(BeanDefinitionDelegate.java:153)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1979)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2768)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2754)
at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2425)
at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2399)
at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1264)
at io.micronaut.context.AbstractBeanDefinition.getBeanForConstructorArgument(AbstractBeanDefinition.java:1014)
at io.micronaut.configuration.hibernate.jpa.$EntityManagerFactoryBean$HibernateStandardServiceRegistry0Definition.doBuild(Unknown Source)
at io.micronaut.context.AbstractParametrizedBeanDefinition.build(AbstractParametrizedBeanDefinition.java:118)
at io.micronaut.context.BeanDefinitionDelegate.build(BeanDefinitionDelegate.java:149)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1979)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2768)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2754)
at io.micronaut.context.DefaultBeanContext.getBeanForDefinition(DefaultBeanContext.java:2425)
at io.micronaut.context.DefaultBeanContext.getBeanInternal(DefaultBeanContext.java:2399)
at io.micronaut.context.DefaultBeanContext.getBean(DefaultBeanContext.java:1264)
at io.micronaut.context.AbstractBeanDefinition.getBeanForConstructorArgument(AbstractBeanDefinition.java:1014)
at io.micronaut.configuration.hibernate.jpa.$EntityManagerFactoryBean$HibernateMetadataSources1Definition.doBuild(Unknown Source)
at io.micronaut.context.AbstractParametrizedBeanDefinition.build(AbstractParametrizedBeanDefinition.java:118)
at io.micronaut.context.BeanDefinitionDelegate.build(BeanDefinitionDelegate.java:149)
at io.micronaut.context.DefaultBeanContext.doCreateBean(DefaultBeanContext.java:1979)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingletonInternal(DefaultBeanContext.java:2768)
at io.micronaut.context.DefaultBeanContext.createAndRegisterSingleton(DefaultBeanContext.java:2754)
at io.micronaut.context.DefaultBeanContext.loadContextScopeBean(DefaultBeanContext.java:2292)
at io.micronaut.context.DefaultBeanContext.initializeContext(DefaultBeanContext.java:1562)
at io.micronaut.context.DefaultApplicationContext.initializeContext(DefaultApplicationContext.java:234)
at io.micronaut.context.DefaultBeanContext.readAllBeanDefinitionClasses(DefaultBeanContext.java:2905)
at io.micronaut.context.DefaultBeanContext.start(DefaultBeanContext.java:231)
at io.micronaut.context.DefaultApplicationContext.start(DefaultApplicationContext.java:180)
at io.micronaut.runtime.Micronaut.start(Micronaut.java:71)
at org.wcode.author.service.ApplicationKt.main(Application.kt:10)
Caused by: java.net.UnknownHostException: st-database
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:220)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:609)
at org.postgresql.core.PGStream.createSocket(PGStream.java:231)
at org.postgresql.core.PGStream.<init>(PGStream.java:95)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:98)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:213)
... 46 common frames omitted
Files definition
Following is the definition of the files that I'm using on my current project.
docker-compose.yml
version: '3.8'
services:
st-author-service:
image: author-service:latest
environment:
- JDBC_URL=jdbc:postgresql://st-database:5432/sentency_db
- JDBC_USER=<USERNAME>
- JDBC_PASSWORD=<PASSWORD>
- JDBC_DRIVER=org.postgresql.Driver
depends_on:
- st-database
networks:
- database-network
- internal
st-database:
image: postgres:13.3-alpine
restart: always
env_file:
- database.env # configure postgres
volumes:
- database-data:/var/lib/postgresql/data/
networks:
- database-network
volumes:
database-data:
networks:
database-network:
driver: bridge
external: false
internal:
application.yml
micronaut:
application:
name: authorService
server:
port: 7000
datasources:
default:
url: ${JDBC_URL:`jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE`}
username: ${JDBC_USER:sa}
password: ${JDBC_PASSWORD:""}
driverClassName: ${JDBC_DRIVER:org.h2.Driver}
schema-generate: CREATE_DROP
jpa:
default:
entity-scan:
packages: 'org.wcode.author.service.domain.entities'
properties:
hibernate:
hbm2ddl:
auto: update
show_sql: true
Tests
I tested the application.yml pointing to a PostgreSQL database deployed on Docker and it worked, the problem started when I tried to use docker-compose to structure the deployment. To check if the network definition was working I deployed pgAdmin together with that configuration and I was able to connect to the database.
After that specific line:
Caused by: java.net.UnknownHostException: st-database
I come to the conclusion that the issue was caused by the host not being found.
I already made a lot of research and haven't found a way to solve that problem. I can't see what I'm missing.
Thanks in advance for all the help.
Update
I was able to get the connection log from the PostgreSQL container and only pgAdmin connection is appearing, the service is not even reaching the container to connect. The network being used is the same. Here is the log:
2021-08-10 15:43:13.465 GMT [100] LOG: connection received: host=sentency-deploy_pgAdmin_1.sentency-deploy_database-network port=40848
2021-08-10 15:43:13.466 GMT [100] LOG: connection authorized: user=user database=postgres
2021-08-10 15:43:13.478 GMT [101] LOG: connection received: host=sentency-deploy_pgAdmin_1.sentency-deploy_database-network port=40850
2021-08-10 15:43:13.479 GMT [101] LOG: connection authorized: user=user database=sentency_db
2021-08-10 15:43:13.491 GMT [102] LOG: connection received: host=sentency-deploy_pgAdmin_1.sentency-deploy_database-network port=40852
2021-08-10 15:43:13.493 GMT [102] LOG: connection authorized: user=user database=postgres
2021-08-10 15:43:17.750 GMT [103] LOG: connection received: host=sentency-deploy_pgAdmin_1.sentency-deploy_database-network port=40858
Now I think the problem could be the way I'm building the image. My current Dockerfile is:
FROM ghcr.io/graalvm/graalvm-ce:ol7-java11-21.2.0 as build
WORKDIR /author-service
COPY . /author-service
RUN yum install -y -q xz
RUN curl -sL -o - https://github.com/upx/upx/releases/download/v3.96/upx-3.96-amd64_linux.tar.xz | tar xJ
RUN gu install native-image
RUN ./gradlew assemble
RUN native-image --no-fallback --static -jar /author-service/build/libs/author-service-*-all.jar service
RUN ./upx-3.96-amd64_linux/upx -7 /author-service/service
FROM scratch
EXPOSE 7000:7000
COPY --from=build /author-service/service /service
ENTRYPOINT ["./service"]
That's giving me an image of 27.67MB.
From the docker compose documentation: depends_on does not wait for [st-database] to be “ready”. So maybe this is caused by the Postgres database not being ready to accept connections at the time your app tries to. The UnknownHostException is a bit strange in that case but that may depend on when docker is binding the port. To fix the problem you would need to write a script that waits for readiness as in the second example here.

PostgresSql terminates with signal 11 when run with docker-compose

I'm trying to run a simple postgres with docker-compose, which is more challenging than I thought. I'm running it on a Raspberry Pi 4 Model B Rev 1.1 with docker-compose at 1.27.4.
It fails after calling docker-compose up with the following output:
mydb_1 | Success. You can now start the database server using:
mydb_1 |
mydb_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
mydb_1 |
mydb_1 | waiting for server to start....[36] LOG: starting PostgreSQL 13.1 on arm-unknown-linux-musleabihf, compiled by gcc (Alpine 10.2.1_pre1) 10.2.1 20201203, 32-bit
mydb_1 | [36] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
mydb_1 | ......[36] LOG: startup process (PID 37) was terminated by signal 11: Segmentation fault
mydb_1 | [36] LOG: aborting startup due to startup process failure
mydb_1 | [36] LOG: database system is shut down
mydb_1 | pg_ctl: could not start server
mydb_1 | Examine the log output.
mydb_1 | stopped waiting
docker-compose.yaml
version: '3'
services:
mydb:
image: postgres:13.1-alpine
environment:
- POSTGRES_USER=synapse
- POSTGRES_PASSWORD=test
volumes:
- synapse-db:/var/lib/postgresql/data
volumes:
synapse-db:
Any ideas why this might happen?
I had the same issue with the latest images (I tried all from 9 to 13).
Looks like the problem is the latest build, so I replaced 9-alpine version which was commited a couple of days ago with 9.4.23-alpine which is 3 months old and the problem disappeared.
So try to pick any version older than a week here https://hub.docker.com/r/arm32v7/postgres/tags?page=1&ordering=last_updated
The option
security_opt:
- seccomp:unconfined
in docker-compose worked for me.
I'm experiencing the same issue here even when removing the old database by removing the volume which contains it.
I tried with postgres:12-alpine and postgres:9-alpine and same result:
...
waiting for server to start....LOG: startup process (PID 30) was terminated by signal 11: Segmentation fault
LOG: aborting startup due to startup process failure
LOG: database system is shut down
pg_ctl: could not start server
...
After investigation, this is again the alpine:3.13 hell on raspbian as described here: https://github.com/docker-library/redis/issues/269#issuecomment-778588162
Just install the latest version of libseccomp as described in this comment and restart everything, it should work...:
wget http://ftp.us.debian.org/debian/pool/main/libs/libseccomp/libseccomp2_2.5.2-2_armhf.deb
sudo dpkg -i libseccomp2_2.5.2-2_armhf.deb

Gitlab CI services are stopped after bizzar psql call

I am trying to use services in gitlab ci, namely, postgres. Anyway, postgres service doesn't seem to be running, although I just copied what is there in Gitlab CI docs. In logs, after service logs that it started, some psql command (I don't know where it came from) gives an name resolution error. If I am doing something wrong here, what is the way to run postgres service in gitlab-ci?
Below are .gitlab-ci.yml file and logs:
.gitlab-ci.yml
image: ubuntu
services:
- name: postgres:12.2-alpine
alias: postgres
variables:
POSTGRES_DB: badr
POSTGRES_USER: badr
POSTGRES_PASSWORD: badr
PGHOST: postgres
POSTGRES_HOST_AUTH_METHOD: trust
stages:
- test
test db:
stage: test
before_script:
- until (echo > /dev/tcp/postgres/5432) >/dev/null 2>&1;do >&2 echo "service not ready...sleeping";sleep 5;done
script:
- echo "connected to...$PGHOST"
- sleep 10
logs
Running with gitlab-runner 13.2.0-rc2 (45f2b4ec)
on docker-auto-scale fa6cab46
Preparing the "docker+machine" executor
00:55
Using Docker executor with image ubuntu ...
Starting service postgres:12.2-alpine ...
Pulling docker image postgres:12.2-alpine ...
Using docker image sha256:ae192c4d3adaebbbf2f023e1e50eaadfabccb6b08c855ac13d6ce2232381a58a for postgres:12.2-alpine ...
WARNING: Service postgres:12.2-alpine is already created. Ignoring.
Waiting for services to be up and running...
*** WARNING: Service runner-fa6cab46-project-14794655-concurrent-0-f52b350b86ad38db-postgres-0 probably didn't start properly.
Health check error:
service "runner-fa6cab46-project-14794655-concurrent-0-f52b350b86ad38db-postgres-0-wait-for-service" timeout
Health check container logs:
Service container logs:
2020-07-31T09:15:00.377204181Z ********************************************************************************
2020-07-31T09:15:00.377254629Z WARNING: POSTGRES_HOST_AUTH_METHOD has been set to "trust". This will allow
2020-07-31T09:15:00.377259167Z anyone with access to the Postgres port to access your database without
2020-07-31T09:15:00.377262471Z a password, even if POSTGRES_PASSWORD is set. See PostgreSQL
2020-07-31T09:15:00.377265670Z documentation about "trust":
2020-07-31T09:15:00.377269181Z https://www.postgresql.org/docs/current/auth-trust.html
2020-07-31T09:15:00.377272282Z In Docker's default configuration, this is effectively any other
2020-07-31T09:15:00.377276152Z container on the same system.
2020-07-31T09:15:00.377295876Z
2020-07-31T09:15:00.377299453Z It is not recommended to use POSTGRES_HOST_AUTH_METHOD=trust. Replace
2020-07-31T09:15:00.377302412Z it with "-e POSTGRES_PASSWORD=password" instead to set a password in
2020-07-31T09:15:00.377305641Z "docker run".
2020-07-31T09:15:00.377308656Z ********************************************************************************
2020-07-31T09:15:00.404620899Z The files belonging to this database system will be owned by user "postgres".
2020-07-31T09:15:00.406021814Z This user must also own the server process.
2020-07-31T09:15:00.406074886Z
2020-07-31T09:15:00.406083517Z The database cluster will be initialized with locale "en_US.utf8".
2020-07-31T09:15:00.406087263Z The default database encoding has accordingly been set to "UTF8".
2020-07-31T09:15:00.406090884Z The default text search configuration will be set to "english".
2020-07-31T09:15:00.406094281Z
2020-07-31T09:15:00.406097490Z Data page checksums are disabled.
2020-07-31T09:15:00.406101511Z
2020-07-31T09:15:00.406197662Z fixing permissions on existing directory /var/lib/postgresql/data ... ok
2020-07-31T09:15:00.406858429Z creating subdirectories ... ok
2020-07-31T09:15:00.407274720Z selecting dynamic shared memory implementation ... posix
2020-07-31T09:15:00.428414929Z selecting default max_connections ... 100
2020-07-31T09:15:00.506801199Z selecting default shared_buffers ... 128MB
2020-07-31T09:15:00.689382376Z selecting default time zone ... UTC
2020-07-31T09:15:00.695744690Z creating configuration files ... ok
2020-07-31T09:15:01.009439741Z running bootstrap script ... ok
2020-07-31T09:15:01.355673765Z sh: locale: not found
2020-07-31T09:15:01.355836607Z 2020-07-31 09:15:01.355 UTC [30] WARNING: no usable system locales were found
2020-07-31T09:15:01.784080826Z performing post-bootstrap initialization ... ok
2020-07-31T09:15:02.416545146Z syncing data to disk ... ok
2020-07-31T09:15:02.416652656Z
2020-07-31T09:15:02.416854775Z initdb: warning: enabling "trust" authentication for local connections
2020-07-31T09:15:02.416911707Z You can change this by editing pg_hba.conf or using the option -A, or
2020-07-31T09:15:02.416917642Z --auth-local and --auth-host, the next time you run initdb.
2020-07-31T09:15:02.416962149Z
2020-07-31T09:15:02.416967325Z Success. You can now start the database server using:
2020-07-31T09:15:02.416970415Z
2020-07-31T09:15:02.416990907Z pg_ctl -D /var/lib/postgresql/data -l logfile start
2020-07-31T09:15:02.416995097Z
2020-07-31T09:15:02.440378884Z waiting for server to start....2020-07-31 09:15:02.440 UTC [35] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.2.0) 9.2.0, 64-bit
2020-07-31T09:15:02.442773414Z 2020-07-31 09:15:02.442 UTC [35] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-07-31T09:15:02.461804500Z 2020-07-31 09:15:02.461 UTC [36] LOG: database system was shut down at 2020-07-31 09:15:01 UTC
2020-07-31T09:15:02.465323529Z 2020-07-31 09:15:02.465 UTC [35] LOG: database system is ready to accept connections
2020-07-31T09:15:02.524643142Z done
2020-07-31T09:15:02.524766601Z server started
2020-07-31T09:15:02.537508874Z psql: error: could not connect to server: could not translate host name "postgres" to address: Name does not resolve
*********
Pulling docker image ubuntu ...
Using docker image sha256:1e4467b07108685c38297025797890f0492c4ec509212e2e4b4822d367fe6bc8 for ubuntu ...
Preparing environment
00:02
Getting source from Git repository
00:01
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/badrmoh/cicdtest/.git/
Created fresh repository.
Checking out 604433de as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ until (echo > /dev/tcp/postgres/5432) >/dev/null 2>&1;do >&2 echo "service not ready...sleeping";sleep 5;done
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
gitlab-runner
The problem was in PGHOST variable. It seems it is used internally by postgres container that is why it fails to start.
The solution is to set PGHOST in script directive:
image: ubuntu
services:
- name: postgres:9
alias: postgres
variables:
POSTGRES_DB: badr
POSTGRES_USER: badr
POSTGRES_PASSWORD: badr
POSTGRES_HOST_AUTH_METHOD: trust
stages:
- test
test db:
stage: test
before_script:
- export PGHOST=postgres
- until (echo > /dev/tcp/$PGHOST/5432) >/dev/null 2>&1;do >&2 echo "service $PGHOST not ready...sleeping";sleep 5;done
script:
- echo "connected to...$PGHOST"
- sleep 10
Note: You can't use variables directive within jobs in this case since it seems to be populated before even starting services itself.

Docker Postgresql ERROR HINT: The server must be started by the user that owns the data directory postgresql rhel

I have a docker compose file where i am launching PostgreSQL with shared volume. But i am continuously getting the bellow ERROR.
2018-10-11 14:57:01.757 GMT [81] LOG: skipping missing configuration
file "/postgresql/data/postgresql.auto.conf"
| 2018-10-11 14:57:01.768 GMT [81] FATAL: data directory "/postgresql/data" has wrong ownership
| 2018-10-11 14:57:01.768 GMT [81] HINT: The server must be started by the user that owns the data directory.
My docker compose file as below
addb:
image : postgres
networks:
- private
ports:
- "5432:5432"
volumes:
- /mnt/shared/app_data/db/postgres/data_db:/postgresql/data
depends_on:
- sol-server
I am using RHEL 7.5 and Docker version 18.06.1-ce, build e68fc7. Any Idea how i can resolve the above issue.
You should mount your volume with the database in /var/lib/postgresql/data
instead of /postgresql/data or give a enviroment PGDATA=/postgresql/data