Postgres, Go Docker compose wait-for-it.sh no such file or directory - postgresql

Been racking my head about this one for almost two days.
I'm new to Docker and Docker Compose, and trying to run my image on an EC2 instance running Postgres and Go. When I run docker-compose up, the db service runs successfully, but not the app service.
When I try to run the services separately using:
docker-compose up db all is good
then run:
docker-compose up app I get...
app_1 | wait-for-it.sh: waiting 15 seconds for db:5432
app_1 | wait-for-it.sh: db:5432 is available after 0 seconds
app_1 | ./wait-for-it.sh: line 174: /go/src/github.com/MY_USERNAME/MY_APP_DIR/EXECUTABLE: No such file or directory
some_name-golang_app_1 exited with code 127
Dockerfile-alternate:
FROM golang:latest
EXPOSE 8080
WORKDIR /go/src/github.com/MY_USERNAME/MY_APP_DIR
ADD . /go/src/github.com/MY_USERNAME/MY_APP_DIR
# Install all dependencies of the current project.
RUN go get -v
RUN go build
docker-compose.yml
version: '3'
services:
db:
image: postgres
environment:
POSTGRES_DB: dbname
POSTGRES_USER: miller
POSTGRES_PASSWORD: miller
ports:
- "6000:5432"
app:
build:
context: .
dockerfile: Dockerfile-alternate
command: ["./wait-for-it.sh", "db:5432", "--", "./EXECUTABLE"]
volumes:
- .:/go/src/github.com/gregpmillr/volume
ports:
- "80:8080"
depends_on:
- db
links:
- db
Interestingly enough, if I run docker run -it --rm MY_USERNAME/custom-go-image then I actually do see the EXECUTABLE file, and can run ./EXECUTABLE successfully... Well sort-of, I get a no such host error but pretty sure that's because I'm not starting them at the same time using docker-compose.
Any thought on this issue? Tips / resources would be great. I've been heading down rabbit holes of googling issues and leading nowhere. It's probably something small that I'm missing as per usual. Thanks!!
I'm looking to run postgres first (works correctly now), then run the go server which connects to postgres.
UPDATE 1
command: [ "sh", "-c", "cd /go/src/github.com/gregpmillr/volume && ls -l" ]
will give the following output:
db_1 | 2018-08-19 11:19:48.828 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2018-08-19 11:19:48.828 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2018-08-19 11:19:48.831 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2018-08-19 11:19:48.857 UTC [21] LOG: database system was shut down at 2018-08-19 01:03:35 UTC
db_1 | 2018-08-19 11:19:48.880 UTC [1] LOG: database system is ready to accept connections
app_1 | total 36
app_1 | -rw-rw-r-- 1 1000 1000 238 Aug 19 11:19 Dockerfile-alternate
app_1 | -rw-rw-r-- 1 1000 1000 260 Aug 17 19:24 Dockerrun.aws.json
app_1 | -rw-rw-r-- 1 1000 1000 62 Aug 17 18:56 README.md
app_1 | drwxrwxr-x 8 1000 1000 4096 Aug 17 19:00 app
app_1 | drwxrwxr-x 2 1000 1000 4096 Aug 17 19:34 config
app_1 | -rwxrwxr-x 1 1000 1000 708 Aug 17 19:48 deploy.sh
app_1 | -rw-rw-r-- 1 1000 1000 548 Aug 19 11:19 docker-compose.yml
app_1 | -rw-rw-r-- 1 1000 1000 1188 Aug 17 19:00 main.go
app_1 | -rwxrwxr-x 1 1000 1000 4079 Aug 17 19:00 wait-for-it.sh
app_1 exited with code 0
Dockerrun file is unnecessary as I'm not using elastic beanstalk atm.
UPDATE 2
Solved. See accepted answer. More specifically, see https://docs.docker.com/storage/ and https://docs.docker.com/storage/volumes/ for more info on Volumes. Thanks for the help!

You've added the go source code into your image and compiled it inside the image as part of your build. Then you overlaid that same path with a volume containing your source code (apparently without a compiled binary) by including the following:
volumes:
- .:/go/src/github.com/gregpmillr/Tranquility-Online-Golang
You either need the compiled binary in that volume, or skip mounting the volume into the container since it is blocking access to the files in your image.

Related

Can't Access Web (Flask) Application from Google Cloud Platform's VM SSH Link

My goal is to run a docker-compose cluster on a VM from Google Cloud Platform. I have successfully installed docker and docker-compose:
$ uname -a
Linux instance-6 4.15.0-1083-gcp #94~16.04.1-Ubuntu SMP Sat Sep 5 22:53:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ docker -v
Docker version 19.03.13, build 4484c46d9d
$ docker-compose -v
docker-compose version 1.27.3, build 4092ae5d
I am following the basic tutorial to create a docker-compose cluster using: https://docs.docker.com/compose/gettingstarted/ (Steps #1-#4).
My app.py file is:
import time
import redis
from flask import Flask
app = Flask(__name__)
cache = redis.Redis(host='redis', port=6379)
def get_hit_count():
retries = 5
while True:
try:
return cache.incr('hits')
except redis.exceptions.ConnectionError as exc:
if retries == 0:
raise exc
retries -= 1
time.sleep(0.5)
#app.route('/')
def hello():
count = get_hit_count()
return 'Hello World! I have been seen {} times.\n'.format(count)
My requirements.txt file is:
flask
redis
My Dockerfile is:
FROM python:3.7-alpine
WORKDIR /code
ENV FLASK_APP app.py
ENV FLASK_RUN_HOST 0.0.0.0
RUN apk add --no-cache gcc musl-dev linux-headers
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
EXPOSE 5000
COPY . .
CMD ["flask", "run"]
And, my docker-compose.yml is:
version: '3'
services:
web:
build: .
ports:
- "5000:5000"
redis:
image: "redis:alpine"
Running docker-compose up gives me the correct output. One of the outputs points to where the web_1 is running.
$ docker-compose up
...
web_1 | * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
...
After pressing the link http://0.0.0.0:5000/, GCP doesn't connect. It tries to go to the URL: https://ssh.cloud.google.com/devshell/proxy?authuser=2&devshellProxyPath=%2F&port=5000&environment_name&environment_id, but then it gives the error: 500. That’s an error. There was an error. Please try again later. That’s all we know.
Going to the external IP address and putting in port 5000 also doesn't return anything. (http://IPAddress:500)
I checked the ports:
$ sudo docker-compose ps
Name Command State Ports
composetest_redis_1 docker-entrypoint.sh redis ... Up 6379/tcp
composetest_web_1 flask run Up 0.0.0.0:5000->5000/tcp
I'm not sure what the reason it. I'm guessing it is the firewall configuration from GCP. Everything is just the default settings. I also allowed HTTP and HTTPS requests in the Compute Engine VM Instance settings. Would really appreciate more guidance on what to do. Thanks in advance!
See below for full output:
$ sudo docker-compose up
Starting composetest_redis_1 ... done
Starting composetest_web_1 ... done
Attaching to composetest_redis_1, composetest_web_1
redis_1 | 1:C 23 Sep 2020 21:40:27.816 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 23 Sep 2020 21:40:27.816 # Redis version=6.0.8, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 23 Sep 2020 21:40:27.816 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 23 Sep 2020 21:40:27.818 * Running mode=standalone, port=6379.
redis_1 | 1:M 23 Sep 2020 21:40:27.818 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 23 Sep 2020 21:40:27.818 # Server initialized
redis_1 | 1:M 23 Sep 2020 21:40:27.818 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 23 Sep 2020 21:40:27.819 * Loading RDB produced by version 6.0.8
redis_1 | 1:M 23 Sep 2020 21:40:27.819 * RDB age 27 seconds
redis_1 | 1:M 23 Sep 2020 21:40:27.819 * RDB memory usage when created 0.77 Mb
redis_1 | 1:M 23 Sep 2020 21:40:27.819 * DB loaded from disk: 0.000 seconds
redis_1 | 1:M 23 Sep 2020 21:40:27.819 * Ready to accept connections
web_1 | * Serving Flask app "app.py"
web_1 | * Environment: production
web_1 | WARNING: This is a development server. Do not use it in a production deployment.
web_1 | Use a production WSGI server instead.
web_1 | * Debug mode: off
web_1 | * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
Need to properly allow traffic through that port by:
Creating a firewall rule with a tag
Including that tag in the VM's Network settings
Link here: Network Tags

Docker | Postgres Database is uninitialized and superuser password is not specified

I am using docker-compose.yml to create multiple running containers but failing to start Postgres docker server, with following logs and yes I have searched many related SO posts, but they didn't helped me out.
Creating network "complex_default" with the default driver
Creating complex_server_1 ... done
Creating complex_redis_1 ... done
Creating complex_postgres_1 ... done
Attaching to complex_postgres_1, complex_redis_1, complex_server_1
postgres_1 | Error: Database is uninitialized and superuser password is not specified.
postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
postgres_1 |
postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
postgres_1 | connections without a password. This is *not* recommended.
postgres_1 |
postgres_1 | See PostgreSQL documentation about "trust":
postgres_1 | https://www.postgresql.org/docs/current/auth-trust.html
complex_postgres_1 exited with code 1
below is my docker-compose configuration:
version: '3'
services:
postgres:
image: 'postgres:11-alpine'
redis:
image: 'redis:latest'
server:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
as well as package.json inside server directory is following:
{
"dependencies": {
"body-parser": "^1.19.0",
"cors": "^2.8.4",
"express": "^4.16.3",
"nodemon": "^2.0.4",
"pg": "7.4.3",
"redis": "^2.8.0"
},
"scripts": {
"dev": "nodemon",
"start": "node index.js"
}
}
also for better consideration, I have attached my hands-on project structure:
A year ago it were actually working fine, Does anyone have any idea, what's going wrong here inside my docker-compose file now.
A year ago it were actually working fine, Does anyone have any idea, what's going wrong here inside my docker-compose file now.
Seems like you pulled the fresh image, where in the new image you should specify Postgres user password. You can look into Dockerhub, the image is update one month ago
postgress-11-alpine
db:
image: postgres:11-alpine
restart: always
environment:
POSTGRES_PASSWORD: example
As the error message is self expalinatory
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
or POSTGRES_HOST_AUTH_METHOD=trust use this which is not recommended.
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
connections without a password. This is *not* recommended.
POSTGRES_PASSWORD
This environment variable is required for you to use the PostgreSQL image. It must not be empty or undefined. This environment variable sets the superuser password for PostgreSQL. The default superuser is defined by the POSTGRES_USER environment variable.
Environment Variables
#Adiii yes, you are nearly right, so I have to explicitly mentioned the environment also for the postgres image but with no db parent tag.
So here, I am explicitly mentioning the docker-compose.yaml config to help others for better understanding, also now I am using recent stable postgres image version 12-alpine, currently latest is postgres:12.3
version: '3'
services:
postgres:
image: 'postgres:12-alpine'
environment:
POSTGRES_PASSWORD: postgres_password
redis:
image: 'redis:latest'
server:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
and so after docker-compose up the creating and running logs were as following:
PS E:\docker\complex> docker-compose up
Creating network "complex_default" with the default driver
Creating complex_postgres_1 ... done
Creating complex_redis_1 ... done
Creating complex_server_1 ... done
Attaching to complex_redis_1, complex_postgres_1, complex_server_1
postgres_1 | The files belonging to this database system will be owned by user "postgres".
postgres_1 | This user must also own the server process.
postgres_1 |
postgres_1 | The database cluster will be initialized with locale "en_US.utf8".
postgres_1 | The default database encoding has accordingly been set to "UTF8".
postgres_1 | The default text search configuration will be set to "english".
postgres_1 |
postgres_1 | Data page checksums are disabled.
postgres_1 |
postgres_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
postgres_1 | creating subdirectories ... ok
postgres_1 | selecting dynamic shared memory implementation ... posix
postgres_1 | selecting default max_connections ... 100
postgres_1 | selecting default shared_buffers ... 128MB
postgres_1 | selecting default time zone ... UTC
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # Redis version=6.0.6, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 05 Aug 2020 14:24:48.692 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
postgres_1 | creating configuration files ... ok
redis_1 | 1:M 05 Aug 2020 14:24:48.693 * Running mode=standalone, port=6379.
redis_1 | 1:M 05 Aug 2020 14:24:48.693 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 05 Aug 2020 14:24:48.694 # Server initialized
redis_1 | 1:M 05 Aug 2020 14:24:48.694 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 05 Aug 2020 14:24:48.694 * Ready to accept connections
postgres_1 | running bootstrap script ... ok
server_1 |
server_1 | > # dev /app
server_1 | > nodemon
server_1 |
postgres_1 | performing post-bootstrap initialization ... sh: locale: not found
postgres_1 | 2020-08-05 14:24:50.153 UTC [29] WARNING: no usable system locales were found
server_1 | [nodemon] 2.0.4
server_1 | [nodemon] to restart at any time, enter `rs`
server_1 | [nodemon] watching path(s): *.*
server_1 | [nodemon] watching extensions: js,mjs,json
server_1 | [nodemon] starting `node index.js`
postgres_1 | ok
server_1 | Listening
postgres_1 | syncing data to disk ... ok
postgres_1 |
postgres_1 |
postgres_1 | Success. You can now start the database server using:
postgres_1 |
postgres_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
postgres_1 |
postgres_1 | initdb: warning: enabling "trust" authentication for local connections
postgres_1 | You can change this by editing pg_hba.conf or using the option -A, or
postgres_1 | --auth-local and --auth-host, the next time you run initdb.
postgres_1 | waiting for server to start....2020-08-05 14:24:51.634 UTC [34] LOG: starting PostgreSQL 12.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
postgres_1 | 2020-08-05 14:24:51.700 UTC [34] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-08-05 14:24:51.981 UTC [35] LOG: database system was shut down at 2020-08-05 14:24:50 UTC
postgres_1 | 2020-08-05 14:24:52.040 UTC [34] LOG: database system is ready to accept connections
postgres_1 | done
postgres_1 | server started
postgres_1 |
postgres_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
postgres_1 |
postgres_1 | waiting for server to shut down....2020-08-05 14:24:52.121 UTC [34] LOG: received fast shutdown request
postgres_1 | 2020-08-05 14:24:52.186 UTC [34] LOG: aborting any active transactions
postgres_1 | 2020-08-05 14:24:52.188 UTC [34] LOG: background worker "logical replication launcher" (PID 41) exited with exit code 1
postgres_1 | 2020-08-05 14:24:52.188 UTC [36] LOG: shutting down
postgres_1 | 2020-08-05 14:24:52.669 UTC [34] LOG: database system is shut down
postgres_1 | done
postgres_1 | server stopped
postgres_1 |
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: starting PostgreSQL 12.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-08-05 14:24:52.832 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-08-05 14:24:52.954 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres_1 | 2020-08-05 14:24:53.136 UTC [43] LOG: database system was shut down at 2020-08-05 14:24:52 UTC
postgres_1 | 2020-08-05 14:24:53.194 UTC [1] LOG: database system is ready to accept connections
Hope this would help manyone.
Adding on ArifMustafa's answer, This worked for me.
postgres:
image: 'postgres:12-alpine'
environment:
POSTGRES_PASSWORD: mypassword
expose:
- 5432
volumes:
- postgres_data:/var/lib/postgres/data/
volumes:
postgres_data:
In my case there was an error about the POSTGRES_PASSWORD (in docker-compose.yml) and the DATABASE settings for PASSWORD in the project_level settings.py file.
After providing a password in docker-compose.yml
db: # For the PostgreSQL database
image: postgres:11
environment:
- POSTGRES_PASSWORD=example
It is necessary to provide the same password to enable access to the POSTGRES database between the two containers (in my case the web application and the database it depended on). In the project_level settings.py:
# Database
# https://docs.djangoproject.com/en/2.2/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'example',
'HOST': 'db',
'PORT': 5432
}
}
To resolve the error when using the command,
docker pull postgres
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
or POSTGRES_HOST_AUTH_METHOD=trust use this which is not recommended.
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all connections without a password. This is not recommended.
Here comes the solution for this:
$ docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
The default postgres user and database are created in the entrypoint with initdb.
The postgres database is a default database meant for use by users, utilities and third party applications.
postgresql.org/docs

How to fix "psql: error: xxx and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?"

How to fix this error?
psql: error: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
We use Docker, Postgres12, Wercker.
When we run this command in Wercker, we can't create a database.
But we can create it in my local Ubuntu environment.
sudo docker exec docker_postgres_1 psql -U postgres -c 'create database postgres_dev'
First, you need to make sure the socket file is located in /var/run/postgresql/.s.PGSQL.5432. To check that
$ cat /var/run/postgresql/.s.PGSQL.5432
if result shows something, then the problem is anything else. But, if file is not there you need to check /tmp dir (specially for OSX Homebrew users)
$ cd /tmp
$ l
total 16
drwxrwxrwt 7 root wheel 224B Mar 11 08:03 .
drwxr-xr-x 6 root wheel 192B Jan 23 18:35 ..
-rw-r--r-- 1 root wheel 65B Nov 7 22:59 .BBE72B41371180178E084EEAF106AED4F350939DB95D3516864A1CC62E7AE82F
srwxrwxrwx 1 shiva wheel 0B Mar 11 08:03 .s.PGSQL.5432
-rw------- 1 shiva wheel 57B Mar 11 08:03 .s.PGSQL.5432.lock
drwx------ 3 shiva wheel 96B Mar 10 17:11 com.apple.launchd.C1tUB2MvF8
drwxr-xr-x 2 root wheel 64B Mar 10 17:10 powerlog
Now, there are two ways you can solve the error
Solution One
You can change the application configuration to see for sockets at /tmp/.s.PGSQL.5432
For Rails Users
# config/database.yml
default: &default
adapter: postgresql
pool: 5
# port:
timeout: 5000
encoding: utf8
# min_messages: warning
socket: /tmp/.s.PGSQL.5432
Solution Two
You can create symlinks to the expected location
$ sudo mkdir /var/pgsql_socket
$ sudo ln /tmp/.s.PGSQL.5432 /var/pgsql_socket/
```
Then the error should go.
Hope this helps.

Postgres shuts down immediately when started with docker-compose

Postgres shuts down immediately when started with docker-compose. The yaml file used is below
version: '2'
services:
postgres:
image: postgres:9.5
container_name: local-postgres9.5
ports:
- "5432:5432"
The log when docker-compose up command is executed
Creating local-postgres9.5
Attaching to local-postgres9.5
local-postgres9.5 | The files belonging to this database system will be owned by user "postgres".
local-postgres9.5 | This user must also own the server process.
local-postgres9.5 |
local-postgres9.5 | The database cluster will be initialized with locale "en_US.utf8".
local-postgres9.5 | The default database encoding has accordingly been set to "UTF8".
local-postgres9.5 | The default text search configuration will be set to "english".
local-postgres9.5 |
local-postgres9.5 | Data page checksums are disabled.
local-postgres9.5 |
local-postgres9.5 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
local-postgres9.5 | creating subdirectories ... ok
local-postgres9.5 | selecting default max_connections ... 100
local-postgres9.5 | selecting default shared_buffers ... 128MB
local-postgres9.5 | selecting dynamic shared memory implementation ... posix
local-postgres9.5 | creating configuration files ... ok
local-postgres9.5 | creating template1 database in /var/lib/postgresql/data/base/1 ... ok
local-postgres9.5 | initializing pg_authid ... ok
local-postgres9.5 | initializing dependencies ... ok
local-postgres9.5 | creating system views ... ok
local-postgres9.5 | loading system objects' descriptions ... ok
local-postgres9.5 | creating collations ... ok
local-postgres9.5 | creating conversions ... ok
local-postgres9.5 | creating dictionaries ... ok
local-postgres9.5 | setting privileges on built-in objects ... ok
local-postgres9.5 | creating information schema ... ok
local-postgres9.5 | loading PL/pgSQL server-side language ... ok
local-postgres9.5 | vacuuming database template1 ... ok
local-postgres9.5 | copying template1 to template0 ... ok
local-postgres9.5 | copying template1 to postgres ... ok
local-postgres9.5 | syncing data to disk ... ok
local-postgres9.5 |
local-postgres9.5 | WARNING: enabling "trust" authentication for local connections
local-postgres9.5 | You can change this by editing pg_hba.conf or using the option -A, or
local-postgres9.5 | --auth-local and --auth-host, the next time you run initdb.
local-postgres9.5 |
local-postgres9.5 | Success. You can now start the database server using:
local-postgres9.5 |
local-postgres9.5 | pg_ctl -D /var/lib/postgresql/data -l logfile start
local-postgres9.5 |
local-postgres9.5 | ****************************************************
local-postgres9.5 | WARNING: No password has been set for the database.
local-postgres9.5 | This will allow anyone with access to the
local-postgres9.5 | Postgres port to access your database. In
local-postgres9.5 | Docker's default configuration, this is
local-postgres9.5 | effectively any other container on the same
local-postgres9.5 | system.
local-postgres9.5 |
local-postgres9.5 | Use "-e POSTGRES_PASSWORD=password" to set
local-postgres9.5 | it in "docker run".
local-postgres9.5 | ****************************************************
local-postgres9.5 | waiting for server to start....LOG: database system was shut down at 2016-05-16 16:51:54 UTC
local-postgres9.5 | LOG: MultiXact member wraparound protections are now enabled
local-postgres9.5 | LOG: database system is ready to accept connections
local-postgres9.5 | LOG: autovacuum launcher started
local-postgres9.5 | done
local-postgres9.5 | server started
local-postgres9.5 | ALTER ROLE
local-postgres9.5 |
local-postgres9.5 |
local-postgres9.5 | /docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
local-postgres9.5 |
local-postgres9.5 | LOG: received fast shutdown request
local-postgres9.5 | LOG: aborting any active transactions
local-postgres9.5 | LOG: autovacuum launcher shutting down
local-postgres9.5 | LOG: shutting down
local-postgres9.5 | waiting for server to shut down....LOG: database system is shut down
local-postgres9.5 | done
local-postgres9.5 | server stopped
local-postgres9.5 |
local-postgres9.5 | PostgreSQL init process complete; ready for start up.
local-postgres9.5 |
local-postgres9.5 | LOG: database system was shut down at 2016-05-16 16:51:55 UTC
local-postgres9.5 | LOG: MultiXact member wraparound protections are now enabled
local-postgres9.5 | LOG: database system is ready to accept connections
local-postgres9.5 | LOG: autovacuum launcher started
Postgres seems to work fine when a container is started using the same image with docker run
docker run --name local-postgres9.5 -p 5432:5432 postgres:9.5
If you look at your log output, the following lines appear towards the end:
local-postgres9.5 | server stopped
local-postgres9.5 |
local-postgres9.5 | PostgreSQL init process complete; ready for start up.
Apparently, stopping and restarting the Postgres server is part of the initialisation process. In fact, the second-to-last line says
local-postgres9.5 | LOG: database system is ready to accept connections.
I'm using the same Postgres docker image version of yours (9.5) and I was running into the same problem.
The first time the container is created, Postgres run a series of commands and in the end, it just sends a shutdown signal and the server becomes unresponsive (but it works the second time onwards).
After several trials, I figured that once I tried to connect to the server, before it was ready to accept connections, PostgresDB would shutdown unexpectedly, and the same would happen for any client trying to connect remotely (outside the container).
I came across this error in the following scenario - I have two containers: one for the PostgresDB itself and other containing a Postgres client (psql) that tries to connect to the first container to create some users, databases and run a liquibase script that creates all the schemas.
At first, I was using a loop in my bash script checking if the server was available every 5 seconds, then after the first attempt, Postgres issued a signal to shutdown and becomes unresponsive.
I could get rid of this error by adding a healthcheck in my docker-compose.yml:
Checkout the CHANGE 1 and CHANGE 2 comments below
version: '3.9'
services:
postgres-db:
container_name: ${POSTGRES_HOST}
image: postgres:9.5
restart: always
ports:
- "5432:5432"
command: postgres
expose:
- 5432
environment:
POSTGRES_USER: "${POSTGRES_USER}"
POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
POSTGRES_DB: "${POSTGRES_DB}"
#this ENV variable is only required for the healthcheck section - if you don't specify it, the check command will fail stating the root user doesn't exist in posgres
PGUSER: "postgres"
healthcheck:
#CHANGE 1: this command checks if the database is ready, right on the source db server
test: [ "CMD-SHELL", "pg_isready" ]
interval: 5s
timeout: 5s
retries: 5
liquibase:
container_name: liquibase-schema-config
image: company/liquibase
build:
context: ./liquibase
environment:
- PGPASSWORD=${POSTGRES_PASSWORD}
- PGPORT=${POSTGRES_PORT}
- PGHOST=${POSTGRES_HOST}
- PGUSER=${POSTGRES_USER}
- PGDATABASE=${POSTGRES_DB}
- JDBC_URL=jdbc:postgresql://${POSTGRES_HOST}:${POSTGRES_PORT}/
- LIQUIBASE_HOME=${LIQUIBASE_HOME}
depends_on:
#CHANGE 2: it prevents issuing a request while the server is starting to depend on the healthy status of postgres-db
postgres-db:
condition: service_healthy
EDIT: There was another issue preventing Docker from accessing VPN protected sites (and even internet sites) while using Cisco AnyConnect on Linux (it doesn't happen on MacOS). To get around this unwanted behavior I installed openconnect sudo apt install openconnect on my Ubuntu and stopped using AnyConnect.
Hope it helps!
I tried your docker-compose and the service seems running in the container:
root#0afe99de0f0b:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
postgres 1 0.5 0.8 227148 16128 ? Ss 14:42 0:00 postgres
postgres 74 0.0 0.0 227148 1768 ? Ss 14:42 0:00 postgres: checkpointer process
postgres 75 0.0 0.0 227148 1772 ? Ss 14:42 0:00 postgres: writer process
postgres 76 0.0 0.0 227148 1768 ? Ss 14:42 0:00 postgres: wal writer process
postgres 77 0.0 0.1 227576 2720 ? Ss 14:42 0:00 postgres: autovacuum launcher process
postgres 78 0.0 0.0 82132 1888 ? Ss 14:42 0:00 postgres: stats collector process
root 79 2.0 0.0 21820 1984 ? Ss 14:42 0:00 /bin/bash
root 84 0.0 0.0 19092 1296 ? R+ 14:42 0:00 ps aux
Anyway, for my project I use another image for postgresql: https://github.com/sameersbn/docker-postgresql. This one works fine.
Add password under postgres service in docker-compose.yml as given in the screenshot. Thank you.
click to see the screenshot
version: '3'
services:
postgres:
image: postgres
environment:
- POSTGRES_PASSWORD=postgres_password

Running SonarQube with Docker in CI/CD pipeline

I'm trying to get SonarQube stood up and scanning applications via Docker containers on an EC2 instance. I've spent the past day poring over SonarQube and Postgres documentation and am having very little luck.
The most sensible guide I've found is the docker-sonarqube project maintained by SonarSource. More specifically, I am following the SonarQube/Postgres guide using docker-compose.
My docker-compose.yml file looks identical to the one provided by SonarSource:
sonarqube:
build: "5.2"
ports:
- "9000:9000"
links:
- db
environment:
- SONARQUBE_JDBC_URL=jdbc:postgresql://db:5432/sonar
volumes_from:
- plugins
db:
image: postgres
volumes_from:
- datadb
environment:
- POSTGRES_USER=sonar
- POSTGRES_PASSWORD=sonar
datadb:
image: postgres
volumes:
- /var/lib/postgresql
command: /bin/true
plugins:
build: "5.2"
volumes:
- /opt/sonarqube/extensions
- /opt/sonarqube/lib/bundled-plugins
command: /bin/true
docker ps -a yields:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2d003aef18f2 dockersonarqube_sonarqube "./bin/run.sh" 47 seconds ago Up 46 seconds 0.0.0.0:9000->9000/tcp dockersonarqube_sonarqube_1
c7d5043f4381 dockersonarqube_plugins "./bin/run.sh /bin/tr" 48 seconds ago Exited (0) 46 seconds ago dockersonarqube_plugins_1
590c72b4a723 postgres "/docker-entrypoint.s" 48 seconds ago Up 47 seconds 5432/tcp dockersonarqube_db_1
c105e6aebe09 postgres "/docker-entrypoint.s" 49 seconds ago Exited (0) 48 seconds ago dockersonarqube_datadb_1
Latest output from the sonarqube_1 container is:
sonarqube_1 | 2016.01.20 17:49:09 INFO web[o.s.s.a.TomcatAccessLog] Web server is started
sonarqube_1 | 2016.01.20 17:49:09 INFO web[o.s.s.a.EmbeddedTomcat] HTTP connector enabled on port 9000
sonarqube_1 | 2016.01.20 17:49:09 INFO app[o.s.p.m.Monitor] Process[web] is up
What does concern me is the latest output from the db_1 container:
PostgreSQL init process complete; ready for start up.
LOG: database system was shut down at 2016-01-20 17:48:40 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
ERROR: relation "schema_migrations" does not exist at character 21
STATEMENT: select version from schema_migrations
ERROR: relation "schema_migrations" does not exist at character 21
STATEMENT: select version from schema_migrations
ERROR: relation "schema_migrations" does not exist at character 21
STATEMENT: select version from schema_migrations
ERROR: relation "schema_info" does not exist at character 15
STATEMENT: SELECT * FROM "schema_info" LIMIT 1
Navigating to http://my.instance.ip:9000 is unsuccessful. I am able to hit the respective ports of other running containers from the same machine.
Could anyone help to point me in the right direction? Any other guides or documentation that may serve me better? I also see issues with the documentation stating that analyzing a project begins with mvn sonar:sonar, but I'll defer that for now. Thank you very much in advance!
Use this image
I modified this image to talk to a RDS instance.
EC2(docker-sonar)<==> RDS postgres