I have some docker-compose.yml file. In this file is defined db service (postgres:9.6):
db:
image: postgres:9.6
volumes:
- ./test/data/postgresql/:/var/lib/postgresql/data:delegated
- ./test/bootstrap/postgres:/docker-entrypoint-initdb.d:delegated
ports:
- 15432:5432
environment:
- POSTGRES_DB=test
- POSTGRES_USER=test
- PGDATA=/var/lib/postgresql/data/testdata
And some sh script located in ./test/bootstrap/postgres. In this script i try to connect to postgres by psql like that: psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" -h localhost -p 5432 (I know that -h localhost -p 5432 is not necessary but i need it).
When my container is up error occured:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
But after this i restart (after first start this script not executed) my db container then inside in container manually run sh script and it executed successfully. It can connect to localhost and port 5432 even if it is first start.
If i connect in script like that psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" there is no connection error.
Can someone explain me this strange behavior? Why i can't connect to localhost:5432 when container is starting but if connect manually from container it connect successfully?
Thanks!
My understanding is that during the initialization of the container the server is listening on Unix-domain socket.
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
The scripts in the docker-entrypoint-init.d directory are executed while the server runs in this mode. You're trying to connect via TCP/IP by specifying -h localhost.
Try to remove that argument to connect to the server using Unix sockets.
try to add a sleep command in your script to give enougth time to your database to start correctly.
According to docs at hub.docker.com/_/postgres, to add a custom script to run when your container starts, create a shell script like the one below and create a custom Dockerfile where you'll copy the shell script into /docker-entrypoint-initdb.d/
Assuming you save the following script as initdatabase.sh in the same folder as your custom Dockerfile also defined below in example, when you build the image, you'll have created your custom image which runs the following sql statements defined below.
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE USER user;
CREATE DATABASE customdatabase;
GRANT ALL PRIVILEGES ON DATABASE bloodapp TO user;
// add other sql statements here
EOSQL
FROM mdillon/postgis:9.6-alpine
COPY initdatabase.sh /docker-entrypoint-initdb.d/
Related
Assume we need to run second postgres instance in our production environment. First of them (postgres-one) already running and has few databases & data in them. Now I want to update my docker-compose.yaml file and add configuration for the second (postgres-two). Moreover I have to grab some databases info from postgres-one and copy it to postgres-two. Here how I'm trying to achieve this:
docker-compose.yaml
postgres-two:
image: postgres:12.5
depends_on:
postgres-one:
condition: service_started
...
ports:
- "5433:5432"
command: bash -c "chmod +x /usr/local/bin/init.sh && /usr/local/bin/init.sh"
volumes:
- ./data/postgres-two/init-db/init.sh:/usr/local/bin/init.sh
init.sh
#!/bin/bash
# allows you to skip the password prompt for pg_dump
echo "postgres-one:5432:dbname1_one:dbuser1_one:dbpass1_one" > ~/.pgpass
echo "postgres-one:5432:dbname2_one:dbuser2_one:dbpass2_one" >> ~/.pgpass
chmod 600 ~/.pgpass
# gets the data from external database & copies it to internal
pg_dump -h postgres-one -U dbuser1_one dbname1_one | psql -h localhost -U dbuser1_two -d dbname1_two
pg_dump -h postgres-one -U dbuser2_one dbname2_one| psql -h localhost -U dbuser2_two -d dbname2_two
But when I run this I get the error:
psql: error: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
I already tried it w/o -h localhost before, it gave me this as I remember:
psql: error: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
If I add find /var/run/postgresql/ -name .s.PGSQL.5432 in the beginning of the init.sh it'll show nothing. So, as I understand I can't proceed further with psql because postgres server is not running at the moment. And of course I can't run it with postgres / pg_ctl commands because they can't be executed by root:
"root" execution of the PostgreSQL server is not permitted.
And of course docker containers run as the root user by default, and if I change the user it also give me errors such as:
chmod: changing permissions of '/usr/local/bin/init.sh': Operation not permitted
Am I doing something wrong? Or maybe I can get the dumps and apply them in other ways.. somehow?
I think server is failing to start because of volume mapping,
You can refer to this yml,
https://github.com/khezen/compose-postgres/blob/master/docker-compose.yml
Nvm, I solved the problem by placing the init.sh script into the docker-entrypoint-initdb.d folder. But then I also had an access issue with pg_dump because for some reason .pgpass didn't work in my case. For those, who will face the same problem, I fixed it with PGPASSWORD env like that:
PGPASSWORD="dbpass1_one" pg_dump -h postgres-one -U dbuser1_one dbname1_one | psql -h localhost -U dbuser1_two -d dbname1_two
I've got a simple Code Build project for a Django App and I want to start a Postgres database in a docker container for testing.
The commands are:
docker run --name django-test-db -d -p 5432:5432 -e POSTGRES_PASSWORD=djangotest -e POSTGRES_USER=configuration-service-master -e POSTGRES_DB=configuration-service postgres:12-alpine
docker exec django-test-db psql -U configuration-service-master -c \"\\dn\"
On my local machine everything works quite fine, but in AWS CodeBuild I'm receiving the error:
psql: error: could not connect to server: could not connect to server:
No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Try two things:
Connect with '-h' flag to specify host so the connection goes through TCP/IP instead of Unix Socket
psql -h localhost -p 5432
Enable privileged mode on Build project:
https://docs.aws.amazon.com/codebuild/latest/userguide/change-project.html#change-project-console
I use the official Postgres image from the Docker Hub
docker pull postgres
I start my container in my machine on local:
docker run --name some-postgres -e POSTGRES_USER=user -e POSTGRES_PASSWORD=password -e POSTGRES_DB=test postgres
The container would have to create my test base with my user on port 542
I have this error when I want to connect on my db
$ sudo docker run -it postgres /bin/bash
root#ef4407c26a96:/# su postgres
postgres#ef4407c26a96:/$ psql
psql: error: could not connect to server: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
postgres#ef4407c26a96:/$
Please provide the database that you want to connect to. As you've provided POSTGRES_DB=test, use it when you wish to connect to psql:
psql -d test -U user
Also, I was able to connect with psql both as root and postgres user.
I'm trying to connect postgresql and pgadmin4 work together.
pgadmin4 works fine but when I try to create a new server I have 2 problems:
if the postgres container is at other port that is not 5432 it dont recognize that port. It show this error: could not connect to server: Connection refused
Is the server running on host "172.17.0.5" and accepting
TCP/IP connections on port 5431?
if the postgres container is at port 5432 the error is FATAL: password authentication failed for user "example".
I execute this command to get postgres container: docker run -p 5431:5432 --name postgres2 -e POSTGRES_PASSWORD=ad1234 -d postgres.
I try, following other responses in stackoverflow, adding this command -c"listen_addresses='*'" and I enter in the config file too but noone of this work to me.
Hope you can help me, thanks.
EDIT [Solved]
Ok I solved, it was a big fail by my part. I was using 172.17.0.5 (the IP container address) and what I need to use to connect is 172.17.01 (the Gateway).
Thanks for you time.
I have reproduce your scenario this way:
# docker run -p 5431:5432 --name postgres2 -e POSTGRES_PASSWORD=ad1234 -d postgres
# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d4030c577a24 postgres "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:5431->5432/tcp postgres2
# sudo -u postgres psql -h localhost -p 5431
could not change directory to "/root": Permission denied
Password:
psql (10.5, server 11.2 (Debian 11.2-1.pgdg90+1))
WARNING: psql major version 10, server major version 11.
Some psql features might not work.
Type "help" for help.
postgres=# CREATE DATABASE mytestdb;
CREATE DATABASE
postgres=# \q
Now starting docker for pgadmin and being able to connect to postgresql:
docker run -p 80:80 --link postgres2 -e "PGADMIN_DEFAULT_EMAIL=user#domain.com" -e "PGADMIN_DEFAULT_PASSWORD=SuperSecret" -d dpage/pgadmin4
With the above command you can link the postgres2 docker to the pgadmin docker and then on creating a connection on pgadmin4 you should use:
host name/address: postgres2
port: 5432
Maintenance database: postgres
username: postgres
with that, I've connected to Postgres from pgadmin4
As far as I know, docker PostgreSQL comes by default with localhost only connection and if you want to add remote connection you should add "listen_addresses = '*'" to postgresql.conf
I used to use a script (/docker-entrypoint-initdb.d/db_init.sh) to loop through database dumps in a folder copied into the postgres container and restoring them. It used to work nicely but recently it stopped working.
I get following error:
postgres_server_1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/db_init.sh
postgres_server_1 | --> db_init.sh: Found /dumps/dataset_1.dump as dataset_1
postgres_server_1 | psql: could not connect to server: Connection refused
postgres_server_1 | Is the server running on host "localhost" (127.0.0.1) and accepting
postgres_server_1 | TCP/IP connections on port 5432?
The db_init.sh script loops through a folder containing database dumps and checks if the database exists already. If not it restores the dump.
/docker-entrypoint-initdb.d/db_init.sh content:
shopt -s nullglob
for i in /dumps/*.dump;
do
db_name=${i##*/}
db_name=${db_name%.dump}
echo "--> db_init.sh: Found $i as $db_name"
if psql -U postgres -h localhost -lqt | cut -d \| -f 1 | grep -qw ${db_name}; then
echo "--> db_init.sh: Database ${db_name} already exists."
else
echo "--> db_init.sh: Setting up database: ${db_name}"
createdb -U postgres -h localhost-T template0 ${db_name}
psql -U postgres -h localhost ${db_name} < ${i}
fi
done
echo "--> db_init.sh: Setup complete."
I am using docker-compose to start the postgres container (and some others).
The docker-compose.yml content:
version: '3'
services:
postgres_server:
image: postgres
volumes:
- /etc/localtime:/etc/localtime:ro
- ./data/dumps:/dumps:ro
- ./scripts/db_init.sh:/docker-entrypoint-initdb.d/db_init.sh
environment:
- TZ=Europe/Berlin
restart: always
volumes:
postgres_data:
driver: local
Now what I don't understand is why there seems to be a connection error usually associated with trying to connect to a postgres database from a different machine or container. But the script itself is running in the postgres container and a volume connects the directory containing the dumps into the container.
Running the psql command from within the container using docker exec -it container_name bash works fine and the dumps are there. Why do the psql commands work when executing them manually from within the container but not when executed via /docker-entrypoint-initdb.d/db_init.sh?
It looks like this commit has broken your script.
Explanation:
PostgreSQL may accept connections not only via TCP/IP, but also via UNIX socket. The -h localhost argument tells psql to use TCP connections rather than UNIX socket.
If you look into the current docker-entrypoint.sh version, you will see, that during the execution of scripts in /docker-entrypoint-initdb.d PostgreSQL listens only on the UNIX socket, and the startup log says:
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
This means that psql -h localhost will not connect to the database, as PostgreSQL does not listen on IP socket. You must use psql without -h localhost option to make it use UNIX socket instead of TCP connections.
But why running psql -h localhost manually works?
If you look into the docker-entrypoint.sh again, you will see that when all init scripts are executed, PostgreSQL is being stopped and then started again in normal (operational) mode, in which it listens both on UNIX and IP sockets:
LOG: listening on IPv4 address "0.0.0.0", port 5432
LOG: listening on IPv6 address "::", port 5432
LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
So, when the startup process is complete, you may connect to PostgreSQL using TCP connections, thus, entering into the container and running psql -h localhost succeeds.