I've created a postgreSQL db structure locally, I want this structure to be the table structure of a postgreSQL instance that I will be running from a docker instance (without manually restoring the dump). I stored the structure in a dump and would normally restore this using psql myDB < myDB_dump.sql.
I got a working PostgreSQL instance running on Docker - with myDB and a user that can access the db.
docker-compose.yml:
version: '3'
services:
db:
image: postgres
restart: unless-stopped
ports:
- ${DB_PORT}:${DB_PORT}
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USER}
POSTGRES_DB: ${DB_DATABASE}
Dockerfile:
# psql
RUN apt-get update && apt-get install -y postgresql-client
FROM postgres
ADD init.sql /docker-entrypoint-initdb.d
RUN ./scripts/init_db.sh
init.sql:
CREATE USER my_user WITH PASSWORD my_user;
CREATE DATABASE myDB;
GRANT ALL PRIVILEGES ON DATABASE myDB TO my_user;
Is the best way to implement an existing table structure into this docker instance through a dump file, if so: how? Is there a better way to copy a db structure? If so, what would be the best way?
Related
This question already has answers here:
How to create User/Database in script for Docker Postgres
(9 answers)
Closed 8 months ago.
I have a very complex environment, there are two services which named as A service and B service depend on service postgresql, but A and B use difference super user. so once I start the service postgresql, I need create another superuser for service B.
This is a part of docker compose file :
postgres:
image: postgres:13.4
container_name: postgresql
hostname: postgresql
volumes:
- 'postgres_data:/var/lib/postgresql/data'
environment:
POSTGRES_DB: keycloak
POSTGRES_USER: keycloak
POSTGRES_PASSWORD: password
command: ["CREATE ROLE postgres LOGIN SUPERUSER; | ALTER USER postgres CREATEDB CREATEROLE LOGIN INHERIT REPLICATION BYPASSRLS;"]
enter image description here
In the environments, I already setup the default user, password and DB in service postgresql for service A, BUT I’d like to create another super user for service B once service postgresql started, does there any one know how to write the commands. many thanks.
I think this grammer is wrong: command: ["CREATE ROLE postgres LOGIN SUPERUSER; | ALTER USER postgres CREATEDB CREATEROLE LOGIN INHERIT REPLICATION BYPASSRLS;"]
The issue with running commands on container startup is that you need the database to be up to be able to run the commands. By overriding the command: on the container, you replace the normal command which is to start Postgres. So with a new command, Postgres will never start.
You can run the command from a different container once Postgres is started. The command-runner container uses psql to connect to the database and run the command. It also uses the postgres docker image, but because it overrides the command on the container, this container doesn't start a database. It only runs the command.
Here's what I've come up with.
version: "3"
services:
postgres:
image: postgres
environment:
- POSTGRES_USER=keycloak
- POSTGRES_PASSWORD=password
- POSTGRES_DB=keycloak
command-runner:
image: postgres
command: /bin/sh -c 'sleep 10 && PGPASSWORD=password psql -U keycloak -h postgres -d keycloak -c "CREATE ROLE postgres LOGIN SUPERUSER; ALTER USER postgres CREATEDB CREATEROLE LOGIN INHERIT REPLICATION BYPASSRLS;"'
depends_on:
- postgres
I had to put in a sleep 10 command in the command-runner container for it to wait for Postgres to be ready to accept connections.
I also removed the | you had in your command before ALTER USER. I got a syntax error on it.
I have a docker image that's not accepting credentials for a user that is defined in the yaml docker-compose file. When I go to the docker console for the container and check users it only lists postgres. Not sure what I am missing - here's the yaml file:
version: '3.8'
services:
db:
container_name: drewreport_container
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: mpassword
POSTGRES_USER: thedrewreport
POSTGRES_DB: thedrewreportdb
ports:
- "5432:5432"
volumes:
- thedrewreportdata:/var/lib/postgresql/data/
volumes:
thedrewreportdata:
Any ideas?
I can't reproduce your problem. Running docker-compose up, I see:
Creating network "docker_default" with the default driver
Creating volume "docker_thedrewreportdata" with default driver
Creating docker_client_1 ...
Creating docker_db_1 ...
Creating docker_client_1 ... done
Creating docker_db_1 ... done
Attaching to docker_db_1, docker_client_1
[...]
db_1 | 2021-07-19 23:03:39.676 UTC [1] LOG: database system is ready to accept connections
If I then connect with psql, I can authenticate using the username
and password you've defined in your docker-compose.yml:
# psql -h localhost -U thedrewreport thedrewreportdb
Password for user thedrewreport:
psql (13.3 (Debian 13.3-1.pgdg100+1))
Type "help" for help.
thedrewreportdb=# \du
List of roles
Role name | Attributes | Member of
---------------+------------------------------------------------------------+-----------
thedrewreport | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
thedrewreportdb=#
Note that any volumes specified in your docker-compose.yml will
persist between a docker-compose down and a docker-compose up, so
if you ever brought your stack up with different credentials, those
will never be replaced unless your explicitly destroy the volume by
running docker-compose down -v.
You can tell that docker-compose is re-using a volume if you don't
see a message like this when you run docker-compose up:
Creating volume "docker_thedrewreportdata" with default driver
If existing, the existing DB data you mount to /var/lib/postgresql/data/ will take precedence over the environment variables to initialize it.
You have 2 options:
Update the existing DB data to add your user / password / database. To do so you can use docker compose exec db bash and then connect using psql command to make your changes.
Delete or move your existing thedrewreportdata local volume, for instance updating it to ./thedrewreportdata_postgres:/var/lib/postgresql/data/
Once done, you can use docker compose exec db psql --username thedrewreport --dbname thedrewreportdb to doublecheck you can connect with your credentials to the updated DB.
I'm trying to initialize a database without using the entry point directory.
Here is a minimal Dockerfile:
FROM postgres:latest
POSTGRES_DB db
POSTGRES_USER user
POSTGRES_PASSWORD password
ADD db.sql /directory/
ADD script.sh /directory/
CMD ["sh", "/directory/script.sh"]
# Or ENTRYPOINT ["/directory/script.sh"]?
And script.sh:
psql -d db -U user < /directory/db.sql
This does not work because postgres isn't up when the script is run.
How can I run db.sql without using /docker-entrypoint-initdb.d?
If you look at the standard postgres image's entrypoint script the mechanism to support the /docker-entrypoint-initdb.d directory is pretty intricate: it needs to bootstrap the database directory and initial user and database, then it starts the database server in the background and runs everything in that directory, and finally runs the database for real. If you're trying to replicate this setup, you have to do all of these steps yourself.
There are other ways to set up a database, though. You can create an empty database and then run your application's migrations normally to create the initial schema. If you have an SQL file that normally you'd run against a database running the psql client tool, you can do the exact same thing with Docker
docker run -d -p 5432:5432 --name postgres postgres:12
psql -h localhost < db.sql
I am not able to set Password for Postgres using Docker-compose. Postgres is loading without password and with the default user name "postgres", non of the environment variables below seems to applied. below is the db service of my docker-compose.yml file: (version 3)
db:
image: postgres
container_name: postgres
environment:
POSTGRES_USER: user
POSTGRES_PASSWORD: pass
POSTGRES_DB: db
restart: unless-stopped
volumes:
- ./postgres-data:/var/lib/postgresql/data
ports:
- "5432:5432"
Note I tried using the "-POSTGRES_USER=" as well, it didn't work
Also, I deleted all old containers/volumes.
Any idea?
The problem should be with the volume attached. When your container start it will add the credentials you give him, but then the volume will be attached and that will cause this information being rewritten.
For more information have a look at https://github.com/docker-library/postgres/issues/203#issuecomment-255200501.
The main reason being use of ':' instead of "=" in the environment section.
Ideally it should look like this:
db:
image: postgres
container_name: postgres
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD= pass
- POSTGRES_DB= db
restart: unless-stopped
volumes:
- ./postgres-data:/var/lib/postgresql/data
ports:
- "5432:5432"
Your configuration works fine for me. I suspect you are not using the complete set of correct credentials, which includes the username, password, and database name. If I take your example docker-compose.yaml and run it without modifications, I can connect to the database db like this with username user and password pass:
$ psql -h localhost -U user db
Password for user user:
psql (9.5.7, server 9.6.1)
WARNING: psql major version 9.5, server major version 9.6.
Some psql features might not work.
Type "help" for help.
db=#
Had the same issue.
Couldn't solve it for 2 weeks.
Read almost everything related to it.
And after I finished all PosgreSQL server related processes on local machine, everything goes well.
Start postgres instance:-
docker run --name postgres-0 -e POSTGRES_PASSWORD=mypassword -p 5433:5433 -d postgres
Now we can check docker all running container by this command:-
docker ps
I have a postgres:9.5.6-alpine container, and another container, named web, which has to be linked to it.
I want to run a script named create_db.sh in postgres container after it has been started and docker-entrypoint.sh has been executed, in order to create a db and a user and restore a backup.
My docker-compose.yml (postgres part):
postgres:
build: ./postgres
container_name: postgres
volumes:
- /shared_folder/postgresql:/var/lib/postgresql
ports:
- "5432:5432"
command: sh /home/create_db.sh
The content of create_db.sh is:
#!/bin/sh
psql -d template1 -U postgres
psql --command "CREATE USER user WITH PASSWORD 'userpassword';"
psql --command "CREATE DATABASE userdb;"
psql --command "GRANT ALL PRIVILEGES ON DATABASE userdb to user;"
psql --command "\q;"
psql -U user -d userdb -f /var/lib/postgresql/backup.sql
exit
When i run docker-compose build and then docker-compose up i get this:
Attaching to postgres, web
postgres | psql: could not connect to server: No such file or directory
postgres | Is the server running locally and accepting
postgres | connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I've understood this is because when I launch create_db.sh postgres server is not ready, but so how can i run it after the docker-entrypoint.sh of the container?
You are overriding the original command and you do not start postgres in this script which is why your database is not available.
You can put your database initialization into the container's entrypoint directory: /docker-entrypoint-initdb.d. This executes all *.sh and *.sql files in this directory and does not touch the original command.
All files in this directory are automatically executed in the alphabetical order on container creation. Therefore, create a volume to add your scripts / sql files to the entrypoint and let the container execute them. This is described in the official postgres documentation, section "How to extend this image".
Your compose file then changes to something like this:
postgres:
build: ./postgres
volumes:
- /shared_folder/postgresql:/var/lib/postgresql
- ./db-init-scripts:/docker-entrypoint-initdb.d
ports:
- "5432:5432"
whereas a local directory, e.g. db-init-scripts, contains your initialization scripts (rename it if you want). Copy create_db.sh to this folder and it will be automatically executed when you create a new container.
Several database-images watch this entrypoint-directory, which is very convenient.
Your container_name: postgres seems redundant.
If you are getting an error /bin/sh: bad interpreter: Permission denied change execution permission to your script file first, since Docker copies over permissions:
chmod +x your/script.sh