Initialize PostgreSQL Container with docker-entrypoint-initdb.d script - postgresql

I am trying to create a PostgreSQL 11.5 docker container. In doing so, I want to run a SQL script that creates the necessary users, tables, etc. However, whenever the container starts I see the following error:
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
****************************************************
WARNING: No password has been set for the database.
This will allow anyone with access to the
Postgres port to access your database. In
Docker's default configuration, this is
effectively any other container on the same
system.
Use "-e POSTGRES_PASSWORD=password" to set
it in "docker run".
****************************************************
waiting for server to start....2019-09-16 17:16:26.568 UTC [42] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-09-16 17:16:26.677 UTC [43] LOG: database system was shut down at 2019-09-16 17:16:25 UTC
2019-09-16 17:16:26.691 UTC [42] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
/docker-entrypoint-initdb.d/init.sql: Permission denied
My Dockerfile looks like this:
FROM postgres:11.5
ADD ./scripts/init.sql /docker-entrypoint-initdb.d/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 5432
CMD ["postgres"]
And, my init.sql file looks like this:
CREATE USER mydb WITH PASSWORD 'password';
CREATE DATABASE mydb;
GRANT ALL PRIVILEGES ON DATABASE mydb TO mydb;
You'll notice neither of them does anything terribly complicated. However, I'm still getting the permission denied error. I've connected to the running container and confirmed that the init.sql file is in place on the filesystem. Any idea what I could be doing wrong here?

So from this Dockerfile I assume the user is postgress.
Try with this Dockerfile
FROM postgres:11.5
USER postgres
RUN whoami
ADD ./scripts/init.sql /docker-entrypoint-initdb.d/
ENTRYPOINT ["docker-entrypoint.sh"]
EXPOSE 5432
CMD ["postgres"]
update:
Seems like the file not owned by Postgres user.
Try to set permission
ADD ./scripts/init.sql /docker-entrypoint-initdb.d/
RUN chown postgres:postgres /docker-entrypoint-initdb.d/init.sql

Initialize Postgres container with Data
Create a docker-compose.yml
version: '2'
services:
postgress-postgresql:
image: postgres:11.3
volumes:
# - ~/volumes/jhipster/postgress/postgresql/:/var/lib/postgresql/data/
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
environment:
- POSTGRES_USER=postgress
- POSTGRES_PASSWORD=
ports:
- 5432:5432
Create a init.sql with the script
CREATE USER platops WITH PASSWORD 'platops';
CREATE DATABASE platopsdb;
GRANT ALL PRIVILEGES ON DATABASE platopsdb TO platops;
RUN with docker-compose up -d

I had the same issue, mounted a .sh file via docker volumes. I checked permissions via ls -lah, in my case it was just -rw-r-----.
Using chmod 644 filename solved my issue.

The underlaying problem in our case was that the sql script was stored on a ntfs partition mounted with ntfs-3g which by default has got permissions' functionality disabled (https://superuser.com/questions/451475/chmod-doesnt-work). Running it on a normal ext4 partition solved the problem.

Disclaimer: I am aware it is not an answer to the question but it might shed some light why for some people it works and for others it does not.
Inside our team, it's working perfectly for the guy who's username is "admin" (literally). It does not work for me nor for out deployment server. Where our usernames are different.
Using "sudo" did not have any impact. It did not break his it did not fix ours.
My machine and his are MacOS. The server is Ubuntu.

Related

Docker runs PostgreSQL in "trust" mode

I am currently learning docker and trying to run a docker container with the PostgreSQL database. I managed that once, and everything seemed to work fine. After some time, I tried to run another docker container with almost identical settings, however, it didn't go as expected. My problem is that now, whenever I try to run PostgreSQL container, initdb initializes the database in "trust" mode and accepts any connections without the password.
So far, I've tried running the command from the console:
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -p 32000:5432 -d postgres:14.5-alpine
As well as running the docker-compose.yaml:
services:
db:
container_name: Test_container
image: postgres:14.5-alpine
restart: unless-stopped
ports:
- "32000:5432"
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: mysecretpassword
Additionally, I tried ordering tags differently, different images, and different values, cleaning docker: removing all containers, images, and volumes, and even reinstalling docker, however, whenever I inspect logs of a newly created container, I get:
sh: locale: not found
2022-08-16 09:35:50.709 UTC [30] WARNING: no usable system locales were found
performing post-bootstrap initialization ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
syncing data to disk ... ok
One of my assumptions was that docker, for some reason, doesn't see the password I am specifying and thus starts the database in "trust" mode, however, if I add
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test_db
to the docker-compose.yaml, test_db database is being created.
I'd appreciate any suggestions on how to make docker run PostgreSQL containers not in a "trust" mode as it should by default if the password is specified.
Juan González pointed out:
Note 1: The PostgreSQL image sets up trust authentication locally so you may notice a password is not required when connecting from localhost (inside the same container). However, a password will be required if connecting from a different host/container.
So, according to the docs, I updated my docker-compose.yaml file:
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test_db
POSTGRES_HOST_AUTH_METHOD: scram-sha-256
POSTGRES_INITDB_ARGS: --auth-host=scram-sha-256
and once again tried swapping order and\or removing POSTGRES_INITDB_ARGS, but database still runs in "trust" mode.
As stated in Postgres' DockerHub documentation:
Note 1: The PostgreSQL image sets up trust authentication locally so you may notice a password is not required when connecting from localhost (inside the same container). However, a password will be required if connecting from a different host/container.
However, if you don't want trust mode even in local connections, you can set the POSTGRES_HOST_AUTH_METHOD environment variable to override this behavior. More info at the documentation mentioned above.
As #jjanes pointed out in the comment to my question, the solution is to add POSTGRES_INITDB_ARGS: --auth=scram-sha-256 which would set both local and host types of connections.

initdb in Postgres.app does not create postgres &template1 database

I installed postgres.app 2.4.2 on macos(big sur 11.4).
I wanted to initdb with locale=C, so run the command initdb -D "/Users/xxx/Library/Application Support/Postgres/var-131" -U postgres --encoding=UTF-8 --locale=C --auth-local=trust and the logging
(base) > $ initdb -D "/Users/xxx/Library/Application Support/Postgres/var-13_1" -U postgres --encoding=UTF-8 --locale=C --auth-local=trust
The files belonging to this database system will be owned by user "yy".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /Users/xxx/Library/Application Support/Postgres/var-13_1 ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Hongkong
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
pg_ctl -D '/Users/xxx/Library/Application Support/Postgres/var-13_1' -l logfile start
In PG document(https://www.postgresql.org/docs/13/app-initdb.html), it said:
generating the shared catalog tables (tables that belong to
the whole cluster rather than to any particular database),
and creating the template1 and postgres databases
It was wired that there was no postgres and template1 database by default.
I tried to login using psql and it displayed role "username" does not exist.
I also tried createuser -U postgres -s $USER and it showed the error
(base) > $ createuser -U postgres -s $USER
createuser: error: could not connect to database template1: FATAL: Peer authentication failed for user "postgres"
I was very confused, and how can I got to solved the question or should I switch to homebrew version?

PostgreSQL not creating tables when running on docker

I am trying to run PostgreSQL on a linux container docker in my Windows server, but when I run it, it creates database, but there is no table and no data in it, while it should create all tables and add datas to it using that Actibook_latest.sql
Here's the code of Dockerfile
# Dockerfile
FROM postgres:9.4
RUN mkdir -p /tmp/psql_data/
COPY Actibook_latest.sql /tmp/psql_data/
COPY init_docker_postgres.sh /docker-entrypoint-initdb.d/
EXPOSE 5432
Here's the code of init_docker_postgres.sh
#!/bin/bash
# this script is run when the docker container is built
# it imports the base database structure and create the database for the tests
DATABASE_NAME="postgres"
DB_DUMP_LOCATION="/tmp/psql_data/Actibook_latest.sql"
echo "*** CREATING DATABASE ***"
psql "$DATABASE_NAME" < "$DB_DUMP_LOCATION";
echo "*** DATABASE CREATED! ***"
And here's the code of docker-compose
version: '2'
services:
db:
build: '.\Main Database Backup'
environment:
POSTGRES_DB: ${DB_POSTGRES_APP_DATABASE}
POSTGRES_USER: ${DB_POSTGRES_APP_USER}
POSTGRES_PASSWORD: ${DB_POSTGRES_APP_PASSW}
PGDATA: /var/lib/postgresql/data/pgdata
ports:
- "5432:5432"
restart: unless-stopped
Main Database Backup is the folder that contains that Dockerfile and init_docker_postgres.sh. Also that Actibook_latest.sql contains sql to create the tables, data, etc.
And when I run docker-compose up while other serives go up and running, here's what it shows to logs:
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data/pgdata ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /var/lib/postgresql/data/pgdata/base/1 ... ok
initializing pg_authid ... ok
setting password ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
postgres -D /var/lib/postgresql/data/pgdata
or
pg_ctl -D /var/lib/postgresql/data/pgdata -l logfile start
waiting for server to start....LOG: database system was shut down at 2018-11-14 16:18:07 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
done
server started
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init_docker_postgres.sh
/usr/local/bin/docker-entrypoint.sh: /docker-entrypoint-initdb.d/init_docker_postgres.sh: /bin/bash^M: bad interpreter: No such file or directory
LOG: database system was interrupted; last known up at 2018-11-14 16:18:16 UTC
LOG: database system was not properly shut down; automatic recovery in progress
LOG: record with zero length at 0/16A4780
LOG: redo is not required
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
I'm thinking that there might be a firewall issue since its running perfectly in other servers that I have, the problem appears only in this one. So is that any chance that postgresql in this machine is preventing it?
Update
I tried to restart docker and then used these commands:
docker rm $(docker ps -a -q)
docker rmi $(docker images -a -q)
docker volume rm $(docker volume ls -q)
now its showing this error:
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init_docker_postgres.sh
*** CREATING DATABASE ***
: No such file or directory/init_docker_postgres.sh: line 4: /tmp/psql_data/Actibook_latest.sql
/docker-entrypoint-initdb.d/init_docker_postgres.sh: line 4: $'\r': command not found
*** DATABASE CREATED! ***
and when I go to docker to check for file:
docker exec -it db bin
cd /tmp/psql_data
ls
it shows that there exists Actibook_latest.sql
It seems that your entrypoint init script was not working, make sure data_volume is empty before run your container.
This is because in docker-entrypoint.sh, at line 57, it checks your data volume, if it exists, it won't execute your init scrip.

Cant connect to PostgreSQL database

I have problem connecting to database. I recive message that role "Darko" does not exists. This is the sequence of my commands:
/usr/local/pgsql/bin/pg_ctl start -D /usr/local/pgsql/data -l logfile
Server starting /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data The files belonging to this database system will be owned by user "darko".
This user must also own the server process.
The database cluster will be initialized with locales COLLATE: en_US.UTF-8 TYPE: en_US.UTF-8 MESSAGES: en_US.UTF-8 MONETARY: hr_HR.UTF-8 NUMERIC: hr_HR.UTF-8 TIME: en_US.UTF-8
The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english".
Data page checksums are disabled. initdb: directory "/usr/local/pgsql/data" exists but is not empty
If you want to create a new database system, either remove or empty the directory "/usr/local/pgsql/data" or run initdb with an argument other than "/usr/local/pgsql/data". /usr/local/pgsql/bin/psql psql:
FATAL: role "darko" does not exist
You're trying to initialize Postgres cluster (main directory where all your Postgres data and logs will be stored) in "/usr/local/pgsql/data" directory.
Create another directory, anywhere and pass full path to it to -D option of initdb.
It will initialize your Postgres cluster.
During this process, a default database role will be created. If you run initdb under your regular OS user ("darko") -- it will create database user with the same name. But usually people run posgres tools (including initdb) under a separate OS user, "postgres" -- and if you will do it (like "sudo -u postgres initdb -D ..."), initdb will create database role "postgres" instead. (BTW, in Postgres terminology, database user and database role are the same things).
Then, once initbd has successfully created your Postgres cluster, you can check it using "ls" -- directory must contain subdirectories such as "base", "global", "pg_xlog", config files, etc.
Then you need to run Postgres, it's done using pg_ctl command, and you need to pass path to your cluster, again with -D option:
pg_ctl -D /path/to/cluster
If it runs successfully, open another terminal tab/window and try to connect using psql and corresponding database role (either "darko" or "postgres", based on your previous decision):
psql -U darko template1
"template1" is a database name which is always present, it serves as a template for all future databases you'll create.
Also, before you try to start Postgres with pg_ctl, it's worth to check if you have already some Postgres running:
ps ax | grep postgres
-- if you have another Postgres running, you can shut it down using "pg_ctl stop" with corresponding -D option pointing to the proper cluster directory.
Example (Ubuntu):
$ ps ax | grep postgres | grep D
21996 ? S 23:37 /usr/lib/postgresql/9.6/bin/postgres -D /var/lib/postgresql/9.6/main -c config_file=/etc/postgresql/9.6/main/postgresql.conf
To stop it gracefully, just run:
/usr/lib/postgresql/9.6/bin/pg_ctl -D /var/lib/postgresql/9.6/main stop
initdb is a tool used to create a cluster (see PostgreSQL documentation for cluster definition in this context). It seems your allready did that as your directory isn't empty. Don't do it anymore.
Then, you're trying to connect with psql. By default, it's trying a socket connection with the same name for database user as your OS user. As you didn't create a role named 'darko', it fails.
What you have to do is trying to connect as postgres user. Try this to connect:
sudo -u postgres psql
You should be abble to connect with that line (assuming you didn't change the pg_hba.conf file) and then you should be abble to create your users and databases.
Here's the documentation page for psql.

Creating a database and connection with client for PostgreSQL on OSX

How can I start a PostgreSQL database and connect with a client in terminal?
I've tried this after installing postgreSQL with brew.
which psql
Results in this:
/usr/local/bin/psql
Then I try to start database with:
pg_ctl init -D /usr/local/bin/psql
Which gives:
The files belonging to this database system will be owned by user "matt".
This user must also own the server process.
The database cluster will be initialized with locale "en_AU.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
initdb: could not access directory "/usr/local/bin/psql": Not a directory
pg_ctl: database system initialization failed
You must create a data directory first and input its name after -D. Say if the data directory you created is /usr/local/bin/psql, then your init command will be
pg_ctl init -D /usr/local/bin/psql/data
From the PostgreSQL pg_ctl documentation:
-D datadir
--pgdata datadir
Specifies the file system location of the database configuration files. If this is omitted, the environment variable PGDATA is used.
You are specifying your pgsql executable as your data directory.