I'm trying to set postgresql db setup (version - 10) via ansible and when i was trying to start postgresql db cluster after recovery.conf file i'm getting permission denied error
i'm trying this via anasible and below is the module information
- name: Starting the postgresql db cluster on standby host
command: 'su - postgres -c "pg_ctl -D {{ data_dir }} start"'
when: inventory_hostname == (groups['pgdb']|sort())[1]
Error message:
2021-06-19 13:46:56.129 UTC [90381] DETAIL: The failed archive command was: cp pg_wal/000000010000000000000001 /data/archives/testarchives/000000010000000000000001 && /bin/rsync -a pg_wal/000000010000000000000001 postgres#10.0.1.120:/data/archives/testarchives/000000010000000000000001
2021-06-19 13:46:56.130 UTC [77934] LOG: archiver process (PID 90381) exited with exit code 1
sh: /bin/rsync: Permission denied
2021-06-19 13:47:56.227 UTC [90385] FATAL: archive command failed with exit code 126
It is clearly stating it is not able to execute rsync command as postgres user.But don't know how to fix this
Tried below first but no luck
- name: Starting the postgresql db cluster on standby host
command: "pg_ctl -D {{ data_dir }} start"
when: inventory_hostname == (groups['pgdb']|sort())[1]
become_user: postgres
My recovery.conf file is as eblow
$ cat recovery.conf
standby_mode = 'on'
primary_conninfo = 'user=postgres host=10.0.1.120 port=5432'
restore_command = 'cp -i /data/archives/testarchives/%f %p'
trigger_file = '/tmp/testtrigger_file'
recovery_target_timeline = 'latest'
archive_cleanup_command = 'pg_archivecleanup -d /data/archives/kongarchives %r 2>>/data/archives/cleanup.log'
But if i manually execute the same command then it is working
[ec2-user#ip-10-0-2-98 ~]$ sudo su - postgres
Last login: Sat Jun 19 15:31:25 UTC 2021 on pts/0
[postgres#ip-10-0-2-98 ~]$ cd /data/dbdata/testdata/
[postgres#ip-10-0-2-98]$ cp pg_wal/000000010000000000000004 /data/archives/testarchives/000000010000000000000004 && /bin/rsync -a pg_wal/000000010000000000000004 postgres#10.0.1.120:/data/archives/testarchives/000000010000000000000004
[postgres#ip-10-0-2-98]$
You are probably a victim of SELinux or something like it, in which the same system OS account can have different permissions depending on whether it is running as a daemon or from a command line. Try running below command after this my issue was fixed,
sudo setenforce 0
refer:https://wiki.gentoo.org/wiki/SELinux/Tutorials/Permissive_versus_enforcing
Can you try below.
- name: Starting the postgresql db cluster on standby host
command: 'sudo -u postgres psql -c "pg_ctl -D {{ data_dir }} start"'
when: inventory_hostname == (groups['pgdb']|sort())[1]
Related
I want to change the start command of postgres to support SSL in the default docker image
db.Docerfile
FROM postgres:14.5-alpine
COPY ./.docker/dev/init-database.sh /docker-entrypoint-initdb.d/
COPY ./.docker/dev/migrations/database_schema.tar ./
COPY ./.docker/dev/certs/out/server.key /var/lib/postgresql
COPY ./.docker/dev/certs/out/server.crt /var/lib/postgresql
COPY ./.docker/dev/certs/out/myCA.crt /var/lib/postgresql
COPY ./.docker/dev/certs/out/myCA.crl /var/lib/postgresql
COPY ./.docker/dev/certs/out/news_user.key ./
COPY ./.docker/dev/certs/out/news_user.crt ./
RUN chown 0:70 /var/lib/postgresql/server.key && chmod 640 /var/lib/postgresql/server.key
RUN chown 0:70 /var/lib/postgresql/server.crt && chmod 640 /var/lib/postgresql/server.crt
RUN chown 0:70 /var/lib/postgresql/myCA.crt && chmod 640 /var/lib/postgresql/myCA.crt
RUN chown 0:70 /var/lib/postgresql/myCA.crl && chmod 640 /var/lib/postgresql/myCA.crl
RUN chown 0:70 ./news_user.key && chmod 640 ./news_user.key
RUN chown 0:70 ./news_user.crt && chmod 640 ./news_user.crt
RUN chown postgres:postgres /docker-entrypoint-initdb.d/init-database.sh
EXPOSE 5432
CMD postgres -c ssl=on -c ssl_cert_file=/var/lib/postgresql/server.crt -c ssl_key_file=/var/lib/postgresql/server.key -c ssl_ca_file=/var/lib/postgresql/myCA.crt -c ssl_crl_file=/var/lib/postgresql/myCA.crl
When I run this image, I get an error saying
"root" execution of the PostgreSQL server is not permitted.
news_database | The server must be started under an unprivileged user ID to prevent
news_database | possible system security compromise. See the documentation for
news_database | more information on how to properly start the server.
The default postgres image comes with a user named postgres so I tried adding a user postgres line before CMD
Now it gives me a new error saying
postgres: could not access the server configuration file "/var/lib/postgresql/data/postgresql.conf": No such file or directory
Can someone kindly tell me how to fix this so that the postgres command works?
UPDATE 1
I am not running the dockerfile directly, I run it via a docker-compose.yml file since I have a python script that accesses the database in another container
docker.compose.yml
version: "3.8"
services:
news_database:
build:
context: ../..
dockerfile: ./.docker/dev/db.Dockerfile
container_name: news_database
restart: unless-stopped
env_file:
- .env
ports:
- "5432:5432"
volumes:
- news_db:/var/lib/postgresql/data
# https://stackoverflow.com/a/55835081/5371505
# https://stackoverflow.com/questions/72213661/test-connection-to-postgres-with-ssl-with-the-command-line
healthcheck:
test: ["CMD-SHELL", "pg_isready -d 'hostaddr=$DATABASE_HOST user=$DATABASE_USER port=$DATABASE_PORT dbname=$DATABASE_NAME'"]
interval: 5s
timeout: 5s
retries: 5
# Dont add a 'restart' policy to the app because we run it as a cronjob regardless of whether it succeeds or fails
news_app:
# https://stackoverflow.com/questions/65594752/docker-compose-how-to-reference-files-in-other-directories
build:
context: ../..
dockerfile: ./.docker/dev/app.Dockerfile
env_file:
- .env
image: news_app
container_name: news_app
depends_on:
news_database:
condition: service_healthy
volumes:
news_db:
driver: local
I run the above file with this command
docker compose -f ".docker/dev/docker-compose.yml" up -d --build news_database && docker compose -f ".docker/dev/docker-compose.yml" logs --follow
I see a couple of problems with your Dockerfile. First, I've stripped out a few bits to reduce the complexity to something I can reproduce locally, so I have:
FROM postgres:14.5-alpine
COPY ./certs/out/server.key /var/lib/postgresql
COPY ./certs/out/server.crt /var/lib/postgresql
RUN chown 0:70 /var/lib/postgresql/server.key && chmod 640 /var/lib/postgresql/server.key
RUN chown 0:70 /var/lib/postgresql/server.crt && chmod 640 /var/lib/postgresql/server.crt
EXPOSE 5432
CMD postgres -c ssl=on -c ssl_cert_file=/var/lib/postgresql/server.crt -c ssl_key_file=/var/lib/postgresql/server.key -c ssl_ca_file=/var/lib/postgresql/myCA.crt -c ssl_crl_file=/var/lib/postgresql/myCA.crl
If we build a pgtest image from this Dockerfile and run it, we see:
$ docker build -t pgtest .
...
$ docker run --rm pgtest
"root" execution of the PostgreSQL server is not permitted.
The server must be started under an unprivileged user ID to prevent
possible system security compromise. See the documentation for
more information on how to properly start the server.
What's going on here? It turns out this is caused by how you've written your CMD directive. You need to use the "exec style", which means passing in a JSON array rather than a string:
CMD ["postgres", \
"-c", "ssl=on", \
"-c", "ssl_cert_file=/var/lib/postgresql/server.crt", \
"-c", "ssl_key_file=/var/lib/postgresql/server.key"]
This is necessary because there's a check in the entrypoint script for
the image that looks like this:
if [ "$1" = 'postgres' ] && ! _pg_want_help "$#"; then
When you write:
CMD postgres -c ...
This wraps the command in a call to sh -c, so $1 will be sh,
thus skipping the important database initialization logic. When we use
the exec format of the CMD statement, $1 will be postgres, as
expected.
With that fix, the image runs correctly:
$ docker run --rm -e POSTGRES_PASSWORD=secret pgtest
.
.
.
PostgreSQL init process complete; ready for start up.
2022-08-28 12:48:48.784 UTC [1] LOG: starting PostgreSQL 14.5 on x86_64-pc-linux-musl, compiled by gcc (Alpine 11.2.1_git20220219) 11.2.1 20220219, 64-bit
2022-08-28 12:48:48.784 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2022-08-28 12:48:48.784 UTC [1] LOG: listening on IPv6 address "::", port 5432
2022-08-28 12:48:48.786 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-08-28 12:48:48.789 UTC [48] LOG: database system was shut down at 2022-08-28 12:48:48 UTC
2022-08-28 12:48:48.792 UTC [1] LOG: database system is ready to accept connections
I followed the steps provided in https://www.health-samurai.io/fhirbase website, using the Fhirbase in Docker and Download Fhirbase both are working perfectly without any issue.
Now, I want to put Fhirbase code (https://github.com/fhirbase/fhirbase?_ga=2.137200179.2086351274.1637554537-194486469.1637215452) into my private gitlab repo and I want run my gitlab fhirbase code In Docker using Dockerfile.
I have created a Dockerfile as below --
FROM postgres:10.5
WORKDIR /fhirbase
COPY demo/bundle.ndjson.gzip .
RUN apt-get update
RUN apt-get install -y git
RUN git clone https://gitlab.com/username/fhirbase.git /usr/bin/fhirbase
CMD cd /usr/bin/fhirbase
RUN ["chmod", "+x", "/usr/bin/fhirbase"]
CMD make
CMD hash -r
CMD make install
RUN mkdir /pgdata && chown postgres:postgres /pgdata
USER postgres
RUN PGDATA=/pgdata /docker-entrypoint.sh postgres & \
until psql -U postgres -c '\q'; do \
>&2 echo "Postgres is starting up in first..."; \
sleep 5; \
done && \
psql -U postgres -c 'create database fhirbase;' && \
fhirbase -d fhirbase init && \
fhirbase -d fhirbase load --mode=insert ./bundle.ndjson.gzip; \
pg_ctl -D /pgdata stop
#EXPOSE 3000
EXPOSE 3000
CMD pg_ctl -D /pgdata start && until psql -U postgres -c '\q'; do \
>&2 echo "Postgres is starting up..."; \
sleep 5; \
done && \
exec fhirbase -d fhirbase web
When I run the above dockerfile getting the below error--
D:\FhirBase\fhirbase>docker run d6ba47544a36
waiting for server to start....2021-11-26 12:37:23.322 UTC [11] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-11-26 12:37:23.323 UTC [11] LOG: listening on IPv6 address "::", port 5432
2021-11-26 12:37:23.390 UTC [11] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-11-26 12:37:23.483 UTC [12] LOG: database system was shut down at 2021-11-26 12:28:08 UTC
2021-11-26 12:37:23.556 UTC [11] LOG: database system is ready to accept connections
done
server started
/bin/sh: 1: exec: fhirbase: Permission denied
Please help me regarding the dockerfile.
I have tried to run Postgres 12 on the docker and the following files I have created. I do not understand where do I made a mistake and what is an issue of PostgreSQL file permission.
Dockerfile:
FROM postgres:12.0-alpine
USER root
RUN chmod 0775 /var/lib/postgresql
RUN chown postgres /var/lib/postgresql
USER postgres
# RUN chmod 0775 /var/lib/postgresql
# RUN chown postgres /var/lib/postgresql
RUN ls -l /var/lib/postgresql
# RUN pgctl -D /usr/local/psql/data initdb &&\
RUN initdb -D /var/lib/postgresql/data &&\
echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf && \
echo "listen_addresses='*'" >> /var/lib/postgresql/data/postgresql.conf && \
pg_ctl start && \
psql -U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'test'" | grep -q 1 || psql -U postgres -c "CREATE DATABASE test" && \
psql --command "ALTER USER postgres WITH ENCRYPTED PASSWORD '123';"
# RUN initdb /var/lib/postgresql/data &&\
# echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf && \
# echo "listen_addresses='*'" >> /var/lib/postgresql/data/postgresql.conf && \
# pg_ctl start && \
# psql -U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'test'" | grep -q 1 || psql -U postgres -c "CREATE DATABASE test" && \
# psql --command "ALTER USER postgres WITH ENCRYPTED PASSWORD '123';"
# Add VOLUMEs to allow backup of config, logs and databases
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
EXPOSE 5432
docker-compose.yml
version: "3.7"
services:
postgresql:
build:
context: .
dockerfile: ./../postgresql/Dockerfile
volumes:
- ../postgresql/db_data:/var/lib/postgresql/data
ports:
- 5432:5432
I have run following command in terminal
docker-compose up
The following error occurs.
Building mydb_postgresql
Step 1/6 : FROM postgres:12-alpine
---> ecb176ff304a
Step 2/6 : USER postgres
---> Running in ee9fe8ed246f
Removing intermediate container ee9fe8ed246f
---> 8dfa002a5fab
Step 3/6 : RUN ls -l /var/lib/postgresql
---> Running in 08d98e1ea7b2
total 4
drwxrwxrwx 2 postgres postgres 4096 Mar 23 23:58 data
Removing intermediate container 08d98e1ea7b2
---> 548bb96ed8db
Step 4/6 : RUN initdb -D /var/lib/postgresql/data && echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf && echo "listen_addresses='*'" >> /var/lib/postgresql/data/postgresql.conf && pg_ctl start && psql -U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'mydb'" | grep -q 1 || psql -U postgres -c "CREATE DATABASE mydb" && psql --command "ALTER USER postgres WITH ENCRYPTED PASSWORD '123';"
---> Running in dbee0a2346df
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... sh: locale: not found
2020-04-17 18:16:13.112 UTC [11] WARNING: no usable system locales were found
ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2020-04-17 18:16:13.427 UTC [15] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.2.0) 9.2.0, 64-bit
2020-04-17 18:16:13.427 UTC [15] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-04-17 18:16:13.427 UTC [15] LOG: listening on IPv6 address "::", port 5432
2020-04-17 18:16:13.433 UTC [15] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-04-17 18:16:13.447 UTC [16] LOG: database system was shut down at 2020-04-17 18:16:13 UTC
2020-04-17 18:16:13.451 UTC [15] LOG: database system is ready to accept connections
done
server started
CREATE DATABASE
ALTER ROLE
Removing intermediate container dbee0a2346df
---> 8fd449f2a9e2
Step 5/6 : VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
---> Running in 76f4b6ee235e
Removing intermediate container 76f4b6ee235e
---> 343b2ddca9d2
Step 6/6 : EXPOSE 5432
---> Running in 306d1fd18818
Removing intermediate container 306d1fd18818
---> aa732eb5b2b6
Successfully built aa732eb5b2b6
Successfully tagged services_mydb_postgresql:latest
WARNING: Image for service mydb_postgresql was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating services_mydb_postgresql_1 ... done
Attaching to services_mydb_postgresql_1
mydb_postgresql_1 | chmod: /var/lib/postgresql/data: Operation not permitted
services_mydb_postgresql_1 exited with code 1
I don't know what is going on and why my process has stopped.
I use postgresql on Debian.
The postgresql service can not start after I edit the config file:
#data_directory = '/var/lib/postgresql/9.4/main' # use data in another directory
data_directory = '/opt/data/postgresql/data'
(yeah,I just use custom directory instead of the default data_directory)
I find the log in /var/log/syslog
Sep 14 10:22:17 thinkserver-ckd postgresql#9.4-main[11324]: Error: could not exec /usr/lib/postgresql/9.4/bin/pg_ctl /usr/lib/postgresql/9.4/bin/pg_ctl start -D /opt/data/postgresql/data -l /var/log/postgresql/postgresql-9.4-main.log -s -o -c config_file="/etc/postgresql/9.4/main/postgresql.conf" :
Sep 14 10:22:17 thinkserver-ckd systemd[1]: postgresql#9.4-main.service: control process exited, code=exited status=1
Sep 14 10:22:17 thinkserver-ckd systemd[1]: Failed to start PostgreSQL Cluster 9.4-main.
Sep 14 10:22:17 thinkserver-ckd systemd[1]: Unit postgresql#9.4-main.service entered failed state.
And nothing in /var/log/postgresql/postgresql-9.4-main.log
Thanks.
I finally got this answer:
What this error means in PostgreSQL?
#langton 's answer.
He said that
you should run pg_upgradecluster or similar, or just create a new cluster with pg_createcluster (these commands are for debian systems - you didn't specify your OS)
So I executed the command:
pg_createcluster -d /opt/data/postgresql/data -l /opt/data/postgresql/log 9.4 ckd
And then :
service postgresql restart
it started!
If downtime is allowed and you already have databases with data in the old cluster location you only need to physically copy the data to the new location.
This is a more or less common operation if you partition is out of space.
# Check that current data directory is the same that
# the one in the postgresql.conf config file
OLD_DATA_DIR=$(sudo -u postgres psql --no-psqlrc --no-align --tuples-only --quiet -c "SHOW data_directory;")
echo "${OLD_DATA_DIR}"
CONFIG_FILE=$(sudo -u postgres psql --no-psqlrc --no-align --tuples-only --quiet -c "SHOW config_file;")
echo "${CONFIG_FILE}"
# Stop PostgreSLQ
systemctl stop postgresql
# Change the data directory in the config
# Better to do it with an editor, instead of sed
NEW_DATA_DIR='/opt/data/postgresql/data'
sed -i "s%data_directory = '${OLD_DATA_DIR}'%data_directory = '${NEW_DATA_DIR}'%" "${CONFIG_FILE}"
# Move/Copy the data for example using rsync
rsync -av --dry-run "${OLD_DATA_DIR}" "${NEW_DATA_DIR}"
# Take care with the classical issues of rsync and end backslashes
rsync -av "${OLD_DATA_DIR}" "${NEW_DATA_DIR}"
# Rename the old dir, just to avoid missunderstandings and set
# check the permissions on the new one
# Start postgres
systemctl start postgresql
# Check that everything goes well and eventually drop the old data
# Make sure that the logs and everything else is where you want.
I have troubles restarting a dockerized postgres database (I use Core OS). The database is started in a bash script using the command
# boot.sh
sudo -i -u postgres /usr/lib/postgresql/9.3/bin/postgres -D /var/lib/postgresql/9.3/main -c config_file=/etc/postgresql/9.3/main/postgresql.conf
which works. I have another script called by confd which is run when some etcd keys change (this part is ok, the file is correctly called) and must restart postgres (not reload, because some config changes require a restart). Here are the main options I tried, which failed...
# restart.sh
sudo -u postgres /usr/lib/postgresql/9.3/bin/pg_ctl --pgdata=/var/lib/postgresql/9.3/main restart
systematically raises an error:
%FATAL: lock file "postmaster.pid" already exists
%HINT: Is another postmaster (PID 273) running in data directory "/var/lib/postgresql/9.3/main"?
Furthermore,
# restart.sh
rm /var/lib/postgresql/9.3/main/postmaster.pid
sudo -i -u postgres /usr/lib/postgresql/9.3/bin/postgres -D /var/lib/postgresql/9.3/main -c config_file=/etc/postgresql/9.3/main/postgresql.conf
,
rm /var/lib/postgresql/9.3/main/postmaster.pid
/etc/init.d/postgresql start
,
/etc/init.d/postgresql restart
and
exec su postgres -c "/usr/lib/postgresql/9.3/bin/postgres -D /var/lib/postgresql/9.3/main -c config_file=/etc/postgresql/9.3/main/postgresql.conf"
fail with
ERROR exit status 1
Any thought? Thank you in advance!
For me, changing the config and doing
$ docker restart <postgres_container>
on the host works just fine.