I'm making a backup from my postgresql database (v11) using pg_basebackup like this:
pg_basebackup -h localhost -p 5432 -U postgres -D /tmp/backup -Ft -z -Xs -P
The backup process is completed just fine without any errors.Then as a test I'd like to recover from the backup and for that I do the following steps:
Shutdown the postgres instance.
Delete everything from /var/lib/postgresql/data folder
rm -rf /var/lib/postgresl/data/*
Untar the base archive there:
tar xvf /tmp/backup/base.tar.gz -C /var/lib/postgresql/data
Untar the wal archives to a temporary directory
tar xvf /tmp/backup/pg_wal.tar.gz -C /tmp/archived_wals
Create a file recovery.conf in the /var/lib/postgresql/data folder:
touch /var/lib/postgresql/data/recovery.conf
chown postgres:postgres /var/lib/postgresql/data/recovery.conf
Specify the recovery command in the recovery.conf file:
restore_command = 'cp /path/archived_wals/%f "%p" '
Restart the postgres instance, and wait for the recovery to be finished.
However when postgres starts up I get the following error in the logs:
2020-02-04 11:34:52.599 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-02-04 11:34:52.599 UTC [1] LOG: listening on IPv6 address "::", port 5432
2020-02-04 11:34:52.613 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-02-04 11:34:52.709 UTC [18] LOG: database system was interrupted; last known up at 2020-02-04 08:39:54 UTC
cp: can't stat '/tmp/archived_wals/00000002.history': No such file or directory
2020-02-04 11:34:52.735 UTC [18] LOG: starting archive recovery
2020-02-04 11:34:52.752 UTC [18] LOG: invalid checkpoint record
2020-02-04 11:34:52.752 UTC [18] FATAL: could not locate required checkpoint record
2020-02-04 11:34:52.752 UTC [18] HINT: If you are not restoring from a backup, try removing the file "/var/lib/postgresql/data/backup_label".
2020-02-04 11:34:52.753 UTC [1] LOG: startup process (PID 18) exited with exit code 1
2020-02-04 11:34:52.753 UTC [1] LOG: aborting startup due to startup process failure
2020-02-04 11:34:52.769 UTC [1] LOG: database system is shut down
I guess I'm missing something, maybe I need to do additional configuration to postgres to be able to restore a pg_basebackup backup?
Try this:
/pgsql-11/bin/pg_basebackup -D /pgdata/pg11/data -X stream --waldir=/pglog/pg11/wal_log/ -P -c fast -h remoteserver\localserver -U username
Let me know if that works.
After all, it was my bad.
I enclosed the destination placeholder within % instead of double quotes in recovery.conf.
However the way I described for the backup and restore is valid and since I corrected the typo it works just fine for me, so I leave the question as it is for further reference.
Related
I'm trying to use the official postgres docker files (with the aim of extending them) but if I run them as a non-root local user, they simply refuse to start.
i.e. If I follow the basic instructions from https://hub.docker.com/_/postgres and run:
docker run -e POSTGRES_PASSWORD=mysecretpassword postgres
then I get:
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....2023-02-18 09:55:58.427 UTC [48] LOG: starting PostgreSQL 15.2 (Debian 15.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-02-18 09:55:58.452 UTC [48] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-02-18 09:55:58.525 UTC [51] LOG: database system was shut down at 2023-02-18 09:55:49 UTC
2023-02-18 09:55:58.550 UTC [48] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
2023-02-18 09:55:58.603 UTC [48] LOG: received fast shutdown request
waiting for server to shut down....2023-02-18 09:55:58.625 UTC [48] LOG: aborting any active transactions
2023-02-18 09:55:58.626 UTC [48] LOG: background worker "logical replication launcher" (PID 54) exited with exit code 1
2023-02-18 09:55:58.626 UTC [49] LOG: shutting down
2023-02-18 09:55:58.650 UTC [49] LOG: checkpoint starting: shutdown immediate
2023-02-18 09:55:58.835 UTC [49] LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.047 s, sync=0.023 s, total=0.209 s; sync files=2, longest=0.012 s, average=0.012 s; distance=0 kB, estimate=0 kB
2023-02-18 09:55:58.840 UTC [48] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2023-02-18 09:55:58.967 UTC [1] LOG: starting PostgreSQL 15.2 (Debian 15.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2023-02-18 09:55:58.968 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2023-02-18 09:55:58.969 UTC [1] LOG: could not create IPv6 socket for address "::": Address family not supported by protocol
2023-02-18 09:55:59.017 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2023-02-18 09:55:59.063 UTC [62] LOG: database system was shut down at 2023-02-18 09:55:58 UTC
2023-02-18 09:55:59.091 UTC [1] LOG: database system is ready to accept connections
and everything is happy.
However, if I following the instructions under "Arbitrary --user Notes"
and run:
docker run -it --rm --user "$(id -u):$(id -g)" -v /etc/passwd:/etc/passwd:ro -e POSTGRES_PASSWORD=mysecretpassword postgres
(or without the it or the rm or with just the user and not the group - makes no difference)
then I get:
chmod: changing permissions of '/var/lib/postgresql/data': Operation not permitted
chmod: changing permissions of '/var/run/postgresql': Operation not permitted
The files belonging to this database system will be owned by user "richard".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... initdb: error: could not change permissions of directory "/var/lib/postgresql/data": Operation not permitted
What am I missing / doing wrong?
Note that Switching Between Root and Non-Root Users in Docker is very out of date and that the answer given as a comment under How to create a postgres container with a non-root user? is simply what I'm trying to do here.
Running unprivileged, the official postgres image needs to run from a fixed UID - matching their image, you can't just re-use your own uid and /etc/passwd.
Try this:
docker run -it --rm --user "999:999" -e POSTGRES_PASSWORD=mysecretpassword postgres
I am trying to write a docker entrypoint script, that’s able to start postgreSQL and creating a new named database.
I feel like there is a much better way, than mine. I am happy for every input.
Issues:
a few times I have to restart the container befor it creates the databas and does not crash. Some times I don't know why it workes. I can't recreate it right now.
Freequently I get messages like [unknown]#[unknown] LOG: incomplete startup packet but there shouldn't be any.
Reason for custom docker image: I need PostgreSQL to use ssl for every connection. The official image is sadly not able to allow ssl with real certificates.
Dockerfile:
FROM debian:10.9
RUN apt update && apt install -y \
postgresql postgresql-contrib \
openssl \
net-tools netcat \
vim
RUN mkdir database_stuff
COPY --chown=postgres:postgres ./entrypoint.sh /database_stuff/entrypoint.sh
RUN chmod 777 /database_stuff/entrypoint.sh
COPY --chown=postgres:postgres ./certificates/cert.pem /database_stuff/cert.pem
RUN chmod 600 /database_stuff/cert.pem
COPY --chown=postgres:postgres ./certificates/key.pem /database_stuff/key.pem
RUN chmod 600 /database_stuff/key.pem
COPY --chown=postgres:postgres ./configuration/postgresql.conf /etc/postgresql/11/main/postgresql.conf
COPY --chown=postgres:postgres ./configuration/pg_hba.conf /etc/postgresql/11/main/pg_hba.conf
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
USER postgres
CMD ["/usr/lib/postgresql/11/bin/postgres", "-D", "/var/lib/postgresql/11/main/", "-c", "config_file=/etc/postgresql/11/main/postgresql.conf"]
ENTRYPOINT ["/database_stuff/entrypoint.sh"]
entrypoint.sh script
#!/usr/bin/env bash
echo "START entrypoint.sh"
echo "start postgrSQL"
/usr/lib/postgresql/11/bin/postgres -D /var/lib/postgresql/11/main/ -c config_file=/etc/postgresql/11/main/postgresql.conf &
until nc -z db 5432; do
sleep 0.1
done
echo "create Database"
psql -U postgres -c 'create database part_screening;'
psql -U postgres -c "create user alstom with encrypted password 'Laser765_f';"
echo "END entrypoint.sh .... Stay open"
tail -f /dev/null
just a log with the unknown#unknown issue
START entrypoint.sh
start postgrSQL
create Database
CREATE DATABASE
CREATE ROLE
END entrypoint.sh .... Stay open
2021-08-02 17:08:52.280 UTC [7] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-08-02 17:08:52.280 UTC [7] LOG: listening on IPv6 address "::", port 5432
2021-08-02 17:08:52.284 UTC [7] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-08-02 17:08:52.300 UTC [10] LOG: database system was shut down at 2021-08-02 17:06:42 UTC
2021-08-02 17:08:52.305 UTC [7] LOG: database system is ready to accept connections
2021-08-02 17:08:52.330 UTC [18] [unknown]#[unknown] LOG: incomplete startup packet
2021-08-02 17:08:53.145 UTC [38] [unknown]#[unknown] LOG: incomplete startup packet
I'm doing the same steps as described in: https://medium.com/better-programming/connect-from-local-machine-to-postgresql-docker-container-f785f00461a7
but when I try to connect to my postgres (with password mysecretpassword) from Intellij I get the following error:
The specified database user/password combination is rejected: [28P01] FATAL: password authentication failed for user "postgres"
Of course I can connect to my db from cmd command:
$ psql -h localhost -p 5432 -U postgres -W Password for user postgres:
psql (9.5.5, server 10.3 (Debian 10.3-1.pgdg90+1)) WARNING: psql major version 9.5, server major version 10. Some psql features might not work.
Type "help" for help. postgres=# \l
My container is up:
What is going on? I don't have any idea... I use postgres driver 42.2.5 in Intellij
Logs from container:
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....2020-09-30 11:17:42.613 UTC [45] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-09-30 11:17:42.618 UTC [45] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-09-30 11:17:42.642 UTC [46] LOG: database system was shut down at 2020-09-30 11:17:42 UTC
2020-09-30 11:17:42.649 UTC [45] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
2020-09-30 11:17:42.700 UTC [45] LOG: received fast shutdown request
waiting for server to shut down....2020-09-30 11:17:42.705 UTC [45] LOG: aborting any active transactions
2020-09-30 11:17:42.708 UTC [45] LOG: background worker "logical replication launcher" (PID 52) exited with exit code 1
2020-09-30 11:17:42.708 UTC [47] LOG: shutting down
2020-09-30 11:17:42.737 UTC [45] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2020-09-30 11:17:42.836 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-09-30 11:17:42.836 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-09-30 11:17:42.836 UTC [1] LOG: listening on IPv6 address "::", port 5432
2020-09-30 11:17:42.845 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-09-30 11:17:42.881 UTC [54] LOG: database system was shut down at 2020-09-30 11:17:42 UTC
2020-09-30 11:17:42.888 UTC [1] LOG: database system is ready to accept connections
create dir for volume: C:\Users{user}\docker\postgres_data\data
Connect to Postgres on Windows:
docker run -p 5432:5432 --name postgres -v C:\Users\{user}\docker\postgres_data\data:/var/lib/postgresql/data -e POSTGRES_PASSWORD=root -d postgres:11.3
Connect to Postgres from CMD and create DB (only after first creation container on empty volume):
docker ps
get the output:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e626f4c7b958 postgres:11.3 "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:5432->5432/tcp postgres
Run:
docker exec -it {CONTAINER ID} bash
psql -h localhost -U postgres
psql -U postgres
Connect from Intellij with PASSWORD=root:
It seems that you have running postgresql instance on 5432 port. Check what is running on port 5432:
sudo lsof -i :5432
If there are any records, you could manually kill the processes that are using this port (man kill) and re-run docker container. In my case there was another local instance of postgresql running on this port, so the following helps me:
sudo pkill -u postgres
Or (more preferable) just specify a different port mapping for your docker postgresql (for e.g. 5433):
docker run -p 5433:5432 ...other_your_flags... postgres:latest
try to remove your PostgreSQL the one you have installed locally, if it exists. Because my problem was precisely that the idea did not understand where to connect, to the docker survey or to that installed database on my computer
I am trying to set up a postgres container to start and run initializing the creation of a table. I've succeeded with the straight image from docker but now that I am trying to extend the image a little to create tables when it's produced and I can't get it running. Based off what I've read here How to create User/Database in script for Docker Postgres, this is what I have:
Dockerfile:
FROM library/postgres
COPY init.sql /docker-entrypoint-initdb.d/
init.sql:
CREATE TABLE incident_disposition (
incident_disposition_code VARCHAR,
incident_disposition_code_description VARCHAR
);
From what I understand, FROM library . . . pulls the postgres image from docker hub and the COPY pushes my init.sql script into the entry point so there is no need for a big dockerfile correct?
I then build the image no issue:
Build
docker build -t my_postgres_image .
But when I run I get the issues:
Run
docker run --name testing my_postgres_image --publish 8000:8080 --detach -e POSTGRES_PASSWORD=postgres -d postgres
Errors from logs
Error: Database is uninitialized and superuser password is not specified.
You must specify POSTGRES_PASSWORD to a non-empty value for the
superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
connections without a password. This is *not* recommended.
See PostgreSQL documentation about "trust":
https://www.postgresql.org/docs/current/auth-trust.html
Attempt from comment:
docker container logs testing
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/lib/postgresql/data -l logfile start
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2020-03-26 14:06:51.064 UTC [46] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-03-26 14:06:51.072 UTC [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-03-26 14:06:51.108 UTC [47] LOG: database system was shut down at 2020-03-26 14:06:50 UTC
2020-03-26 14:06:51.119 UTC [46] LOG: database system is ready to accept connections
done
server started
/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
CREATE TABLE
2020-03-26 14:06:51.231 UTC [46] LOG: received fast shutdown request
waiting for server to shut down....2020-03-26 14:06:51.232 UTC [46] LOG: aborting any active transactions
2020-03-26 14:06:51.233 UTC [46] LOG: background worker "logical replication launcher" (PID 53) exited with exit code 1
2020-03-26 14:06:51.234 UTC [48] LOG: shutting down
2020-03-26 14:06:51.290 UTC [46] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2020-03-26 14:06:51.345 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2020-03-26 14:06:51.345 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2020-03-26 14:06:51.345 UTC [1] LOG: listening on IPv6 address "::", port 5432
2020-03-26 14:06:51.361 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-03-26 14:06:51.387 UTC [64] LOG: database system was shut down at 2020-03-26 14:06:51 UTC
2020-03-26 14:06:51.398 UTC [1] LOG: database system is ready to accept connections
2020-03-26 14:07:27.715 UTC [72] ERROR: relation "incident_disposition" does not exist at character 15
2020-03-26 14:07:27.715 UTC [72] STATEMENT: select * from incident_disposition;
In addition to comments
Due to recent docker image's updates postgres images do not allow to connect to DB without a password from anywhere. So you need to specify username/password
docker run -p 8000:8080 -e POSTGRES_PASSWORD=postgres --name testing -d my_postgres_image
Or if you still don't want to use password, you can just set POSTGRES_HOST_AUTH_METHOD=trust environment variable:
docker run -p 8000:8080 -e POSTGRES_HOST_AUTH_METHOD=trust --name testing -d my_postgres_image
It is a typical Initialization scripts issue.
You can file the full explaination in postgresql docker page. https://hub.docker.com/_/postgres
Here is the brief intro:
1. One common problem is that if one of your /docker-entrypoint-initdb.d scripts fails (which will cause the entrypoint script to exit) and your orchestrator restarts the container with the already initialized data directory, it will not continue on with your scripts.
note:
in your case, you may need clean the historical docker containers(stopped) by
step 1: docker ps |grep
step 2: docker rm -f -v
Or if you are using docker-compose, the historical orchestrator could be easily removed by docker-compose down -v.
I am trying to create a hot_standby server, and I receive the following error after pg_basebackup completes. Notice I use a shell script, replicator.sh, to start the replication. Can anyone give me some insight?
My specs:
Debian Wheezy 7.6
Postgresql 9.3
Database size: ~115GB
Error:
postgres#database-master:/etc/postgresql/9.3/main$ sh replicator.sh
Stopping PostgreSQL
[ ok ] Stopping PostgreSQL 9.3 database server: main.
Cleaning up old cluster directory
Starting base backup as replicator
Password:
113720266/113720266 kB (100%), 1/1 tablespace
NOTICE: WAL archiving is not enabled; you must ensure that all required WAL segments are copied through other means to complete the backup
pg_basebackup: base backup completed
Starting Postgresql
[....] Starting PostgreSQL 9.3 database server: main[....] The PostgreSQL server failed to start.
Please check the log output: 2014-09-11 17:56:33 UTC LOG: database system was interrupted; last
known up at 2014-09-11 16:54:29 UTC 2014-09-11 17:56:33 UTC LOG: creating missing WAL directory
"pg_xlog/archive_status" 2014-09-11 17:56:33 UTC LOG: incomplete startup packet 2014-09-11 17:56:33
UTC LOG: invalid checkpoint record 2014-09-11 17:56:33 UTC FATAL: could not locate required
checkpoint record 2014-09-11 17:56:33 UTC HINT: If you are not restoring from a backup, try
removing the file "/var/lib/p[FAILesql/9.3/main/backup_label". 2014-09-11 17:56:33 UTC LOG: startup
process (PID 21972) exited with exit code 1 2014-09-11 17:56:33 UTC LOG: aborting startup due to
startup process failure ... failed! failed!
Contents of replicator.sh:
#!/bin/bash
echo Stopping PostgreSQL
/etc/init.d/postgresql stop
echo Cleaning up old cluster directory
rm -rf /var/lib/postgresql/9.3/main
echo Starting base backup as replicator
pg_basebackup -h 123.456.789.123 -D /var/lib/postgresql/9.3/main -U replicator -v -P
echo Writing recovery.conf file
sudo -u postgres bash -c "cat > /var/lib/postgresql/9.3/main/recovery.conf <<- _EOF1_
standby_mode = 'on'
primary_conninfo = 'host=123.456.789.123 port=5432 user=replicator password=XXXXX sslmode=require'
trigger_file = '/tmp/postgresql.trigger'
_EOF1_
"
echo Starting Postgresql
/etc/init.d/postgresql start
Thank you,
Jake
My best guess from the above is that the pg_basebackup failed and your shell script doesn't check for error return codes or use set -e to automatically abort after errors, so it just carried on regardless.
It's also possible that you don't have WAL archiving configured, or don't have a restore_command set in the replica. In that case, the transaction logs required to start the base backup will not be available and startup will fail.
I strongly recommend that you:
Use pg_basebackup -X stream so that the required transaction logs get copied along with the backup; and
Use set -e in your shell script, or test for errors with a suitable if ! pg_basebackup .... ; then block.