I am trying to configure replication between two Postgresql docker containers. All tutorials out there - which are based on regular VM, not containers - states that a backup from the Master has to be done in the Replica / StandBy, through a command like this:
pg_basebackup -h 192.168.0.108 -U replicator -p 5432 -D $PGDATA -Fp -Xs -P -R
Unfortunately, this throws an error:
pg_basebackup: error: directory "/var/lib/postgresql/data" exists but is not empty
I cannot delete the content of this folder because the Postgresql service will crash and I will be kicked out of the container.
So, how can I do this given that I cannot stop the Postgresql service because of the containerized application?
These instructions following the percona docs for configuration
postgres replication.
Create a network for the postgres servers:
docker network create pgnet
Start the primary database server:
docker run -d \
--network pgnet \
-v primary_data:/var/lib/postgresql/data \
--name pg_primary \
-e POSTGRES_PASSWORD=secret \
-e POSTGRES_DB=exampledb \
docker.io/postgres:14
Create a replication user and configure pg_hba.conf to allow
replication access:
docker exec pg_primary \
psql -U postgres -c \
"CREATE USER replicator WITH REPLICATION ENCRYPTED PASSWORD 'secret'"
docker exec pg_primary \
sh -c 'echo host replication replicator 0.0.0.0/0 md5 >> $PGDATA/pg_hba.conf'
docker exec pg_primary \
psql -U postgres -c 'select pg_reload_conf()'
Prepare the pgdata directory for the replica server. We do this by running the pg_basebackup command in an ephemeral container:
docker run -it --rm \
--network pgnet \
-v replica_data:/var/lib/postgresql/data \
--name pg_replica_init \
docker.io/postgres:14 \
sh -c 'pg_basebackup -h pg_primary -U replicator -p 5432 -D $PGDATA -Fp -Xs -P -R -W'
(The -W in the above command forces pg_basebackup to prompt for a
password. There are other ways you could provide the password; the
postgres docs have details.)
Start the replica:
docker run -d \
--network pgnet \
-v replica_data:/var/lib/postgresql/data \
--name pg_replica \
docker.io/postgres:14
Verify that replication is working. Create a table in the primary:
docker exec pg_primary \
psql -U postgres exampledb -c 'create table example_table (id int, name text)'
See that it shows up in the replica:
docker exec pg_replica \
psql -U postgres exampledb -c '\d'
You should see as output:
List of relations
Schema | Name | Type | Owner
--------+---------------+-------+----------
public | example_table | table | postgres
(1 row)
Related
docker run -d --rm --name dummy -v postgres_volume:/root alpine tail -f /dev/null
docker cp ./data dummy:/root
docker stop dummy
docker run -p 5432:5432 -v postgres_volume:/var/lib/postgresql -e POSTGRES_PASSWORD=password postgres
The above commands giving error: mkdir: cannot create directory ‘/var/lib/postgresql’: Permission denied.
./data in the docker cp command contains the Postgres backup.
What am I missing?
You (or rather postgres) are missing permissions for the files you have copied. This should fix it:
docker run --rm -v postgres_volume:/var/lib/postgresql postgres /bin/sh -c \
'chown --recursive $(id -u postgres):$(id -g postgres) /var/lib/postgresql'
I want to connect to psql using password on kubectl exec command on kubernetes like
kubectl exec -it postgres -- bash \`psql -h $IP -U admin --password password -p $PORT dbname\`
I tried to command
kubectl exec -it $podid -- bash \`psql -h $IP -U admin --password -p $PORT postgresdb\`
and
kubectl exec -it $podid -- bash -c "psql -- dbname=postgresql://postgres:password#$ip:$port/postgresdb -c 'INSERT INTO public."user" (user_id,user_pw,user_nm,email,phone,is_admin) VALUES ('admin','admin','admin','admin#admin.com',NULL,true);'"
but these command did not work.
How can I connect to psql using kubernetes commands and password?
try
kubectl exec -it <postgresspod-name> bash or sh
when you inside the container you can just do
PGPASSWORD=<password> psql -h <host or service name or localhost> -U <username> <dbname>
another option is
kubectl exec -it postgres -- bash \`PGPASSWORD=<password> psql -h <host or service name or localhost> -U <username> <dbname>\`
The accepted answer did not work for me (command PGPASSWORD=<...> not found).
This worked :
kubectl exec -ti postgres -- env PGPASSWORD=$PASSWD psql psql -h <host> -U <username> <dbname>
As you asking only for command I assume you have all Deployments, Services, etc.
Please execute command below:
$ kubectl exec -ti <postgresspod-name> -- psql -h <node/host IP> -U postgresadmin --password -p <port> postgresdb
Password for user postgresadmin:
After you will type password you will be connected to Postgres
For me only this works
kubectl exec postgres-postgresql-0 -n nte -- sh -c 'PGPASSWORD=postgres psql -U "postgres" -d "database_name" -c "select * from table_name"'
You can try this way :
kubectl exec -i <plsql podname> -- psql -h <host or service name or localhost> -u <username> -p<password>
On Gitlab-CI I set up a postgres service for my database and would like to inspect the config file of it.
For this I let postgres return the location of the config file but when I go to that directory, it is empty.
How can I access it?
.gitlab-ci.yaml:
image: maven:3.5.3-jdk-8
services:
- postgres
variables:
POSTGRES_DB: custom_db
POSTGRES_USER: custom_user
POSTGRES_PASSWORD: custom_pass
connect:
image: postgres
script:
- export PGPASSWORD=$POSTGRES_PASSWORD
- psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT 'OK' AS status;"
- psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SHOW config_file;"
- cd /var/lib/postgresql/data
- dir
- ls -a
- cat postgresql.conf
The respective job output:
Running with gitlab-runner 11.8.0 (4745a6f3)
on docker-auto-scale 72989761
Using Docker executor with image postgres ...
Starting service postgres:latest ...
Pulling docker image postgres:latest ...
Using docker image sha256:30bf4f039abe0affe9fe4f07a13b00ea959299510626d650c719fb10c4f41470 for postgres:latest ...
Waiting for services to be up and running...
Pulling docker image postgres ...
Using docker image sha256:30bf4f039abe0affe9fe4f07a13b00ea959299510626d650c719fb10c4f41470 for postgres ...
Running on runner-72989761-project-7829066-concurrent-0 via runner-72989761-srm-1551974294-08e28deb...
Cloning repository...
Cloning into '/builds/kimtang/SpringBootTimeWithSwagger'...
Checking out 1399a232 as master...
Skipping Git submodules setup
$ export PGPASSWORD=$POSTGRES_PASSWORD
$ psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT 'OK' AS status;"
status
--------
OK
(1 row)
$ psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SHOW config_file;"
config_file
------------------------------------------
/var/lib/postgresql/data/postgresql.conf
(1 row)
$ cd /var/lib/postgresql/data
$ dir
$ ls -a
.
..
$ cat postgresql.conf
cat: postgresql.conf: No such file or directory
ERROR: Job failed: exit code 1
Why does it state it is in /var/lib/postgresql/data but then can not be found?
You're connected to a remote docker instance via psql and you're checking a local directory. If you really want to check what's going on on the service docker image then ssh into the worker and then use the docker exec -i -t <container_name> /bin/sh command to log into the container. You will have to make the job run for a long time though so put some sleep in there.
Im getting errors trying to run my docker containers. I need the postgres and redis connected to my server application.
docker pull postgres
docker rm -f syda-postgres
docker run -p 30203:5432 --name syda-postgres -e POSTGRES_PASSWORD=password POSTGRES_USER=root POSTGRES_DB=syda postgres
docker pull redis
docker rm -f syda-inmemory
docker run -d -p 30204:6379 --name syda-inmemory redis redis-server --appendonly yes
docker pull docker.url.ee/syda/server:latest
docker rm -f syda-server
docker run -d -p 30202:8080 --name syda-server --link syda-postgres:postgres --link syda-inmemory:redis \docker.url.ee/syda/server:latest
This is the error im getting:
Error: No such container: syda-postgres
docker: invalid reference format: repository name must be lowercase.
See 'docker run --help'.
Error: No such container: syda-server
docker: Error response from daemon: could not get container for syda-postgres: No such container: syda-postgres.
See 'docker run --help'.
docker run -p 30203:5432 --name syda-postgres -e POSTGRES_PASSWORD=password POSTGRES_USER=root POSTGRES_DB=syda postgres
That tries to run a container from the image named POSTGRES_USER=root with the command/arguments to the entrypoint POSTGRES_DB=syda postgres. You need to pass the -e for each variable like:
docker run -p 30203:5432 --name syda-postgres \
-e POSTGRES_PASSWORD=password -e POSTGRES_USER=root -e POSTGRES_DB=syda \
postgres
Also, note that links are deprecated, you should use a shared network for communicating between containers. This is often done with a compose file. If you need to do it from a script, you could run:
docker pull postgres
docker pull redis
docker pull docker.url.ee/syda/server:latest
docker rm -f syda-postgres
docker rm -f syda-inmemory
docker rm -f syda-server
docker network rm syda-net
docker network create syda-net
docker run -p 30203:5432 --net syda-net --name syda-postgres \
-e POSTGRES_PASSWORD=password -e POSTGRES_USER=root -e POSTGRES_DB=syda \
postgres
docker run -d -p 30204:6379 --net syda-net --name syda-inmemory \
redis redis-server --appendonly yes
docker run -d -p 30202:8080 --net syda-net --name syda-server \
docker.url.ee/syda/server:latest
I want to spin-up a docker for postgres:latest but let the data be stored on the host. This is how I do it right now but doesn't seem to work. Any ideas?
docker run --name name1 -e POSTGRES_PASSWORD=PASSWORD1 -e POSTGRES_USER=USER1 -e PGDATA=/my/local/path/postgresql -e POSTGRES_DB=DB1 -d -P postgres:latest
You need to mount the host directory as a data volume first with -v host_path:contiainer_path:mode.
docker run \
--name name1 \
-v /my/local/path/postgresql:/data \
-e POSTGRES_PASSWORD=PASSWORD1 \
-e POSTGRES_USER=USER1 \
-e PGDATA=/data \
-e POSTGRES_DB=DB1 \
-d -P postgres:latest
The path should exist in the container image.