I create backups like this: docker exec DOCKER pg_dump -U USER -F t DB | gzip > ./FILE.tar.gz
What's the best way to restore the database given that the database runs within a container?
For your case:
docker exec -it <CONTAINER> gunzip < backup.tar.gz | pg_restore -U <USER> -F t -d <DB>
Remote restore is also available if your container is public facing and remote connections are allowed in pg_hba.conf for postresql:
gunzip < backup.tar.gz | pg_restore -U <USER> -F t -d <DB> -h <HOST_IP> -p 5432
As a rule of thumb, it is good idea to document your backup and restore commands specific to the project.
How take backup of the data which is existing in the running PostgreSQL container
Create some folder in your root
mkdir -p '/myfolder/bdbackup'
download the postgres image which you are using and execute the following command
docker run --name demo1 -e POSTGRES_PASSWORD=passowrd -v /myfolder/bdbackup:/var/lib/postgresql/data -d postgres
docker exec -it demo1 psql -U postgres
Back up will be stored in the following folder /myfolder/bdbackup
you can kill the container and stop the container any time but data will be stored in the host.
and once again re-run the postgres the container with same command
docker run --name demo2 -e POSTGRES_PASSWORD=passowrd -v /myfolder/bdbackup:/var/lib/postgresql/data -d postgres
docker exec -it demo1 psql -U postgres
and execute following query select * from emp;
you can see the data has restored...
Related
I'm having trouble in creating a Postgres DB using this bash script:
#! /bin/bash
docker pull postgres
docker run --name coverage-postgres -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgres
export CONTAINER_ID=$(sudo docker ps -a | grep coverage-postgres | head -c12)
sleep 2s
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "create user coverage_user with password 'password';"
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "create database coverage owner coverage_user;"
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "grant all privileges on database coverage to coverage_user;"
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "\c coverage coverage_user" # it seems useless...
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "CREATE TABLE IF NOT EXISTS postal_codes (id,...;"
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "CREATE UNIQUE INDEX ... ;"
# exit from container
exit
# restart container
docker start $CONTAINER_ID
In particular, the database is created, the user is created, the table is created but... it's not in the coverage db but in postgres db.
I've tried to add "CREATE TABLE coverage.postal_codes" but coverage is a db and not a schema and it didn't work.
I've tried to use: psql -U coverage_user but the system tells me that database coverage_user doesn't exist.
So of course I thought "I have to specify the database of course!". Then I've tried to use: psql -U coverage as the name of the database but this time, the system makes fun of me and, changing its mind, tells me that the role coverage doesn't exists.
I tried a workaround: within the command -c "\c coverage coverage_user" I concatenated the other commands this way:
-c "\c coverage coverage_user; CREATE TABLE...; CREATE UNIQUE INDEX...;"
but, of course, neither this worked at all.
I make a premise: I know there are other ways to do this but I would like to understand what I am missing with these specific commands.
Solution
#! /bin/bash
docker pull postgres
docker run --name coverage-postgres -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgres
export CONTAINER_ID=$(sudo docker ps -a | grep coverage-postgres | head -c12)
sleep 2s
docker exec -it $CONTAINER_ID psql -U postgres -c "create user coverage_user with password 'password';"
sleep 0.5
docker exec -it $CONTAINER_ID psql -U postgres -c "create database coverage owner coverage_user;"
sleep 0.5
docker exec -it $CONTAINER_ID psql -U postgres -c "grant all privileges on database coverage to coverage_user;"
sleep 0.5
docker exec -it $CONTAINER_ID psql -U coverage_user -c "CREATE TABLE IF NOT EXISTS postal_codes (id int)" coverage
Explanation
https://www.postgresql.org/docs/9.2/app-psql.html
psql [option...] [dbname [username]]
Just add dbname after options. And change user as -U option. You can pass dbname also as an rgument -d
I have multiple Environment Variables defined on my Postgres container, such as POSTGRES_USER. The container is running and I want to connect to Postgres from the command line using exec.
I'm unable to connect with the following:
docker exec -it <CONTAINER-ID> psql -U $POSTGRES_USER -d <DB NAME>
I understand that the variable is defined on the container and the following does work:
docker exec -it <CONTAINER-ID> bash -c 'psql -U $POSTGRES_USER -d <DB NAME>'
Is there a way for me to execute the psql command directly from docker exec and call the environment variable on the container?
docker exec -it <CONTAINER-ID> psql -U ????? -d <DB NAME>
Depending on your use case, what you could do, instead of passing a user to the psql command is to define the environment variable PGUSER to the container at boot time.
This way, it will be the default user for PostgreSQL, if you do not specify any, so you won't even have to specify it in order to connect:
$ docker run --name postgres -e POSTGRES_PASSWORD=bar -e POSTGRES_USER=foo -e PGUSER=foo -d postgres
e250f0821613a5e2021e94772a732f299874fc7a16b340ada4233afe73744423
$ docker exec -ti postgres psql -d postgres
psql (12.4 (Debian 12.4-1.pgdg100+1))
Type "help" for help.
postgres=#
The reason this isn't working for you is because when you run the command
docker exec -it <CONTAINER-ID> psql -U $POSTGRES_USER -d <DB NAME>
You're running it on your host. So, $POSTGRES_USER refers to the environment variable on your host, not your container. That variable isn't set on your host.
The second command
docker exec -it <CONTAINER-ID> bash -c 'psql -U $POSTGRES_USER -d <DB NAME>'
works because you're passing the command in the quotes to the shell in the container, where that variable actually exists.
The method in the second command is the way to do what you're trying to do, unless you set the variable on your host somehow and make sure it has the same value as it does in your container.
The easiest way to do this would be to reference your host variable at image build time.
So, in your Dockerfile, if you write ENV POSTGRES_USER=${POSTGRES_USER} it will look in the host environment for that value, and use it.
If you set the variables this way, then your command will work.
I have a dump of my production db, which I can restore easily in my docker container with: docker exec -it my_db_container pg_restore --user=my_user --dbname=dbname sql/current.dump. Everything works, data are here.
But when I re-dump my local database from the docker with docker exec -it my_db_container -U my-user -F c -b dbname > docker/db/current_stripped.dump back to project folder, my dump file is created (with appropriate size and content) but I can not use it for restoring (docker exec -it whasq-db pg_restore --user=my-user --dbname=dbname sql/current_stripped.dump) it again to a fresh db due to an error: pg_restore: [custom archiver] could not read from input file: end of file however the restore command is the same (except the my_user which is postgres in production) as used in production env.
I had the same problem and solved it as follows. I switched to the -f or --file option instead of piping (taken from https://stackoverflow.com/a/51073680/6593069). And I created the dump file inside the container and then copied it to my host.
That would mean in your case:
docker exec -it my_db_container -U my-user -F c -b dbname -f current_stripped.dump
docker cp my_db_container:current_stripped.dump .
i am trying to use Docker. So i installed in Docker postgresql image.
Until now, when i imported osm data into postql i used this command:
psql -U postgres mydb
CREATE EXTENSION postgis;
osm2pgsql -U postgres -d mydb -s -S ./osm_stylesheet /home/ramnikov/Downloads/hessen-latest.osm
How can i do the same inside Docker after this command
$ sudo docker exec -it postgresql sudo -u postgres psql
or before this command ?
Tnx
Andrey
You can execute osm2pgsql outside of Docker:
-H|--host Database server host name or socket location.
As well as psql:
-h, --host=HOSTNAME database server host or socket directory
Like this:
psql -h dockerIP -U postgres -d mydb -c 'create extension postgis'
osm2pgsql -H dockerIP -U postgres -d mydb -s -S ./osm_stylesheet /home/ramnikov/Downloads/hessen-latest.osm
I would like to have a way to enter into the Postgresql container and get a data dump from it.
Use the following command from a UNIX or a Windows terminal:
docker exec <container_name> pg_dump <schema_name> > backup
The following command will dump only inserts from all tables:
docker exec <container_name> pg_dump --column-inserts --data-only <schema_name> > inserts.sql
I have container named postgres with mounted volume -v /backups:/backups
To backup gziped DB my_db I use:
docker exec postgres pg_dump -U postgres -F t my_db | gzip >/backups/my_db-$(date +%Y-%m-%d).tar.gz
Now I have
user#my-server:/backups$ ls
my_db-2016-11-30.tar.gz
Although the mountpoint solution above looked promising, the following is the only solution that worked for me after multiple iterations:
docker run -it -e PGPASSWORD=my_password postgres:alpine pg_dump -h hostname -U my_user my_db > backup.sql
What was unique in my case: I have a password on the database that needs to be passed in; needed to pass in the tag (alpine); and finally the hosts version of the psql tools were different to the docker versions.
This one, using container_name instead of database_scheme's one, works for me:
docker exec {container_name} pg_dump -U {user_name} > {backup_file_name}
In instance, for me, database name, user and password are supposed declared in docker-compose.yaml
I wish it could help someone
for those who suffered with permissions, I used this following command with success to perform my dump:
docker exec -i MY_CONTAINER_NAME /bin/bash -c "PGPASSWORD=MY_PASSWORD pg_dump -Fc -h localhost -U postgres MY_DB_NAME" > /home/MY_USER/db-$(date +%d-%m-%y).backup
This will mount the pwd and include your environment variables
docker run -it --rm \
--env-file <(env) \
-w /working \
--volume $(pwd):/working \
postgres:latest /usr/bin/pg_dump -Fc -h localhost -U postgres MY_DB_NAME" > /working/db-$(date +%d-%m-%y).backup
Another workaround method is to start postgre sql with a mountpoint to the location of the dump in docker.
like docker run -v <location of the files>.
Then perform a docker inspect on the docker running container
docker inspect <image_id>
you can find "Volumes" tag inside and a corresponding location.Go to the location and you can find all the postgresql/mysql files.It worked for me.Let us know if that worked for you also.
Good luck
To run the container that has the Postgres user and password, you need to have preconfigured variables as container environment variable.
For example:
docker run -it --rm --link <container_name>:<data_container_name> -e POSTGRES_PASSWORD=<password> postgres /usr/bin/pg_dump -h <data_container_name> -d <database_name> -U <postgres_username> > dump.sql