I'm moving an application from Heroku into Docker containers, and am trying to figure out how to migrate the data from my Heroku db into my new Postgres container.
Attempt so far
I have copied a dump of the db into the Postgres container.
docker cp latest.dump mycontainer:/latest.dump
However when I try to pg_restore it, I'm getting errors as I attempt to run a docker exec command.
docker exec <pg_container> pg_restore -d <db_name> latest.dump
pg_restore: [archiver (db)] connection to database "<db_name>" failed: FATAL: role "root" does not exist
Or when I try to run with the db user:
docker exec <pg_container> -u <db_user> pg_restore -d <db_name> latest.dump
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"-u\": executable file not found in $PATH": unknown
Question
What is the correct way to go about loading this data into the new database within the docker container?
I was able to find the solution as a one line command:
docker exec <container_name> pg_restore --verbose --clean --no-acl --no-owner -h postgres --dbname=postgresql://<user>:<password>#127.0.0.1:<port>/<db_name> latest.dump
latest.dump is the name of my db dump from Heroku so that can be changed to whatever the name of your dump is.
Related
I used to create postgresql database for my old project with command:
docker run --name oldpostgresqldb -e POSTGRES_USER=oldadmin -e POSTGRES_PASSWORD=secret -p 5432:5432 -v /data:/var/lib/postgresql/data -d postgres
Then create database with command:
docker exec -i oldpostgresqldb psql -U oldadmin -c "CREATE DATABASE oldDB WITH ENCODING='UTF8' OWNER=oldadmin;"
When I started my new project stoped and removed all containers, images, volumes etc. and even used docker system prune.
Now I am trying to create new container and db with commands:
docker run --name newpostgresqldb -e POSTGRES_USER=newadmin -e POSTGRES_PASSWORD=secret -p 5432:5432 -v /data:/var/lib/postgresql/data -d postgres
and
docker exec -i newpostgresqldb psql -U newadmin -c "CREATE DATABASE newDB WITH ENCODING='UTF8' OWNER=newadmin;"
But I receive psql: error: FATAL: role "newadmin" does not exist.
Furthermore, I still can manage oldDB with oldadmin user in newpostgresqldb container, because user oldadmin is still exists.
How can I delete old data and create new user and database using docker?
Thanks to #Ay0 I've found a solution. I just needed to clear content of /data folder in the host directory manualy.
I have a dockerized Postgres db. I have successfully executed
docker exec -t your-db-container pg_dumpall -c -U postgres > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql
and generated .sql file. But when I'm trying to restore with the following script:
cat my_dump.sql | docker exec -i your-db-container psql --username="myusername" mydb
I get:
ERROR: syntax error at or near "pg_dumpall"
LINE 1: pg_dumpall: error: could not connect to database "template1"...
What am I missing?
The dump file contains an error message from the backup procedure instead of containing a database dump.
docker exec combines the standard output and the standard error. Its output cannot be trusted for a backup file.
Aside from solving the root problem that pg_dumpall in the container cannot connect to template1, you want a more sophisticated dump procedure that cannot create this situation where a shell error message ends up where SQL statements should be.
I'm new with docker and i have to learn at my new job, so i should restore a sql file into a postgres container, when i type the command:
docker exec -i postgres-container pg_restore -U postgres -d postgres /var/lib/postgresql/data/_postgres_2020-11-09T02_00_06Z.sql
i get the following message:
pg_restore: error: input file appears to be a text format dump. Please use psql.
After that i tried to convert that file with psql inside the container but i get this message:
psql: error: FATAL: role "root" does not exist.
How can i get this? can someone help me ?
I got it! activereality's answer solve my problem in this post Backup/Restore a dockerized PostgreSQL database
cat your_dump.sql | docker exec -i your-db-container psql -U postgres -d dbname
I am trying to run postgres in a docker container and create an user & database by linking to the running postgress container. Once the user is created I want to provide those details as part of running a Pact brocker instance. However when I run the shell script I am getting error connecting to the postgres container and running ddl queries.
Below are script and error details.
Script
docker run --name pactbroker-db -e POSTGRES_PASSWORD=XXXXX -e POSTGRES_USER=admin -v ~/pact/data:/var/data -d postgres
docker run -it --link pactbroker-db:postgres --rm postgres sh -c 'export PGPASSWORD=XXXXX; exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U admin'
CREATE USER pactbrokeruser WITH PASSWORD 'YYYYY';
CREATE DATABASE pactbroker WITH OWNER pactbrokeruser;
GRANT ALL PRIVILEGES ON DATABASE pactbroker TO pactbrokeruser;
\q
docker run --name pactbroker --link pactbroker-db:postgres -e PACT_BROKER_DATABASE_USERNAME=pactbrokeruser -e PACT_BROKER_DATABASE_PASSWORD=YYYYY -e PACT_BROKER_DATABASE_HOST=postgres -e PACT_BROKER_DATABASE_NAME=pactbroker -d -p 80:80 dius/pact-broker
Error:
./install_pact_postgress.sh
fae1ae3b569a75c33ed5cb7d0faad665c038eb03d9b62c1e2df694f0ce162c5c
psql: error: could not connect to server: could not connect to server: Connection refused
Is the server running on host "XXX.17.0.2" and accepting
TCP/IP connections on port 5432?
./install_pact_postgress.sh: line 6: CREATE: command not found
./install_pact_postgress.sh: line 7: CREATE: command not found
./install_pact_postgress.sh: line 8: GRANT: command not found
./install_pact_postgress.sh: line 10: q: command not found
I am new to docker and not sure where I am going wrong.
Please help.
For database initialization use official postgres docker image feature, which run .sql scripts found in the /docker-entrypoint-initdb.d/ folder
So create your initialization script at your host machine ~/pact/initscriptdir and mount it to that directory starting postrgres.
docker run --name pactbroker-db -e POSTGRES_PASSWORD=XXXXX -e POSTGRES_USER=admin -v ~/pact/initscriptdir:/docker-entrypoint-initdb.d -d postgres
I usually attach to postgres by typing in the terminal:
psql -U user db
I want to move postgres into docker container:
docker run -v /var/lib/postgresql:/var/lib/postgresql -p 5432:5432 postgres
Then I run command and have an error:
psql -h localhost -p 5432 -U user
psql: FATAL: role "user" does not exist
I expected that -v /var/lib/postgresql:/var/lib/postgresql will make possible to reach db by user. But it doesn't happen. How to make it correctly?