i am trying to use Docker. So i installed in Docker postgresql image.
Until now, when i imported osm data into postql i used this command:
psql -U postgres mydb
CREATE EXTENSION postgis;
osm2pgsql -U postgres -d mydb -s -S ./osm_stylesheet /home/ramnikov/Downloads/hessen-latest.osm
How can i do the same inside Docker after this command
$ sudo docker exec -it postgresql sudo -u postgres psql
or before this command ?
Tnx
Andrey
You can execute osm2pgsql outside of Docker:
-H|--host Database server host name or socket location.
As well as psql:
-h, --host=HOSTNAME database server host or socket directory
Like this:
psql -h dockerIP -U postgres -d mydb -c 'create extension postgis'
osm2pgsql -H dockerIP -U postgres -d mydb -s -S ./osm_stylesheet /home/ramnikov/Downloads/hessen-latest.osm
Related
I take backup using
pg_dump db_production > postgres_db.dump
and then I copy it to localhost using scp.
Now when I import on my local db it gives an error
pg_restore: [archiver] input file appears to be a text format dump. Please use psql.
by using commad line
pg_restore -d db_development postgres_db.dump
From the pg_dump documentation:
Examples
To dump a database called mydb into a SQL-script file:
$ pg_dump mydb > db.sql
To reload such a script into a (freshly created) database named newdb:
$ psql -d newdb -f db.sql
To dump a database into a custom-format archive file:
$ pg_dump -Fc mydb > db.dump
To dump a database into a directory-format archive:
$ pg_dump -Fd mydb -f dumpdir
To reload an archive file into a (freshly created) database named newdb:
$ pg_restore -d newdb db.dump
From the pg_restore documentation:
Examples
Assume we have dumped a database called mydb into a custom-format dump file:
$ pg_dump -Fc mydb > db.dump
To drop the database and recreate it from the dump:
$ dropdb mydb
$ pg_restore -C -d postgres db.dump
The answer above didn't work for me, this worked:
psql db_development < postgres_db.dump
In order to create a backup using pg_dump that is compatible with pg_restore you must use the --format=custom / -Fc when creating your dump.
From the docs:
Output a custom-format archive suitable for input into pg_restore.
So your pg_dump command might look like:
pg_dump --file /tmp/db.dump --format=custom --host localhost --dbname my-source-database --username my-username --password
And your pg_restore command:
pg_restore --verbose --clean --no-acl --no-owner --host localhost --dbname my-destination-database /tmp/db.dump
For me when i try to restore from remote host i used
psql -U username -p 5432 -h 10.10.10.1 -d database < db.dump
worked fine. And if not remote just following command worked.
psql -d database < db.dump
For me, It's working like this one.
C:\Program Files\PostgreSQL\12\bin> psql -U postgres -p 5432 -d dummy -f C:\Users\Downloads\d2cm_test.sql
If you restore .SQL file.
Create a new database in pgAdmin.
Go to the terminal and navigate the folder/directory where your .sql file is located. And then write the following command in terminal.
Syntax:
supername user postgres psql newDatabasename < inputfile.sql
Examaple:
sudo -u postgres psql newDb < restoreDb.sql
I've got same error when tried to backup db with DBeaver. If anyone uses DBeaver interface instead of command line on Windows, make sure your selected format as tar during backup and restore settings.
If you have a full DB dump:
PGPASSWORD="your_pass" psql -h "your_host" -U "your_user" -d "your_database" -f backup.sql
If you have schemas kept separately, however, that won't work. Then you'll need to disable triggers for data insertion, akin to pg_restore --disable-triggers. You can then use this:
cat database_data_only.gzip | gunzip | PGPASSWORD="your_pass" psql -h "your_host" -U root "your_database" -c 'SET session_replication_role = replica;' -f /dev/stdin
On a side note, it is a very unfortunate downside of postgres, I think. The default way of creating a dump in pg_dump is incompatible with pg_restore. With some additional keys, however, it is. WTF?
if you use pg_dump with -Fp to backup in plain text format, use following command:
cat db.txt | psql dbname
to copy all data to your database with name dbname
psql -U <username> -d <database-name> -h <host-name> -f <backup.sql>
Providing a simple one line answer which worked for me and will work for you too for most cases
psql -U username -d database_name < dump_file.sql
If above gives role related errors then replace username with postgres.
psql -U postgres -d database_name < dump_file.sql
Probably when you create a backup you want to restore it in another network or create a remote restoration.
We need to create a backup file using the --format=custom [-Fc] to restore it using pg_restore. We can use a connection string postgresql://<user>:<pass>#localhost:5432/<dbname> and replace <user>, <pass>, and <dbname> with your information.
pg_dump -v -Fc \
postgresql://<user>:<pass>#localhost:5432/<dbname> \
> db-20211122-163508.sql
To restore we will call it using --clean [-c] and --create [-C] to drop the database before restoring. Replace <user>, <host>, <port>, and <dbname> with your information.
pg_restore -vcC \
-U <user> \
-h <host> \
-p <port> \
-d <dbname> \
< db-20211122-163508.sql
If you backup with this way, I think this will be more easy to import database.
pg_dump -h (remote db address) -a --column-inserts -U postgres (database name) > (file name).sql
For import,
psql
-f (file name).sql
--host (remote db address)
--port 5432
--username postgres
--password (your password)
--dbname (database you want to import)
I've been struggling with this as well. This is the combination of dump & restore commands that worked for me:
pg_dump -Ft -C -h database_host -U username database > DATA.dump
To restore
pg_restore -x --no-owner -d database DATA.dump
Remove the -x flag if you want to keep the same access privileges (ACLs) in your DB. You must have the same roles and users in the database for this.
https://www.postgresql.org/docs/15/app-pgdump.html
https://www.postgresql.org/docs/15/app-pgrestore.html
here is the solution,
pg_restore -U username -p 5432 -h 10.10.10.1 -d database_name < dump_file
I create backups like this: docker exec DOCKER pg_dump -U USER -F t DB | gzip > ./FILE.tar.gz
What's the best way to restore the database given that the database runs within a container?
For your case:
docker exec -it <CONTAINER> gunzip < backup.tar.gz | pg_restore -U <USER> -F t -d <DB>
Remote restore is also available if your container is public facing and remote connections are allowed in pg_hba.conf for postresql:
gunzip < backup.tar.gz | pg_restore -U <USER> -F t -d <DB> -h <HOST_IP> -p 5432
As a rule of thumb, it is good idea to document your backup and restore commands specific to the project.
How take backup of the data which is existing in the running PostgreSQL container
Create some folder in your root
mkdir -p '/myfolder/bdbackup'
download the postgres image which you are using and execute the following command
docker run --name demo1 -e POSTGRES_PASSWORD=passowrd -v /myfolder/bdbackup:/var/lib/postgresql/data -d postgres
docker exec -it demo1 psql -U postgres
Back up will be stored in the following folder /myfolder/bdbackup
you can kill the container and stop the container any time but data will be stored in the host.
and once again re-run the postgres the container with same command
docker run --name demo2 -e POSTGRES_PASSWORD=passowrd -v /myfolder/bdbackup:/var/lib/postgresql/data -d postgres
docker exec -it demo1 psql -U postgres
and execute following query select * from emp;
you can see the data has restored...
I've got my own machine with postgres dmp file, which I want to restore on the remote virtual machine (e.g. ip is 192.168.0.190 and postgres port is 5432) in my network. Is it possible to restore this dump using pg_restore without copying dump to remote machine? Because the size of dump about 12GB and the disk space on the virtual machine is 20GB.
Thanks
You can run a restore over the network without copying the dump to the remote host.
Just invoke pg_restore with -h <hostname> and -p <port> (and probably -U <username> to authenticate as different user) on the host you got the dump file, for example:
pg_restore -h 192.168.0.190 -p 5432 -d databasename -U myuser mydump.dump
References:
pg_restore documentation
Alternatively, you can use psql:
psql -h 192.168.0.190 -p 5432 -d <dbname> -U <username> -W -f mydump.dump
An example for a remote RDS instance on AWS
psql -h mydb.dsdreetr34.eu-west-1.rds.amazonaws.com -p 5432 -d mydbname -U mydbuser -W -f mydatabase-dump.sql
-f, --file=FILENAME execute commands from file, then exit
-W, --password force password prompt (should happen automatically)
You can pass the password parameter in your script before "pg_restore" using PGPASSWORD="your_database_password"
I run this and works to me:
scp backup.dump user#remotemachine:~
ssh user#remotemachine "pg_restore -h localhost -p 5432 -U databaseuser -W -F c -d databasename -v backup.dump"
You can write a script to automate this.
I have a backup of a database that I would to restore to a postgres database running inside a docker container.
I'm using docker-machine on OS X.
Postgres image is postgres:9.4.
This is the script I've come up with so far:
pg_restore --verbose --clean --no-acl --no-owner \
-h tcp://`docker-machine ip default`:5432 \
-U postgres \
-d tonsser-api_development latest.dump
But that doesn't work. I get the error:
pg_restore: connecting to database for restore
pg_restore: [archiver (db)] connection to database "tonsser-api_development" failed: could not translate host name "tcp://192.168.99.100:5432" to address: nodename nor servname provided, or not known
There seems to be no proper solution to do this on runtime, however copying the dumpfile to the container:
docker cp dumpfile DBcontainer:/dumpfile
and restore from within the container:
docker exec -i -t DBcontainer /bin/bash
psql DBname < dumpfile
worked well for a one time use...
You have to use a connection string, e.g. $DATABASE_URL
The following works well for me:
docker-compose exec fooapp pg_restore
--verbose --clean --no-acl --no-owner
-d $DATABASE_URL foo.dump
For extra points, you can create a rake task (assuming you're using a Rails app), as follows:
namespace :db do
desc 'Import a given file into the database'
task :import, [:path] => :environment do |_t, args|
dump_path = args.path
system 'pg_restore --verbose --clean --no-acl --no-owner -d $DATABASE_URL '\
+ dump_path
end
end
Put that file in lib/tasks, then you can call something like:
docker-compose exec fooapp bundle exec rails db:import[foo.dump]
Maybe because you have to specify host and port separately?
-h `docker-machine ip default` -p 5432
See docs
I've got my own machine with postgres dmp file, which I want to restore on the remote virtual machine (e.g. ip is 192.168.0.190 and postgres port is 5432) in my network. Is it possible to restore this dump using pg_restore without copying dump to remote machine? Because the size of dump about 12GB and the disk space on the virtual machine is 20GB.
Thanks
You can run a restore over the network without copying the dump to the remote host.
Just invoke pg_restore with -h <hostname> and -p <port> (and probably -U <username> to authenticate as different user) on the host you got the dump file, for example:
pg_restore -h 192.168.0.190 -p 5432 -d databasename -U myuser mydump.dump
References:
pg_restore documentation
Alternatively, you can use psql:
psql -h 192.168.0.190 -p 5432 -d <dbname> -U <username> -W -f mydump.dump
An example for a remote RDS instance on AWS
psql -h mydb.dsdreetr34.eu-west-1.rds.amazonaws.com -p 5432 -d mydbname -U mydbuser -W -f mydatabase-dump.sql
-f, --file=FILENAME execute commands from file, then exit
-W, --password force password prompt (should happen automatically)
You can pass the password parameter in your script before "pg_restore" using PGPASSWORD="your_database_password"
I run this and works to me:
scp backup.dump user#remotemachine:~
ssh user#remotemachine "pg_restore -h localhost -p 5432 -U databaseuser -W -F c -d databasename -v backup.dump"
You can write a script to automate this.