I have access to two postgres database servers on different hosts. On server A I access the client using:
psql -h localhost -U user -W db_name
db_name=>
And on the second host B I access the client using (docker image):
docker run -it --rm --network fiware_default jbergknoff/postgresql-client\
postgresql://postgres:password#postgres-db:5432/postgres
postgres=#
Now I need to dump database file copied from A (now on B) using:
psql -U postgres -d targetdb -f sourcedb.sql
However, the command psql isn´t recognised second host B. I mean I cannot run commands using psql B
what is then the difference between psqland postgres-clienthere please?
The docker image postgresql-client has psql defined as an entrypoint. See https://github.com/jbergknoff/Dockerfile/blob/master/postgresql-client/Dockerfile#L3 .
So you basically ran psql psql and psql does not understand that. Just leave psql out and start straight with the args.
You can read up on CMD vs ENTRYPOINT here What is the difference between CMD and ENTRYPOINT in a Dockerfile? or here http://goinbigdata.com/docker-run-vs-cmd-vs-entrypoint/ .
Related
I have multiple Environment Variables defined on my Postgres container, such as POSTGRES_USER. The container is running and I want to connect to Postgres from the command line using exec.
I'm unable to connect with the following:
docker exec -it <CONTAINER-ID> psql -U $POSTGRES_USER -d <DB NAME>
I understand that the variable is defined on the container and the following does work:
docker exec -it <CONTAINER-ID> bash -c 'psql -U $POSTGRES_USER -d <DB NAME>'
Is there a way for me to execute the psql command directly from docker exec and call the environment variable on the container?
docker exec -it <CONTAINER-ID> psql -U ????? -d <DB NAME>
Depending on your use case, what you could do, instead of passing a user to the psql command is to define the environment variable PGUSER to the container at boot time.
This way, it will be the default user for PostgreSQL, if you do not specify any, so you won't even have to specify it in order to connect:
$ docker run --name postgres -e POSTGRES_PASSWORD=bar -e POSTGRES_USER=foo -e PGUSER=foo -d postgres
e250f0821613a5e2021e94772a732f299874fc7a16b340ada4233afe73744423
$ docker exec -ti postgres psql -d postgres
psql (12.4 (Debian 12.4-1.pgdg100+1))
Type "help" for help.
postgres=#
The reason this isn't working for you is because when you run the command
docker exec -it <CONTAINER-ID> psql -U $POSTGRES_USER -d <DB NAME>
You're running it on your host. So, $POSTGRES_USER refers to the environment variable on your host, not your container. That variable isn't set on your host.
The second command
docker exec -it <CONTAINER-ID> bash -c 'psql -U $POSTGRES_USER -d <DB NAME>'
works because you're passing the command in the quotes to the shell in the container, where that variable actually exists.
The method in the second command is the way to do what you're trying to do, unless you set the variable on your host somehow and make sure it has the same value as it does in your container.
The easiest way to do this would be to reference your host variable at image build time.
So, in your Dockerfile, if you write ENV POSTGRES_USER=${POSTGRES_USER} it will look in the host environment for that value, and use it.
If you set the variables this way, then your command will work.
I create backups like this: docker exec DOCKER pg_dump -U USER -F t DB | gzip > ./FILE.tar.gz
What's the best way to restore the database given that the database runs within a container?
For your case:
docker exec -it <CONTAINER> gunzip < backup.tar.gz | pg_restore -U <USER> -F t -d <DB>
Remote restore is also available if your container is public facing and remote connections are allowed in pg_hba.conf for postresql:
gunzip < backup.tar.gz | pg_restore -U <USER> -F t -d <DB> -h <HOST_IP> -p 5432
As a rule of thumb, it is good idea to document your backup and restore commands specific to the project.
How take backup of the data which is existing in the running PostgreSQL container
Create some folder in your root
mkdir -p '/myfolder/bdbackup'
download the postgres image which you are using and execute the following command
docker run --name demo1 -e POSTGRES_PASSWORD=passowrd -v /myfolder/bdbackup:/var/lib/postgresql/data -d postgres
docker exec -it demo1 psql -U postgres
Back up will be stored in the following folder /myfolder/bdbackup
you can kill the container and stop the container any time but data will be stored in the host.
and once again re-run the postgres the container with same command
docker run --name demo2 -e POSTGRES_PASSWORD=passowrd -v /myfolder/bdbackup:/var/lib/postgresql/data -d postgres
docker exec -it demo1 psql -U postgres
and execute following query select * from emp;
you can see the data has restored...
I tried:
$ alias psql="docker exec -ti pg-hello-phoenix sh -c 'exec psql -h localhost -p 5432 -U postgres'"
$ mix ecto.create
but got:
** (RuntimeError) could not find executable psql in path, please guarantee it is available before running ecto commands
lib/ecto/adapters/postgres.ex:106: Ecto.Adapters.Postgres.run_with_psql/2
lib/ecto/adapters/postgres.ex:83: Ecto.Adapters.Postgres.storage_up/1
lib/mix/tasks/ecto.create.ex:34: anonymous fn/2 in Mix.Tasks.Ecto.Create.run/1
(elixir) lib/enum.ex:604: Enum."-each/2-lists^foreach/1-0-"/2
(elixir) lib/enum.ex:604: Enum.each/2
(mix) lib/mix/cli.ex:58: Mix.CLI.run_task/2
(elixir) lib/code.ex:363: Code.require_file/2
Also I tried to create symlink /usr/local/bin/psql:
#!/usr/bin/env bash
docker exec -ti pg-hello-phoenix sh -c "exec psql -h localhost -p 5432 -U postgres $#"
and then:
$ sudo chmod +x /usr/local/bin/psql
check:
$ which psql
/usr/local/bin/psql
$ psql --version
psql (PostgreSQL) 9.5.1
run again:
$ mix ecto.create
** (Mix) The database for HelloPhoenix.Repo couldn't be created, reason given: cannot enable tty mode on non tty input
.
Container with PostgreSQL is launched:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
013464d7227e postgres "/docker-entrypoint.s" 47 minutes ago Up 47 minutes 5432/tcp pg-hello-phoenix
I was able to do this by going into /config/.exs In my case it was development, so /config/dev.exs and left the hostname as localhost but added another setting for port: 32768 because that's the port that docker exposed.
Make sure to put a space between the port: and the number (not string). Otherwise it won't work.
Worked as usual after that. The natural assumption is that the username / password matches on the container as well.
To me I did the following:
sudo docker exec -it postgres-db bash
After I got the interactive shell
psql -h localhost -p 5432 -U postgres
Then I create my db manually:
CREATE DATABASE cars_dev;
Then finally:
mix ecto.migrate
Everything worked fine after that :) hope it helps.
I would like to have a way to enter into the Postgresql container and get a data dump from it.
Use the following command from a UNIX or a Windows terminal:
docker exec <container_name> pg_dump <schema_name> > backup
The following command will dump only inserts from all tables:
docker exec <container_name> pg_dump --column-inserts --data-only <schema_name> > inserts.sql
I have container named postgres with mounted volume -v /backups:/backups
To backup gziped DB my_db I use:
docker exec postgres pg_dump -U postgres -F t my_db | gzip >/backups/my_db-$(date +%Y-%m-%d).tar.gz
Now I have
user#my-server:/backups$ ls
my_db-2016-11-30.tar.gz
Although the mountpoint solution above looked promising, the following is the only solution that worked for me after multiple iterations:
docker run -it -e PGPASSWORD=my_password postgres:alpine pg_dump -h hostname -U my_user my_db > backup.sql
What was unique in my case: I have a password on the database that needs to be passed in; needed to pass in the tag (alpine); and finally the hosts version of the psql tools were different to the docker versions.
This one, using container_name instead of database_scheme's one, works for me:
docker exec {container_name} pg_dump -U {user_name} > {backup_file_name}
In instance, for me, database name, user and password are supposed declared in docker-compose.yaml
I wish it could help someone
for those who suffered with permissions, I used this following command with success to perform my dump:
docker exec -i MY_CONTAINER_NAME /bin/bash -c "PGPASSWORD=MY_PASSWORD pg_dump -Fc -h localhost -U postgres MY_DB_NAME" > /home/MY_USER/db-$(date +%d-%m-%y).backup
This will mount the pwd and include your environment variables
docker run -it --rm \
--env-file <(env) \
-w /working \
--volume $(pwd):/working \
postgres:latest /usr/bin/pg_dump -Fc -h localhost -U postgres MY_DB_NAME" > /working/db-$(date +%d-%m-%y).backup
Another workaround method is to start postgre sql with a mountpoint to the location of the dump in docker.
like docker run -v <location of the files>.
Then perform a docker inspect on the docker running container
docker inspect <image_id>
you can find "Volumes" tag inside and a corresponding location.Go to the location and you can find all the postgresql/mysql files.It worked for me.Let us know if that worked for you also.
Good luck
To run the container that has the Postgres user and password, you need to have preconfigured variables as container environment variable.
For example:
docker run -it --rm --link <container_name>:<data_container_name> -e POSTGRES_PASSWORD=<password> postgres /usr/bin/pg_dump -h <data_container_name> -d <database_name> -U <postgres_username> > dump.sql
I have some .sql files with thousands of INSERT statements in them and need to run these inserts on my PostgreSQL database in order to add them to a table. The files are that large that it is impossible to open them and copy the INSERT statements into an editor window and run them there. I found on the Internet that you can use the following by navigating to the bin folder of your PostgreSQL install:
psql -d myDataBase -a -f myInsertFile
In my case:
psql -d HIGHWAYS -a -f CLUSTER_1000M.sql
I am then asked for a password for my user, but I cannot enter anything and when I hit enter I get this error:
psql: FATAL: password authentication failed for user "myUsername"
Why won't it let me enter a password. Is there a way round this as it is critical that I can run these scripts?
I got around this issue by adding a new entry in my pg_hba.conf file with the following structure:
# IPv6 local connections:
host myDbName myUserName ::1/128 trust
The pg_hba.conf file can usually be found in the 'data' folder of your PostgreSQL install.
Of course, you will get a fatal error for authenticating, because you do not include a user name...
Try this one, it is OK for me :)
psql -U username -d myDataBase -a -f myInsertFile
If the database is remote, use the same command with host
psql -h host -U username -d myDataBase -a -f myInsertFile
You should do it like this:
\i path_to_sql_file
See:
You have four choices to supply a password:
Set the PGPASSWORD environment variable. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-envars.html
Use a .pgpass file to store the password. For details see the manual: http://www.postgresql.org/docs/current/static/libpq-pgpass.html
Use "trust authentication" for that specific user: http://www.postgresql.org/docs/current/static/auth-methods.html#AUTH-TRUST
Since PostgreSQL 9.1 you can also use a connection string: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
Use this to execute *.sql files when the PostgreSQL server is located in a difference place:
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
Then you are prompted to enter the password of the user.
EDIT: updated based on the comment provided by #zwacky
If you are logged in into psql on the Linux shell the command is:
\i fileName.sql
for an absolute path and
\ir filename.sql
for the relative path from where you have called psql.
export PGPASSWORD=<password>
psql -h <host> -d <database> -U <user_name> -p <port> -a -w -f <file>.sql
Via the terminal log on to your database and try this:
database-# >#pathof_mysqlfile.sql
or
database-#>-i pathof_mysqlfile.sql
or
database-#>-c pathof_mysqlfile.sql
You can give both user name and PASSSWORD on the command line itself with the "-d" parameter
psql -d "dbname='urDbName' user='yourUserName' password='yourPasswd' host='yourHost'" -f yourFileName.sql
you could even do it in this way:
sudo -u postgres psql -d myDataBase -a -f myInsertFile
If you have sudo access on machine and it's not recommended for production scripts just for test on your own machine it's the easiest way.
2021 Solution
if your PostgreSQL database is on your system locally.
psql dbname < sqldump.sql username
If its hosted online
psql -h hostname dbname < sqldump.sql username
If you have any doubts or questions, please ask them in the comments.
Walk through on how to run an SQL on the command line for PostgreSQL in Linux:
Open a terminal and make sure you can run the psql command:
psql --version
which psql
Mine is version 9.1.6 located in /bin/psql.
Create a plain textfile called mysqlfile.sql
Edit that file, put a single line in there:
select * from mytable;
Run this command on commandline (substituting your username and the name of your database for pgadmin and kurz_prod):
psql -U pgadmin -d kurz_prod -a -f mysqlfile.sql
The following is the result I get on the terminal (I am not prompted for a password):
select * from mytable;
test1
--------
hi
me too
(2 rows)
psql -h localhost -d userstoreis -U admin -p 5432 -a -q -f /home/jobs/Desktop/resources/postgresql.sql
Parameter explanations:
-h PostgreSQL server IP address
-d database name
-U user name
-p port which PostgreSQL server is listening on
-f path to SQL script
-a all echo
-q quiet
You can open a command prompt and run as administrator. Then type
../bin>psql -f c:/...-h localhost -p 5432 -d databasename -U "postgres"
Password for user postgres: will show up.
Type your password and enter. I couldn't see the password what I was typing, but this time when I press enter it worked. Actually I was loading data into the database.
I achived that wrote (located in the directory where my script is)
::someguy#host::$sudo -u user psql -d my_database -a -f file.sql
where -u user is the role who owns the database where I want to execute the script then the psql connects to the psql console after that -d my_database loads me in mydatabase finally -a -f file.sql where -a echo all input from the script and -f execute commands from file.sql into mydatabase, then exit.
I'm using:
psql (PostgreSQL) 10.12
on (Ubuntu 10.12-0ubuntu0.18.04.1)
A small improvement in #wingman__7 's 2021 answer: if your username contains certain characters (an underscore in my case), you need to pass it with the -U flag.
This worked for me:
$ psql -h db.host -d db_name -U my_user < query.sql
Try using the following command in the command line console:
psql -h localhost -U postgres -f restore.sql