I'm trying to get Postgresql 10.0 working in a docker container. I have the following Dockerfile:
FROM postgres:10.0-alpine
RUN apk add openrc --no-cache
USER postgres
RUN /etc/init.d/postgresql start
RUN psql --command "IF NOT EXISTS (SELECT FROM pg_catalog.pg_user WHERE usename = 'user') THEN CREATE USER user WITH SUPERUSER ENCRYPTED PASSWORD 'password'; END IF;"
RUN createdb main
EXPOSE 5432
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
CMD ["/usr/lib/postgresql/10.0/bin/postgres", "-D", "/var/lib/postgresql/10.0/main", "-c", "config_file=/etc/postgresql/10.0/main/postgresql.conf"]
I got following error:
/bin/sh: /etc/init.d/postgresql: not found
Seems that /etc/init.d/postgresql is really missing. What am I doing wrong?
I finished with this Dockerfile:
FROM postgres:10.0-alpine
USER postgres
RUN chmod 0700 /var/lib/postgresql/data &&\
initdb /var/lib/postgresql/data &&\
echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf &&\
echo "listen_addresses='*'" >> /var/lib/postgresql/data/postgresql.conf &&\
pg_ctl start &&\
psql -U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'main'" | grep -q 1 || psql -U postgres -c "CREATE DATABASE main" &&\
psql -c "ALTER USER postgres WITH ENCRYPTED PASSWORD 'mysecurepassword';"
EXPOSE 5432
This dockerfile create database 'main' (if not exists), starts postgres and sets default user password
I was able to do it using exec commands, so I can run postgres in a running container, in this case alpine 3.7. Note $CONTAINER_NAME is the container id from docker ps:
# Install postgresql, create user, db & start the daemon (for testing)
sudo docker exec $CONTAINER_NAME sh -c 'apk add postgresql'
sudo docker exec $CONTAINER_NAME sh -c 'addgroup -S postgres && adduser -S postgres -G postgres'
sudo docker exec $CONTAINER_NAME sh -c 'mkdir -p /var/lib/postgresql/data'
sudo docker exec $CONTAINER_NAME sh -c 'mkdir -p /run/postgresql/'
sudo docker exec $CONTAINER_NAME sh -c 'chown -R postgres:postgres /run/postgresql/'
sudo docker exec $CONTAINER_NAME sh -c 'chmod -R 777 /var/lib/postgresql/data'
sudo docker exec $CONTAINER_NAME sh -c 'chown -R postgres:postgres /var/lib/postgresql/data'
sudo docker exec --user postgres $CONTAINER_NAME sh -c 'initdb /var/lib/postgresql/data'
sudo docker exec --user postgres $CONTAINER_NAME sh -c 'echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf'
sudo docker exec --user postgres $CONTAINER_NAME sh -c 'pg_ctl start -D /var/lib/postgresql/data -l /var/lib/postgresql/log.log'
sudo docker exec --user postgres $CONTAINER_NAME sh -c "psql --command \"ALTER USER postgres WITH ENCRYPTED PASSWORD 'buildpgpass';\""
sudo docker exec --user postgres $CONTAINER_NAME sh -c "psql --command \"CREATE DATABASE builddb;\""
You can also include this in your Dockerfile like so:
# Postgres
RUN apk add postgresql=11.1-r0
RUN (addgroup -S postgres && adduser -S postgres -G postgres || true)
RUN mkdir -p /var/lib/postgresql/data
RUN mkdir -p /run/postgresql/
RUN chown -R postgres:postgres /run/postgresql/
RUN chmod -R 777 /var/lib/postgresql/data
RUN chown -R postgres:postgres /var/lib/postgresql/data
RUN su - postgres -c "initdb /var/lib/postgresql/data"
RUN echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf
RUN su - postgres -c "pg_ctl start -D /var/lib/postgresql/data -l /var/lib/postgresql/log.log && psql --command \"ALTER USER postgres WITH ENCRYPTED PASSWORD 'postgres';\" && psql --command \"CREATE DATABASE builddb;\""
This seems to work great if you want to test something out on the fly.
There are multiple problems with your dockerfile. I guess you are trying to start the server when building the image to create the main database. This won't work since each command in the dockefile will execute in its own layer, and thus when you reach RUN psql ... the database won't be started since it was started in a different layer. So you need to group the commands in one line.
Second problem is that the file /etc/init.d/postgresql does not exist. The server can be started using the postgres command:
RUN postgres &\
psql --command "IF NOT EXISTS (SELECT FROM pg_catalog.pg_user WHERE usename = 'user')\
THEN CREATE USER user WITH SUPERUSER ENCRYPTED PASSWORD 'password'; END IF;" &\
createdb main
Related
I am running postgreSQL on docker.
In order to access the container I run
winpty docker exec -it postgres_db bash
Now that I am inside the docker container, I run this to access as user 'postgres' in postgres server
psql -U postgres
Then what I want to do is create a database
create database test;
I can I do this sequentially in a script?
I want something like this:
#!/bin/bash
winpty docker run -p 8005:5432 --name postgres_db -e POSTGRES_PASSWORD=password -d postgres
sleep 5
winpty docker exec -it postgres_db bash -c "psql -U postgres" [-c "create database test;"]
create database cannot be executed if im not inside "psql -U postgres". Obviously the last line -c "create database test;" is wrong, just to make you understand what I want to do.
winpty docker exec -it postgres_db bash -c "psql -U postgres -c 'CREATE DATABASE test;'"
Both bash -c and psql -c take one shell word as input. A single or double quoted string is one word. You need to nest the quotes, just like the -c commands are nested.
Alternative:
winpty docker exec -it postgres_db bash -c "psql -U postgres -c \"CREATE DATABASE test;\""
I want to execute these lines in the below .sh file:
#!/bin/sh +x
sudo su postgres
psql -U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'test_db'" | grep -q 1 || psql -U postgres -c "CREATE DATABASE test_db"
But only 'sudo su postgres' is executing and the 2nd line is not executing. Can someone help me execute those 2 lines
Just pass your psql command to su:
sudo su postgres -c "psql -U postgres -tc \"SELECT 1 FROM pg_database WHERE datname = 'test_db'\" | \
grep -q 1 || psql -U postgres -c \"CREATE DATABASE test_db\""
If you want to pass the datname value as a parameter, replace it by $1 in the script ($1, $2, etc. expand as the first, second, etc. parameters you pass to your script):
sudo su postgres -c "psql -U postgres -tc \"SELECT 1 FROM pg_database WHERE datname = '$1'\" | \
grep -q 1 || psql -U postgres -c \"CREATE DATABASE $1\""
and call your script like this:
./script.sh test_db
You can even have optional parameters. For instance, if you want an optional parameter for the table name (pg_database in your example):
db=pg_database
if [ -n "$2" ]; then db="$2"; fi
sudo su postgres -c "psql -U postgres -tc \"SELECT 1 FROM $db WHERE datname = '$1'\" | \
grep -q 1 || psql -U postgres -c \"CREATE DATABASE $1\""
and call your script like this:
./script.sh test_db
to use the default, else:
./script.sh test_db other_pg_database
I suggest you update your sudo configuration so you can directly express that you only need to as postgres and not root:
sudo -u postgres ...
For example:
user host = (postgres) command
where user is your user, host is the host name you want this apply (ALL?), and command is the name of your command (ALL?) possible prefixed with "NOPASSWD: " if you don't want to require a password.
Then do the action and deal with the error if needed instead of guarding against it:
sudo -u postgres bash -c "psql -U postgres -c "CREATE DATABASE $1"
Possible single quoting the $1 and if you want that to be robust, escape any single quotes in the database name. I showed you a shell (bash) in the above, so can easily tag on error handling.
I'm having trouble in creating a Postgres DB using this bash script:
#! /bin/bash
docker pull postgres
docker run --name coverage-postgres -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgres
export CONTAINER_ID=$(sudo docker ps -a | grep coverage-postgres | head -c12)
sleep 2s
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "create user coverage_user with password 'password';"
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "create database coverage owner coverage_user;"
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "grant all privileges on database coverage to coverage_user;"
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "\c coverage coverage_user" # it seems useless...
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "CREATE TABLE IF NOT EXISTS postal_codes (id,...;"
sleep 0.5
sudo docker exec -it $CONTAINER_ID psql -U postgres -c "CREATE UNIQUE INDEX ... ;"
# exit from container
exit
# restart container
docker start $CONTAINER_ID
In particular, the database is created, the user is created, the table is created but... it's not in the coverage db but in postgres db.
I've tried to add "CREATE TABLE coverage.postal_codes" but coverage is a db and not a schema and it didn't work.
I've tried to use: psql -U coverage_user but the system tells me that database coverage_user doesn't exist.
So of course I thought "I have to specify the database of course!". Then I've tried to use: psql -U coverage as the name of the database but this time, the system makes fun of me and, changing its mind, tells me that the role coverage doesn't exists.
I tried a workaround: within the command -c "\c coverage coverage_user" I concatenated the other commands this way:
-c "\c coverage coverage_user; CREATE TABLE...; CREATE UNIQUE INDEX...;"
but, of course, neither this worked at all.
I make a premise: I know there are other ways to do this but I would like to understand what I am missing with these specific commands.
Solution
#! /bin/bash
docker pull postgres
docker run --name coverage-postgres -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgres
export CONTAINER_ID=$(sudo docker ps -a | grep coverage-postgres | head -c12)
sleep 2s
docker exec -it $CONTAINER_ID psql -U postgres -c "create user coverage_user with password 'password';"
sleep 0.5
docker exec -it $CONTAINER_ID psql -U postgres -c "create database coverage owner coverage_user;"
sleep 0.5
docker exec -it $CONTAINER_ID psql -U postgres -c "grant all privileges on database coverage to coverage_user;"
sleep 0.5
docker exec -it $CONTAINER_ID psql -U coverage_user -c "CREATE TABLE IF NOT EXISTS postal_codes (id int)" coverage
Explanation
https://www.postgresql.org/docs/9.2/app-psql.html
psql [option...] [dbname [username]]
Just add dbname after options. And change user as -U option. You can pass dbname also as an rgument -d
I tried:
ssh root#mytutumnode "docker exec -ti mydockercontainerid su postgres -c 'psql mydatabasename'" < mydump.sql
but it executes only
ssh root#mytutumnode "docker exec -ti mydockercontainerid su postgres -c 'psql mydatabasename'"
... and omits < mydump.sql part.
psql --list and alikes do work.
Here's the Makefile i ended up using:
# Convenience cli shortcuts
LOCAL_DOCKER_IP = 192.168.99.100
STAGING_IP = DOCKER_NODE_IP_HERE
RELEASE_IP = IP_FROM_THE_SERVER_WITH_THE_ORIGINAL_DB
DOCKER_DB_ID = $(shell ssh root#$$STAGING_IP "docker ps | grep database" | awk '{print $$1}')
download_staging_db:
psql -h $(LOCAL_DOCKER_IP) -p 5432 -U postgres -c 'drop database mydb'
psql -h $(LOCAL_DOCKER_IP) -p 5432 -U postgres -c 'create database mydb'
ssh root#$(STAGING_IP) "docker exec $(DOCKER_DB_ID) su postgres -c 'pg_dump mydb'" | psql -h $(LOCAL_DOCKER_IP) -p 5432 -U postgres mydb
upload_release_db_to_staging_db:
ssh root#$(RELEASE_IP) "cd / && su postgres -c 'pg_dump my db'" > temp.sql
scp temp.sql root#$(STAGING_IP):/temp.sql
ssh root#$(STAGING_IP) "docker cp /temp.sql $(DOCKER_DB_ID):/temp.sql"
ssh root#$(STAGING_IP) "docker exec $(DOCKER_DB_ID) su postgres -c 'drop database mydb'"
ssh root#$(STAGING_IP) "docker exec $(DOCKER_DB_ID) su postgres -c 'create database mydb'"
ssh root#$(STAGING_IP) "docker exec $(DOCKER_DB_ID) su postgres -c 'psql mydb < /temp.sql'"
rm temp.sql
I using docker postgres:9.4 image.
I need to know how to backup and restore volume only container.
Created Volume container:
docker run -v /var/lib/postgresql/data --name dbdata postgres:9.4 /bin/true
Using the volume
docker run --name=postgres --volumes-from=dbdata -d -p 6432:5432 postgres:9.4
Backup Volume container
docker run --volumes-from dbdata -v $(pwd):/backup postgres:9.4 tar cvf /backup/backup.tar /var/lib/postgresql/data
Restore volume in new container
docker run --name=dbdata-new --volumes-from dbdata -v $(pwd):/backup ubuntu:14.04 /bin/sh -c 'cd /var/lib/postgresql/data && tar xvf /backup/backup.tar'
Use in the new volume in creating new postgres container:
docker run --name=postgres-new --volumes-from=dbdata-new -d -p 7532:5432 postgres:9.4
Issue: I get the below error in the logs when I run the new container.
initdb: directory "/var/lib/postgresql/data" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "/var/lib/postgresql/data" or run initdb
with an argument other than "/var/lib/postgresql/data"
Not sure what I am doing wrong. Can someone please point out where I am making mistake.
Could not easily reproduce the issue following the steps with very rudiment data (one record one table one new db):
psql -U postgres -h $(boot2docker ip || echo 'localhost') -p 6432 -c "CREATE DATABASE ttt;"
psql -U postgres -h $(boot2docker ip || echo 'localhost') -p 6432 -d ttt -c "CREATE table a(b int); insert into a(b) values(1);"
psql -U postgres -h $(boot2docker ip || echo 'localhost') -p 6432 -d ttt -c "select * from a;"
when I start the postgres-new I get no exceptions in logs and the data seems to be there:
$ psql -U postgres -h $(boot2docker ip || echo 'localhost') -p 7532 -d ttt -c "select * from a;"
b
---
1
(1 row)
data-new --volumes-from dbdata -v $(pwd):/backup ubuntu:14.04 /bin/sh -c 'cd /var/lib/postgresql/data && tar xvf /backup/backup.tar'
The bold text was the problem.
By untaring the backup in the folder /var/lib/postgresql/data ,the tar command was creating /var/lib/postgresql/data inside the above folder.
Thank you for all the help MyKola.