I am creating a docker image for postgresql installation from sources. Here is my Dockerfile which I created according to the postgresql documentation:
FROM ubuntu
RUN apt-get update && apt-get install gcc zlib1g-dev libreadline6-dev apt-utils make -y
RUN mkdir -p /tmp/downloads
ADD https://ftp.postgresql.org/pub/source/v9.6.6/postgresql-9.6.6.tar.gz /tmp/downloads
RUN cd /tmp/downloads && tar -zxf postgresql-9.6.6.tar.gz
RUN cd /tmp/downloads/postgresql-9.6.6 && make configure
RUN cd /tmp/downloads/postgresql-9.6.6 && ./configure
RUN cd /tmp/downloads/postgresql-9.6.6 && make
RUN cd /tmp/downloads/postgresql-9.6.6 && su
RUN cd /tmp/downloads/postgresql-9.6.6 && make install
RUN cd /tmp/downloads/postgresql-9.6.6 && adduser postgres
RUN cd /tmp/downloads/postgresql-9.6.6 && mkdir /usr/local/pgsql/data
RUN cd /tmp/downloads/postgresql-9.6.6 && chown postgres /usr/local/pgsql/data
RUN cd /tmp/downloads/postgresql-9.6.6 && su postgres
RUN cd /tmp/downloads/postgresql-9.6.6 && /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
RUN cd /tmp/downloads/postgresql-9.6.6 && /usr/local/pgsql/bin/createdb test
RUN cd /tmp/downloads/postgresql-9.6.6 && /usr/local/pgsql/bin/psql test
So when I run docker build -t roksolanad/psql:latest . I get errors:
initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.
The command '/bin/sh -c cd /tmp/downloads/postgresql-9.6.6 && /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data' returned a non-zero code: 1
So how can I fix my Dockerfile? I would be very grateful for some help!
Problem is that each RUN creates its own shell. Which means that username changes only persistent for that one instance. This is different from any filesystem changes which will persist.
Try chaining your commands together like this instead:
RUN cd /tmp/downloads/postgresql-9.6.6 &&\
make configure &&\
./configure &&\
make &&\
su &&\
make install &&\
adduser postgres &&\
mkdir /usr/local/pgsql/data &&\
chown postgres /usr/local/pgsql/data
RUN cd /tmp/downloads/postgresql-9.6.6 &&\
su postgres &&\
/usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data &&\
/usr/local/pgsql/bin/createdb test &&\
/usr/local/pgsql/bin/psql test
As said by #Harald Nordgren, you need sum up all the RUN commands in one command, if possible.
Along with that there are couple of things which are causing the failures
1) adding "postgres" user:
"adduser" expects the additional parameters to be configured when you add a user, as you have mentioned in the comments like asking for password and so. So you need to modify the command to disable the arguments as well as the password like below:
adduser postgres --gecos '' --disabled-login
2) executing postgress using root user
When you execute the command "su postgres", it executes with root user. But whereas we have changed the permissions in the above command "chown postgres /usr/local/pgsql/data"
For this you need a execute the command as "postgres" user which can be enabled by adding USER in dockerfile.
Finally your Dockerfile looks something like this:
FROM ubuntu
RUN apt-get update && apt-get install gcc zlib1g-dev libreadline6-dev apt-utils make -y
RUN mkdir -p /tmp/downloads
ADD https://ftp.postgresql.org/pub/source/v9.6.6/postgresql-9.6.6.tar.gz /tmp/downloads
RUN cd /tmp/downloads && tar -zxf postgresql-9.6.6.tar.gz
RUN cd /tmp/downloads/postgresql-9.6.6 &&\
make configure &&\
./configure &&\
make &&\
su &&\
make install &&\
adduser postgres --gecos '' --disabled-login &&\
mkdir /usr/local/pgsql/data &&\
chown postgres /usr/local/pgsql/data
USER postgres
#use below command only if it is necessary, it is similar to "cd" linux command
WORKDIR /tmp/downloads/postgresql-9.6.6
RUN /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
With this I am able to create a Docker image and tested as well. Adding more information which may be useful.
# docker run -it postgres:2.0 /bin/bash
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ ps -eaf
UID PID PPID C STIME TTY TIME CMD
postgres 1 0 0 10:58 ? 00:00:00 /bin/bash
postgres 9 1 0 10:59 ? 00:00:00 ps -eaf
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start
server starting
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ ps -eaf
UID PID PPID C STIME TTY TIME CMD
postgres 1 0 0 10:58 ? 00:00:00 /bin/bash
postgres 13 1 0 10:59 ? 00:00:00 /usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data
postgres 15 13 0 10:59 ? 00:00:00 postgres: checkpointer process
postgres 16 13 0 10:59 ? 00:00:00 postgres: writer process
postgres 17 13 0 10:59 ? 00:00:00 postgres: wal writer process
postgres 18 13 0 10:59 ? 00:00:00 postgres: autovacuum launcher process
postgres 19 13 0 10:59 ? 00:00:00 postgres: stats collector process
postgres 20 1 0 10:59 ? 00:00:00 ps -eaf
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ /usr/local/pgsql/bin/createdb test
postgres#8354d83023f9:/tmp/downloads/postgresql-9.6.6$ /usr/local/pgsql/bin/psql test
psql (9.6.6)
Type "help" for help.
test=#
test=#
There are other ways to build this docker image but this way I am able to.
Related
I am working on a docker container which initialises postgre sql db from a sql dump file.However while running the init script I am facing below exception .
Attaching to postgres
postgres | psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: No such file or directory
postgres | Is the server running locally and accepting connections on that socket?
postgres exited with code 2
DockerFile:
FROM postgres:latest
## Some utilities
RUN apt-get update -y && \
apt-get install -y build-essential libfuse-dev libcurl4-openssl-dev libxml2-dev pkg-config libssl-dev mime-support automake libtool wget tar git unzip
RUN apt-get install lsb-release -y && apt-get install zip -y && apt-get install vim -y
## Install S3 Fuse
RUN rm -rf /usr/src/s3fs-fuse
RUN git clone https://github.com/s3fs-fuse/s3fs-fuse/ /usr/src/s3fs-fuse
WORKDIR /usr/src/s3fs-fuse
RUN ./autogen.sh && ./configure && make && make install
## Create folder
RUN mkdir /data
WORKDIR /data
RUN mkdir out
## change workdir to /
WORKDIR /
RUN rm -rf 1-restore.sh
ADD docker-entrypoint-initdb.d/1-restore.sh 1-restore.sh
RUN chmod 777 1-restore.sh
ENTRYPOINT ["/bin/bash", "1-restore.sh" ]
Below is my init Shell script file:
#!/bin/bash
set -e
export PGPASSWORD=$POSTGRES_PASSWORD
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" < $DUMP_FILE
set -e
export PGPASSWORD=$POSTGRES_PASSWORD
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
SET client_encoding TO 'UTF8';
\t
\a
\o $OUT_FOLDER/export_json.sql;
SELECT '\o $OUT_FOLDER/' || table_name || '.json;' || E'\n' || 'SELECT row_to_json(r) FROM "' || table_name || '" AS r;'
FROM information_schema.tables
WHERE table_schema = 'public'
ORDER BY table_name;
\i $OUT_FOLDER/export_json.sql;
EOSQL
rm $OUT_FOLDER/export_json.sql
echo ""TestMessage">>test.txt
I checked the other so ans but nothing yet worked for me .
I have a python application that interacts with the postgresql database and i need to run it all in one docker container.
I get a connection error when running the container:
...
File "/usr/lib/python3.6/asyncio/tasks.py", line 358, in wait_for
return fut.result()
File "/usr/lib/python3.6/asyncio/base_events.py", line 787, in create_connection
', '.join(str(exc) for exc in exceptions)))
OSError: Multiple exceptions: [Errno 111] Connect call failed ('::1', 5432), [Errno 111] Connect call failed ('127.0.0.1', 5432)
My Dockerfile:
FROM postgres:10.0-alpine
RUN apk add --update --no-cache g++ alpine-sdk
RUN apk --no-cache add python3-dev
RUN apk add --no-cache python3 && \
python3 -m ensurepip && \
rm -r /usr/lib/python*/ensurepip && \
pip3 install --upgrade pip setuptools && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache
ADD app /app/
RUN chmod -R 777 /app
WORKDIR /app
ADD requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
USER postgres
RUN chmod 0700 /var/lib/postgresql/data &&\
initdb /var/lib/postgresql/data &&\
echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf &&\
echo "listen_addresses='*'" >> /var/lib/postgresql/data/postgresql.conf &&\
pg_ctl start &&\
psql --command "ALTER USER postgres WITH ENCRYPTED PASSWORD 'postgres';"
EXPOSE 5432
EXPOSE 80
CMD ["python3", "main.py"]
Although this is not recommended, it's doable. The problem is pg_ctl in RUN instruction is executed at build time, not in the container. You need to run it with CMD.
You can have a script like
pg_ctl start
psql --command "ALTER USER postgres WITH ENCRYPTED PASSWORD 'postgres';"
python3 main.py
COPY the script in the image and at the end of the dockerfile, `CMD ["./script.sh"]
#Anton, it is not recommended to run multiple processes inside a docker container. Have a look at this link https://docs.docker.com/config/containers/multi-service_container/ it will explain more and demonstrate a manner to accomplish this. You probably are aware, but your container will be running the postgresql instance and have the data in it, so if you ever re-create the container, you will lose any data in that container.
I'm trying to get Postgresql 10.0 working in a docker container. I have the following Dockerfile:
FROM postgres:10.0-alpine
RUN apk add openrc --no-cache
USER postgres
RUN /etc/init.d/postgresql start
RUN psql --command "IF NOT EXISTS (SELECT FROM pg_catalog.pg_user WHERE usename = 'user') THEN CREATE USER user WITH SUPERUSER ENCRYPTED PASSWORD 'password'; END IF;"
RUN createdb main
EXPOSE 5432
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql"]
CMD ["/usr/lib/postgresql/10.0/bin/postgres", "-D", "/var/lib/postgresql/10.0/main", "-c", "config_file=/etc/postgresql/10.0/main/postgresql.conf"]
I got following error:
/bin/sh: /etc/init.d/postgresql: not found
Seems that /etc/init.d/postgresql is really missing. What am I doing wrong?
I finished with this Dockerfile:
FROM postgres:10.0-alpine
USER postgres
RUN chmod 0700 /var/lib/postgresql/data &&\
initdb /var/lib/postgresql/data &&\
echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf &&\
echo "listen_addresses='*'" >> /var/lib/postgresql/data/postgresql.conf &&\
pg_ctl start &&\
psql -U postgres -tc "SELECT 1 FROM pg_database WHERE datname = 'main'" | grep -q 1 || psql -U postgres -c "CREATE DATABASE main" &&\
psql -c "ALTER USER postgres WITH ENCRYPTED PASSWORD 'mysecurepassword';"
EXPOSE 5432
This dockerfile create database 'main' (if not exists), starts postgres and sets default user password
I was able to do it using exec commands, so I can run postgres in a running container, in this case alpine 3.7. Note $CONTAINER_NAME is the container id from docker ps:
# Install postgresql, create user, db & start the daemon (for testing)
sudo docker exec $CONTAINER_NAME sh -c 'apk add postgresql'
sudo docker exec $CONTAINER_NAME sh -c 'addgroup -S postgres && adduser -S postgres -G postgres'
sudo docker exec $CONTAINER_NAME sh -c 'mkdir -p /var/lib/postgresql/data'
sudo docker exec $CONTAINER_NAME sh -c 'mkdir -p /run/postgresql/'
sudo docker exec $CONTAINER_NAME sh -c 'chown -R postgres:postgres /run/postgresql/'
sudo docker exec $CONTAINER_NAME sh -c 'chmod -R 777 /var/lib/postgresql/data'
sudo docker exec $CONTAINER_NAME sh -c 'chown -R postgres:postgres /var/lib/postgresql/data'
sudo docker exec --user postgres $CONTAINER_NAME sh -c 'initdb /var/lib/postgresql/data'
sudo docker exec --user postgres $CONTAINER_NAME sh -c 'echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf'
sudo docker exec --user postgres $CONTAINER_NAME sh -c 'pg_ctl start -D /var/lib/postgresql/data -l /var/lib/postgresql/log.log'
sudo docker exec --user postgres $CONTAINER_NAME sh -c "psql --command \"ALTER USER postgres WITH ENCRYPTED PASSWORD 'buildpgpass';\""
sudo docker exec --user postgres $CONTAINER_NAME sh -c "psql --command \"CREATE DATABASE builddb;\""
You can also include this in your Dockerfile like so:
# Postgres
RUN apk add postgresql=11.1-r0
RUN (addgroup -S postgres && adduser -S postgres -G postgres || true)
RUN mkdir -p /var/lib/postgresql/data
RUN mkdir -p /run/postgresql/
RUN chown -R postgres:postgres /run/postgresql/
RUN chmod -R 777 /var/lib/postgresql/data
RUN chown -R postgres:postgres /var/lib/postgresql/data
RUN su - postgres -c "initdb /var/lib/postgresql/data"
RUN echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf
RUN su - postgres -c "pg_ctl start -D /var/lib/postgresql/data -l /var/lib/postgresql/log.log && psql --command \"ALTER USER postgres WITH ENCRYPTED PASSWORD 'postgres';\" && psql --command \"CREATE DATABASE builddb;\""
This seems to work great if you want to test something out on the fly.
There are multiple problems with your dockerfile. I guess you are trying to start the server when building the image to create the main database. This won't work since each command in the dockefile will execute in its own layer, and thus when you reach RUN psql ... the database won't be started since it was started in a different layer. So you need to group the commands in one line.
Second problem is that the file /etc/init.d/postgresql does not exist. The server can be started using the postgres command:
RUN postgres &\
psql --command "IF NOT EXISTS (SELECT FROM pg_catalog.pg_user WHERE usename = 'user')\
THEN CREATE USER user WITH SUPERUSER ENCRYPTED PASSWORD 'password'; END IF;" &\
createdb main
How can i enable the Postgis extension in a Dockerfile? The Postgres installation is already working
this is the command in Ubuntu:
sudo -u postgres psql -c "CREATE EXTENSION postgis;"
My Dockerfile:
# Set the base image to Ubuntu
FROM ubuntu:14.04
# Update the repository sources list
RUN apt-get update -y
################## BEGIN INSTALLATION ######################
# Postgres with Postgis
# Install wget
RUN apt-get install wget -y
# Setup Postgres repository
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
# Add Postgres repository
RUN sh -c "echo "deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main" >> /etc/apt/sources.list.d/postgresql.list"
# Update repository
RUN apt-get update -y
# Install Postgres with Postgis
RUN apt-get install postgresql-9.3-postgis-2.1 -y
# Change rights for start Postgresql
RUN chmod +x /etc/init.d/postgresql
# Start Postgresql
CMD service postgresql start && tail -F /var/lib/postgresql/data/serverlog
The solution for enable an extension
# Enable Postgis
RUN service postgresql start \
&& sudo -u postgres psql -c "CREATE EXTENSION postgis;"
RUN service postgresql stop
If you see other Dockerfiles using that psql -c command, like apache/marmotta Dockerfile, you will see lines like:
RUN service postgresql start \
&& psql --command "CREATE USER $DB_USER WITH PASSWORD '$DB_PASS';" \
&& psql --command "CREATE DATABASE $DB_NAME WITH OWNER $DB_USER;"
USER root
RUN service postgresql stop
In other words, you need to make sure the postgresql service is running for those commands to succeed.
What I mean is that I want to create a docker image for postgis that will be completely usable right after build. So that if user runs
docker run -e POSTGRES_USER=user somepostgis
the user database would be created and extensions already installed?
The official postgres image can't be used for that AFAIK.
Basically need to write script and tell that it would be entrypoint. This script should create database and create extensions with porstgres server running on different port and then restart it on port 5432.
But I don't know sh enough and docker to do that. Right now it's saying that there is no pg_ctl command
If you want to help you can fork
FROM ubuntu:15.04
#ENV RELEASE_NAME lsb_release -sc
#RUN apt-get update && apt-get install wget
#RUN echo "deb http://apt.postgresql.org/pub/repos/apt ${RELEASE_NAME}-pgdg main" >> /etc/apt/sources.list
#RUN cat /etc/apt/sources.list
#RUN wget --quiet -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc | sudo apt-key add -
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
postgresql-9.4-postgis-2.1 \
curl \
&& curl -o /usr/local/bin/gosu -SL "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture)" \
&& curl -o /usr/local/bin/gosu.asc -SL "https://github.com/tianon/gosu/releases/download/1.2/gosu-$(dpkg --print-architecture).asc" \
&& gpg --verify /usr/local/bin/gosu.asc \
&& rm /usr/local/bin/gosu.asc \
&& chmod +x /usr/local/bin/gosu \
&& apt-get purge -y --auto-remove curl
RUN mkdir /docker-entrypoint-initdb.d
COPY docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
RUN chmod +x /docker-entrypoint.sh
RUN ls -l /docker-entrypoint.sh
EXPOSE 5432
CMD ["postgres"]
So I'm trying to do somethink like that, but it doesn't work.
#!/bin/bash
${POSTGRES_DB:=$POSTGRES_USER}
gosu postgres pg_ctl start -w -D ${PGDATA} -0 "-p 5433"
gosu postgres createuser ${POSTGRES_USER}
gosu postgres createdb ${POSTGRES_DB} -s -E UTF8
gosu postgres psql -d ${POSTGRES_DB} -c "create extension if not exists postgis;"
gosu postgres psql -d ${POSTGRES_DB} -c "create extension if not exists postgis_topology;"
pg_ctl -w restart