I know there are probably many ways to do this. What I am looking for is a way to do it using (preferably) only my DockerFile and one container.
Here is my current dockerfile:
FROM postgres:latest
ENV POSTGRES_USER=myuser
ENV POSTGRES_PASSWORD=mypassword
Here is the command I used to build this container:
docker built -t my_db .
And here is the command that I use to run the container:
docker run -p 5432:5432 my_db
What I would like to do is have the data stored in the container if possible, but I don't seem to understand how or where postgres stores it's data. I saw on another stack overflow post that postgres will store it by default in /var/lib/postgresql/data however when I look in that folder I see nothing. I can however verify that postgres is running because I am using a client called teamSQL and from that client I can create tables and insert/read data.
I can also verify that when i stop the container and restart the data is definitely not persisted.
Note: this is running in OSx but I don't think that is relevant.
You should use Docker volumes, so when you stop your container, data will persist on host machine, and when you start container again data will be mounted to it
docker volume create pgdata
docker run -p 5432:5432 -v pgdata:/var/lib/postgresql/data my_db
Related
So I'm setting up with Docker swarm.
I am now cool with the docker stack deploy -c docker-compose.yml myapp command which replaces my former docker-compose up.
But one of my service is my DB and I need to pgrestore inside it.
Previously with compose, I would run:
docker-compose run --rm postgres pg_restore --rest-of-command
How can I do the same with stack deploy?
Unfortunately, the container created with compose is not the same as the one from stack deploy: the first one is called myapp_postgres while the second myapp_postgres.1.zamd6kb6cy4p8mtfha0gn50vh.
I guess I could write something like docker exec 035803286af0 but then I loose all the benefits of the config from docker-compose.yml, which in this case is:
postgres:
env_file:
- ./.env
image: postgres:11.0-alpine
volumes:
- "..:/app" # toe make the dump accessible to the container
- "/var/run/postgresql:/var/run/postgresql"
So this solution is not very IaC.
So ain't there a docker service run or something?
Thanks
You can follow docker image docs (Initialization scripts section):
and create *.sh script under /docker-entrypoint-initdb.d which will run pg_restore ... when Postgres container will run as part of the Docker service.
It doesn't seem to be a direct answer to your question, however it may achieve your goal of restoring the dump during Postgres initialization.
I want to build postgres docker container for testing some issue.
I have:
Archived folder of postgres files(/var/lib/postgres/data/)
Dockerfile that place folder into doccker postgres:latest.
I want:
Docker image that reset self-state after recreate image.
Container that have database state based on passed into the container postgres files
I don't want to wait for a long time operation of backup and restore existing database in /docker-entrypoint-initdb.d initialization script.
I DON'T WANT TO USE VOLUMES because I don't need to store new data between restart (That's why this post is different from How to use a PostgreSQL container with existing data?. In that post volumes are used)
My suggestion is to copy postgres files(/var/lib/postgres/data/) from host machine into docker's /var/lib/postgres/data/ in build phase.
But postgres docker replace this files when initdb phase is executing.
How to ask Postgres docker not overriding database files?
e.g.
Dockerfile
FROM postgres:latest
COPY ./postgres-data.tar.gz /opt/pg-data/
WORKDIR /opt/pg-data
RUN tar -xzf postgres-data.tar.gz
RUN mv ./data/ /var/lib/postgresql/data/pg-data/
Run command
docker run -p 5432:5432 -e PGDATA=/var/lib/postgresql/data/pg-data --name database-immage1 database-docker
If you don't really need to create a custom image with the database snapshot you could use volumes. Un-tar the database files somewhere on the host say ~/pgdata then run the image. Example:
docker run -v ~/pgdata:/var/lib/postgresql/data/ -p 5432:5432 postgres:9.5
The files must be compatible with the postgres version of the image so use the same image version as the archived database.
If, instead, you must recreate the image you don't need to uncompress the database archive. The ADD instruction will do
that for you. Make sure the tar does not contain any leading directory.
The Dockerfile:
FROM postgres:latest
ADD ./postgres-data.tar.gz /var/lib/postgresql/data/
Build it:
docker build . -t database-docker
Run without overriding the environment variable PGDATA. Note that you copy the files in /var/lib/postgresql/data but the PGDATA points to /var/lib/postgresql/data/pg-data.
Run the container:
docker run -p 5432:5432 --name database-image1 database-docker
I'm using the official postgresql docker image to start a container.
Afterwards, I install some software and use psql to create some tables etc. I am doing this by first starting the postgres container as follows:
docker run -it --name="pgtestcontainer" -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:9.6.6
Then I attach to this container with
docker exec -it pgtestcontainer bash
and I install software, create db tables etc.
Afterwards, I first quit from the second terminal session (that I used to install software) and do a ctrl + c in the first one to stop the postgres container.
At this point my expectation is that if I commit this postgres image with
docker commit xyz...zxy pg-commit-test
and then run a new container based on the committed image with:
docker run -it --name="modifiedcontainer" -e POSTGRES_PASSWORD=postgres -p 5432:5432 pg-commit-test
then I should have all the software and tables in place.
The outcome of the process above is that the software I've installed is in the modifiedcontainer but the sql tables etc are gone. So my guess is my approach is more or less correct but there is something specific to postgres docker image I'm missing.
I know that it creates the db from scratch if no external directory or docker volume is bound to
/var/lib/postgresql/data
but I'm not doing that and after the commit I'd expect the contents of the db to stay as they are.
How do I follow the procedure above (or the right one) and keep the changes to database(s)?
The postgres Dockerfile creates a mount point at /var/lib/postgresql/data which you must mount an external volume onto if you want persistent data.
ENV PGDATA /var/lib/postgresql/data
RUN mkdir -p "$PGDATA" && chown -R postgres:postgres "$PGDATA" && chmod 777 "$PGDATA" # this 777 will be replaced by 700 at runtime (allows semi-arbitrary "--user" values)
VOLUME /var/lib/postgresql/data
https://docs.docker.com/engine/reference/builder/#notes-about-specifying-volumes
You can create a volume using
docker volume create mydb
Then you can use it in your container
docker run -it --name="pgtestcontainer" -v mydb:/var/lib/postgresql/data -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:9.6.6
https://docs.docker.com/engine/admin/volumes/volumes/#create-and-manage-volumes
In my opinion, the best way is to create your own image with a /docker-entrypoint-initdb.d folder and your script inside.
Look How to extend this image
But without volume you can't (I think) save your datas.
I solved this by passing PGDATA parameter with a value that is different than the path that is bound to docker volume as suggested in one of the responses to this question.
I'm a bit new to Docker.
I have two containers running using docker-compose.
One is the API and the other is the actual application.
I want to add a new DB container using the Postgres official image.
It's a bit hard to find a simple tutorial on how to create the container and populate it with a predefined sql file (of schemas and data).
When I start with "CMD /etc/init.d/postgresql start" in the Dockerfile I get an error saying: "No PostgreSQL clusters exist; see "man pg_createcluster" ... (warning)."
Since it takes me too much time to get things going I was wondering if it might be better to get an Ubuntu image and install Postgres on my own since there is only one source on how to use the image - docker hub, and I don't seem to understand it that well.
Any ideas or simple steps on how to compose and 'configure' this image?
If you want populate your database with some file, A simply way to do this is:
How to extend this image
If you would like to do additional initialization in an image derived
from this one, add one or more *.sql, *.sql.gz, or *.sh scripts under
/docker-entrypoint-initdb.d (creating the directory if necessary).
After the entrypoint calls initdb to create the default postgres user
and database, it will run any *.sql files and source any *.sh scripts
found in that directory to do further initialization before starting
the service.
Dockerfile
FROM postgres:alpine
COPY init.sql /docker-entrypoint-initdb.d/init.sql
docker-compose.yml
version: '3'
services:
app:
//your app definition
postgres:
build: .
Pull the postgres image
docker pull postges:14.2
Create the service with the below command
docker service create --name postgres --network my_overlay --env "POSTGRES_PASSWORD=password" --publish 5432:5432 postgres:14.2
Try to connect using userName as postgres and password as password to the default postgres db.
jdbc:postgresql://127.0.0.1:5432/postgres // JDBC connection
I want to make two minor improvements on the official MongoDB docker so that it starts with the --auth enabled and uses a separate data container to store the data. What's the best way to do this?
If all are set, how should I start the shell? Will it be possible for someone without a username and password to access any of the databases available? Which directory should I backup?
EDIT
Apparently, this is not enough:
docker run --name mymongoname1 -v /my/local/dir:/data/db -d -P mongo:latest
OK, so partial answer, because I haven't messed around with docker auth.
Containerising storage is done with a storage container. That's basically a container created off a token instance, with some volumes assigned.
So for elasticsearch (which I know isn't mongo, but it is at least a NoSQL db) I've been using:
docker create -v /es_data:/es_data --name elasticsearch_data es-base /bin/true
Then:
docker run -d -p 9200:9200 --vols-from elasticsearch_data elasticsearch-2.1.0
This connects the container volume to my es container - in this example it passes through a host volume, but you don't actually need to any more, because the container can hold the data in the docker filesystem. (And then I think you can push the data container around too, but I've not got that far!)
If you run ps -a you will see the data container in Created state. Just watch if you're doing a cleanup script that you don't delete it, because unlike running containers, you can freely delete it...