I create my container like this:
docker run --name postgresql -itd --restart always \
--publish 5432:5432 \
--volume /my/local/host:/var/lib/postgresql \
sameersbn/postgresql:9.4-11
but, when I do ls on the root directory, I see something like this:
drwxrwxr-x 3 messagebus messagebus 4,0K Ιαν 10 00:44 host/
or, in other words, I cannot access the /my/local/host directory. I have no idea about the messagebus user. is that normal? if this is the case, then how could I move the database from one machine to another in the future?
Try using a data container to hold your DB data. The pattern is described in the docs and is designed to promote clean separation between run-time and data.
$ docker create -v /var/lib/postgresql --name dbdata sameersbn/postgresql:9.4-11
$ docker run --name postgresql1 -itd --restart always \
--publish 5432:5432 \
--volumes-from dbdata
sameersbn/postgresql:9.4-11
A separate data container makes backup and recovery simpler and more obvious
docker run --volumes-from dbdata -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /var/lib/postgresql
The following posting I think gives a good explanation of data containers:
https://medium.com/#ramangupta/why-docker-data-containers-are-good-589b3c6c749e#.zarkr5sxc
Related
This question already has answers here:
docker postgres with initial data is not persisted over commits
(5 answers)
Closed 1 year ago.
I just create a container like:
docker run --name mypostgres \
--publish 5432:5432 \
--net=apinetwork \
-e POSTGRES_PASSWORD=pwd \
-e POSTGRES_DB=db_uso \
-d postgres
I connect in the database and create some tables and objects...
After this I commit;
docker commit mypostgres user/mypostgres
But when I create other container from this image:
docker run --name mypostgres2 \
--publish 5432:5432 \
--net=apinetwork \
-e POSTGRES_PASSWORD=mysecretpassword \
-e POSTGRES_DB=correntista \
-d user/mypostgres
I don't see my objects that I created inside!
Please help me.
It's most likely you are not seeing the changes because the volume created with the original container is not being used by the new container.
See Persist the DB Docker tutorial.
You need to explicitly create a volume, and make sure it is being used in the new container.
$ docker volume create my-pg-db
$ docker run --name mypostgres \
--publish 5432:5432 \
--net=apinetwork \
-v my-pg-db:/var/lib/postgresql/data \
-e POSTGRES_PASSWORD=pwd \
-e POSTGRES_DB=db_uso \
-d postgres
This means it may not even be necessary to make a copy of your container, if all the changes you are doing are on the DB. The volume which contains the DB is what you need to preserve!
And make sure you mount and use it whenever you want to start the container.
The reason data is not persisted is explained in this other SO question:
docker postgres with initial data is not persisted over commits
The problem is that the postgres Dockerfile declares "/var/lib/postgresql/data" as a volume. This is a just a normal directory that lives outside of the Union File System used by images. Volumes live until no containers link to them and they are explicitly deleted.
which is why you need externally managed volumes like #smac90 wrote.
Source:
https://github.com/docker-library/postgres/blob/5c0e796bb660f0ae42ae8bf084470f13417b8d63/Dockerfile-debian.template#L186
I want to create a docker container or image and have tensorflow and mongodb installed, I have seen that there are docker images for each application, but I need them to be working together, from a mongodb database I must extract the data to feed a model created in tensorflow.
Then I want to know if it is possible to have a configuration like that, since I have tried with a ubuntu container and inside it to install the applications I need, but I don't know if there is another way to do it.
Thanks.
Interesting that I find this post, and just found one solution for myself. Maybe not the one for you, BTW.
What I did is: docker pull mongo and run as daemon:
#!/bin/bash
export VOLUME='/home/user/code'
docker run -itd \
--name mongodb \
--publish 27017:27017 \
--volume ${VOLUME}:/code \
mongo
Here
the 'd' in '-itd' means running as daemon (like service, not
interactive).
The --volume may not be used.
Then docker pull tensorflow/tensorflow and run it with:
#!/bin/bash
export VOLUME='/home/user/code'
docker run \
-u 1000:1000 \
-it --rm \
--name tensorflow \
--volume ${VOLUME}:/code \
-w /code \
-e HOME=/code/tf_mongodb \
tensorflow/tensorflow bash
Here
the -u make docker bash with same ownership as host machine;
the --volume make host folder /home/user/code mapping to /code in docker;
the -w work make docker bash start from /code, which is /home/user/code in host;
the -e HOME= option sign bash $HOME folder such that later you can pip install.
Now you have bash prompt such that you can
create virtual env folder under /code (which is mapping to /home/user/code),
activate venv,
pip install pymongo,
then you can connect to mongodb you run in docker (localhost may not work, please use host IP address).
I have performed the following steps,
docker run -d --name demo-mongo -p 27017:27017 -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=secret -e MONGO_INITDB_DATABASE=testdb mongo ** to create a new mongodb container
Create data-base inside the running container, by connecting to it using a mongo client
docker commit demo-mongo demo-mongo-updated ** create image from the running container
However, docker does not by default (which seems obvious) retain the data of the newly created data-base (likely to be retained in /data/db) in the newly created image.
Is it possible by any means to preserve the state of a container while creating an image from the same.
You can create a volume that points to a host directory and mount it to the container:
docker run -v <PATH_TO_THE_HOST_DIR>:/data -d --name demo-mongo -p 27017:27017 -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=secret -e MONGO_INITDB_DATABASE=testdb mongo
The next time you create a mongo container pointing to the same host path, it will be available on the container.
I am having some difficulty with docker and the postgres image from the Docker Hub. I am developing an app and using the postgres docker to store my development data. I am using the following command to start my container:
sudo docker run --name some-postgresql -e POSTGRES_DB=AppDB -e POSTGRES_PASSWORD=App123! -e POSTGRES_USER=appuser -e PGDATA="/pgdata" --mount source=mydata,target=/home/myuser/pgdata -p 5432:5432/tcp postgres
When I finish working on my app, I usually have to run "docker container prune", in order to free up the container name and be able to run it again later. This worked until recently, when I upgraded my postgres image to run version 11 of PostgreSQL. Now, when I start my container and create data in it, the next time I use it the data is gone. I've been reading about volumes in the docker documentation cannot find anything that can tell my why this is not working. Can anyone please shed some light on this?
Specify a volume mount with -v $PGDATA_HOST:/var/lib/postgresql/data.
The default PGDATA inside the container is /var/lib/postgresql/data so there is no need to change that if you're not modifying the Docker image.
e.g. to mount the data directory on the host at /srv/pgdata/:
$ PGDATA_HOST=/srv/pgdata/
$ docker run -d -p 5432:5432 --name=some-postgres \
-e POSTGRES_PASSWORD=secret \
-v $PGDATA_HOST:/var/lib/postgresql/data \
postgres
The \ are only needed if you break the command over multiple lines, which I did here for the sake of clarity.
since you specified -e PGDATA="/pgdata", the database data will be written to /pgdata within the container. If you want the files in /pgdata to survive container deletion, that location must be a docker volume. To make that location a docker volume, use --mount source=mydata,target=/pgdata.
In the end, it would be simpler to just run:
sudo docker run --name some-postgresql -e POSTGRES_DB=AppDB -e POSTGRES_PASSWORD=App123! -e POSTGRES_USER=appuser --mount source=mydata,target=/var/lib/postgresql/data -p 5432:5432/tcp postgres
With the advent of
docker volume create
for storage only containers, I'm still using the old way for running postgres on my machine for small applications without a dockerfile:
# MAKE MY DATA STORE
STORAGE_DIR=/home/username/mydockerdata/pgdata
docker create -v $STORAGE_DIR:/var/lib/postgresql/data --name mypgdata ubuntu true
# CREATE THE PG
docker run --name mypg -e POSTGRES_PASSWORD=password123 -d -p 5432:5432 --volumes-from mypgdata library/postgres:9.5.4
# RUN IT
docker start mypg
# docker stop mypg
I have 4 questions:
How could I move the old way to store my data in a local, persistent container to modern volumes?
The permissions my way have always seemed whacky:
$ ls -lah $STORAGE_DIR/..
drwx------ 19 999 root 4.0K Aug 28 10:04 pgdata
Should I do this differently?
Does my networking look correct here? This will be visible only on the machine hosting docker, or is this also published to all machines on my wifi network?
Besides the weak password, standard ports, default usernames, for example here, are there other security fears in doing this for personal use only that I should be aware of?
Create a new volume and copy the data over. Then run your container with the new volume definition.
docker volume create --name mypgdata
docker run --rm \
-v $STORAGE_DIR:/data \
-v mypgdata:/datanew ubuntu \
sh -c 'tar -C /data -cf - . | tar -C /datanew -xvf -'
docker run --rm -v mypgdata:/data ubuntu ls -l /data
The permissions are normal. UID 999 is the postgres user that the postgres image creates.
Port 5432 will be accessible on all your docker hosts interfaces. If you only want it to be available on localhost use --port 127.0.0.1:5432:5432
Moving to listening on localhost mitigates most security issues, until someone gains access to your docker host. General security is a bit too broad a topic for a dot point.