I have a docker-compose file that links a seed script with a mongo image from docker's public registry. I can populate the database with:
docker-compose up
docker-compose run seed_script
I can connect to this mongo container with the mongo cli from my host system and verify that the seed script is working and there's data in the database.
When I'm done seeding, I can see the mongo container ID with docker ps. I stop the container by pressing Ctrlc in the docker-compose terminal and commit the changes to the mongo container:
docker commit <mongo container ID> mongo-seeded-data
However, when I run that container individually, it's empty:
docker run -p 27017:27017 mongo-seeded-data
mongo
> show dbs
local 0.000GB
If I bring up the docker-compose containers again and use my host mongo client, I can see the data:
docker-compose up
mongo
> show dbs
seeded_db 0.018GB
local 0.000GB
I committed the container with the data in it. Why is it not there when I bring up the container? What is docker-compose doing differently than docker run?
Because there is a VOLUME defined in it.
docker commit saves the content of the overlay fs as a new image, but VOLUMES transcend the overlay fs. That's why it doesn't get saved.
I had the same problem in the past and resolved patching the mongo image by using a different data directory, so that the data would get written inside the container instead of the VOLUME.
Related
I'm trying to docker-containerize PostgreSQL server and this container will have many other applications as well. The need is that, PostgreSQL server data should be mapped to the host volume so that when container is stopped, we won't lose the data. Also that, the next time when we start the container, the same directory can be mapped again and postgres can use the old data. Below is the DOCKERFILE. Note that I'm using ubuntu 22.04 on the host.
FROM ubuntu:22.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt install -y postgresql
ENTRYPOINT ["tail", "-f", "/dev/null"]
Docker image is built using the command
docker build -t pg_test .
and the container is run using the command
docker run --name test -v /home/me/data:/var/lib/postgresql/14/main pg_test
'/home/me/data' is the host directory which is empty where I want to map the postgres server data. '/var/lib/postgresql/14/main' is the directory inside the docker container where the postgres is supposed to store the data.
Once the docker container starts, I enter the docker container using the command
docker exec -it test bash
and once I'm inside, I'm trying to start the PostgreSQL service. But PostgreSQL fails to start as there is no data in '/var/lib/postgresql/14/main' directory. I understand that since I have mapped an empty host directory to '/var/lib/postgresql/14/main' directory, postgres doesn't have the files required to start.
I understand that I'm doing it the wrong way, but I couldn't find a way around it. Can anyone please help me to do this the right way, if there is one?
Any help would be appreciable.
You should use the postgres docker image, it will set up the db for you when you start the container, you can find instructions on https://hub.docker.com/_/postgres
If you must use a custom image, you will need to initialize the db yourself, usually by running initdb or whatever your system provides.
But really you should use the appropriate docker image, and if you need more services you start them in their own container and connect them to the postgres one
I have a container called "postgres", build with plain docker command, that has a configured PostgreSQL inside it. Also, I have a docker-compose setup with two services - "api" and "nginx".
How to add the "postgres" container to my existing docker-compose setup as a service, without rebuilding? The PostgreSQL database is configured manually, and filled with data, so rebuilding is a really, really bad option.
I went through the docker-compose documentation, but found no way to do this without a re-build, sadly.
Unfortunately this is not possible.
You don't refer containers on docker-compose, you use images.
You need to create a volume and/or bind mount it to keep your database data.
This is because containers do not save data, if you have filled it with data and did not make a bind mount or a volume to it, you will lose everything on using docker container stop.
Recommendation:
docker cp
Docker cp will copy the contents from container to host. https://docs.docker.com/engine/reference/commandline/container_cp/
Create a folder to save all your PostgreSQL data (ex: /home/user/postgre_data/)
Save the contents of your PostgreSQL container data to this folder (docker hub postgres page for further reference: ;
Run a new PostgreSQL (same version) container with a bind mount poiting to the new folder;
This will maintain all your data and you will be able to volume or bind mount it to use on docker-compose.
Reference of docker-compose volumes: https://docs.docker.com/compose/compose-file/#volumes
Reference of postgres docker image: https://hub.docker.com/_/postgres/
Reference of volumes and bind mounts: https://docs.docker.com/storage/bind-mounts/#choosing-the--v-or---mount-flag
You can save this container in a new image using docker container commit and use that newly created image in your docker-compose
docker container commit [OPTIONS] CONTAINER [REPOSITORY[:TAG]]
I however prefer creating images with the use of Dockerfiles and scripts to fill my data etc.
I want to use docker run a vm mongodb, at the same time, the mongo configure file use my own defined configure file to archive replication and open authentication.
Scanning some files but don't resolve the problem.
Any ideas?
The docker mongo image has a docker-entrypoint.sh it calls in the Dockerfile
Check if you can:
create your own image which would create the right user and restart mongo with authentication on: see "umputun/mongo-auth" and its init.sh script
or mount a createUser.js script in docker-entrypoint-initdb.d.
See "how to make a mongo docker container with auth"
i have a running Mongo DB Container called xyz from offical Mongo Image. i created the container with docker run -d -p 21707:21707 mongo
In this container i created 2 collections with sample data.
Now i want to extract this container in a new image on dockerhub.
I used docker commit and created a new image, pushed it on the docker hub. If i pull the image on a other system and create a new container from this image, there are no data like in my origin container xyz.
After research i found out that the mongo image could used with a volume, but i missed this step in the past... I think the container use /data/db as standard volume(docker inspect), on commit this volume is not attached to the new image?!
Also i tried docker export/import with the same problem mentioned above!
Now my Question, how could i reach to migrate this "xyz" running container with my sample data in a new image/container?
Thanks a lot!
I used docker commit and created a new image, pushed it on the docker hub. If i pull the image on a other system and create a new container from this image, there are no data like in my origin container xyz.
The mongo docker image writes its data into a volume. As a result, those data won't be saved to a new docker image created with docker commit. docker export won't produce your data for the same reason.
how could i reach to migrate this "xyz" running container with my sample data in a new image/container?
What you want is either:
create a new container re-using the first container's volume → see --volumes-from
save the first container's volume data into a directory on your docker host, and create a new container mounting this directory into the container → the docker run -v option
Also, this SO question might help you figure out volumes.
I wanted to make an image to test things around authentication and I has the same issue. I solved it this way:
Dockerfile
FROM mongo
COPY setup.sh /
RUN chmod +x /setup.sh
RUN /setup.sh
ENTRYPOINT ["/usr/bin/mongod"]
CMD ["--auth", "--dbpath=/containerdb"]
setup.sh
#!/bin/bash
mkdir containerdb
mongod --auth --dbpath=/containerdb &
until mongo admin --eval 'db.createUser({"user":"root", "pwd":"password", "roles": [ "root" ] })'
do
echo 'Mongo not yet ready, waiting 1 sec until retry'
sleep 1
done
mongod --shutdown --dbpath=/containerdb
Bim! When you run this image you have a mongodb with authentication activated and a root user.
The key is to workaround the VOLUME in the base Dockerfile by storing the data somewhere else.
It is quite easy to run MongoDB containerised using docker. Though each time you start a new mongodb container, you will get new empty database.
What should I do in order to keep the database content between container restarts? I tried to bind external directory to container using -v option but without any success.
I tried using the ehazlett/mongodb image and it worked fine.
With this image, you can easily specify where mongo store its data with DATA_DIR env variable. I am sure it must not be very difficult to change on your image too.
Here is what I did:
mkdir test; docker run -v `pwd`/test:/tmp/mongo -e DATA_DIR=/tmp/mongo ehazlett/mongodb
notice the `pwd` in within the -v, as the server and the client might have different path, it is important to specify the absolute path.
With this command, I can run mongo as many time as I want and the database will always be store in the ./test directory I just created.
When using the official Mongo docker image, which is i.e. version mongo:4.2.2-bionic as writing this answer, and using docker-compose, you can achieve persistent data storage using this docker-compose.yml file example.
In the official mongo image, data is stored in the container under the root directory in the folder /data/db by default.
Map this folder to a folder in your local working directory called data (in this example).
Make sure ports are set and mapped, default 27017-27019:27017-27019.
Example of my docker-compose.yml:
version: "3.2"
services:
mongodb:
image: mongo:4.2.2-bionic
container_name: mongodb
restart: unless-stopped
ports:
- 27017-27019:27017-27019
volumes:
- ./data:/data/db
Run docker-compose up in the directory where the yml file is located to run the mongodb container with persistent storage. If you do not have the official image yet, it will pull it from Dockerhub first.
Old post but may be someone still need quick and easy solution...
The easiest way I found is using binding to volume.
Following that way you can easily attach existing MongoDB data; and it will live even after you destroying the container.
Create volume that points to your folder (may include existing db). In my case it's done under Windows, but you can do it on any file system:
docker volume create --opt type=none --opt o=bind --opt device=d:/data/db db
Create/run docker container with MongoDB using that volume binding:
docker run --name mongodb -d -p 27017:27017 -v db:/data/db mongo