I have an app that runs as a docker container, the app connects to another container which is Mongodb, I want to mount the mongodb volume data/db to Azure share files. I am using docker compose file to define my containers. Assuming that I already have a storage account linked to the app, this is how I define my docker volume in docker-compose
database:
image: "mongo:latest"
container_name: database
ports:
- "27017:27017"
volumes:
- ${WEBAPP_STORAGE_HOME}/data/db:/data/db
In Unable to mount azure file shares as mongodb volume in azure container instances it was mentioned that mounting data/db is not recommended and it won't work. My question is:
How can I mount my mongodb files to Azure files ? how to perform back-ups to those files? and if i want to restore a backup to the database would it be possible to just upload the files in the azure files and see them in my mongodb ?
For your issue, just as I said in another issue you find. To mount the Azure File Share to Web App will override the existing files in the image. So you need to change the MongoDB data path when the Azure File Share is already mounted.
The example docker-compose file like this:
version: '3.7'
services:
web:
image: mongo:latest
ports:
- "27017:27017"
volumes:
- fileshare:/data/mongodb
And create the Web App with this docker-compose file and set the start file with value mongod --dbpath=/data/mongodb --bind_ip_all. For example, if you use the Azure CLI, the create command like this:
az webapp create -g group_name --plan plan_name -n web_name --multicontainer-config-type compose --multicontainer-config-file docker-compose-file --startup-file "mongod --dbpath=/data/mongodb --bind_ip_all"
Finally, you need to set the file share mount like below:
Or follow the steps through CLI in Configure your app with Azure Storage.
Related
I am using mongo db in the docker container. I am using docker compose to spin up mongo. Now we have old mongo containers running and authentication is not enforced. In order to use authentication enforced and start up script I am using .env filein my docker compose file as below. But .env file and startup script are taking place only if I change the volume. By using same volume both .env file and startup script is not taking any effect. Is there any way to use the same volume and create users using .env and also use start up script.
services:
mongo:
image: mongo
container_name: mongot
restart: always
env_file:
- .env
ports:
- 27019:27017
volumes:
- /data/db8/configdb:/data/configdb
- /data/db8/db:/data/db
- $PWD/mongoentry/:/docker-entrypoint-initdb.d/
network_mode: "bridge"
command: mongod
Environment files and startup scripts are only used when creating a new database. If a database already exists, they aren't used.
In your case, you already have a database in the volume, so they aren't used. But if you change the volume so no database exists, Mongo creates a new one using your values and scripts.
From the docs:
When you start the mongo image, you can adjust the initialization of
the MongoDB instance by passing one or more environment variables on
the docker run command line. Do note that none of the variables below
will have any effect if you start the container with a data directory
that already contains a database
I have a docker-compose.yml which I am using to deploy to a remote host from my local (Mac) machine using docker context. The compose config is as follows:
database:
image: postgres:14.2
restart: on-failure
volumes:
- ./db-data:/var/lib/postgresql/data
environment:
POSTGRES_DB: db
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
In order to persist data, I have defined a volume ./db-data:/var/lib/postgresql/data. This db-data folder does not exist in my local machine. I want delete this mount completely because I don't want any of the previously persisted data. I know I can define a new volume directory but I would like to use the same directory name (db-data). I have tried the following:
docker compose down --volume --remove-orphans - when I recreate new container, previously persisted data still exists
There is no folder called ./db-data in my Mac working directory.
I tried searching var/lib/docker in my Mac. But that directory does not exists.
Docker for Mac app doesn't list any volumes
There is no db-data in the remote host where the database is deployed
Running docker inspect <container-id> listed the mount directory for the container. The mount directory resembled absolute path of my local computer. For example it was like /Users/<user-name>/dir/db-data. When I saw this I assumed this had to be in the local computer due to the prefix Users/<user-name> but this path was actually found in the root of the remote machine.
Thats because the directory for docker volumes is in the docker vm for MACOS.
Where is /var/lib/docker on Mac/OS X
You would have to follow this to see the volume
I have this docker-compose:
version: "2"
services:
api:
build: .
ports:
- "3007:3007"
links:
- mongo
mongo:
image: mongo
volumes:
- /data/mongodb/db:/data/db
ports:
- "27017:27017"
The volumes, /data/mongodb/db:/data/db, is the first part (/data/mongodb/db) where the data is stored inside the image and the second part (/data/db) where it's stored locally?
It works on production (ubuntu) but when i run it on my dev-machine (mac) I get:
ERROR: for mongo Cannot start service mongo: error while creating mount source path '/data/mongodb/db': mkdir /data/mongodb: permission denied
Even if I run it as sudo. I've added the /data directory in the "File Sharing"-section in the docker-program on the mac.
Is the idea to use the same docker-compose on both production and development? How do I solve this issue?
Actually it's the other way around (HOST:CONTAINER), /data/mongodb/db is on your host machine and /data/db is in the container.
You have added the /data in the shared folders of your dev machine but you haven't created /data/mongodb/db, that's why you get a permission denied error. Docker doesn't have the rights to create folders.
I get the impression you need to learn a little bit more about the fundamentals of Docker to fully understand what you are doing. There are a lot of potential pitfalls running Docker in production, and my recommendation is to learn the basics really well so you know how to handle them.
Here is what the documentation says about volumes:
[...] specify a path on the host machine (HOST:CONTAINER)
So you have it the wrong way around. The first part is the past on the host, e.g. your local machine, and the second is where the volume is mounted within the container.
Regarding your last question, have a look at this article: Using Compose in production.
Since Docker-Compose syntax version 3.2, you can use a long syntax of the volume property to specify the type of volume. This allows you to create a "Bind" volume, which effectively links a folder from a container to a folder in your host.
Here is an example :
version : "3.2"
services:
mongo:
container_name: mongo
image: mongo
volumes:
- type: bind
source: /data
target: /data/db
ports:
- "42421:27017"
source is the folder in your host and target the folder in your container
More information avaliable here : https://docs.docker.com/compose/compose-file/#long-syntax
I am playing with MongoDB and Docker and at this point I am trying to create a useful image for myself to use at work. I have created the following Dockerfile:
FROM mongo:2.6
VOLUME /data/db /data/configdb
CMD ["mongod"]
EXPOSE 27017
And I have added it to my docker-compose.yml file:
version: '2'
services:
### PHP/Apache Container
php-apache:
container_name: "php55-dev"
image: reynierpm/php55-dev
ports:
- "80:80"
environment:
PHP_ERROR_REPORTING: 'E_ALL & ~E_DEPRECATED & ~E_NOTICE'
volumes:
- ~/mmi:/var/www
- ~/data:/data
links:
- mongodb
### MongoDB Container
mongodb:
container_name: "mongodb"
build: ./mongo
environment:
MONGODB_USER: "xxxx"
MONGODB_DATABASE: "xxxx"
MONGODB_PASS: "xxxx"
ports:
- "27017:27017"
volumes:
- ~/data/mongo:/data/db
I have some questions regarding this setup I have made:
Do I need VOLUME /data/db /data/configdb at the Dockerfile or would be enough to have this line ~/data/mongo:/data/configdb at docker-compose.yml?
I am assuming (and I took it from here) that as soon as I build the Mongo image I will be creating a database and giving full permissions to the user with password as it's on the environment variables? I am right? (I couldn't find anything helpful here)
How do I import a current mongo backup (several JSON files) into the database that should be created on the mongo container? I believe I need to run mongorestore command but how? do I need to create an script and run it each time the container start? or should I run during image build? What's the best approach?
Do I need VOLUME /data/db /data/configdb at the Dockerfile or would be enough to have this line ~/data/mongo:/data/configdb at docker-compose.yml?
VOLUME is not required when you are mounting a host directory but it is helpful as metadata. VOLUME does provide some special "copy data on volume creation" semantics when mounting a Docker volume (non host dir) which will impact your data initialisation method choice.
am assuming (and I took it from here) that as soon as I build the Mongo image I will be creating a database and giving full permissions to the user with password as it's on the environment variables? I am right? (I couldn't find anything helpful here)
MONGO_USER, MONGO_DATABASE and MONGO_PASS do not do anything in the official mongo Docker image or to mongod itself.
The mongo image has added support for similar environment variables:
MONGO_INITDB_ROOT_USERNAME
MONGO_INITDB_ROOT_PASSWORD
MONGO_INITDB_DATABASE
How do I import a current mongo backup (several JSON files) into the database that should be created on the mongo container? I believe I need to run mongorestore command but how? do I need to create an script and run it each time the container start? or should I run during image build? What's the best approach?
Whether you initialise data at build or runtime is up to your usage. As mentioned previously, Docker can copy data from a specified VOLUME into a volume it creates. If you are mounting a host directory you probably need to do the initialisation at run time.
mongorestore requires a running server to restore to. During a build you would need to launch the server and restore in the same RUN step. At runtime you might need to include a startup script that checks for existence of your database.
Mongo is able to initialise any empty directory into a blank mongo instance so you don't need to be worried about mongo not starting.
It is quite easy to run MongoDB containerised using docker. Though each time you start a new mongodb container, you will get new empty database.
What should I do in order to keep the database content between container restarts? I tried to bind external directory to container using -v option but without any success.
I tried using the ehazlett/mongodb image and it worked fine.
With this image, you can easily specify where mongo store its data with DATA_DIR env variable. I am sure it must not be very difficult to change on your image too.
Here is what I did:
mkdir test; docker run -v `pwd`/test:/tmp/mongo -e DATA_DIR=/tmp/mongo ehazlett/mongodb
notice the `pwd` in within the -v, as the server and the client might have different path, it is important to specify the absolute path.
With this command, I can run mongo as many time as I want and the database will always be store in the ./test directory I just created.
When using the official Mongo docker image, which is i.e. version mongo:4.2.2-bionic as writing this answer, and using docker-compose, you can achieve persistent data storage using this docker-compose.yml file example.
In the official mongo image, data is stored in the container under the root directory in the folder /data/db by default.
Map this folder to a folder in your local working directory called data (in this example).
Make sure ports are set and mapped, default 27017-27019:27017-27019.
Example of my docker-compose.yml:
version: "3.2"
services:
mongodb:
image: mongo:4.2.2-bionic
container_name: mongodb
restart: unless-stopped
ports:
- 27017-27019:27017-27019
volumes:
- ./data:/data/db
Run docker-compose up in the directory where the yml file is located to run the mongodb container with persistent storage. If you do not have the official image yet, it will pull it from Dockerhub first.
Old post but may be someone still need quick and easy solution...
The easiest way I found is using binding to volume.
Following that way you can easily attach existing MongoDB data; and it will live even after you destroying the container.
Create volume that points to your folder (may include existing db). In my case it's done under Windows, but you can do it on any file system:
docker volume create --opt type=none --opt o=bind --opt device=d:/data/db db
Create/run docker container with MongoDB using that volume binding:
docker run --name mongodb -d -p 27017:27017 -v db:/data/db mongo