Windows Docker mongo container doesn't work with volume mount - mongodb

I have the following docker command
docker run -v //c/data:/data/db mongo
and I get the following error response from docker / mongo
MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=8706cbf1b78f
db version v3.4.2
git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
OpenSSL version: OpenSSL 1.0.1t 3 May 2016
allocator: tcmalloc
modules: none
build environment:
distmod: debian81
distarch: x86_64
target_arch: x86_64
options: {}
wiredtiger_open config: create,cache_size=478M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
WiredTiger error (1) [1489982988:687653][1:0x7fec9df0ccc0], connection: /data/db/WiredTiger.wt: handle-open: open: Operation not permitted
Assertion: 28595:1: Operation not permitted src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 267
exception in initAndListen: 28595 1: Operation not permitted, terminating
shutdown: going to close listening sockets...
removing socket file: /tmp/mongodb-27017.sock
shutdown: going to flush diaglog...
now exiting
shutting down with code:100
now when I remove the volume mongo works but I need to persist my data so I need to mount the volume somehow, just not sure what i am doing wrong at this point.
I do see the files appearing in my folder but not sure why I get the 100 error

To get around this, you can employ a tool like rsync to move the db files into mapped directory while Mongo is running. The underlying bug has to do with latency between the Windows mapped volume and that bind path within the container. Offloading the work to rsync decouples the latency from Mongo's runtime requirements.
Example
Create a basic Dockerfile like this:
FROM mongo:latest
RUN apt-get update && \
apt-get install -y \
rsync
ADD init.sh /init.sh
Where init.sh is:
#!/bin/bash
migrate_db() {
while true
do
rsync -avh /data/db/* /data/mapped-db
sleep 5
done
}
migrate_db &
#Execute a command
mongod --smallfiles --logpath=/dev/null --verbose &
#Wait
wait $!
Then, when launching the container, just start with ./init.sh as your ENTRYPOINT.

Related

loading docker container from existing project, mongo will not connect to database Ubuntu 18.04

At first I had a problem where the mongod service stopped working altogether. Thereupon I completely uninstalled mongodb and reinstalled it and followed instructions per this thread: https://askubuntu.com/questions/921753/failed-to-start-mongod-service-unit-mongod-service-not-found
This caused the mongod service to finally start.
In order for it to work, I also had to follow instructions from this thread:
mongo - couldn't connect to server 127.0.0.1:27017
So far this configuration has worked only 1 time and that was after I followed the instructions to remove the .lock file in the mongod directory and then repair mongod.
I also tried to remove the sock file like stated here: MongoDB Failing to Start - ***aborting after fassert() failure
Full project reset
Then I tried just disconnecting the project entirely; close nodemon for backend, exit ng serve on frontend, stop and remove mongo docker container... And tried again. Here is my step-by-step process after everything should have been fixed:
Restart mongod service
sudo service mongod restart
Restart frontend (executed from frontend directory)
ng serve frontend
Restart backend (executed from backend directory)
npx nodemon backend
Restart mongo container (executed from deploy directory, mongo is the docker container name, I'm building it from existing provided deploy script)
sudo docker-compose up -d mongo
sudo docker ps -a # check container was created and is running
Debugging information:
sudo service mongod status
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset:
Active: active (running) since Thu 2020-01-09 11:21:43 CET; 3min 40s ago
Docs: https://docs.mongodb.org/manual
Main PID: 17242 (mongod)
CGroup: /system.slice/mongod.service
└─17242 /usr/bin/mongod --config /etc/mongod.conf
janv. 09 11:21:43 junior-LIFEBOOK-E752 systemd[1]: Started MongoDB Database Server
sudo mongod2020-01-09T11:25:45.983+0100 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] MongoDB starting : pid=17727 port=27017 dbpath=/data/db 64-bit host=junior-LIFEBOOK-E752
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] db version v4.2.2
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] git version: a0bbbff6ada159e19298d37946ac8dc4b497eadf
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.1.1d 10 Sep 2019
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] allocator: tcmalloc
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] modules: none
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] build environment:
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] distmod: ubuntu1804
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] distarch: x86_64
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] target_arch: x86_64
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] options: {}
2020-01-09T11:25:45.987+0100 E STORAGE [initandlisten] Failed to set up listener: SocketException: Address already in use
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] now exiting
2020-01-09T11:25:45.987+0100 I CONTROL [initandlisten] shutting down with code:48
sudo mongo
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("8e813fc2-049c-440d-83ce-0b1d8516c4d0") }
MongoDB server version: 4.2.2
Server has startup warnings:
2020-01-09T11:21:43.352+0100 I STORAGE [initandlisten]
2020-01-09T11:21:43.352+0100 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-01-09T11:21:43.352+0100 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-01-09T11:21:44.848+0100 I CONTROL [initandlisten]
2020-01-09T11:21:44.848+0100 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2020-01-09T11:21:44.848+0100 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2020-01-09T11:21:44.848+0100 I CONTROL [initandlisten]
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
So something weird is happening here, it doesn't find the database from the project and I have no clue why. As far as I can see, it creates a new database including my system details.
How can I fix this please and connect to the project database? I can't move further until I do.
it's been resolved. First I found out I didn't need to have the mongod service running locally because mongo was already included in the docker container. But actually this wasn't a database issue at all. There were a few things going wrong like there was a proxy but the command had been changed to execute the connection, endpoints were on localhost but weren't supposed to be, and the content I was expecting to see had been hidden by a careless push from another developer. Fun times! But we can move on now. Thanks to the people who tried to help!

mongodb works when executing the binary, but not when starting the service ("Unable to determine status of lock file")

I know this is a duplicate, but all answers suggest either changing owner of the data directory to mongodb:mongodb or permissions to 777. I've tried both and it hasn't worked.
I have a machine whose sole purpose is to run mongodb. I'm logged
in as root. I'd like to change my data directory to /root/mongodb. I have updated dbpath in /etc/mongod.conf.
If I run systemctl start mongod or sudo systemctl start mongod, I get the following error:
2018-02-07T18:39:53.246+0100 I STORAGE [initandlisten] exception in initAndListen: 28596 Unable to determine status of lock file in the data directory /root/mongodb: boost::filesystem::status: Permission denied: "/root/mongodb/mongod.lock", terminating
I've tried chown -R mongodb:mongodb /root/mongodb and chmod -R 777 /root/mongodb, but still see the same error. This are the permissions after running both:
drwxrwxrwx 4 mongodb mongodb 4096 Feb 7 18:27 mongodb
However, if I run mongodb --dbpath=/root/mongodb, it starts up fine.
Version info:
db version v3.4.11
git version: 34f5bec2c9d827d71828fe858167f89a28b29a2a
OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
allocator: tcmalloc
modules: none
build environment:
distmod: ubuntu1604
distarch: x86_64
target_arch: x86_64
Any thoughts on how to fix this?

mongodb server is not starting, mac os

When running:
mongod
I get:
2017-04-30T11:19:56.234+0600 I CONTROL [initandlisten] MongoDB starting : pid=1160 port=27017 dbpath=/data/db 64-bit host=Interstellar
2017-04-30T11:19:56.234+0600 I CONTROL [initandlisten] db version v3.4.4
2017-04-30T11:19:56.234+0600 I CONTROL [initandlisten] git version: 888390515874a9debd1b6c5d36559ca86b44babd
2017-04-30T11:19:56.234+0600 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.2k 26 Jan 2017
2017-04-30T11:19:56.234+0600 I CONTROL [initandlisten] allocator: system
2017-04-30T11:19:56.234+0600 I CONTROL [initandlisten] modules: none
2017-04-30T11:19:56.234+0600 I CONTROL [initandlisten] build environment:
2017-04-30T11:19:56.235+0600 I CONTROL [initandlisten] distarch: x86_64
2017-04-30T11:19:56.235+0600 I CONTROL [initandlisten] target_arch: x86_64
2017-04-30T11:19:56.235+0600 I CONTROL [initandlisten] options: {}
2017-04-30T11:19:56.235+0600 I STORAGE [initandlisten] exception in
initAndListen: 29 Data directory /data/db not found., terminating
2017-04-30T11:19:56.235+0600 I NETWORK [initandlisten] shutdown: going to close listening sockets...
2017-04-30T11:19:56.235+0600 I NETWORK [initandlisten] shutdown: going to flush diaglog...
2017-04-30T11:19:56.235+0600 I CONTROL [initandlisten] now exiting
2017-04-30T11:19:56.235+0600 I CONTROL [initandlisten] shutting down with code:100
I'm having trouble with this, badly need help
The problem is that if you run "mongod" the default path is /usr/data. NEVERTHELESS macOS Catalina does not accept creating a writable folder there. So after long research, in both here, and reddit, I finally got my server (mongod as well as Mongo) up and running. Try the following these steps:
mkdir -p /System/Volumes/Data/data/db
sudo chown -Rv <your_user_name> /System/Volumes/Data/data/db
sudo mongod --dbpath /System/Volumes/Data/data/db
This may run mongod and start the server. Open another terminal tab, and type Mongo.
First step
brew services start mongodb/brew/mongodb-community#4.4
Second step
cd /Users
whoami
cd to your whoami directory
mkdir data
cd data
mkdir db
mongod --dbpath ~/data/db
Third step
mongo
The problem is that there is no /data/db directory. Please issue this command:
sudo mkdir -p /data/db
If you're not worried about opening up permissions too widely, but just want to ensure that mongodb has permissions to start, you can also issue these commands. They may fix any problems you have if the above command alone doesn't solve things.
cd /
sudo chmod -R 777 data

Can't run mongod ***aborting after fassert() failure on mac

I was working through the mongodb installation video on Mongodb university.
https://www.youtube.com/watch?v=_WJ8m5QHvwc
I am a mac user and encountered this problem when I initially tried to run mongod command for the first time. I am a new mongodb user and I would appreciate help with this. Thank you.
Edit: The first step I took was finding the download page on google. I selected the first organic link. I did not download the cloud service. I downloaded it and installed it onto my machine. I downloaded the most recent production version. I used the mac download.
Next, after the downloading, I went into the downloads directory in my terminal to untar the file by
tar xvf mongodb-osx-ssl-x86_64-3.4.3.tgz
That expanded the tarball. I entered the directory using
cd mongodb-osx-ssl-x86_64-3.4.3
and looked around. I saw a licence, a readme, and a bin directory.
I went into the bin directory.
cd bin
Next, I looked inside the bin directory
ls -l
I saw a bunch of mongo programs. In here I found mongo (shell) and mongod (server). Now I understand that the server will put its data in /data/db by default.
Next I proceeded to make that directory. I went root in order to do this by running
sudo bash
I then ran
mkdir -p /data/db
chmod 777 /data
chmod 777 /data/db
Next, I exited the root shell. I then looked inside the bin directory.
Roberts-MacBook-Air:Downloads robertchecco$ cd mongodb-osx-x86_64-3.4.3
Roberts-MacBook-Air:mongodb-osx-x86_64-3.4.3 robertchecco$ ls
GNU-AGPL-3.0 README bin
MPL-2 THIRD-PARTY-NOTICES
Roberts-MacBook-Air:mongodb-osx-x86_64-3.4.3
robertchecco$ cd bin
Roberts-MacBook-Air:bin robertchecco$ ls
bsondump mongodump mongoimport mongoreplay mongostat
mongo mongoexport mongooplog mongorestore mongotop
mongod mongofiles mongoperf mongos
Roberts-MacBook-Air:bin robertchecco$ ./mongod
2017-03-28T23:07:48.262-0400 I CONTROL [initandlisten] MongoDB starting : pid=77503 port=27017 dbpath=/data/db 64-bit host=Roberts-MacBook-Air.local
2017-03-28T23:07:48.263-0400 I CONTROL [initandlisten] db version v3.4.3
2017-03-28T23:07:48.263-0400 I CONTROL [initandlisten] git version: f07437fb5a6cca07c10bafa78365456eb1d6d5e1
2017-03-28T23:07:48.264-0400 I CONTROL [initandlisten] OpenSSL version: OpenSSL 0.9.8zh 14 Jan 2016
2017-03-28T23:07:48.264-0400 I CONTROL [initandlisten] allocator: system
2017-03-28T23:07:48.264-0400 I CONTROL [initandlisten] modules: none
2017-03-28T23:07:48.264-0400 I CONTROL [initandlisten] build environment:
2017-03-28T23:07:48.264-0400 I CONTROL [initandlisten] distarch: x86_64
2017-03-28T23:07:48.264-0400 I CONTROL [initandlisten] target_arch: x86_64
2017-03-28T23:07:48.264-0400 I CONTROL [initandlisten] options: {}
2017-03-28T23:07:48.265-0400 W - [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty.
2017-03-28T23:07:48.267-0400 E NETWORK [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Permission denied
2017-03-28T23:07:48.267-0400 I - [initandlisten] Fatal Assertion 28578 at src/mongo/util/net/listen.cpp 194
2017-03-28T23:07:48.267-0400 I - [initandlisten]
***aborting after fassert() failure

How do I configure mongo to run in docker to using an external drive on windows

I am trying to setup a docker image that will run a mongo instance which points to a drive location that isn't within the default docker container.
I used the following commands to create the issue:
docker run -p 27017:27017 --name mongo_test -v //c/Users/usrname/mongo_test:/data/db -d mongo
the container instantly fails, when I run the docker container logs mongo_test command I get back the following dump:
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=4f14ca0711c7
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] db version v3.4.2
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1t 3 May 2016
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] allocator: tcmalloc
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] modules: none
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] build environment:
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] distmod: debian81
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] distarch: x86_64
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] target_arch: x86_64
2017-03-13T04:33:33.358+0000 I CONTROL [initandlisten] options: {}
2017-03-13T04:33:33.363+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=478M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2017-03-13T04:33:33.384+0000 E STORAGE [initandlisten] WiredTiger error (17) [1489379613:384669][1:0x7f1e6c4f5cc0], connection: /data/db/WiredTiger.wt: handle-open: open: File exists
2017-03-13T04:33:33.385+0000 I STORAGE [initandlisten] WiredTiger message unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.1
2017-03-13T04:33:33.386+0000 E STORAGE [initandlisten] WiredTiger error (1) [1489379613:386165][1:0x7f1e6c4f5cc0], connection: /data/db/WiredTiger.wt: handle-open: open: Operation not permitted
2017-03-13T04:33:33.386+0000 I - [initandlisten] Assertion: 28595:1: Operation not permitted src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 267
2017-03-13T04:33:33.387+0000 I STORAGE [initandlisten] exception in initAndListen: 28595 1: Operation not permitted, terminating
2017-03-13T04:33:33.387+0000 I NETWORK [initandlisten] shutdown: going to close listening sockets...
2017-03-13T04:33:33.387+0000 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27017.sock
2017-03-13T04:33:33.387+0000 I NETWORK [initandlisten] shutdown: going to flush diaglog...
2017-03-13T04:33:33.387+0000 I CONTROL [initandlisten] now exiting
2017-03-13T04:33:33.387+0000 I CONTROL [initandlisten] shutting down with code:100
I'm currently running windows 10 enterprise edition which came preinstalled with hyper-v instead of virutalbox. From what I've read virtualbox wont support the externally mapped drive for docker, but hyper-v should. I've also granted the Hyper-V Administrators user group full control of the file, as well as ensuring that docker has the drive mapped.
It can map the drive correctly, as I can run a LS command + it creates a journal folder, mongod.lock, WiredTiger, WiredTiger.lock, WiredTiger.wt, and WiredTiger.wt.1 file in the directory I mapped before it errors.
Any help would be great.
This won't work. MongoDB and mapped volumes in Docker on Windows are not compatible, as noted for example in the mongo docker image documentation:
WARNING (Windows & OS X): The default Docker setup on Windows and OS X uses a VirtualBox VM to host the Docker daemon. Unfortunately, the mechanism VirtualBox uses to share folders between the host system and the Docker container is not compatible with the memory mapped files used by MongoDB (see vbox bug, docs.mongodb.org and related jira.mongodb.org bug). This means that it is not possible to run a MongoDB container with the data directory mapped to the host.