I installed mongodb and then I created a service to run it.
Starting the service works without problem but while trying to shutdown it, I get an error code by windows. I checked the log file and this is what I get:
2017-02-23T08:36:51.518+0100 I CONTROL [serviceStopWorker] shutting down with code:49
(Translation)
Error 1067: the process terminated unexpectedly
Service path:
"C:\Program Files\MongoDB\Server\3.4\bin\mongod.exe" --config "C:\Program Files\MongoDB\Server\3.4\bin\mongod.cfg" --service
mongodb.cfg
systemLog:
destination: file
path: "C:\\data\\log\\mongod.log"
storage:
dbPath: "C:\\data\\db"
I tried to find the error codes but it doesn't appear.
https://docs.mongodb.com/manual/reference/exit-codes/
The only reference I could find is this. A test with questions/answers about exit codes. (Question #7)
http://www.sanfoundry.com/mongodb-questions-answers-exit-codes-statutes/
7. 49 code is returned by mongod from _________ Service control Manager.
a) Linux
b) Windows
c) Ubuntu
d) All of the mentioned
View Answer
Answer:a
Explanation:Custom read preferences and write concerns evaluate tags sets in different ways.
But I am on a Windows 7 machine. So that's quite odd.
Windows 7 x64 SP1
MongoDB 3.4.2
Any idea?
LOG
2017-02-23T08:36:48.484+0100 I CONTROL [main] Trying to start Windows service 'MongoDB'
2017-02-23T08:36:48.485+0100 I CONTROL [initandlisten] MongoDB starting : pid=17856 port=27017 dbpath=C:\data\db 64-bit host=FRAmdsWS430
2017-02-23T08:36:48.485+0100 I CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2017-02-23T08:36:48.485+0100 I CONTROL [initandlisten] db version v3.4.2
2017-02-23T08:36:48.485+0100 I CONTROL [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
2017-02-23T08:36:48.485+0100 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1u-fips 22 Sep 2016
2017-02-23T08:36:48.485+0100 I CONTROL [initandlisten] allocator: tcmalloc
2017-02-23T08:36:48.485+0100 I CONTROL [initandlisten] modules: none
2017-02-23T08:36:48.486+0100 I CONTROL [initandlisten] build environment:
2017-02-23T08:36:48.486+0100 I CONTROL [initandlisten] distmod: 2008plus-ssl
2017-02-23T08:36:48.486+0100 I CONTROL [initandlisten] distarch: x86_64
2017-02-23T08:36:48.486+0100 I CONTROL [initandlisten] target_arch: x86_64
2017-02-23T08:36:48.486+0100 I CONTROL [initandlisten] options: { config: "C:\Program Files\MongoDB\Server\3.4\bin\mongod.cfg", service: true, storage: { dbPath: "C:\data\db" }, systemLog: { destination: "file", path: "C:\data\log\mongod.log" } }
2017-02-23T08:36:48.488+0100 I - [initandlisten] Detected data files in C:\data\db created by the 'mmapv1' storage engine, so setting the active storage engine to 'mmapv1'.
2017-02-23T08:36:48.497+0100 I JOURNAL [initandlisten] journal dir=C:\data\db\journal
2017-02-23T08:36:48.497+0100 I JOURNAL [initandlisten] recover : no journal files present, no recovery needed
2017-02-23T08:36:48.636+0100 I JOURNAL [durability] Durability thread started
2017-02-23T08:36:48.636+0100 I JOURNAL [journal writer] Journal writer thread started
2017-02-23T08:36:48.693+0100 I CONTROL [initandlisten]
2017-02-23T08:36:48.693+0100 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-02-23T08:36:48.693+0100 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2017-02-23T08:36:48.693+0100 I CONTROL [initandlisten]
2017-02-23T08:36:48.938+0100 W FTDC [initandlisten] Failed to initialize Performance Counters for FTDC: WindowsPdhError: PdhExpandCounterPathW failed with 'Das angegebene Objekt wurde nicht auf dem Computer gefunden.' for counter '\Memory\Available Bytes'
2017-02-23T08:36:48.938+0100 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory 'C:/data/db/diagnostic.data'
2017-02-23T08:36:48.940+0100 I NETWORK [thread1] waiting for connections on port 27017
2017-02-23T08:36:48.940+0100 I STORAGE [initandlisten] Service running
2017-02-23T08:36:51.412+0100 I CONTROL [serviceShutdown] got SERVICE_CONTROL_STOP request from Windows Service Control Manager, will terminate after current cmd ends
2017-02-23T08:36:51.412+0100 I NETWORK [serviceShutdown] shutdown: going to close listening sockets...
2017-02-23T08:36:51.412+0100 I NETWORK [serviceShutdown] closing listening socket: 456
2017-02-23T08:36:51.413+0100 I NETWORK [serviceShutdown] shutdown: going to flush diaglog...
2017-02-23T08:36:51.413+0100 I FTDC [serviceShutdown] Shutting down full-time diagnostic data capture
2017-02-23T08:36:51.413+0100 I STORAGE [serviceShutdown] shutdown: waiting for fs preallocator...
2017-02-23T08:36:51.413+0100 I STORAGE [serviceShutdown] shutdown: final commit...
2017-02-23T08:36:51.438+0100 I JOURNAL [serviceShutdown] journalCleanup...
2017-02-23T08:36:51.439+0100 I JOURNAL [serviceShutdown] removeJournalFiles
2017-02-23T08:36:51.439+0100 I JOURNAL [serviceShutdown] old journal file will be removed: C:\data\db\journal\j._0
2017-02-23T08:36:51.439+0100 I JOURNAL [serviceShutdown] Terminating durability thread ...
2017-02-23T08:36:51.516+0100 I JOURNAL [journal writer] Journal writer thread stopped
2017-02-23T08:36:51.516+0100 I JOURNAL [durability] Durability thread stopped
2017-02-23T08:36:51.516+0100 I STORAGE [serviceShutdown] shutdown: closing all files...
2017-02-23T08:36:51.518+0100 I STORAGE [serviceShutdown] closeAllFiles() finished
2017-02-23T08:36:51.518+0100 I STORAGE [serviceShutdown] shutdown: removing fs lock...
2017-02-23T08:36:51.518+0100 I CONTROL [serviceShutdown] now exiting
2017-02-23T08:36:51.518+0100 I CONTROL [serviceStopWorker] shutting down with code:49
This seems to be a bug introduced in MongoDB version 3.4.2. Check out https://jira.mongodb.org/browse/SERVER-6065
Related
For context: I am creating a website in html+ejs+mongoose which I had created locally on my pc. Since I want to see the website running on another device besides my local computer, I uploaded the files on GitHub and then cloned it into an online IDE, in this case, GoormIDE, to do that.
As the title above had said, I can't run mongod in a container even though I have enabled mongoDB in the container setup. I created a file which is mongod which runs the code mongod --nojournal so that I will not have journal files in my directory. I kept receiving this error code:
2020-12-27T05:24:49.128+0000 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] MongoDB starting : pid=802 port=27017 dbpath=/data/db 64-bit host=goorm
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] db version v4.0.22
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] git version: 1741806fb46c161a1d42870f6e98f5100d196315
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.1.1 11 Sep 2018
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] allocator: tcmalloc
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] modules: none
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] build environment:
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] distmod: ubuntu1804
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] distarch: x86_64
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] target_arch: x86_64
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] 1024 MB of memory available to the process out of 7771 MB total system memory
2020-12-27T05:24:49.135+0000 I CONTROL [initandlisten] options: { storage: { journal: { enabled: false } } }
2020-12-27T05:24:49.135+0000 I STORAGE [initandlisten] exception in initAndListen: NonExistentPath: Data directory /data/db not found. Create the missing directory or specify another path using (1) the --dbpath command line option, or (2) by adding the 'storage.dbPath' option in the configuration file., terminating
2020-12-27T05:24:49.136+0000 I REPL [initandlisten] Stepping down the ReplicationCoordinator for shutdown, waitTime: 10000ms
2020-12-27T05:24:49.136+0000 I NETWORK [initandlisten] shutdown: going to close listening sockets...
2020-12-27T05:24:49.136+0000 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27017.sock
2020-12-27T05:24:49.136+0000 I NETWORK [initandlisten] Shutting down the global connection pool
2020-12-27T05:24:49.136+0000 I - [initandlisten] Killing all operations for shutdown
2020-12-27T05:24:49.136+0000 I NETWORK [initandlisten] Shutting down the ReplicaSetMonitor
2020-12-27T05:24:49.136+0000 I CONTROL [initandlisten] Shutting down free monitoring
2020-12-27T05:24:49.136+0000 I FTDC [initandlisten] Shutting down full-time data capture
2020-12-27T05:24:49.136+0000 I STORAGE [initandlisten] Shutting down the HealthLog
2020-12-27T05:24:49.136+0000 I - [initandlisten] Dropping the scope cache for shutdown
2020-12-27T05:24:49.136+0000 I CONTROL [initandlisten] now exiting
2020-12-27T05:24:49.136+0000 I CONTROL [initandlisten] shutting down with code:100
Because of this, I can't start the website because it needs mongoDB/mongoose in order to run the server.
It seems that mongod/mongoDB doesn't have a data directory. Any ideas?
I already found the answer guys!
I just need to put -dbpath ./data/db after mongod
With that, I created a directory of data/db where I will store all the data from mongoDB.
so now I edited the file that I created named "mongod" with: mongod --nojournal -dbpath ./data/db and everything works once again. I feel pretty silly not knowing this but I hope I will never make this mistake again.
yesterday when i was about to install mongodb, i created /data/db with the right permission and i typed mongod but it got stuck on [ftdc] Marking collection local.oplog.rs as collection version: since 1 hour, thanks if you take time to help me
I was having the same problem. Open a new terminal, cd into the same path and then run "mongo" command. Then it worked!
I think we have had the same problem and I found the solution for myself.
But I beg my pardon if I am wrong)
If you want to get access to Mongodb you have to type mongo instead of mongod.
I was doing a step by step tutorial on installation of MongoDB on Centos 7 and playing around with it.
At some point I've decided to shutdown the server with:
db.shutdownServer()
And after that I've had trouble starting/logging into the database with mongod.
But the thing is, that mongod starts the Mongodb server in your current terminal.
[root#ip-*-*-*-* tmp]# mongod
2020-06-09T19:52:14.630+0000 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-06-09T19:52:14.632+0000 W ASIO [main] No TransportLayer configured during NetworkInterface startup
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] MongoDB starting : pid=27618 port=27017 dbpath=/data/db 64-bit host=ip-*-*-*-*.eu-central-1.compute.internal
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] db version v4.2.7
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] git version: 51d9fe12b5d19720e72dcd7db0f2f17dd9a19212
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] allocator: tcmalloc
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] modules: none
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] build environment:
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] distmod: rhel70
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] distarch: x86_64
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] target_arch: x86_64
2020-06-09T19:52:14.632+0000 I CONTROL [initandlisten] options: { storage: { dbPath: "/data/db" } }
2020-06-09T19:52:14.633+0000 I STORAGE [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-06-09T19:52:14.633+0000 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=256M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
2020-06-09T19:52:15.356+0000 I STORAGE [initandlisten] WiredTiger message [1591732335:356822][27618:0x7f9b34a81c40], txn-recover: Recovering log 6 through 7
2020-06-09T19:52:15.412+0000 I STORAGE [initandlisten] WiredTiger message [1591732335:412272][27618:0x7f9b34a81c40], txn-recover: Recovering log 7 through 7
2020-06-09T19:52:15.500+0000 I STORAGE [initandlisten] WiredTiger message [1591732335:500634][27618:0x7f9b34a81c40], txn-recover: Main recovery loop: starting at 6/22528 to 7/256
2020-06-09T19:52:15.609+0000 I STORAGE [initandlisten] WiredTiger message [1591732335:609015][27618:0x7f9b34a81c40], txn-recover: Recovering log 6 through 7
2020-06-09T19:52:15.672+0000 I STORAGE [initandlisten] WiredTiger message [1591732335:672877][27618:0x7f9b34a81c40], txn-recover: Recovering log 7 through 7
2020-06-09T19:52:15.727+0000 I STORAGE [initandlisten] WiredTiger message [1591732335:727038][27618:0x7f9b34a81c40], txn-recover: Set global recovery timestamp: (0, 0)
2020-06-09T19:52:15.735+0000 I RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0)
2020-06-09T19:52:15.739+0000 I STORAGE [initandlisten] Timestamp monitor starting
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten]
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten]
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** WARNING: This server is bound to localhost.
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** Remote systems will be unable to connect to this server.
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** Start the server with --bind_ip <address> to specify which IP
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** addresses it should serve responses from, or with --bind_ip_all to
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** bind to all interfaces. If this behavior is desired, start the
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** server with --bind_ip 127.0.0.1 to disable this warning.
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten]
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten]
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten]
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2020-06-09T19:52:15.741+0000 I CONTROL [initandlisten]
2020-06-09T19:52:15.743+0000 I SHARDING [initandlisten] Marking collection local.system.replset as collection version: <unsharded>
2020-06-09T19:52:15.744+0000 I STORAGE [initandlisten] Flow Control is enabled on this deployment.
2020-06-09T19:52:15.744+0000 I SHARDING [initandlisten] Marking collection admin.system.roles as collection version: <unsharded>
2020-06-09T19:52:15.745+0000 I SHARDING [initandlisten] Marking collection admin.system.version as collection version: <unsharded>
2020-06-09T19:52:15.746+0000 I SHARDING [initandlisten] Marking collection local.startup_log as collection version: <unsharded>
2020-06-09T19:52:15.746+0000 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
2020-06-09T19:52:15.748+0000 I SHARDING [LogicalSessionCacheRefresh] Marking collection config.system.sessions as collection version: <unsharded>
2020-06-09T19:52:15.749+0000 I SHARDING [LogicalSessionCacheReap] Marking collection config.transactions as collection version: <unsharded>
2020-06-09T19:52:15.749+0000 I NETWORK [listener] Listening on /tmp/mongodb-27017.sock
2020-06-09T19:52:15.749+0000 I NETWORK [listener] Listening on 127.0.0.1
2020-06-09T19:52:15.749+0000 I NETWORK [listener] waiting for connections on port 27017
2020-06-09T19:52:16.000+0000 I SHARDING [ftdc] Marking collection local.oplog.rs as collection version: <unsharded>
And I just got stuck at this point and pressed ctrl+C
2020-06-09T19:56:26.172+0000 I CONTROL [signalProcessingThread] got signal 2 (Interrupt), will terminate after current cmd ends
2020-06-09T19:56:26.172+0000 I NETWORK [signalProcessingThread] shutdown: going to close listening sockets...
2020-06-09T19:56:26.176+0000 I NETWORK [listener] removing socket file: /tmp/mongodb-27017.sock
2020-06-09T19:56:26.176+0000 I - [signalProcessingThread] Stopping further Flow Control ticket acquisitions.
2020-06-09T19:56:26.176+0000 I CONTROL [signalProcessingThread] Shutting down free monitoring
2020-06-09T19:56:26.176+0000 I FTDC [signalProcessingThread] Shutting down full-time diagnostic data capture
2020-06-09T19:56:26.180+0000 I STORAGE [signalProcessingThread] Deregistering all the collections
2020-06-09T19:56:26.180+0000 I STORAGE [signalProcessingThread] Timestamp monitor shutting down
2020-06-09T19:56:26.180+0000 I STORAGE [signalProcessingThread] WiredTigerKVEngine shutting down
2020-06-09T19:56:26.180+0000 I STORAGE [signalProcessingThread] Shutting down session sweeper thread
2020-06-09T19:56:26.180+0000 I STORAGE [signalProcessingThread] Finished shutting down session sweeper thread
2020-06-09T19:56:26.180+0000 I STORAGE [signalProcessingThread] Shutting down journal flusher thread
2020-06-09T19:56:26.264+0000 I STORAGE [signalProcessingThread] Finished shutting down journal flusher thread
2020-06-09T19:56:26.264+0000 I STORAGE [signalProcessingThread] Shutting down checkpoint thread
2020-06-09T19:56:26.264+0000 I STORAGE [signalProcessingThread] Finished shutting down checkpoint thread
2020-06-09T19:56:26.268+0000 I STORAGE [signalProcessingThread] shutdown: removing fs lock...
2020-06-09T19:56:26.268+0000 I CONTROL [signalProcessingThread] now exiting
2020-06-09T19:56:26.268+0000 I CONTROL [signalProcessingThread] shutting down with code:0
And at this point I realized that I had that fatal extra letter and I was able to access the database with
[root#ip-*-*-*-* tmp]# mongo
After many hours and few reinstalls, I decided to look and view if some files was "corrupt", I found that MongoDB doesn't delete its folder in /var/lib/, so I deleted it and when I reinstalled MongoDB again it worked like a charm.
I am trying to install MEAN stack components. So I tried to install mongodb. The packages from the repository installed correctly but running it seems to have issues.
Post installation I tried:
sudo systemctl enable mongodb
which is resulted in:
● mongodb.service
Loaded: masked (/dev/null)
Active: failed (Result: exit-code) since Thu 2016-10-27 15:22:36 EDT; 17min ago
Main PID: 22947 (code=exited, status=14)
Hint: Some lines were ellipsized, use -l to show in full.
and then I tried: sudo systemctl enable mongodb
Executing /usr/sbin/update-rc.d mongodb defaults
Executing /usr/sbin/update-rc.d mongodb enable
Failed to execute operation: Operation not supported
Any idea why it is not starting? I can't figure it out. I am using Xubuntu 15.04.
Log File says:
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] MongoDB starting : pid=26138 port=27017 dbpath=/var/lib/mongodb 64-bit host=System-Product-Name
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] db version v3.2.10
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] git version: 79d9b3ab5ce20f51c272b4411202710a082d0317
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] allocator: tcmalloc
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] modules: none
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] build environment:
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] distmod: ubuntu1404
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] distarch: x86_64
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] target_arch: x86_64
2016-10-27T15:47:02.747-0400 I CONTROL [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2016-10-27T15:47:02.764-0400 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=4G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten]
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten]
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten]
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2016-10-27T15:47:02.883-0400 I CONTROL [initandlisten]
2016-10-27T15:47:02.884-0400 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongodb/diagnostic.data'
2016-10-27T15:47:02.884-0400 I NETWORK [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-10-27T15:47:02.918-0400 I NETWORK [initandlisten] waiting for connections on port 27017
2016-10-27T15:48:23.409-0400 I CONTROL [signalProcessingThread] got signal 2 (Interrupt), will terminate after current cmd ends
2016-10-27T15:48:23.409-0400 I FTDC [signalProcessingThread] Shutting down full-time diagnostic data capture
2016-10-27T15:48:23.413-0400 I CONTROL [signalProcessingThread] now exiting
2016-10-27T15:48:23.413-0400 I NETWORK [signalProcessingThread] shutdown: going to close listening sockets...
2016-10-27T15:48:23.413-0400 I NETWORK [signalProcessingThread] closing listening socket: 6
2016-10-27T15:48:23.413-0400 I NETWORK [signalProcessingThread] closing listening socket: 7
2016-10-27T15:48:23.413-0400 I NETWORK [signalProcessingThread] removing socket file: /tmp/mongodb-27017.sock
2016-10-27T15:48:23.413-0400 I NETWORK [signalProcessingThread] shutdown: going to flush diaglog...
2016-10-27T15:48:23.413-0400 I NETWORK [signalProcessingThread] shutdown: going to close sockets...
2016-10-27T15:48:23.413-0400 I STORAGE [signalProcessingThread] WiredTigerKVEngine shutting down
2016-10-27T15:48:23.511-0400 I STORAGE [signalProcessingThread] shutdown: removing fs lock...
2016-10-27T15:48:23.512-0400 I CONTROL [signalProcessingThread] dbexit: rc: 0
I have a MMS replica set deployed with 3 instances. It was working fine until I shut it down this morning in order to do some changes and restarted it. From logs, it feels that there was an unclean shutdown of replica set. The mongod server on primary is failing to start now.
Here are the full logs
2015-08-20T07:08:41.389+0000 W - [initandlisten] Detected unclean shutdown - /data/XXXXXXXXX/mongod.lock is not empty.
2015-08-20T07:08:41.406+0000 I JOURNAL [initandlisten] journal dir=/data/XXXXXXXXX/journal
2015-08-20T07:08:41.406+0000 I JOURNAL [initandlisten] recover begin
2015-08-20T07:08:41.406+0000 I JOURNAL [initandlisten] info no lsn file in journal/ directory
2015-08-20T07:08:41.406+0000 I JOURNAL [initandlisten] recover lsn: 0
2015-08-20T07:08:41.406+0000 I JOURNAL [initandlisten] recover /data/XXXXXXXXX/journal/j._0
2015-08-20T07:08:41.407+0000 I JOURNAL [initandlisten] recover cleaning up
2015-08-20T07:08:41.407+0000 I JOURNAL [initandlisten] removeJournalFiles
2015-08-20T07:08:41.641+0000 I JOURNAL [initandlisten] recover done
2015-08-20T07:08:41.641+0000 I JOURNAL [initandlisten] preallocating a journal file /data/XXXXXXXXXXX/journal/prealloc.0
2015-08-20T07:08:44.074+0000 I - [initandlisten] File Preallocator Progress: 744488960/1073741824 69%
2015-08-20T07:08:47.176+0000 I - [initandlisten] File Preallocator Progress: 901775360/1073741824 83%
2015-08-20T07:08:50.274+0000 I - [initandlisten] File Preallocator Progress: 1027604480/1073741824 95%
2015-08-20T07:09:09.057+0000 I JOURNAL [durability] Durability thread started
2015-08-20T07:09:09.057+0000 I JOURNAL [journal writer] Journal writer thread started
2015-08-20T07:09:09.060+0000 I CONTROL [initandlisten] MongoDB starting : pid=25507 port=27000 dbpath=/data/XXXXXXXXXXXXX 64-bit host=CH$
2015-08-20T07:09:09.060+0000 I CONTROL [initandlisten] db version v3.0.2
2015-08-20T07:09:09.060+0000 I CONTROL [initandlisten] git version: 6201872043ecbbc0a4cc169b5482dcf385fc464f
2015-08-20T07:09:09.060+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
2015-08-20T07:09:09.060+0000 I CONTROL [initandlisten] build info: Linux ip-10-229-1-2 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10 19:11:08 U$
2015-08-20T07:09:09.060+0000 I CONTROL [initandlisten] allocator: tcmalloc
2015-08-20T07:09:09.060+0000 I CONTROL [initandlisten] options: { config: "/data/XXXXXXXXXXXX/automation-mongod.conf", net: { port: 270$
2015-08-20T07:09:09.074+0000 I - [initandlisten] Invariant failure _name == nsToDatabaseSubstring( ns ) src/mongo/db/catalog/database.c$
2015-08-20T07:09:09.091+0000 I CONTROL [initandlisten]
0xf4f859 0xef0031 0xed4b52 0x91e106 0x91e18f 0x920033 0x922cb0 0x808701 0x7d4ba4 0x7f0503489ec5 0x805d17
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"B4F859"},{"b":"400000","o":"AF0031"},{"b":"400000","o":"AD4B52"},{"b":"400000","o":"51E106"},{"b":"400000","o$
mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf4f859]
mongod(_ZN5mongo10logContextEPKc+0xE1) [0xef0031]
mongod(_ZN5mongo15invariantFailedEPKcS1_j+0xB2) [0xed4b52]
mongod(_ZNK5mongo8Database13getCollectionERKNS_10StringDataE+0x36) [0x91e106]
mongod(_ZN5mongo8Database30_getOrCreateCollectionInstanceEPNS_16OperationContextERKNS_10StringDataE+0x1F) [0x91e18f]
mongod(_ZN5mongo8DatabaseC1EPNS_16OperationContextERKNS_10StringDataEPNS_20DatabaseCatalogEntryE+0x1E3) [0x920033]
mongod(_ZN5mongo14DatabaseHolder6openDbEPNS_16OperationContextERKNS_10StringDataEPb+0x150) [0x922cb0]
mongod(_ZN5mongo14DatabaseHolder6openDbEPNS_16OperationContextERKNS_10StringDataEPb+0x150) [0x922cb0]
mongod(_ZN5mongo13initAndListenEi+0xC01) [0x808701]
mongod(main+0x134) [0x7d4ba4]
libc.so.6(__libc_start_main+0xF5) [0x7f0503489ec5]
mongod(+0x405D17) [0x805d17]
----- END BACKTRACE -----
2015-08-20T07:09:09.091+0000 I - [initandlisten]
***aborting after invariant() failure
Any idea how can I fix it? I have been trying to fix it since past 4 hours but nothing seems to be working.
So, it turned out that issue was in .ns file of one of the database. I had to delete that file and restart the server. Server started successfully however, the database whose .ns I deleted was lost.
For newbies in mongoDB .ns file is the namespace file that mongo creates for each database in its data (/data) directory.
I am a total mongo newbie who is very confused, frustrated and frankly lost...
I have tried installing / uninstalling mongo, then reinstalling. AT one point it seemed to be working as I followed a tutorial but ultimately wouldn't after I returned to use it in a node app I am trying to create. Now that I have it installed I created the /data/db but it keeps failing when I go to run "mongod"
name$ mongod
2015-06-28T15:13:34.503-0400 E NETWORK [initandlisten] listen(): bind() failed errno:48 Address already in use for socket: 0.0.0.0:27017
2015-06-28T15:13:34.503-0400 E NETWORK [initandlisten] addr already in use
2015-06-28T15:13:34.513-0400 I JOURNAL [initandlisten] journal dir=/data/db/journal
2015-06-28T15:13:34.513-0400 I JOURNAL [initandlisten] recover : no journal files present, no recovery needed
2015-06-28T15:13:34.541-0400 I JOURNAL [durability] Durability thread started
2015-06-28T15:13:34.541-0400 I JOURNAL [journal writer] Journal writer thread started
2015-06-28T15:13:34.541-0400 I CONTROL [initandlisten] MongoDB starting : pid=4041 port=27017 dbpath=/data/db 64-bit host=Vickens-MBP.home
2015-06-28T15:13:34.541-0400 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2015-06-28T15:13:34.541-0400 I CONTROL [initandlisten]
2015-06-28T15:13:34.541-0400 I CONTROL [initandlisten]
2015-06-28T15:13:34.541-0400 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
2015-06-28T15:13:34.542-0400 I CONTROL [initandlisten] db version v3.0.4
2015-06-28T15:13:34.542-0400 I CONTROL [initandlisten] git version: nogitversion
2015-06-28T15:13:34.542-0400 I CONTROL [initandlisten] build info: Darwin yosemitevm.local 14.3.0 Darwin Kernel Version 14.3.0: Mon Mar 23 11:59:05 PDT 2015; root:xnu-2782.20.48~5/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
2015-06-28T15:13:34.542-0400 I CONTROL [initandlisten] allocator: system
2015-06-28T15:13:34.542-0400 I CONTROL [initandlisten] options: {}
2015-06-28T15:13:34.551-0400 I CONTROL [initandlisten] now exiting
2015-06-28T15:13:34.551-0400 I NETWORK [initandlisten] shutdown: going to close listening sockets...
2015-06-28T15:13:34.551-0400 I NETWORK [initandlisten] shutdown: going to flush diaglog...
2015-06-28T15:13:34.551-0400 I NETWORK [initandlisten] shutdown: going to close sockets...
2015-06-28T15:13:34.551-0400 I STORAGE [initandlisten] shutdown: waiting for fs preallocator...
2015-06-28T15:13:34.552-0400 I STORAGE [initandlisten] shutdown: final commit...
2015-06-28T15:13:34.555-0400 I JOURNAL [initandlisten] journalCleanup...
2015-06-28T15:13:34.555-0400 I JOURNAL [initandlisten] removeJournalFiles
2015-06-28T15:13:34.556-0400 I JOURNAL [initandlisten] Terminating durability thread ...
2015-06-28T15:13:34.663-0400 I JOURNAL [journal writer] Journal writer thread stopped
2015-06-28T15:13:34.663-0400 I JOURNAL [durability] Durability thread stopped
2015-06-28T15:13:34.664-0400 I STORAGE [initandlisten] shutdown: closing all files...
2015-06-28T15:13:34.665-0400 I STORAGE [initandlisten] closeAllFiles() finished
2015-06-28T15:13:34.665-0400 I STORAGE [initandlisten] shutdown: removing fs lock...
2015-06-28T15:13:34.665-0400 I CONTROL [initandlisten] dbexit: rc: 48
Looks like something is running on 0.0.0.0:27017,
run command to kill that process first
kill $(lsof -t -i:27017)
then start you mongo server
sudo rm /var/lib/mongodb/mongod.lock
mongod --repair
sudo service mongodb start