Change storage engine of MongoDB - mongodb

I STORAGE [initandlisten] exception in initAndListen: 28574 Cannot start server. Detected data files in /data/db created by storage engine 'wiredTiger'. The configured storage engine is 'mmapv1'., terminating ....
https://docs.mongodb.com/manual/tutorial/change-standalone-wiredtiger/
I found this link thinking of any help but it says that you have to start mongod first which is not starting right now.

If you have multiple nodes in a replica set, changing from MMAPv1 to WiredTiger is as simple as for each node :
- stop the node
- delete the database file
- restart the node on WiredTiger (with --storageEngine wiredTiger if version less than 3.2)
The node will now replicate all it's database from the primary using the new format.
If you have a single node installation. You need to use mongodump/mongorestore to dump your database, stop the standalone node, remove the database directory, restart the node with WiredTiger and restore your backup.

Related

Can MongoDB failover in case of disk failure

From online document, it seems only when MongoDB instance is stopped and no heartbeat is detected, then election for failover will happen. But in case of bad disk or bad disk sector and MongoDB write failure to journal or datafile, how will MongoDB response? Will MongoDB instance crash and hence failover can happen after?
Today in MOngoDB bare metal setup, generally how system admin detect and handle disk failure? Thanks
Since MongoDB 4.2, Storage Node Watchdog is available in the community servers. From the linked page:
The Storage Node Watchdog monitors the following MongoDB directories to detect filesystem unresponsiveness:
The --dbpath directory
The journal directory inside the --dbpath directory if journaling is enabled
The directory of --logpath file
The directory of --auditPath file
If any of the filesystems containing the monitored directories become unresponsive, the Storage Node Watchdog terminates the mongod and exits with a status code of 61. If the mongod is the primary of a replica set, the termination initiates a failover, allowing another member to become primary.
One caveat:
If any of its monitored directories is a symlink to other volumes, the Storage Node Watchdog does not monitor the symlink target.

Using Mongo Cache as a replacement for redis

Currently, i am writing data into redis which is reduced to mongo every 2 minutes. I am wondering if i can use Mongo Cache instead of redis here. This will save the cost of 2 dbs in production.Also,the mongodb page says :
The WiredTiger cache stores uncompressed data and provides in-memory-like performance
I want to test it but i am not able to find any documentation for playing around with WiredTiger Cache.
Note: Wiredtiger is default mongo storage engine.
You can run Mongo DB as a replacement for Redis by specifying the In Memory storage engine:
mongod --storageEngine inMemory --dbpath <path>
You can read more about it here.
I may be wrong, but I believe that you will still need to run 2 Mongo DB servers, one running in Wired Tiger for disk storage, and the other running with In Memory storage for your cache service.

Read concern level of majority error while upgrading Sharded Cluster from 3.2 to 3.4

I have upgraded MongodDB sharded cluster having two replica sets from 3.2 to 3.4. Current storage engine is MMAPv1. After successfully upgrading all the secondary, primary, config server and mongos to 3.4, when I run config server using following command.
sudo mongod --configsvr
I keep getting following Error.
SHARDING [shard registry reload] Periodic reload of shard registry failed :: caused by :: 115 could not get updated shard list from config server due to Current storage engine does not support majority readConcerns; will retry after 30s
And also I am unable to connect mongos with config server. When I try to connect it using following command
sudo mongos --configdb [ip-of-my-config-server]:27019
It gives me following error.
BadValue: configdb supports only replica set connection string
I suppose mongos is unable to connect to config server due to the majority readConcerns error on config server.
MongoDB manual says
"When reading from the replica set config servers, MongoDB 3.4 uses a Read Concern level of "majority"."
And to use a read concern level of "majority", WiredTiger must be used as storage engine.
So it seems I have to switch to WiredTiger storage engine to make it work. But when I was going to switch to WiredTiger storage engine of a secondary replica set member, according to manual
"This procedure completely removes a secondary replica set member’s data"
So I am stuck halfway. Situation is
Config server is giving error regarding majority readConcerns.
I have to switch to WiredTiger to get rid of it.
Switching to WiredTiger will remove data from secondary members.
Data will not be replicated back to secondary members during this switching to WiredTiger procedure because of config server error and eventually I will be ended up losing all the data (Please correct if I am wrong).
Questions are:
Can I make MongoDB 3.4 to use a Read Concern level of "local" when reading from the replica set config servers?
How can I switch to WiredTiger engine without losing data in my scenario?
You could migrate each node in the replica set as if it was a standalone, by using mongodump to back up the data, restarting with WiredTiger and a blank data directory, then using mongorestore to populate the new database.
This isn't normally recommended for replica set nodes, but only because it's just easier to wipe the data on a secondary and let it resync from the other nodes. Doing it this way will work just fine, but involves a little more fiddly work for you with the dump and restore tools.

mongod unclean shutdown detected

I try to start mongod.exe but I have and I get the following error:
C:\MongoDB\Server\30\bin>mongod.exe
2015-12-16T19:12:17.108+0100 I CONTROL 2015-12-16T19:12:17.110+0100 W CONTROL 32-bit servers don't have journaling enabled by default.
Please use --journal if you want durability.
2015-12-16T19:12:17.110+0100 I CONTROL
2015-12-16T19:12:17.120+0100 I CONTROL Hotfix KB2731284 or later update is not installed, will zero-out data files
2015-12-16T19:12:17.132+0100 I STORAGE [initandlisten] **************
2015-12-16T19:12:17.132+0100 I STORAGE [initandlisten] Error: journal files are present in journal directory, yet starting without journaling enabled.
2015-12-16T19:12:17.133+0100 I STORAGE [initandlisten] It is recommended that you start with journaling enabled so that recovery may occur.
2015-12-16T19:12:17.133+0100 I STORAGE [initandlisten] **************
2015-12-16T19:12:17.135+0100 I STORAGE [initandlisten] exception in initAndListen: 13597 can't start without --journal enabled when journal/ files are present, terminating
2015-12-16T19:12:17.135+0100 I CONTROL [initandlisten] dbexit: rc: 100
I also tried to run it with --repair but then I get the same error.
Finally, I tried to delete the mongod.lock file but I still get the error.
How should I fix the unclean shutdown?
The solution to this problem is mongod --repair. This command automatically shuts down the all processes and repairs Mongodb issues. You can find more details in the official documentation.
Ok, to get some confusion right here. Journal files are not there to annoy you. They hold data not yet applied to the datafiles, but already received and acknowledged by the server. The mongod process finished a request after applying the data to the journal, but before applying them to the data files.
This behavior is configured by the chosen write concern.
Bottom line: special measurements were taken to make the data in the journal durable, you should not ignore that.
So you should create a configuration file containing this (among other things, if one already exists):
storage:
journal:
enabled: true
Please follow the documentation on running MongoDB on windows to the letter. Adjust the configuration file with options according to your needs.
If you are absolutely, positively sure that you do not need journaling, you can start mongodb with the --journal command line option just once, shut the instance down after the journal was successfully applied and remove the journal files then. Expect any write with a write concern involving the journal to fail, however.
Note 1 You are using the 32-bit version of MongoDB, which is only suitable for testing. Note that the 32-bit version only supports up to 2Gb of data.
Note 2 MongoDB is VERY well documented. You really should read the manual from top to bottom – it get's you started fast enough with providing a lot of information on the internals.
start cmd shell as admin and call start_mongo. This should fix it
Same error.
It' permission issue. If you get this error on Windows platform you should do all operations with administrator privilegies.
On Linux run
mongod --repair
but you should run it with sudo or under root. If under root you should change ownership of the files in data DB directory. Do it carefully or another error will appear.
Try removing the lock file, then running with --repair.
Here's what the Mongo Docs say about recovering data / restarting after an unexpected shutdown.

MongoDB multi-granularity locking

MongoDB uses multi-granularity locking [1] that allows operations to lock at the global, database or collection level, and allows for individual storage engines to implement their own concurrency control below the collection (i.e., at the document-level in WiredTiger).
Besides I've read that there are two possible Storage Engines for MongoDB: MMAPv1 (default) and WiredTiger. From MongoDB 3.0, the first one uses collection-level locking, the second one document-level locking. What does it mean that MongoDB allows operations to lock at the GLOBAL, DATABASE or COLLECTION level? It means that I can choose lock granularity? If yes how I can do it? Is this in contrast with lock-granularity of the chosen storage engine (for example document-level in WiredTiger)?
And how can I change my storage engine from MMAPv1 to WiredTiger?
Thanks in advance.
Answer to
And how can I change my storage engine from MMAPv1 to WiredTiger?
This page explains it:
Start 3.0 mongod. Ensure that the 3.0 mongod is running with the default MMAPv1 engine.
Export the data using mongodump.
mongodump --out <exportDataDestination>
Specify additional options as appropriate, such as username and password if running with authorization enabled. See mongodump for available options.
Create data directory for WiredTiger.
Create a new data directory for WiredTiger. Ensure that the user account running mongod has read and write permissions for the new directory.
mongod with WiredTiger will not start with data files created with a different storage engine.
Restart the mongod with WiredTiger.
Restart the 3.0 mongod, specifying wiredTiger as the --storageEngine and the newly created data directory for WiredTiger as the --dbpath. Specify additional options as appropriate.
mongod --storageEngine wiredTiger --dbpath <newWiredTigerDBPath>
You can also specify the options in a configuration file. To specify the storage engine, use the new storage.engine setting.