mongodb: Journal error. Which collection is referred to? - mongodb

We have two collections.
So I have to delete at least one, because mongo won't restart (losing 400gb of data in the process, but that is normal when using mongo).
From this log, I cannot tell which one to delete. Is there a clue I am missing?
2015-01-27T02:03:58.781-0600 Trying to start Windows service 'MongoDB'
2015-01-27T02:03:58.781-0600 Service running
2015-01-27T02:03:58.781-0600 [initandlisten] MongoDB starting : pid=1864 port=27017 dbpath=c:\MongoDB\Data 64-bit host=619723-MONGO-1
2015-01-27T02:03:58.781-0600 [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2015-01-27T02:03:58.781-0600 [initandlisten] db version v2.6.4
2015-01-27T02:03:58.781-0600 [initandlisten] git version: 3a830be0eb92d772aa855ebb711ac91d658ee910
2015-01-27T02:03:58.781-0600 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
2015-01-27T02:03:58.781-0600 [initandlisten] allocator: system
2015-01-27T02:03:58.781-0600 [initandlisten] options: { config: "C:\MongoDB\mongod.cfg", service: true, storage: { dbPath: "c:\MongoDB\Data", directoryPerDB: true }, systemLog: { destination: "file", path: "c:\MongoDB\logs" } }
2015-01-27T02:03:58.890-0600 [initandlisten] journal dir=c:\MongoDB\Data\journal
2015-01-27T02:03:58.968-0600 [initandlisten] dbexception during recovery: 13532 unexpected file in journal directory c:\MongoDB\Data\journal : j._1895 : can't find its preceding file
2015-01-27T02:03:58.984-0600 [initandlisten] exception in initAndListen: 13532 unexpected file in journal directory c:\MongoDB\Data\journal : j._1895 : can't find its preceding file, terminating
2015-01-27T02:03:58.999-0600 [initandlisten] dbexit:
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: going to close listening sockets...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: going to flush diaglog...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: going to close sockets...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: waiting for fs preallocator...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: lock for final commit...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: final commit...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: closing all files...
2015-01-27T02:03:58.999-0600 [initandlisten] closeAllFiles() finished
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: removing fs lock...
2015-01-27T02:03:58.999-0600 [initandlisten] dbexit: really exiting now

Our plan was to reset the databases, but we did a repair instead.

Related

mongodb terminated and won't come back up. Error 13532

Running mongodb 2.6.4 on Win2008-R2.
Full of data.... and today it went down, and won't come back up.
What to do?
2015-01-27T02:03:58.781-0600 Trying to start Windows service 'MongoDB'
2015-01-27T02:03:58.781-0600 Service running
2015-01-27T02:03:58.781-0600 [initandlisten] MongoDB starting : pid=1864 port=27017 dbpath=c:\MongoDB\Data 64-bit host=619723-MONGO-1
2015-01-27T02:03:58.781-0600 [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2015-01-27T02:03:58.781-0600 [initandlisten] db version v2.6.4
2015-01-27T02:03:58.781-0600 [initandlisten] git version: 3a830be0eb92d772aa855ebb711ac91d658ee910
2015-01-27T02:03:58.781-0600 [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
2015-01-27T02:03:58.781-0600 [initandlisten] allocator: system
2015-01-27T02:03:58.781-0600 [initandlisten] options: { config: "C:\MongoDB\mongod.cfg", service: true, storage: { dbPath: "c:\MongoDB\Data", directoryPerDB: true }, systemLog: { destination: "file", path: "c:\MongoDB\logs" } }
2015-01-27T02:03:58.890-0600 [initandlisten] journal dir=c:\MongoDB\Data\journal
2015-01-27T02:03:58.968-0600 [initandlisten] dbexception during recovery: 13532 unexpected file in journal directory c:\MongoDB\Data\journal : j._1895 : can't find its preceding file
2015-01-27T02:03:58.984-0600 [initandlisten] exception in initAndListen: 13532 unexpected file in journal directory c:\MongoDB\Data\journal : j._1895 : can't find its preceding file, terminating
2015-01-27T02:03:58.999-0600 [initandlisten] dbexit:
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: going to close listening sockets...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: going to flush diaglog...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: going to close sockets...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: waiting for fs preallocator...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: lock for final commit...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: final commit...
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: closing all files...
2015-01-27T02:03:58.999-0600 [initandlisten] closeAllFiles() finished
2015-01-27T02:03:58.999-0600 [initandlisten] shutdown: removing fs lock...
2015-01-27T02:03:58.999-0600 [initandlisten] dbexit: really exiting now
Well, google did not seem to have anything useful on mongo error 13532
What we did:
a) Deleted all files in /journal dir
b) Deleted the mongo .lock file
c) From cmd line: mongod --repair --dbpath c:\mongodb\data\
d) Started up mongo service
Up now for several hours OK.
Not sure what the data loss was.

MongoDB instance not starting

I have installed MongoDB on Mac OSX. When I do mongod it does not start any connections; instead it displays the output below. Can someone please help me to resolve this
$ mongod
mongod --help for help and startup options
2015-01-16T09:57:18.994-0500 [initandlisten] MongoDB starting : pid=1571 port=27017 dbpath=/data/db 64-bit host=3c15c2e72774
2015-01-16T09:57:18.995-0500 [initandlisten]
2015-01-16T09:57:18.995-0500 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
2015-01-16T09:57:18.995-0500 [initandlisten] db version v2.6.7
2015-01-16T09:57:18.995-0500 [initandlisten] git version: nogitversion
2015-01-16T09:57:18.995-0500 [initandlisten] build info: Darwin minimavericks.local 13.4.0 Darwin Kernel Version 13.4.0: Sun Aug 17 19:50:11 PDT 2014; root:xnu-2422.115.4~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
2015-01-16T09:57:18.995-0500 [initandlisten] allocator: tcmalloc
2015-01-16T09:57:18.995-0500 [initandlisten] options: {}
2015-01-16T09:57:18.996-0500 [initandlisten] journal dir=/data/db/journal
2015-01-16T09:57:18.996-0500 [initandlisten] recover : no journal files present, no recovery needed
2015-01-16T09:57:19.056-0500 [initandlisten] bad .ns file: /data/db/local.ns
2015-01-16T09:57:19.056-0500 [initandlisten] warning database /data/db local could not be opened
2015-01-16T09:57:19.056-0500 [initandlisten] DBException 10079: bad .ns file length, cannot open database
2015-01-16T09:57:19.056-0500 [initandlisten] exception in initAndListen: 10079 bad .ns file length, cannot open database, terminating
2015-01-16T09:57:19.056-0500 [initandlisten] dbexit:
2015-01-16T09:57:19.056-0500 [initandlisten] shutdown: going to close listening sockets...
2015-01-16T09:57:19.056-0500 [initandlisten] shutdown: going to flush diaglog...
2015-01-16T09:57:19.056-0500 [initandlisten] shutdown: going to close sockets...
2015-01-16T09:57:19.056-0500 [initandlisten] shutdown: waiting for fs preallocator...
2015-01-16T09:57:19.056-0500 [initandlisten] shutdown: lock for final commit...
2015-01-16T09:57:19.056-0500 [initandlisten] shutdown: final commit...
2015-01-16T09:57:19.056-0500 [initandlisten] shutdown: closing all files...
2015-01-16T09:57:19.056-0500 [initandlisten] closeAllFiles() finished
2015-01-16T09:57:19.056-0500 [initandlisten] journalCleanup...
2015-01-16T09:57:19.060-0500 [initandlisten] removeJournalFiles
2015-01-16T09:57:19.060-0500 [initandlisten] shutdown: removing fs lock...
2015-01-16T09:57:19.060-0500 [initandlisten] dbexit: really exiting now
Remove any old or corrupted .ns files in /data/db and then restart mongodb
I had this problem. I solved it by increasing the memory size for the replica using Virtual Box.

Resstarting MongoDB after crash

After a crash of MongoDB I currently cannot restart the database. I cleaned up the .lock file but still get the error:
2014-12-23T19:55:41.922+0100 [initandlisten] MongoDB starting : pid=4880 port=27
017 dbpath=c:/Program Files/MongoDB 2.6 Standard/bin/asd 64-bit host=Lenovo-PC
2014-12-23T19:55:41.922+0100 [initandlisten] targetMinOS: Windows 7/Windows Serv
er 2008 R2
2014-12-23T19:55:41.922+0100 [initandlisten] db version v2.6.6
2014-12-23T19:55:41.922+0100 [initandlisten] git version: 608e8bc319627693b04cc7
da29ecc300a5f45a1f
2014-12-23T19:55:41.922+0100 [initandlisten] build info: windows sys.getwindowsv
ersion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1')
BOOST_LIB_VERSION=1_49
2014-12-23T19:55:41.922+0100 [initandlisten] allocator: system
2014-12-23T19:55:41.922+0100 [initandlisten] options: { storage: { dbPath: "c:/P
rogram Files/MongoDB 2.6 Standard/bin/asd" } }
2014-12-23T19:55:41.938+0100 [initandlisten] exception in initAndListen: 13627 U
nable to create/open lock file: c:/Program Files/MongoDB 2.6 Standard/bin/asd\mo
ngod.lock Zugriff verweigert. Is a mongod instance already running?, terminating
2014-12-23T19:55:41.938+0100 [initandlisten] dbexit:
2014-12-23T19:55:41.938+0100 [initandlisten] shutdown: going to close listening
sockets...
2014-12-23T19:55:41.938+0100 [initandlisten] shutdown: going to flush diaglog...
2014-12-23T19:55:41.938+0100 [initandlisten] shutdown: going to close sockets...
2014-12-23T19:55:41.938+0100 [initandlisten] shutdown: waiting for fs preallocat
or...
2014-12-23T19:55:41.938+0100 [initandlisten] shutdown: lock for final commit...
2014-12-23T19:55:41.938+0100 [initandlisten] shutdown: final commit...
2014-12-23T19:55:41.938+0100 [initandlisten] shutdown: closing all files...
2014-12-23T19:55:41.938+0100 [initandlisten] closeAllFiles() finished
2014-12-23T19:55:41.938+0100 [initandlisten] dbexit: really exiting now
when exciting mongodb with "mongod --dbpath PATH" where path is the path to the folder, where the database date shoulb be positioned.
The system runs under a virtual environment within windows.

how to start mongodb on remote server EC2?

Am new to mongodb. I have installed it on my pc with wamp server and it is working fine. Now am trying to connect with EC2 remotely.
I have gone through this . But still am not getting to connect.
Here is the code
[ec2-user#ip-162-31-45-45 ~]$ mongod
mongod --help for help and startup options
2014-09-07T06:07:44.158+0000 [initandlisten] MongoDB starting : pid=27743 port=27017 dbpath=/data/db 64-bit host=ip-172-31-45-55
2014-09-07T06:07:44.158+0000 [initandlisten] db version v2.6.4
2014-09-07T06:07:44.158+0000 [initandlisten] git version: xxxxx
2014-09-07T06:07:44.158+0000 [initandlisten] build info: xxxxx
2014-09-07T06:07:44.158+0000 [initandlisten] allocator: tcmalloc
2014-09-07T06:07:44.158+0000 [initandlisten] options: {}
2014-09-07T06:07:44.163+0000 [initandlisten] journal dir=/data/db/journal
2014-09-07T06:07:44.163+0000 [initandlisten] recover : no journal files present, no recovery needed
2014-09-07T06:07:44.331+0000 [initandlisten] ERROR: listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:27017
2014-09-07T06:07:44.331+0000 [initandlisten] ERROR: addr already in use
2014-09-07T06:07:44.331+0000 [initandlisten] now exiting
2014-09-07T06:07:44.331+0000 [initandlisten] dbexit:
2014-09-07T06:07:44.331+0000 [initandlisten] shutdown: going to close listening sockets...
2014-09-07T06:07:44.331+0000 [initandlisten] shutdown: going to flush diaglog...
2014-09-07T06:07:44.332+0000 [initandlisten] shutdown: going to close sockets...
2014-09-07T06:07:44.332+0000 [initandlisten] shutdown: waiting for fs preallocator...
2014-09-07T06:07:44.332+0000 [initandlisten] shutdown: lock for final commit...
2014-09-07T06:07:44.332+0000 [initandlisten] shutdown: final commit...
2014-09-07T06:07:44.340+0000 [initandlisten] shutdown: closing all files...
2014-09-07T06:07:44.340+0000 [initandlisten] closeAllFiles() finished
2014-09-07T06:07:44.340+0000 [initandlisten] journalCleanup...
2014-09-07T06:07:44.340+0000 [initandlisten] removeJournalFiles
2014-09-07T06:07:44.344+0000 [initandlisten] shutdown: removing fs lock...
2014-09-07T06:07:44.344+0000 [initandlisten] dbexit: really exiting now
[ec2-user#ip-162-31-45-45 ~]$ sudo service mongod start
Starting mongod: [FAILED]
How to solve this ?
By installing all these from This was solved my problem.
The error says that the address is already in use. Are you sure it's not already running?

mongodb: Is there any way of stoping mongod from trying to set a lock on mongod.lock?

Because of the size of the mongodb database I'm trying to build and because I needed a filesystem that can handle parallel writes, I moved it on a cluster that has lustreFS, which fulfills these needs.
The problem is that I can't start mongod, so I can't connect anything to the database, because mongod can't get a lock on mongod.lock. I don't have root or sudo permissions but this does not seem to be the problem here because permissions are not mentioned, as you can see:
[user#host mongo]$ ./mongod --dbpath=path-to-db
2014-07-28T20:29:58.600+0300 [initandlisten] MongoDB starting : pid=54974 port=27017 dbpath=path-to-db 64-bit host=hostname
2014-07-28T20:29:58.600+0300 [initandlisten]
2014-07-28T20:29:58.600+0300 [initandlisten] ** WARNING: /proc/sys/vm/zone_reclaim_mode is 1
2014-07-28T20:29:58.600+0300 [initandlisten] ** We suggest setting it to 0
2014-07-28T20:29:58.600+0300 [initandlisten] ** http://www.kernel.org/doc/Documentation/sysctl/vm.txt
2014-07-28T20:29:58.600+0300 [initandlisten] db version v2.6.3
2014-07-28T20:29:58.601+0300 [initandlisten] git version: 255f67a66f9603c59380b2a389e386910bbb52cb
2014-07-28T20:29:58.601+0300 [initandlisten] build info: Linux build12.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2014-07-28T20:29:58.601+0300 [initandlisten] allocator: tcmalloc
2014-07-28T20:29:58.601+0300 [initandlisten] options: { storage: { dbPath: "path-to-db" } }
2014-07-28T20:29:58.614+0300 [initandlisten] exception in initAndListen: 10310 Unable to lock file: path-to-db/mongod.lock. Is a mongod instance already running?, terminating
2014-07-28T20:29:58.615+0300 [initandlisten] dbexit:
2014-07-28T20:29:58.615+0300 [initandlisten] shutdown: going to close listening sockets...
2014-07-28T20:29:58.615+0300 [initandlisten] shutdown: going to flush diaglog...
2014-07-28T20:29:58.615+0300 [initandlisten] shutdown: going to close sockets...
2014-07-28T20:29:58.615+0300 [initandlisten] shutdown: waiting for fs preallocator...
2014-07-28T20:29:58.615+0300 [initandlisten] shutdown: lock for final commit...
2014-07-28T20:29:58.615+0300 [initandlisten] shutdown: final commit...
2014-07-28T20:29:58.615+0300 [initandlisten] shutdown: closing all files...
2014-07-28T20:29:58.615+0300 [initandlisten] closeAllFiles() finished
2014-07-28T20:29:58.615+0300 [initandlisten] dbexit: really exiting now
Could you please tell me if there's any way around that lock? And no, there weren't any other mongod instances running.
Solution: File locking was being disabled by cluster admins. After file locking was enabled, mongod started and accepted connections as it should.