MongoDB Unclean Shutdown - Fail to repair - mongodb

I was running MongoDB on Amazon AWS, until one day it ran out of space. Then, the database is completely inaccessible,and I tried to shutdown the database. Having restarted the server and installed larger disk, the MongoDB can't be started.
I tried to repair the database, and the error is shown like below:
Wed Aug 15 10:08:04 [initandlisten] MongoDB starting : pid=1447 port=27017 dbpath=/var/lib/mongodb/ 32-bit
** NOTE: when using MongoDB 32 bit, you are limited to about 2 gigabytes of data
** see http://blog.mongodb.org/post/137788967/32-bit-limitations
** with --dur, the limit is lower
Wed Aug 15 10:08:04 [initandlisten] db version v1.8.3, pdfile version 4.5
Wed Aug 15 10:08:04 [initandlisten] git version: c206d77e94bc3b65c76681df5a6b605f68a2de05
Wed Aug 15 10:08:04 [initandlisten] build sys info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_37
Wed Aug 15 10:08:04 [initandlisten] **
Wed Aug 15 10:08:04 [initandlisten] **
Wed Aug 15 10:08:04 [initandlisten] need to upgrade database md with pdfile version 559903.1345021735, new version: 4.5
Wed Aug 15 10:08:04 [initandlisten] starting upgrade
Wed Aug 15 10:08:04 [initandlisten] md repairDatabase md
Wed Aug 15 10:08:04 [initandlisten] md.system.namespaces Assertion failure isOk() db/pdfile.h 259
0x817c27b 0x819121e 0x83cf525 0x83d6bec 0x83f5420 0x82a4fb7 0x83f8680 0x83fcece 0x8401358 0x8401e29 0x8402abd 0x828f755 0x83b3d67 0x83b931c 0x83bb3df 0x81ce25f 0x81ae714 0x8380f31 0x83831ba 0x83d4faf
mongod(_ZN5mongo12sayDbContextEPKc+0xcb) [0x817c27b]
mongod(_ZN5mongo8assertedEPKcS1_j+0x12e) [0x819121e]
mongod(_ZN5mongo11DataFileMgr7findAllEPKcRKNS_7DiskLocE+0x7a5) [0x83cf525]
mongod(_ZN5mongo13findTableScanEPKcRKNS_7BSONObjERKNS_7DiskLocE+0x13c) [0x83d6bec]
mongod(_ZNK5mongo9QueryPlan9newCursorERKNS_7DiskLocEi+0x830) [0x83f5420]
mongod(_ZN5mongo11UserQueryOp5_initEv+0x437) [0x82a4fb7]
mongod(_ZN5mongo12QueryPlanSet6Runner6initOpERNS_7QueryOpE+0x110) [0x83f8680]
mongod(_ZN5mongo12QueryPlanSet6Runner3runEv+0x2ee) [0x83fcece]
mongod(_ZN5mongo12QueryPlanSet5runOpERNS_7QueryOpE+0x2e8) [0x8401358]
mongod(_ZN5mongo16MultiPlanScanner9runOpOnceERNS_7QueryOpE+0x59) [0x8401e29]
mongod(_ZN5mongo16MultiPlanScanner5runOpERNS_7QueryOpE+0x2d) [0x8402abd]
mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x1575) [0x828f755]
mongod() [0x83b3d67]
mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x70c) [0x83b931c]
mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0x7f) [0x83bb3df]
mongod(_ZN5mongo14DBClientCursor4initEv+0x15f) [0x81ce25f]
mongod(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0x2a4) [0x81ae714]
mongod(_ZN5mongo6Cloner2goEPKcRSsRKSsbbbb+0x451) [0x8380f31]
mongod(_ZN5mongo9cloneFromEPKcRSsRKSsbbbb+0x5a) [0x83831ba]
mongod(_ZN5mongo14repairDatabaseESsRSsbb+0x38f) [0x83d4faf]
Wed Aug 15 10:08:04 [initandlisten] assertion 0 assertion db/pdfile.h:259 ns:md.system.namespaces query:{}
Wed Aug 15 10:08:04 [initandlisten] Assertion: 10290:bad system.namespaces object { $err: "assertion db/pdfile.h:259" }
0x819045e 0x8381181 0x83831ba 0x83d4faf 0x852a30c 0x852b217 0x852e733 0x852f59c 0x85366bb 0xb74a4ce7 0x810b551
mongod(_ZN5mongo11msgassertedEiPKc+0x15e) [0x819045e]
mongod(_ZN5mongo6Cloner2goEPKcRSsRKSsbbbb+0x6a1) [0x8381181]
mongod(_ZN5mongo9cloneFromEPKcRSsRKSsbbbb+0x5a) [0x83831ba]
mongod(_ZN5mongo14repairDatabaseESsRSsbb+0x38f) [0x83d4faf]
mongod(_ZN5mongo11doDBUpgradeERKSsSsPNS_14DataFileHeaderE+0x6c) [0x852a30c]
mongod() [0x852b217]
mongod(_ZN5mongo14_initAndListenEiPKc+0x4b3) [0x852e733]
mongod(ZN5mongo13initAndListenEiPKc+0x2c) [0x852f59c]
mongod(main+0x6bdb) [0x85366bb]
/lib/libc.so.6(_libc_start_main+0xe7) [0xb74a4ce7]
mongod() [0x810b551]
Wed Aug 15 10:08:04 [initandlisten] exception in initAndListen std::exception: bad system.namespaces object { $err: "assertion db/pdfile.h:259" }, terminating
Wed Aug 15 10:08:04 dbexit:
Wed Aug 15 10:08:04 [initandlisten] shutdown: going to close listening sockets...
Wed Aug 15 10:08:04 [initandlisten] shutdown: going to flush diaglog...
Wed Aug 15 10:08:04 [initandlisten] shutdown: going to close sockets...
Wed Aug 15 10:08:04 [initandlisten] shutdown: waiting for fs preallocator...
Wed Aug 15 10:08:04 [initandlisten] shutdown: closing all files...
Wed Aug 15 10:08:04 closeAllFiles() finished
Wed Aug 15 10:08:04 [initandlisten] shutdown: removing fs lock...
Wed Aug 15 10:08:04 dbexit: really exiting now
I have all *.number and *.ns file completely. But it seems the ns file is corrupted. Any way I could repair the database?
Thanks

If repair doesn't work then you are unfortunately out of luck.
I'm presuming that you didn't have journaling enabled or any back-ups. I would strongly encourage you to run with journaling enabled and follow one of the back-up strategies outlined here. The LVM snapshot is easy to implement, fairly quick and reliable.
You are better to run MongoDB on a 64-bit system also. 32-bit limits you to 2gb of data. You say that you "ran out of space"?

Related

Unable to start MongoDB. ERROR: addr already in use

I tried running MongoDB in local server with mongod command and it failed to run. The error:
mongod --help for help and startup options
Sat Jan 31 13:07:16.392 [initandlisten] MongoDB starting : pid=5127 port=27017 dbpath=/data/db/ 64-bit host=leandrotk
Sat Jan 31 13:07:16.392 [initandlisten] db version v2.4.9
Sat Jan 31 13:07:16.392 [initandlisten] git version: nogitversion
Sat Jan 31 13:07:16.392 [initandlisten] build info: Linux orlo 3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 BOOST_LIB_VERSION=1_54
Sat Jan 31 13:07:16.392 [initandlisten] allocator: tcmalloc
Sat Jan 31 13:07:16.392 [initandlisten] options: {}
Sat Jan 31 13:07:16.435 [initandlisten] journal dir=/data/db/journal
Sat Jan 31 13:07:16.435 [initandlisten] recover : no journal files present, no recovery needed
Sat Jan 31 13:07:16.513 [initandlisten] ERROR: listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:27017
Sat Jan 31 13:07:16.514 [initandlisten] ERROR: addr already in use
Sat Jan 31 13:07:16.514 [websvr] ERROR: listen(): bind() failed errno:98 Address already in use for socket: 0.0.0.0:28017
Sat Jan 31 13:07:16.514 [initandlisten] now exiting
Sat Jan 31 13:07:16.514 [websvr] ERROR: addr already in use
Sat Jan 31 13:07:16.514 dbexit:
Sat Jan 31 13:07:16.514 [initandlisten] shutdown: going to close listening sockets...
Sat Jan 31 13:07:16.514 [initandlisten] shutdown: going to flush diaglog...
Sat Jan 31 13:07:16.514 [initandlisten] shutdown: going to close sockets...
Sat Jan 31 13:07:16.514 [initandlisten] shutdown: waiting for fs preallocator...
Sat Jan 31 13:07:16.514 [initandlisten] shutdown: lock for final commit...
Sat Jan 31 13:07:16.514 [initandlisten] shutdown: final commit...
Sat Jan 31 13:07:16.599 [initandlisten] shutdown: closing all files...
Sat Jan 31 13:07:16.599 [initandlisten] closeAllFiles() finished
Sat Jan 31 13:07:16.599 [initandlisten] journalCleanup...
Sat Jan 31 13:07:16.599 [initandlisten] removeJournalFiles
Sat Jan 31 13:07:16.668 [initandlisten] shutdown: removing fs lock...
Sat Jan 31 13:07:16.668 dbexit: really exiting now
you already have a process running
killall mongod
should kill the other process, if you have another kind of process bound to that port you could find it with netstat | grep 27017
Try this:
sudo service mongodb stop
sudo mongod
some version is:
sudo service mongod stop
mongod: mongo damon

Mongo service doesn't connect

I just did a manual installation of mongoDB like the documentation says and when I run ./mongocommand this is what I get.
./mongo
MongoDB shell version: 2.4.8
connecting to: test
It doesn't say anymore and doesn't connect to testdatabase.
Anyone know how can I solve this problem?
Thank's in advance
EDIT: I uninstall all and re-install using Brew. Now this is what I get when I run mongod
all output going to: /usr/local/var/log/mongodb/mongo.log
And this when i run mongo
MongoDB shell version: 2.4.8
connecting to: test
Wed Jan 8 13:35:04.634 Error: couldn't connect to server 127.0.0.1:27017 at src/mongo/shell/mongo.js:145
exception: connect failed
And this is what /usr/local/var/log/mongodb/mongo.logshows
***** SERVER RESTARTED *****
Wed Jan 8 13:34:11.749 [initandlisten] MongoDB starting : pid=745 port=27017 dbpath=/usr/local/var/mongodb 64-bit host=MacBook-Pro-de-Kemical.local
Wed Jan 8 13:34:11.749 [initandlisten]
Wed Jan 8 13:34:11.749 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
Wed Jan 8 13:34:11.749 [initandlisten] db version v2.4.8
Wed Jan 8 13:34:11.749 [initandlisten] git version: nogitversion
Wed Jan 8 13:34:11.749 [initandlisten] build info: Darwin minimountain.local 12.5.0 Darwin Kernel Version 12.5.0: Sun Sep 29 13:33:47 PDT 2013; root:xnu-2050.48.12~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
Wed Jan 8 13:34:11.749 [initandlisten] allocator: tcmalloc
Wed Jan 8 13:34:11.749 [initandlisten] options: { bind_ip: "127.0.0.1", config: "/usr/local/etc/mongod.conf", dbpath: "/usr/local/var/mongodb", logappend: "true", logpath: "/usr/local/var/log/mongodb/mongo.log" }
Wed Jan 8 13:34:11.750 [initandlisten] exception in initAndListen: 10310 Unable to lock file: /usr/local/var/mongodb/mongod.lock. Is a mongod instance already running?, terminating
Wed Jan 8 13:34:11.750 dbexit:
Wed Jan 8 13:34:11.750 [initandlisten] shutdown: going to close listening sockets...
Wed Jan 8 13:34:11.750 [initandlisten] shutdown: going to flush diaglog...
Wed Jan 8 13:34:11.750 [initandlisten] shutdown: going to close sockets...
Wed Jan 8 13:34:11.750 [initandlisten] shutdown: waiting for fs preallocator...
Wed Jan 8 13:34:11.750 [initandlisten] shutdown: lock for final commit...
Wed Jan 8 13:34:11.750 [initandlisten] shutdown: final commit...
Wed Jan 8 13:34:11.750 [initandlisten] shutdown: closing all files...
Wed Jan 8 13:34:11.750 [initandlisten] closeAllFiles() finished
Wed Jan 8 13:34:11.750 dbexit: really exiting now
First of all, have you actually started the "mongod" server process?
If you have, you need to find out what is happening and one way to do this is with the "strace" tool. Running it like follows creates a log file in /tmp/strace.log and that should tell you system level wise what's going on:
strace -o /tmp/strace.log ./mongo

Start mongo failed

I get an error when I try to start mongo using the "mongod" command (on Macintosh).
all output going to: /usr/local/var/log/mongodb/mongo.log
My logs :
***** SERVER RESTARTED *****
Tue Oct 29 13:15:48.783 [initandlisten] MongoDB starting : pid=859 port=27017 dbpath=/usr/local/var/mongodb 64-bit host=MacBook-Pro-de-Anthony
Tue Oct 29 13:15:48.784 [initandlisten] db version v2.4.4
Tue Oct 29 13:15:48.784 [initandlisten] git version: 4ec1fb96702c9d4c57b1e06dd34eb73a16e407d2
Tue Oct 29 13:15:48.784 [initandlisten] build info: Darwin bs-osx-106-x86-64-2.10gen.cc 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
Tue Oct 29 13:15:48.784 [initandlisten] allocator: system
Tue Oct 29 13:15:48.784 [initandlisten] options: { bind_ip: "127.0.0.1", config: "/usr/local/etc/mongod.conf", dbpath: "/usr/local/var/mongodb", logappend: "true", logpath: "/usr/local/var/log/mongodb/mongo.log" }
Tue Oct 29 13:15:48.784 [initandlisten] journal dir=/usr/local/var/mongodb/journal
Tue Oct 29 13:15:48.784 [initandlisten] recover : no journal files present, no recovery needed
Tue Oct 29 13:15:48.801 [initandlisten] ERROR: listen(): bind() failed errno:48 Address already in use for socket: 127.0.0.1:27017
Tue Oct 29 13:15:48.801 [websvr] ERROR: listen(): bind() failed errno:48 Address already in use for socket: 127.0.0.1:28017
Tue Oct 29 13:15:48.801 [initandlisten] ERROR: addr already in use
Tue Oct 29 13:15:48.801 [websvr] ERROR: addr already in use
Tue Oct 29 13:15:48.801 [initandlisten] now exiting
Tue Oct 29 13:15:48.801 dbexit:
Tue Oct 29 13:15:48.801 [initandlisten] shutdown: going to close listening sockets...
Tue Oct 29 13:15:48.801 [initandlisten] shutdown: going to flush diaglog...
Tue Oct 29 13:15:48.801 [initandlisten] shutdown: going to close sockets...
Tue Oct 29 13:15:48.801 [initandlisten] shutdown: waiting for fs preallocator...
Tue Oct 29 13:15:48.801 [initandlisten] shutdown: lock for final commit...
Tue Oct 29 13:15:48.801 [initandlisten] shutdown: final commit...
Tue Oct 29 13:15:48.802 [initandlisten] shutdown: closing all files...
Tue Oct 29 13:15:48.802 [initandlisten] closeAllFiles() finished
Tue Oct 29 13:15:48.802 [initandlisten] journalCleanup...
Tue Oct 29 13:15:48.802 [initandlisten] removeJournalFiles
Tue Oct 29 13:15:48.803 [initandlisten] shutdown: removing fs lock...
Tue Oct 29 13:15:48.803 dbexit: really exiting now
UPDATE:
When I execute the "mongo" command :
MacBook-Pro-de-Anthony:Sites anthonycluse$ mongo
MongoDB shell version: 2.4.4
connecting to: test
Server has startup warnings:
Tue Oct 29 11:43:15 [initandlisten]
Tue Oct 29 11:43:15 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
> show dbs;
anthonycluse (empty)
local (empty)
>
It means that the username is "test" no?
UPDATE 2 :
The "show users;" command :
MacBook-Pro-de-Anthony:Sites anthonycluse$ mongo
MongoDB shell version: 2.4.4
connecting to: test
Server has startup warnings:
Tue Oct 29 11:43:15 [initandlisten]
Tue Oct 29 11:43:15 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
> show users;
>
Is there another process listening on those ports? Try finding out with:
netstat -tulpen | grep 017 # 017 for both port 27017 and 28017
Kill that process if you're sure you don't need it. As answered below by Lix, there may already be a mongo process running.
The answer is right there in side the log file:
Tue Oct 29 13:15:48.801 [websvr] ERROR: listen(): bind() failed errno:48 Address already in use for socket: 127.0.0.1:28017
Specifically this part: "Address already in use for socket: 127.0.0.1:28017"
It seems that there is already a mongo instance running on that default port 27017. It might not even be a mongo instance but some other application that is using that port.
As per the logs it is clear that there is another mongodb instance running on default, i believe you installed mongodb using rpm and started this service using service mongodb start. so when the server restarts by default it starts some of the services, using defaultconfiguration which will be avialble in /etc/mongod.config edit this file or stop the one which is running already so you can avoid the mongod default instamce and made ur version to up and running

Error connecting to local installation of MongoDB thru terminal

Terminal displays following error:
$ mongo
MongoDB shell version: 2.4.5
connecting to: test
Wed Jul 24 14:26:45.779 JavaScript execution failed: Error: couldn't connect to server 127.0.0.1:27017 at src/mongo/shell/mongo.js:L112
exception: connect failed
upon running
mongod run --config /usr/local/etc/mongod.conf
I get following error on console:
about to fork child process, waiting until server is ready for connections.
forked process: 11204
all output going to: /usr/local/var/log/mongodb/mongo.log
ERROR: child process failed, exited with error number 100
The mongo.log shows:
***** SERVER RESTARTED *****
Wed Jul 24 14:39:57.360 [initandlisten] MongoDB starting : pid=11204 port=27017 dbpath=/usr/local/var/mongodb 64-bit host=59.161.67.13.del-cdma.dialup.vsnl.net.in
Wed Jul 24 14:39:57.363 [initandlisten]
Wed Jul 24 14:39:57.363 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
Wed Jul 24 14:39:57.363 [initandlisten] db version v2.4.5
Wed Jul 24 14:39:57.363 [initandlisten] git version: a2ddc68ba7c9cee17bfe69ed840383ec3506602b
Wed Jul 24 14:39:57.363 [initandlisten] build info: Darwin bs-osx-106-x86-64-2.10gen.cc 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
Wed Jul 24 14:39:57.363 [initandlisten] allocator: system
Wed Jul 24 14:39:57.363 [initandlisten] options: { bind_ip: "127.0.0.1", command: [ "run" ], config: "/usr/local/etc/mongod.conf", dbpath: "/usr/local/var/mongodb", fork: "true", journal: "true", logappend: "true", logpath: "/usr/local/var/log/mongodb/mongo.log", port: 27017, quiet: "true" }
Wed Jul 24 14:39:57.367 [initandlisten] journal dir=/usr/local/var/mongodb/journal
Wed Jul 24 14:39:57.368 [initandlisten] recover : no journal files present, no recovery needed
Wed Jul 24 14:39:58.113 [initandlisten] couldn't open /usr/local/var/mongodb/test.0 errno:13 Permission denied
Wed Jul 24 14:39:58.136 [initandlisten] couldn't open /usr/local/var/mongodb/test.0 errno:13 Permission denied
Wed Jul 24 14:39:58.152 [initandlisten] exception in initAndListen: 10085 can't map file memory, terminating
Wed Jul 24 14:39:58.152 dbexit:
Wed Jul 24 14:39:58.152 [initandlisten] shutdown: going to close listening sockets...
Wed Jul 24 14:39:58.153 [initandlisten] shutdown: going to flush diaglog...
Wed Jul 24 14:39:58.153 [initandlisten] shutdown: going to close sockets...
Wed Jul 24 14:39:58.153 [initandlisten] shutdown: waiting for fs preallocator...
Wed Jul 24 14:39:58.153 [initandlisten] shutdown: lock for final commit...
Wed Jul 24 14:39:58.153 [initandlisten] shutdown: final commit...
Wed Jul 24 14:39:58.153 [initandlisten] shutdown: closing all files...
Wed Jul 24 14:39:58.154 [initandlisten] closeAllFiles() finished
Wed Jul 24 14:39:58.154 [initandlisten] journalCleanup...
Wed Jul 24 14:39:58.154 [initandlisten] removeJournalFiles
Wed Jul 24 14:39:58.154 [initandlisten] shutdown: removing fs lock...
Wed Jul 24 14:39:58.154 dbexit: really exiting now
EDIT! ( trying suggestions from answers)
running $ mongod --repair gives the following error:
$ mongod --repair
Wed Jul 24 15:16:51.596 Can't specify both --journal and --repair options.
Run the following command first to start the mongo server
mongod run --config /usr/local/etc/mongod.conf
Assuming you installed mongo using Brew.
More information about the mongod process here
EDIT
Try to repair mongo using :
mongod --repair
Seems to have issues with permissions also, try using sudo to start mongod
You need to make sure that you've actually started the server. It doesn't seem to be running here.

MongoDB Catch-22 won't let me drop huge collection

So I have a 2TB hard drive with 2 collections in 1 mongoDB.
Collection 1 is about 500gb and completely essential
Collection 2 is about 1500gb and needs to be deleted.
I was running a process that accidentally filled up the entire HDD, leaving 0% space. There is literally nothing on the machine apart from a Linux Distro and this mongoDB.
I would like to delete Collection 2.
In order to do this, I would normally just go into mongo and type db.collection.drop()
However, mongoDB is not currently running. In order to start it, I'm using:
[bobby#myPC bin]# ./mongod --dbpath /home/mongo &
However, since there is no space on the HDD, it returns:
[bobby#myPC bin]# Thu Feb 14 17:47:10 [initandlisten] MongoDB starting : pid=1264 port=27017 dbpath=/home/mongo 64-bit host=localhost.localdomain
Thu Feb 14 17:47:10 [initandlisten] db version v2.2.3, pdfile version 4.5
Thu Feb 14 17:47:10 [initandlisten] git version: f570771a5d8a3846eb7586eaffcf4c2f4a96bf08
Thu Feb 14 17:47:10 [initandlisten] build info: Linux ip-10-2-29-40 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_49
Thu Feb 14 17:47:10 [initandlisten] options: { dbpath: "/home/mongo" }
Thu Feb 14 17:47:10 [initandlisten] journal dir=/home/mongo/journal
Thu Feb 14 17:47:10 [initandlisten] recover : no journal files present, no recovery needed
Thu Feb 14 17:47:10 [initandlisten]
Thu Feb 14 17:47:10 [initandlisten] ERROR: Insufficient free space for journal files
Thu Feb 14 17:47:10 [initandlisten] Please make at least 3379MB available in /home/mongo/journal or use --smallfiles
Thu Feb 14 17:47:10 [initandlisten]
Thu Feb 14 17:47:10 [initandlisten] exception in initAndListen: 15926 Insufficient free space for journals, terminating
Thu Feb 14 17:47:10 dbexit:
Thu Feb 14 17:47:10 [initandlisten] shutdown: going to close listening sockets...
Thu Feb 14 17:47:10 [initandlisten] shutdown: going to flush diaglog...
Thu Feb 14 17:47:10 [initandlisten] shutdown: going to close sockets...
Thu Feb 14 17:47:10 [initandlisten] shutdown: waiting for fs preallocator...
Thu Feb 14 17:47:10 [initandlisten] shutdown: lock for final commit...
Thu Feb 14 17:47:10 [initandlisten] shutdown: final commit...
Thu Feb 14 17:47:10 [initandlisten] shutdown: closing all files...
Thu Feb 14 17:47:10 [initandlisten] closeAllFiles() finished
Thu Feb 14 17:47:10 [initandlisten] journalCleanup...
Thu Feb 14 17:47:10 [initandlisten] removeJournalFiles
Thu Feb 14 17:47:10 [initandlisten] shutdown: removing fs lock...
Thu Feb 14 17:47:10 dbexit: really exiting now
I can't remove this collection cause there's no space. And there's no space because I can't remove this collection.
How can I fix this? Is there some way of identifying which mongoDB files in /home/mongo I can take out?
Based on the error message, its complaining about not having enough space for the journal. Its not ideal in a production environment to run with out journaling, but you could try starting with out it while you fix the issue.
./mongod --nojournal --dbpath /home/mongo &
edit: to answer your question though, mongo creates and allocates files per database, not collection. So you wouldn't be able to target a specific file to remove a collection.