I'm getting this below error while restoring dump into mongodb,
Mon Jul 27 14:08:52.936 going into namespace [test.BC_2022_tmp]
Mon Jul 27 14:08:52.936 warning: Restoring to viacrm.BC_2022_tmp without dropping. Restored data will be inserted without raising errors; check your server log
Mon Jul 27 14:08:52.937 file /home/dev/test/BC_2022_tmp.bson empty, skipping
Mon Jul 27 14:08:52.937 Creating index: { key: { _id: 1 }, ns: "viacrm.BC_2022_tmp", name: "_id_" }
Mon Jul 27 14:08:52.938 ERROR: Error creating index test.BC_2022_tmp: 13347 err: "local.oplog.rs missing. did you drop it? if so restart server"
Abandon (core dumped)
Dump Restore command:
mongorestore -d test /home/dev/crm
Please can anyone help me to resolve this issue
Related
When I launch the mongo shell it closes immediately. There is no error. It was working fine (yesterday). My MongoDB server is running perfectly, and my application is connecting to it fine. I can't even launch the shell without connecting to a database:
C:\mongodb\bin> mongo --verbose
MongoDB shell version: 2.4.7
Fri Mar 27 10:35:38.425 versionArrayTest passed
connecting to: test
Fri Mar 27 10:35:38.472 creating new connection to:127.0.0.1:27017
Fri Mar 27 10:35:38.472 BackgroundJob starting: ConnectBG
Fri Mar 27 10:35:38.487 connected connection!
bye
Fri Mar 27 10:35:38.487 freeing 1 uncollected class mongo::DBClientCursor objects
Fri Mar 27 10:35:38.487 freeing 1 uncollected class mongo::DBClientWithCommands objects
Fri Mar 27 10:35:38.487 freeing 1 uncollected class mongo::BSONHolder objects
Similarly when connecting with no db:
C:\mongodb\bin> mongo --nodb --norc --verbose
MongoDB shell version: 2.4.7
Fri Mar 27 10:37:00.513 versionArrayTest passed
bye
Any and all thoughts on how to connect gratefully received.
As, mongod service has stopped; you need to go in the task manager-> Services Tab
-> search for MongoDB
-> Right click and click start.
You will find mongod shell is working as it was working previously.
I hope this helped!
Can anyone say are if there is any practical limit for the number of databases in mongodb? I've started to have serious problems when I passed 120 databases. Simple things like :
> show dbs
Mon Feb 10 16:35:32 DBClientCursor::init call() failed
Mon Feb 10 16:35:32 query failed : admin.$cmd { listDatabases: 1.0 } to: 127.0.0.1:27017
Mon Feb 10 16:35:32 Error: error doing query: failed src/mongo/shell/collection.js:155
Mon Feb 10 16:35:32 trying reconnect to 127.0.0.1:27017
Mon Feb 10 16:35:32 reconnect 127.0.0.1:27017 failed couldn't connect to server 127.0.0.1:27017
>
Mon Feb 10 16:36:01 trying reconnect to 127.0.0.1:27017
Mon Feb 10 16:36:01 reconnect 127.0.0.1:27017 failed couldn't connect to server 127.0.0.1:27017
>
Mon Feb 10 16:37:01 trying reconnect to 127.0.0.1:27017
Mon Feb 10 16:37:01 reconnect 127.0.0.1:27017 ok
and
> getMemInfo()
{ "virtual" : 32, "resident" : 7 }
Mon Feb 10 16:39:00 DBClientCursor::init call() failed
Mon Feb 10 16:39:00 query failed : admin.$cmd { replSetGetStatus: 1.0, forShell: 1.0 } to: 127.0.0.1:27017
> shell
Mon Feb 10 16:39:38 ReferenceError: shell is not defined (shell):1
Mon Feb 10 16:39:38 trying reconnect to 127.0.0.1:27017
Mon Feb 10 16:39:38 reconnect 127.0.0.1:27017 ok
Yet the log file stayed enigmatic
What version of mongodb are you running on what host?
Here is a test on CenOS 6.5, mongodb 2.2 x86_64 direct from EPEL
Here is a sample python script that creates 1000 databases
from pymongo import MongoClient
mc = MongoClient()
for i in range(5000):
print i
mc['db%s'%(i)].test.insert({"test":True})
output:
...snip...
506
Traceback (most recent call last):
File "overload_mongo.py", line 6, in <module>
mc['db%s'%(i)].test.insert({"test":True})
File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 357, in insert
continue_on_error, self.__uuid_subtype), safe)
File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", line 929, in _send_message
raise AutoReconnect(str(e))
pymongo.errors.AutoReconnect: [Errno 104] Connection reset by peer
There it is, looking at the log
ERROR: Uncaught std::exception: boost::filesystem::basic_directory_iterator constructor: Too many open files: "/index/bauman/db/_tmp/esort.1392056635.506/", terminating
The good ole too many open files problem
If you are on a enterprise linux platform, you can drop this file into /etc/security/limits.d/mongodb.conf and start a new session
mongodb hard nofile 99999
mongodb soft nofile 99999
mongodb hard nproc 99999
mongodb soft nproc 99999
I dont know how to achieve a similar result on windows.
The 'problem' lies in that MongoDB wants to memory map every single database file, so you need your hostOS to allow it to do so.
Same code as above
python overload_mongo.py
Output
...snip...
995
996
997
998
999
All better
In dump/enron directory messages.bson and messages.medata.json fiels. It should restore
120,477 documents.
I want to restore data from it.
I input command:
mongorestore -v --db enron --drop dump/enron
After command is finished I get a messagee:
120477 objects found
don't know what to do with file [dump/enron/messages.metadata.json]
But in the collection message I see 112196 documents using:
db.messages.count()
Could you please tell me what's the problem with it?
The output of the command:
c:\mongodb\mongodb-win32-i386-2.0.5\bin>mongorestore -v dump/enron/
Tue Dec 11 14:17:39 creating new connection to:127.0.0.1
Tue Dec 11 14:17:39 BackgroundJob starting: ConnectBG
Tue Dec 11 14:17:39 connected connection!
connected to: 127.0.0.1
Tue Dec 11 14:17:39 dump/enron/messages.bson
Tue Dec 11 14:17:39 going into namespace [enron.messages]
Tue Dec 11 14:17:39 file size: 396236668
126878231/396236668 32%
270206614/396236668 68%
375698921/396236668 94%
381433738/396236668 96%
387378348/396236668 97%
394626836/396236668 99%
120477 objects found
don't know what to do with file [dump/enron/messages.metadata.json]
What does the message: "don't know what to do with file [dump/enron/messages.metadata.json]" mean?
This should work:
ssharma$ mongorestore -d enron --collection messages /dump/enron/messages.bson
connected to: 127.0.0.1
Thu Mar 7 13:05:05 /dump/enron/messages.bson
Thu Mar 7 13:05:05 going into namespace [enron.messages]
Thu Mar 7 13:05:09 74234213/396236668 18% (bytes)
Thu Mar 7 13:05:12 126614885/396236668 31% (bytes)
Thu Mar 7 13:05:15 192098158/396236668 48% (bytes)
Thu Mar 7 13:05:18 208083274/396236668 52% (bytes)
Thu Mar 7 13:05:21 231816712/396236668 58% (bytes)
Thu Mar 7 13:05:24 293564538/396236668 74% (bytes)
Thu Mar 7 13:05:27 356071219/396236668 89% (bytes)
Thu Mar 7 13:05:30 379387449/396236668 95% (bytes)
120477 objects found
Thu Mar 7 13:05:32 Creating index: { key: { _id: 1 }, ns: "enron.messages", name: "_id_" }
ssharma$ ./mongo
MongoDB shell version: 2.2.2
connecting to: test
> use enron
switched to db enron
> db.messages.count()
120477
You need to specify the database and the collection when restoring the bson file.
It works for me like so:
$ mongodump -d mark --collection coll
connected to: 127.0.0.1
DATABASE: mark to dump/mark
mark.coll to dump/mark/coll.bson
1000 objects
and
$ mongorestore -d mark --collection newcoll dump/mark/
connected to: 127.0.0.1
Wed Aug 29 11:48:39 dump/mark/coll.bson
Wed Aug 29 11:48:39 going into namespace [mark.newcoll]
1000 objects found
Can you try -
mongorestore -d enron --collection messages /dump/enron/
I started seeing this in my mongodb production database logs
ns:my_app_production.artists query:{ $query: {}, $orderby: { semester: 1, name: 1 } }
Wed Jul 25 19:20:59 [conn199] Assertion: 10334:Invalid BSONObj spec size: -286331154 (EEEEEEEE) first element:_id: "agelio-batle"
Wed Jul 25 19:20:59 [conn199] assertion 10334 Invalid BSONObj spec size: -286331154 (EEEEEEEE) first element:_id: "agelio-batle"
first i tried running a repair
mongo --repair
...
Wed Jul 25 22:20:39 [initandlisten] Assertion: 10334:Invalid BSONObj spec size: -286331154 (EEEEEEEE) first element:_id: "agelio-batle"
0x467eaa 0x4183ca 0x62dd82 0x643478 0x532b22 0x64d196 0x6578b7 0x65ac31 0x65cd75 0x65d6d9 0x51a419 0x6195f5 0x61b0c5 0x61bd3d 0x4914c8 0x47ad9a 0x5e2e7c 0x5e60b9 0x5e78ad 0x6346a2
[0x467eaa]
# several more what i assume memory addresses omitted
[0x6346a2]
Wed Jul 25 22:20:39 [initandlisten] assertion 10334 Invalid BSONObj spec size: -286331154 (EEEEEEEE) first element:_id: "agelio-batle" ns:my_app_production.artists query:{}
Wed Jul 25 22:20:39 [initandlisten] exception in initAndListen std::exception: nextSafe(): { $err: "Invalid BSONObj spec size: -286331154 (EEEEEEEE) first element:_id: "a...", code: 10334 }, terminating
Wed Jul 25 22:20:39 dbexit:
...
i also tried running db.repairDatabase(); from the mongo shell with the same results.
I have never seen this before. Typically with mongodb a repair fixes most problems so im not sure how to proceed or troubleshoot this. any ideas?
If an overall repair fails you can mongodump out the individual collections with a --repair option and attempt to isolate the issues. You can even pass queries in to filter out the corrupt data from a corrupted collection, essentially working around the bad data, but it is an incremental and often slow process. This is why it is always recommended to take backups and run in replica sets to avoid the scenarion where you are left with a potentially corrupt data set.
That said, if you have no way to restore from a backup or another replica set member, then you can try something like (with the database shut down):
mongodump --dbpath /path/to/source/data/files --repair --db <dbname> --out /path/to/repaired
If that does not work, then to skip the index read (which might be tripping you up):
mongodump --forceTableScan --dbpath /path/to/source/data/files --repair --db <dbname> --out /path/to/repaired
Thanks to Ren in the comments, the other thing you can try is dropping/rebuilding the index. The table scan option (mongodump walks _id index by default) will avoid using the indexes for the dump. So assuming you get the path and the database names right, the second option should work.
so im trying to dump a collection on mongodump through stdout so i can directly pipe it and gzip it. But it's giving me errors.
When i do:
mongodump --db test -u username -p Password -h mymongodatabase.onthecloud:12888 -vvvvv --collection posts --out -
it displays these errors:
Wed Feb 29 21:48:13 creating new connection to:mymongodatabase.onthecloud:12888
Wed Feb 29 21:48:13 BackgroundJob starting: ConnectBG
Wed Feb 29 21:48:13 connected connection!
connected to: mymongodatabase.onthecloud:12888
Wed Feb 29 21:48:13 nextSafe(): { $err: "unauthorized db:test lock type:-1 client:10.3.55.10", code: 10057 }
Wed Feb 29 21:48:13 User Assertion: 13106:nextSafe(): { $err: "unauthorized db:test lock type:-1 client:10.3.55.10", code: 10057 }
assertion: 13106 nextSafe(): { $err: "unauthorized db:test lock type:-1 client:10.3.55.10", code: 10057 }
but when i dunt include '--out -' it dumps the collection fine (doesn't give me aunauthorized error). What's going on?
it was a bug, issued and fixed:
https://jira.mongodb.org/browse/SERVER-5233