I am setting up a Mongo DB database on mongolab. I have 2 small documents in a single collection in my local db server, that I want to upload. I used mongodump to export that to a collection.bson and collection.metadata.json file
When I use mongorestore I get an error:
Failed:database.collection: error restoring from /tmp/mongodump/database/collection.bson: insertion error: EOF
collection.bson is less than 2KB. My research shows that this error shows up when your database is huge, but not very small. I can't get anything for my situation. The common solution of using --batchSize 1 results in the same issue.
After I run mongorestore the collection does exist on the remote, but it is empty (0 documents)
How can I get my tiny local db up on my remote server (on MongoLab).
I am locally running mongo3.4, but my MongoLab instance is 3.2.13. I'm wondering if the problem is the mismatch of versions?
mongorestore command:
mongorestore -h ds111882.mlab.com:11882 -d database -u username -p password /tmp/mongodump/database
mongodump
mongodump -d database -o /tmp/mongodump
Additional info: I imported the mongodump I created into another local database (also running 3.4) and it worked just fine, but restoring to the mongolab server causes the error.
Also a bsondump of the generated collection.bson file creates a json file with the correct 2 documents.
Found the answer, it is an issue with versions. My schema was using the Decimal128 type, which isn't supported in 3.2. Downgrading to Double (didn't need Decimal128 anyway) fixed the issue.
Related
Failed: (AtlasError) getting non-bucket system collections is unsupported
I'm trying to use mongorestore to migrate data from one database to another. Both on Atlas. The dump is working fine, but mongorestore is outputting the below messages. I have no clue what this means and a google search renders nothing remotely close. I've added hyphens to the beginning of each line of the output to make it more readable.
-- using write concern: &{majority false 0}
-- will listen for SIGTERM, SIGINT, and SIGKILL
-- connected to node type: replset
-- The --db and --collection flags are deprecated for this use-case; please use --nsInclude instead, i.e. with --nsInclude=${DATABASE}.${COLLECTION}
-- got error from options parsing: (AtlasError) getting non-bucket system collections is unsupported
-- Failed: (AtlasError) getting non-bucket system collections is unsupported
-- 0 document(s) restored successfully. 0 document(s) failed to restore.
The command I'm running
mongorestore --uri="mongodb+srv://username:password#hostname.mongodb.net/$DEV_DATABASE" \
--preserveUUID \
--drop \
--nsFrom="$PROD_DATABASE.*" \
--nsTo="$DEV_DATABASE.*" \
--verbose \
"dump/$PROD_DATABASE"
I've also tried creating an archive file with mongodump and using that with --archive="filename", as well as piping stdout to mongorestore. I've also checked that the user I'm using has the correct privileges. They have the role of Atlas admin, which I'm assuming is correct. The dev cluster I'm trying to restore to is an M0 if that makes any difference.
I should also point out that I have minimal Mongo management experience, so I'm sure there's something I've overlooked. Thanks for your help.
MongoDB records the collection UUIDs in a separate system collection.
The --preserveUUID option instructs mongorestore to create the collection, and force it to use the UUID from the source system.
The error message indicates that Atlas is refusing to allow you to access or modify that system collection.
Run without the --preserveUUID option when restoring to Atlas.
I have dumped a database from a server using mongodump - the version of mongodb used here is 2.4.
To restore the dump into the latest 3.6.5 mongoDb instance I used the command mongorestore --drop -d database dumpFolder
I got the error:
Failed: myDb.collectx: error creating collection myDb.collectx: error running create command: BSON field 'OperationSessionInfo.create' is a duplicate field
I'm hitting the same thing, and found some detail in an answer on a related site: https://dba.stackexchange.com/questions/201827/failed-to-import-a-mongodb-database-with-duplicate-fields
What I've gathered is that mongodump version 2.2 and later is incompatible with earlier versions of MongoD - so check the version you used to dump the data.
I was able to use bsondump my_collection.bson to get a JSON dump output of the details, so I suspect I can load it from that, even though it's not recommended.
UPDATE:
I was able to use bsondump and mongoimport together with the raw BSON file to get the data loaded:
bsondump my_collection.bson > my_collection.json
mongoimport -d my_db -c my_collection my_collection.json
So a tad inelegant, but I was able to load up the MongoDB 2.4 data into a MongoDB 3.6.5 instance locally.
I am currently trying to import a group of JSON files containing data into my mongo database hosted on IBM Bluemix/Compose.
I have a script that runs through the files creating and then running a mongoimport command to import the files into the database, this works great against my local database(and indeed occasionally against the Compose database) however most of the time I get the following error -
2017-05-09T14:59:02.508+0100 Failed: error connecting to db server:
SSL errors: x509 certificate routines:X509_STORE_add_cert:cert
already in hash table x509 certificate
2017-05-09T14:59:02.508+0100 imported 0 documents
My mongoimport command looks like this -
mongoimport --batchSize 100 --ssl --sslAllowInvalidCertificates --host *censored* --collection Personnel --file data/TestData/Personnel_WICS.json -u admin -p *censored* -d MY_DB --authenticationDatabase admin
Is this a mongoimport error? Perhaps an issue with Compose? Or am I doing something incorrectly with the command?
I should note that the files I am importing range in size from 3mb-100mb, but even reducing the larger file sizes down by splitting them up does not seem to help.
My import script runs one import command immediately following the completion of the previous one, is there maybe some issue in running several back to back imports like this?
For anyone finding this in future - it looks like this may have been caused by a mismatch in mongo versions between the machine I'm running the mongoimport command from and the mongo database hosted in compose.
Compose DB Version: 3.2
Build server machine(running mongoimport): 3.4
Downgrading the build server version has resolved the issue.
My mongodb can't lanuch now, when I want start mongo got error ***aborting after invariant() failure
Now I want to restore collection-0-****.wt file to a new db, is this possible?
As at MongoDB 3.2, only full backups of WiredTiger data directories can be copied into a new instance. WiredTiger collection or index files aren't self-contained; they rely on other metadata in the WiredTiger.* catalog files. The invariant/assertion you are getting on startup is expected if data files are incomplete or inconsistent.
If you want to backup and restore a single collection, you should use mongodump and mongorestore, eg:
mongodump --db test --collection northwind --host host1
mongorestore --db test dump/test/northwind.bson --host host2
For supported full backup procedures, see: MongoDB Backup Methods.
I had the same issue and after spending 5 hours doing everything, found this.
https://medium.com/#imunscarred/repairing-mongodb-when-wiredtiger-wt-file-is-corrupted-9405978751b5
You will need to restore 1 collection at a time(a few at once when you get the hang of it), but it works!
It might be a dead simple question yet I still wanted to ask. I've created a Node.js application and deployed it on Heroku. I've also set up the database connection without having any trouble as well.
However, I cannot get the load the local data in my MongoDB to MongoLab I use on heroku. I've searched google and could not find a useful solution so I ended up trying these commands;
mongodump
And:
mongorestore -h mydburl:mydbport -d mydbname -u myusername -p mypassword --db Collect.1
Now when I run the command mongorestore, I received the error;
ERROR: multiple occurrences
Import BSON files into MongoDB.
When I take a look at the DB file for MongoDB I've specified and used during the local development, I see that there are files Collect.0, Collect.1 and Collect.ns. Now I know that my db name is 'Collect' since when I use the shell I always type `use Collect`. So I specified the db as Collect.1 in command line but I still receive the same errors. Should I remove all the other collect files or there is another way around?
You can't use 'mongorestore' against the raw database files. 'mongorestore' is meant to work off of a dump file generated by 'mongodump'. First us 'mongodump' to dump your local database and then use 'mongorestore' to restore that dump file.
If you go to the Tools tab in the MongoLab UI for your database, and click 'Import / Export' you can see an example of each command with the correct params for your database.
Email us at support#mongolab.com if you continue to have trouble.
-will
This can done by two steps.
1.Dump the database
mongodump -d mylocal_db_name -o dump/
2.Restore the database
mongorestore -h xyz.mongolab.com:12345 -d remote_db_name -u username -p password dump/mylocal_db_name/