I have a mongo server receiving data from servers behind an amazon LB, 2 days ago there was an error and part of the servers sent their data to an old mongo server that had a db of the same name, we realized that and fixed it back right away.
Now i have part of my data stored on the wrong machine.
What I need now is a way to compare the data between the 2 dbs (which each have 2 relevant collections) and insert only the missing data to the correct collection.
I do not care about the unique id mongo gives but i do need to compare by the field "uuid" that we create.
mongo version: 2.4.4
I am new to Mongo and any help would be greatly appreciated.
Yes, you can. Follow these steps...
1 mongoexport and then mongoimport on the basis of fields on which you want to compare and import.
mongoexport --db db_name --collection collection_name --type csv --fields field1,field2,field3 --out /var/www/export.csv
once you get the exported CSV at the specified location. Open and remove unwanted fields...
mongoimport --db db_name --collection collection_name --type csv --file /var/www/export.csv --fields field1,field2,field3 --upsertFields field1,field2,field3
NOTE:
1. If you working in production environment, dealing with huge database then free up your mongo from load of queries and then export else it might get stucked.
Related
I downloaded the wikidata json compressed dump which is 7 gb. I extracted it and its 122 gb ! what is the best way to get this data into mongo ?
My real use case is I just want to query this data in python. Is there a better option ?
PS: I'm using my laptop.
After obtaining the JSON file, you can simply use the load command to MongoDB:
mongoimport --db dbName --collection collectionName --file fileName.json --jsonArray
I use mongochef as a UI client for my mongo database. Now I have collection which consists of 12,000 records. I want to export them using the mongochef.
I have tried with the export option(available) which is working fine up to 3000 documents. But if the number of records gets increasing the system is getting hung up.
Can you please let me know the best way to export all the documents in a nice way using mongochef.
Thanks.
Finally I came to conclusion to use the mongo using terminal which the best way to use(efficient).
read about the primary and secondary databases and executed the following query:
mongoexport --username user --password pass --host host --db database --collection coll --out file_name.json
I am trying to migrate my database from SQL server to mongoDB, I've exported tables from my database, and wrote the corresponding mongoDB import function for it:
mongoimport -d dashboard -c col --type csv --file "C:/Users/sesa356116/text.csv" --headerline -f bu_id,bu_name,bu_id,
this is getting executed, but when I am trying to see the data in robomongo, no records are getting displayed. What I want to know is after this import function are there any further steps that should be followed in order to get the corresponding records, any leads for the same will be helpful.
I just started learning to build nodejs application. I am able to figure how things work so i made decision to test my application with large test data.
I create json file with 1 million records in it.
I import data using
mongoimport --host 127.0.0.1 --port 27017 --collection customers --db Customer --file "path to json file mock.json" --jsonArray
Sample Json file is
[{"fname":"Theresia","lname":"Feest","email":"aileen#okeefe.name"},
{"fname":"Shannon","lname":"Bayer","email":"tristian.barrows#christiansenvandervort.com"},
{"fname":"Cora","lname":"Baumbach","email":"domenico.grimes#lesley.co.uk"},
{"fname":"Carolina","lname":"Hintz","email":"betty#romaguerasenger.us"},
{"fname":"Dovie","lname":"Bartell","email":"rogers_mayert#daniel.biz"}]
but it is taking too much time approx. 14 Hrs.
Please suggest any other optimized ways for the same.
Split your single json file into multiple files and then run multiple parallel mongoimport commands for each of the file.
Is it possible to duplicate collection from single server to cluster? I have a collection on test server (single machine) and I will copy the collection to another server (cluster) and I'm not sure if I can use duplicate collection.
I want to copy collection once.
First use mongoexport on the source server to export the collection to a file.
mongoexport --db yourDB --collection yourCollection --out yourCollection.json
When the collection doesn't already exist on the destination shard or isn't yet configured to be sharded, you should do so now by connecting to the mongos instance with the mongo shell and using the command:
sh.shardCollection( "yourDatabase.yourCollection", { yourDesiredShardKey: 1 } )
Then use mongoimport on the destination to import the collection.
mongoimport --db yourDB --collection yourCollection --file yourCollection.json
Both mongoimport and mongoexport have optional --host and --port parameters to import from / export to a remote server. But I would recommend you to copy the file to the destination server yourself. First, this should be faster. Second, in a securely configured network, you shouldn't be able to access both test and production database from the same machine anyway, at least not without authentication.
For a collection to use a cluster's power, you need to have shard options. Duplicating the test collection won't do that.
You can create a new collection with the right sharding options and then copy items of the test collection into it
db.test.mycollection.find().forEach( function(x){db.otherdb.othercollection.insert(x)} );