I downloaded the wikidata json compressed dump which is 7 gb. I extracted it and its 122 gb ! what is the best way to get this data into mongo ?
My real use case is I just want to query this data in python. Is there a better option ?
PS: I'm using my laptop.
After obtaining the JSON file, you can simply use the load command to MongoDB:
mongoimport --db dbName --collection collectionName --file fileName.json --jsonArray
Related
I have large number of files in database,I need to take backup of specific week files and export to another database.I can dump fs.files based on uploadDate.
./mongodump --port <port> --db <Database> --collection fs.files --query <json> --out <destination>
How can I export the specific fs.chunks data while iterating fs.files in the shell?
Here's a blog post and the gist for a bash script that will do what's asked for here. Run this script from the command line of the mongodb server. It will loop through the fsfiles collection and exporting the files using the mongofiles utility that is included with MongoDB.
Original Blog Post
Gist
So I'm searching where the files are stored after this command
mongoimport --db meteor --collection articles --type csv --headerline --file /data/seeds/articles.csv
I'm using a Docker container from the image mongo:latest
I thought it would be in the /data/db but there isn't my db 'meteor' or my collection 'articles' so I think I'm not in the right folder...
(The application is working perfectly, the data are displayed on the website)
Someone can help me ?
MongoImport imported the csv data into db collection articles.
You will now have to connect to mongod server through mongo shell and run the below commands.
Switch to the db meteor
Use meteor
View collection article
db.article.find()
I have a mongo server receiving data from servers behind an amazon LB, 2 days ago there was an error and part of the servers sent their data to an old mongo server that had a db of the same name, we realized that and fixed it back right away.
Now i have part of my data stored on the wrong machine.
What I need now is a way to compare the data between the 2 dbs (which each have 2 relevant collections) and insert only the missing data to the correct collection.
I do not care about the unique id mongo gives but i do need to compare by the field "uuid" that we create.
mongo version: 2.4.4
I am new to Mongo and any help would be greatly appreciated.
Yes, you can. Follow these steps...
1 mongoexport and then mongoimport on the basis of fields on which you want to compare and import.
mongoexport --db db_name --collection collection_name --type csv --fields field1,field2,field3 --out /var/www/export.csv
once you get the exported CSV at the specified location. Open and remove unwanted fields...
mongoimport --db db_name --collection collection_name --type csv --file /var/www/export.csv --fields field1,field2,field3 --upsertFields field1,field2,field3
NOTE:
1. If you working in production environment, dealing with huge database then free up your mongo from load of queries and then export else it might get stucked.
I just started learning to build nodejs application. I am able to figure how things work so i made decision to test my application with large test data.
I create json file with 1 million records in it.
I import data using
mongoimport --host 127.0.0.1 --port 27017 --collection customers --db Customer --file "path to json file mock.json" --jsonArray
Sample Json file is
[{"fname":"Theresia","lname":"Feest","email":"aileen#okeefe.name"},
{"fname":"Shannon","lname":"Bayer","email":"tristian.barrows#christiansenvandervort.com"},
{"fname":"Cora","lname":"Baumbach","email":"domenico.grimes#lesley.co.uk"},
{"fname":"Carolina","lname":"Hintz","email":"betty#romaguerasenger.us"},
{"fname":"Dovie","lname":"Bartell","email":"rogers_mayert#daniel.biz"}]
but it is taking too much time approx. 14 Hrs.
Please suggest any other optimized ways for the same.
Split your single json file into multiple files and then run multiple parallel mongoimport commands for each of the file.
How is it possible to Import and export the MongoDB from Meteor APP into .json or .csv file ?
You can use mongoexport utility to export into a .json or .csv file and mongoimport to import to your db. Detailed info is found here: http://docs.mongodb.org/v2.2/reference/mongoexport/
If you intend to reimport into mongodb, mongodump might be a bit better since you won't be losing type information
Try mongodump
mongodump --host 127.0.0.1:3001
Here you get BSON, and separate key indexes in json files. Then you can use the dumps to restore your db (if this is your intention):
If you're in the dump directory from above:
mongorestore --host 128.199.224.29:27017 --db meteor .