Is it possible to write JSON that will cause mongoimport to append to existing arrays during an upsert? (mongodb 2.0)
It appears that, as of now (9/26/11) this is not possible. Users with this problem are encouraged to write their own import script.
Related
I'm trying to find a way to import a whole dataset into one MongoDB document. Every solution I've tried only inserts many documents instead of one. What I've tried so far.
mongoimport --db=dbName --collection=collectionName --drop --file=file.json --jsonArray
Thank you in advance!
Problem is solved, thanks to both prasad_ and Joe.
After using the mongoimport that is shown in my question, I just aggregated using $group to get everything into one document. Though there was a problem which Joe commented, that a document is limited to 16MB. Which meant that I had to remove some data from the dataset that I wasn't using.
I'm trying to write a mongodump / mongorestore script that would copy our data from the production environment to staging once a week.
Problem is, I need to filter out one of the collections.
I was sure I'd find a way to apply a query only on a specific collection during the mongodump, but it seems like the query statement affects all cloned collections.
So currently I'm running one dump-restore for all the other collections, and one for this specific collection with a query on it.
Am I missing something? Is there a better way to achieve this goal?
Thanks!
It is possible.
--excludeCollection=<string>
Excludes the specified collection from the mongodump output. To exclude multiple collections, specify the --excludeCollection multiple times.
Example
mongodump --db=test --excludeCollection=users --excludeCollection=salaries
See Details here.
Important mongodump writes to /dump folder. If it's already there, it will overwrite everything.
If you need that data rename the folder or give mongodump an --out directory. Otherwise you don't need to worry.
I want to export around 5000 MongoDB Collections to JSON format using a single command. Is it possible to do so?
Please take a look at this script.
It's a bash script that basically does the following:
Read the collections from MongoDB to a variable
Iterate and call mongoexport for each collection
I am trying to use mongoimport to translate a one-to-many relational structure into mongoDB using csv files. My approach is to import the "one" file then use the upsert option to append the "many" records as a nested array but it looks like it only replaces the original document instead of appending.
Is this a limitation of mongoimport or could I be doing something wrong?
You can do upserts when using mongoimport, but you cannot use complex operators to perform modifications to the data as you would with a normal update operation. This is a limitation of mongoimport - essentially each piece of data you import must be insert ready even though you are using the upsert functionality, which is basically working as a de-duplication mechanism for your input data.
If you wish to merge in a more sophisticated manner then it would be best to use one of the drivers and merge the data using your language of choice. This also has the advantage of avoiding potential issues with type fidelity and allowing you to code around exceptions etc.
Is it possible to do bulk update/upsert (not insert) in MongoDB?
If yes, please point me to any docs related to this?
Thanks
You can use the command line program mongoimport it should be in your MongoDB bin dir ...
There are two options you'll want to look into to use upsert ...
--upsert insert or
update objects that already exist
--upsertFields arg comma-separated fields for the query
part of the
upsert. You should make sure this is indexed
More info here: http://www.mongodb.org/display/DOCS/Import+Export+Tools
Or just do ...
$ mongoimport --help
mongo can execute .js file.
you can push all you update commands in a js file.
t.js
db.record.update({md5:"a35f10a8339ab678612d1f86be08b81a"},{$set:{algres:[]}},false,true);
db.record.update({md5:"a35f10a8339ab678612d1f86be08b81b"},{$set:{algres:[]}},false,true);
then,
mongo 127.0.0.1/test t.js
Bulk updates can also be done in batches as found in the documentation:
MongoDB Bulk Methods
I use these to import CSV files that I need to massage a bit before importing the data. Its kinda slow when dealing with updates, but it did my 50K document updates in about 83 seconds, which is far slower than mongoimport command.