How can I dump an entire MongoDB database as text (plain text, json, or CSV)?
I'm using an application I'm not too familiar with. I'd like to
clear the database
load seed data
dump the whole db as text
do some stuff
dump again
then diff the two!
Using mongodump and bsondump:
Step 1
Dump the entire database to BSON files:
mongodump --db db1
Step 2
Convert each BSON file to JSON file:
for f in dump/db1/*.bson; do bsondump "$f" > "$f.json"; done
Hope it's helps!
You can use mongoexport utility. It will dump out json by default but you can specify to dump out csv format.
mongoexport --help will give you all the options you will need.
You mentioned that you would like to dump all the collections from a database, and mongoexport expects you to specify a database and collection name, since it expects to dump one collection.
You can write a short shell script (if you are on Windows, convert to batch script) to the effect of:
#!/bin/sh
# assuming mongo bin is in your path
host=YOURMONGOHOST
port=YOURMONGOPORT
db=DBYOUWANTTOEXPORT
for c in `mongo --quiet $host:$port/$db --eval 'db.getCollectionNames()' | sed 's/,/ /g'`
do
mongoexport --host $host --port $port -d $db -c $c > $c.json
done
Related
I'm trying to dump an existing MongoDB into a new MongoDB without using dump files at all.
Is there a way to pass data from mongodump into mongorestore using pipes?
something like this:
mongodump --host=0.0.0.0 --port=27017 --db=database_name | mongorestore --host 0.0.0.0 -d database_name --port 27000
Thanks!
I found this solution:
mongodump -vvvvv --host=0.0.0.0 --port=27017 --db=database_name --archive | mongorestore --archive -vvvvv
Explained:
mongodump a MongoDB util to dump (backup) a database
-vvvvv v for verbose, write to stdout the output of the command, more data per v, you can use -v -vv -vvv etc...
--host specify the host of the MongoDB you want to dump
--host specify the port of the MongoDB you want to dump
--archive writes the output to a specified archive file or, if the archive file is unspecified, writes to the standard output (stdout)
| takes the output of the command and passes it as input to the next command
mongorestore a MongoDB util to restore a database
I am using mongodump ( version r3.6.18) and I want to backup a list of collections.
I tried using the --exclude option but the problem I am facing is OSError: [Errno 7] Argument list too long which is an OS error if the argument size increases more than the ARGMAX buffer.
There is a way to backup a list of collections by looping over it, but I want to use the --archive and --gzip option on which it is not working. Can you please provide an alternative for the below problem statement -
I want to backup a list of collections
The backup archive must be a single file (gzip archive)
Here is the bash script that you can use to dump specific collections:
#!/bin/bash
colls=(coll1 coll2 coll3)
for c in ${colls[#]}
do
mongodump --host {YOUR_HOST} -d {YOUR_DB} -u {USERNAME} -p {PASSWORD} -c $c --gzip --out OUTPUT
done
I use the following command to backup my local db
mongodump -h 127.0.0.1 --port 8001 -d meteor -c products --archive --gzip > dump.gz
Then I use the following command to restore on my server
cat dump.gz | ssh root#66.205.148.23 "cat | docker exec -i mongodb mongorestore --archive --gzip"
I want to do the same but with only one collection. Adding the -c parameter to the above commands does not work when trying to restore. I get a message that states that the -c param can only be used with BSON files.
How can I do the above for only one collection using the -c parameter?
Thanks
Use --out option for mongodump instead of --archieve to write BSON files
Specifies the directory where mongodump will write BSON files for the dumped databases. By default, mongodump saves output files in a directory named dump in the current working directory.
To send the database dump to standard output, specify “-” instead of a path. Write to standard output if you want process the output before saving it, such as to use gzip to compress the dump. When writing standard output, mongodump does not write the metadata that writes in a .metadata.json file when writing to files directly.
You cannot use the --archive option with the --out option.
This will create folder dump with BSON files
mongodump -h 127.0.0.1 --port 8001 -d meteor --gzip --out dump
To restore:
mongorestore -h 127.0.0.1 --port 8001 -d meteor --gzip -c collname foo dump/meteor/collname.bson.gz
I was given a data dump of bson files. In the mongo db, the database and the collections exists. These are updates to each of the collections in the database. So, in the given directory, there are about 30 bson files for each collection.
From the command line, I am using ubuntu, how do I append and load? Mongo is on my localhost with no username or password.
Thanks
Took me a while to get around this excuse for an error. In the end I went to the directory outside of my dump folder, and did the following...
For a full DB restore:
mongorestore --drop dump/mydb
Note that the mongodump operation will create individual folders for each database within the dump folder it creates, so you need to specify the full relative path, as above.
For a single collection:
mongorestore --drop -d mydb -c mycollection dump/mydb/mycollection.bson
The usual syntax is:
mongorestore -d dbname -c collectionname dir/file.bson
are you looking for mongorestore? http://www.mongodb.org/display/DOCS/Import+Export+Tools#ImportExportTools-mongorestore
import Bson
mongorestore -d dbname -c collectionname dir/file.bson
import Json
mongoimport --collection NAME --file NAME.
http://docs.mongodb.org/v2.2/reference/mongoimport/
Since Mongo restore does not update the current records this would not be a good choice.
Mongorestore only appends new records as stated:
mongorestore just does inserts with the data to restore; if existing
data (like with the same _id) is there it will not be replaced.
You may wish to build a BSON parser in your language of choice and make a more complex tool than mongorestore, since mongorestore is only designed to "restore" (as the name kinda suggests) a database/collection you will need to write something a little more complicated to do what you want and that depends heavily on your server-side language.
Edit
This is actually better done with mongoexport and mongoimport:
http://www.mongodb.org/display/DOCS/Import+Export+Tools#ImportExportTools-mongoimport
With mongoexport you could export a JSON file and give the command line for it do upserts. So I would personally go back to the person who gave this file and tell them that you actually want a mongo export file instead.
1) Go to the directory where the "dump" folder is located in CMD.
2) Run the mongorestore command.
I have used mongodump to dump my database of mongodb, it created some bson files under dump/mydb
But I don't know how to use them. I tried mongoimport, but seems it can't import bson data. Then how to use these bson files? How to import them to another mongodb?
You need to use mongorestore, not mongoimport ... which is used for things like importing json, or csv, etc.
From the back-up-with-mongodump docs:
mongodump reads data from a MongoDB database and creates high fidelity BSON files which the mongorestore tool can use to populate a MongoDB database.
mongodump and mongorestore are simple and efficient tools for backing
up and restoring small MongoDB deployments, but are not ideal for
capturing backups of larger systems.
You can read more about mongorestore in the docs below; I'd take a look and read up on them as they are very helpful.
http://www.mongodb.org/display/DOCS/Import+Export+Tools#ImportExportTools-mongorestore
You can also check out http://learnmongo.com for tips and help!
I am using mongodump, mongorestore for daily backups and restoring from backup. I have two .bat files:
First, for backup, where you need just specify host database name and backup folder:
SET host=localhost:27020
SET dbNameToDump=MyDB
SET backupsFolder=Backups
mongodump.exe --host %host% --db %dbNameToDump%
SET date="%date:~10,4%-%date:~4,2%-%date:~7,2%.%time:~0,2%-%time:~3,2%"
cd %backupsFolder%
md %date%
xcopy /e ..\dump %date%
rmdir /s /q ..\dump
Above bat file create folder with name like this 2011-03-31.11-17(yyyy-MM-dd.hh-ss) in folder Backups with dumped collections from specified database. In files explorer it looks like so:
Second bat file i use for retore specified dumped files(here you also need specify database name and folder with dumped files):
SET host=localhost:27020
SET dbNameToRestore=MyDB
SET restoreFolder=Restore
mongorestore.exe --host %host% --db %dbNameToRestore% %restoreFolder%
In files explorer:
In additional, i am using windows schedule to automate backup process.
Hope above information will be useful for someone.
As mentioned in the previous answers, you have to use mongorestore instead of mongoimport. Adding to the previous answers, when your mongodb is running, execute the following command to restore your dump from the dump directory,
mongorestore dump
This will import all the collections into your mydb database. However this doesn't drop the database before restoring. If you wish to drop the database before importing,
mongorestore --drop dump
The bson files in the mydb directory will be restored as the collections inside mydb database. For more info on mongorestore check the documentation here.
Use mongorestore. mongoimport works on the output of mongoexport. mongodump & mongorestore work on binary data files while import / export work on json, csv, etc.. (human readable formats)
For resolving this, I copied the dump folder,dbdump(which contains bson files) to bin directory of mongodb and executed the below commands in command prompt:
1.
cd "path to MongoDB's bin folder"
(Example: cd C:\Program Files\MongoDB\Server\3.2\bin)
2.
mongorestore.exe --dir ./directory name --db database-name
(Example: mongorestore --dir ./dbdump --db testdb)
All bson files in the dump folder will be imported into your database.
You can verfiy this by executing the below commands :
cd "path to MongoDB's bin folder"
mongo.exe
show dbs;
For mongo version 3 and above use the command below:
mongorestore --host=localhost --port=27017 --username=root --authenticationDatabase=admin --db=test dump_folder/
Mongo will ask password after that