I have 2 MongoDB collections on 2 different database. DB1-Collection1 is non-sharded, DB2-Collection2 is sharded. I use following command to dump DB1-Collection1.
mongodump --host some_ip:27017 --username some_user --password some_pwd --authenticationDatabase admin --db DB1 --collection col1 --gzip --archive=/root/col1.gz --quiet
Just before I run mongorestore command to restore data to DB2-Collection2, I saw this article,
https://docs.mongodb.com/manual/reference/program/mongodump/
which says
Starting in MongoDB 4.2, mongodump and mongorestore cannot be part of
a backup strategy for sharded clusters. These manual tools do not
maintain the atomicity guarantees of transactions across shards.
DB2 is running MongoDB 4.2, and Collection2 is sharded. Will it be OK if I run mongorestore to restore the data, or what's the proper way to restore data in that case.
Related
In mongodb is it possible to backup and restore a single collection in database?. Recently came across a database which which started as VM1 (primary) and VM2 (secondary) which at some point it time has failed over to VM2 (new primary) and VM1(New Secondary). When I took over the database looked at the collection in a database notices one of the collections in the database VM1 is not there in VM2.
Is there a way I can backup that specific collection in the database delete the collection and then import it to VM2 (New Primary) in the same database?
To export a specific collection use the below syntax
mongodump --port 27020
--db test
--collection restaurants
--out /mydata/restoredata/
To import to another VM - do the below
mongorestore --port 27017 --db test2 --collection rest2 /mydata/restoredata/test/restaurants.bson --drop
More details here -
https://docs.cloudmanager.mongodb.com/tutorial/restore-single-database/
I need to be able to restore a single database, even a single collection from a backup. Since mongodump --oplog only applies to a full instance (Replica Set), I made the following procedure to filter only the entries from the db that I want to restore from the oplog.bson generated by the --oplog option.
Could someone tell me if this is correct or if I'm missing something?
First I restore only the db (the db is test in this example) that I need
mongorestore -d test dump/test
Restore into a random collection the oplog.bson file generated by the --oplog option
mongorestore -d oplog -c oplog dump/oplog.bson
Dump from the restored oplog.bson collection only the documents that refer to the db that I'm restoring
mongodump -d oplog -c oplog -q "{ns:/^test[.]/}" -o oplog
Restore with the --oplogReplay option using the last dump with filtered operations
mongorestore --oplogReplay oplog/oplog
Finally I drop the temporary oplog collection.
mongo --eval "db.getSisterDB('oplog').dropDatabase()"
Thanks in advance!
Basically, that's one way to do it.. Or you can do extra export after dumping specific database.
mongoexport -d local -c oplog.rs --query="{ns:/^test[.]/}" -o oplog.dump
2017-10-10T11:51:50.780+0300 connected to: localhost
2017-10-10T11:51:51.737+0300 local.oplog.rs 0
2017-10-10T11:51:51.938+0300 local.oplog.rs 3
2017-10-10T11:51:51.938+0300 exported 3 records
And you have now direct json dump file what you can read back
I have a collection named dashboard in one db and i want to copy that collection to another db using robomongo. How can i do this? I tried creating a new collection in 2nd db and tried copying but it failed. so somebody please help me
Another db - another connection. But Robomongo works only with one connection in one period of time. That is why it is impossible.
I suggest you to use mongoimport/mongoexoprt tools for your task. They comes with mongo, are located in same folder as mongod.exe and allows to move collections via databases, by exporting to and importing from a json file.
Code sample:
mongoexport --db testFrom --port portFrom --username userFrom --password passwordFrom --collection yourCollection --out test.json
mongoimport --db testTo --port portTo --username userTo --password passwordTo --collection yourCollection --file test.json
When I restore a collection with the following command:
mongorestore --db mydb --drop --collection mycollection --batchSize=100 mycollection.bson
as the original collection is dropped, the indexes are lost. I can see that there is also a mycollection.metadata.json file which contains indexes of this collection, but I cannot find in the documentation how this file can be restored.
All I've found is how to restore an entire database, which restores all collections with metadata from a directory. However, I want to restore only a single collection. How do I do that?
Note: I am using mongo version 3.0.7
You don't need to do something specific to restore metadata.
mongorestore does this for you.
When you restore collection:
mongorestore --collection mycollection --db mydb mycollection.bson
mongorestore checks directory where mycollection.bson exist for mycollection.metadata.json file. Just keep metadata file in the same directory as collection.
I was trying to backup my mongo database on the product sever.and then restore then back to the staging server.
and here comes some problem, there are a lot of collections in db, I want to igonre some collections that I don't want to restore on staging server.
I can approach this by dumpping the staging db, dumpping the producting db, and then restore the prodct to staging useing --drop option. and restore the specified collections in staging db. uh..it's really bad.
1. dump producting db
mongodump --host product-server-host --username abcd --password bcda -d db -o pruduct-dump-dir
2. dump staging db
mongodump --host staging-server-host --username abcd --password bcda -d db -o staging -dump-dir
3. restore all collection, then restore the collection back
restore pruduct-dump-dir to staging server
mongorestore --host staging-server-host --username abcd --password bcda --drop pruduct-dump-dir
mongorestore --host staging-server-host --username abcd --password bcda --drop --collection coll pruducting-dump-dir
Is there any option like ignore-collection when I'm dumpping?
any suggestion will be appreciated :3
Now available from version 3.0.0
--excludeCollection <collection_name>
--excludeCollectionsWithPrefix <collection_prefix>
Repeat to exclude more than 1
Checkout the documentation
mongodump --db test --excludeCollection=users --excludeCollection=salaries
You can add --collection COLLECTION_NAME to dump the collection you need. By default, if you do not specify a collection to dump from a database, MongoDump will dump all collections in that database.
As of Mongo 3.4, you can now specify an --nsExclude <namespace pattern> option when restoring from a Mongo database dump, which will exclude the specified namespaces from the restore operation. This is particularly useful when the mongodump operation has already occurred.
Official documentation here: https://docs.mongodb.com/manual/reference/program/mongorestore/#cmdoption-nsexclude
You can exclude multiple collections with wildcards:
mongorestore --db test --nsExclude 'test.*_tmp'
Or alternatively, specifying multiple --nsExclude options work as well:
mongorestore --db test --nsExclude 'test.collection1' --nsExclude 'test.collection2'
I had to do the same while backing up a mongo db. If you use python (or any other language), you can use similar approach as well. After doing the mongodump, you simple have to remove the unwanted collection's bson & the metadata.json files.
import os
EXCLUDE_COLLECTIONS = ['collection_1', 'collection_2']
db_dump_path = "/Path/to/mongodump"
db_name = "name_of_db"
for collection_name in EXCLUDE_COLLECTIONS:
bson_file_path = os.path.join(db_dump_path, db_name, '{}.bson'.format(collection_name)
meta_file_path = os.path.join(db_dump_path, db_name, '{}.metadata.json'.format(collection_name)
if os.path.exists(bson_file_path) and os.path.exists(meta_file_path):
os.remove(bson_file_path)
os.remove(meta_file_path)