How to restore a mongodb backup on a replicaSet? - mongodb

I made a local backup of my mongodb (version 4) by doing
mongodump -u adminUser --authenticationDatabase admin --gzip --archive=/tmp/file.gz --db <dbname>
Now I would like to restore this data to my remote mongodb, which is a replicaset with two replicas running on a kubernetes cluster.
Normaly I would use
mongorestore -u adminUser --authenticationDatabase admin --gzip --archive=/tmp/file.gz
But how do I do this restore on a replicaset architecture?
Do I have to run a shell on any pod instance?
kubectl exec -it <pod-name> -n mongodb -- bash

You need to connect to a replica set. It needs to be set in connection string uri:
mongorestore --uri="mongodb://adminUser:password#hostname1,hostname2,hostname3/?authSource=admin&replicaSet=<replSetName>"
according documentation you can specify the replica set also with --host parameter
mongorestore -u adminUser --authenticationDatabase admin --host=<replSetName>/hostname1,hostname2,hostname3
mongorestore will connect to the PRIMARY member, data is restored into there and then replicated to SECONDARY members.
If you have to restore a lot of data, then consider Sync by Copying Data Files from Another Member

Related

MongoRestore is giving me the system cannot find the file

I am trying to export my local mongodb data to an atlas cluster and i created a dump and now i am using the command,
mongorestore --host Cluster0-shard-0/cluster0-shard-00-00-qwo7v.mongodb.net:27017,cluster0-shard-00-01-qwo7v.mongodb.net:27017,cluster0-shard-00-02-qwo7v.mongodb.net:27017 --ssl --username username --password <PASSWORD> --authenticationDatabase admin
to try to restore it but is giving me the system cannot find the file specified.
But if i type mongorestore then it works but it doesn't restore to the atlas cloud server whatever.
What am i doing wrong?
Edit: The path i have used is C:\Program Files\MongoDB\Server\4.2\bin\dump\gfg and dump\gfg but it is still not working.
Usage of MongoRestore.exe can be found here.
Standalone MongoDB
When you use mongorestore, you need to provide the location of the file/dump you need to restore
mongorestore --host=mongodb1.example.net --port=27017 --username=user --authenticationDatabase=admin /opt/backup/mongodump-2011-10-24
last portion of the command is missing that defines the location of the data that needs to be restored.
ReplicaSet
To Restore MongoDB on a Replica, You have to stop the mongod and replace the files for mongodb. Please read here for restoring database on replica set.
For restore from standalone to replica,
mongorestore --host myReplSet/mongo0.example.com:27020,mongo1.example.com:27012 --db <dbname> <folder_location>
If this still does not work, check path and ensure its properly escaped (spaces in name) or quoted.
Your code with quotes and path:
mongorestore --host Cluster0-shard-0/cluster0-shard-00-00-qwo7v.mongodb.net:27017,cluster0-shard-00-01-qwo7v.mongodb.net:27017,cluster0-shard-00-02-qwo7v.mongodb.net:27017 --ssl --username username --password <PASSWORD> --authenticationDatabase admin "C:\Program Files\MongoDB\Server\4.2\bin\dump\gfg"

coping mongo collection fails to maintain insertion order

The following command needs to copy a previously "dumped" mongo database from local machine to Mongodb cluster "Atlas", which it does but it fails to maintain the order of documents.
mongorestore --maintainInsertionOrder --ssl --db=myDB --host cluster0-shard-00-00-oko1k.mongodb.net:27017,cluster0-shard-00-01-oko1k.mongodb.net:27017,cluster0-shard-00-02-oko1k.mongodb.net:27017 --authenticationDatabase admin --dir=dump/myDB -u myname --password mypassowrd --drop
Any idea what is the fix so that the order of the collection at the destination is maintained?

MongoDB - dump and restore across different host, db with oplog

Is it possible to take a mongodump and mongorestore it to a different hosts, with different DB names, with oplog enabled?
From: mongodb://user:password#source-hostname:source-port/db1
To: mongodb://user:password#dest-hostname:dest-port/db5
When I do a mongodump with oplog on the source MongoDB, it takes a dump of the entire DB.
mongodump --oplog --host <source-hostname> -u <user> -p <password> --port <source-port> --authenticationDatabase admin
Now for the restore, I want to restore to a different hostname, and the db-name is also different. Is there way to restore the data to this db, with the oplogReplay?
mongorestore --host <dest-host> --port <dest-port> --username <user> --password <password> --authenticationDatabase admin --oplogReplay --db <db5> <path-to-dump>/dump
If I use oplogReplay, I am getting the following error
Can only replay oplog on full restore
I do not want to do a full restore, as it will create the db-name as db1, whereas I want to make use of db5. Also, there are already multiple DBs on this destination host and I do not want to bombard with another new database.
Any suggestions on this issue?
You can't use two options --oplogReplay and --db at the same time.
If you don't want to restore the full DB, simply go to dump/ folder and delete files for DBs other than db5. Then retry mongorestore without --db:
mongorestore --host <dest-host> --port <dest-port> --username <user> --password <password> --authenticationDatabase admin --oplogReplay <path-to-dump>/dump
If this doesn't not work for you, you may need to import oplog collection to a temporary db, and manipulate it to remove all except records for db5.

mongorestore from standalone to replicaset

I have dumped a standalone mongo database (14Gb big) running on the default port as follows:
mongodump --username <username> --password <password> --db <dbname>
and I am trying to restore it onto a replica set with the following:
mongorestore --host myReplSet/127.0.0.1:27020 --drop --username <username> --password <password>
The restore seems to work (it takes a minute or so and the output looks good) the database is created and all the collections are created but there is no data in the collections, they are all empty. What am I doing wrong?
For what it is worth, the local database is showing that it is 10 Gb big (I am not sure how big it was before the restore)
If anybody is facing the same issue. I used a replica set, and tried to do the Op's command. i.e
mongorestore --host myReplSet/mongo0.example.com:27020,mongo1.example.com:27012 --db <dbname> <folder_location>
Instead just go to the primary server and do this
mongorestore --port <if running on port other than 27017> --db <dbname> <location of dump>
This works for me, A possible explanation could be that whenever I write into the primary database, it is automatically copied over to my secondary databases. Hence no need to give any replica set address. This might work if you write in a secondary, I am not sure, haven't tried it. Maybe somebody else can throw a light on it.
Make sure the mongorestore version is the same version as the mongodb.
mongorestore --version
mongo --version
Same thing goes for the mongodump.
mongodump --version
They need to be the same or close enough versions.
These are issues that you will run into on a distributed system.
It will create the collections, but they will be empty if versions don't match.

Mongodump from remote server

We recently ported some data over to MongoDB and are now looking into running daily backups, preferably from a cron job, and restore one of the backups to a secondary mongo database.
Our system is set up as follows:
server 1: the development mongo database
server 2: two mongo databases, one for staging data and one for production
server 3: is where we run all of our cron jobs/batch scripts from.
I checked the mongo docs, and logged into our cron job server and tried to run the following command: (username, host, and password changed for security, I'm not actually connecting to localhost)
mongodump --host 127.0.0.1/development --port 27017 --username user --password pass --out /opt/backup/mongodump-2013-10-07-1
I get the following messages:
Mon Oct 7 10:03:42 starting new replica set monitor for replica set 127.0.0.1 with seed of development:27017
Mon Oct 7 10:03:42 successfully connected to seed development:27017 for replica set 127.0.0.1
Mon Oct 7 10:03:42 warning: node: development:27017 isn't a part of set: 127.0.0.1 ismaster: { ismaster: true, maxBsonObjectSize: 16777216, ok: 1.0 }
Mon Oct 7 10:03:44 replica set monitor for replica set 127.0.0.1 started, address is 127.0.0.1/
Mon Oct 7 10:03:44 [ReplicaSetMonitorWatcher] starting couldn't connect to [127.0.0.1/development:27017] connect failed to set 127.0.0.1/development:27017
I confirmed that I can connect to the mongo database using mongo -u -p ip/development
Our ultimate goal will be to dump the data from the production database and store it in the staging database. These two databases are both located on the same box, if that makes a difference, but for testing purposes I am just trying to get a backup of development test data.
mongo client can parse MongoDB connection string URI, so instead of specifying all connection parameters separately you may pass single connection string URI.
In your case you're trying to pass connection URI as a host, but 127.0.0.1/development is not a valid host name. It means you should specify database parameter separately from the host:
mongodump --host 127.0.0.1 -d development --port 27017 --username user --password pass --out /opt/backup/mongodump-2013-10-07-1
You can use with mongodump with --uri
mongodump --uri "mongodb://usersname:password#127.0.0.1:27100/dbname?replicaSet=replica_name&authSource=admin" --out "C:\Umesh"
All your collections will store inside the out folder it will create directory name as your Database name and all the collections are bson and metadata will store as json format.
For restore
mongorestore --uri "mongodb://usersname:password#127.0.0.1:27100/dbname?replicaSet=replica_name&authSource=admin" -d dbname mongodbumppath
Try this it will work.
This worked for me.
Reference: https://docs.mongodb.com/manual/reference/program/mongodump
Syntax 1:
mongodump --host <hostname:port> --db <database> --username <username> --password <password> --out <path>
Syntax 2:
mongodump -h <hostname:port> -d <database> -u <username> -p <password> -o <path>
Example 1:
mongodump --host 127.0.0.1:27017 --db db_app --username root --password secret --out /backup/db/app-17-03-07
Example 2:
mongodump -h 127.0.0.1:27017 -d db_app -u root -p secret -o /backup/db/app-17-03-07
mongodump --host remotehostip:port --db dbname -u username -p password
Here is an example of exporting collection from node server to local machine:
Host : xxx.xxx.xxx.xx
Port :27017
Username:”XXXX”
Password :”YYYY”
AuthDB : “admin”
“DB”: “mydb”
D:\mongodb-backup>mongodump -h xxx.xxx.xxx.xxx –port 27017 -u “XXXX” -p “YYYY” –authenticationDatabase “admin” –db “mydb”
Use this to get dump using URI:
mongodump --uri=mongodb+srv://john:xxxxxxxxxxxxxxx#cluster0-jdtjt.mongodb.net/sales
You can also use gzip for taking backup of one collection and compressing the backup on the fly
mongodump --db somedb --collection somecollection --out - | gzip > collectiondump.gz
Or with a date in the file name:
mongodump --db somedb --collection somecollection --out - | gzip > dump_`date "+%Y-%m-%d"`.gz
This worked to me like a charm for a single collection with a remote Windows Server.
mongodump --host <remote_ip> --port <mongo_port> --db <remote_db_name> --authenticationDatabase <remote_auth_db> --username <remote_mongo_username> --password <remote_db_pwd> --out <local_DB_backup_folder> --collection <remote_collection_name>
On Mac, this is what worked for me (but be sure to use your own real credentials):
brew tap mongodb/brew
brew install mongodb-community#5.0
brew services start mongodb/brew/mongodb-community
mongodump --uri "mongodb://usersname:password#127.0.0.1:27100/dbname" --out "/Users/some_username/code/mongodb_dumps/dump/"
cd /Users/some_username/code/mongodb_dumps/
mongorestore --nsInclude "*.*"
mongodump --host hostip -d dbname --port portnumber --username username --password password --authenticationDatabase admin -o ./path/of/backupfolder
note: "./path/of/backupfolder" path is in your client
This worked for me:
Step1: Export remote/local DB.
mongodump --uri "mongodb+srv://USER:PASSWORD........." --out "/Users/Hardik/Desktop/mongo_bkp"
Step2: Import
mongorestore ./mongo_bkp/
Posting this here in case it helps somebody.
It was impossible for me to connect using mongodump. I ended up installing the VS Code Mongo extension and it generated the string for me. The command looks like this:
mongodump -o dump_destination --uri "mongodb://<USERNAME>:<PASSWORD>#<HOST>:<PORT>/<DATABASENAME>?authSource=admin&readPreference=primary&ssl=true"