I have two mongodbs, one is on my server and another is on my laptop.
how do I sync from server to local once a day? What is python code to do it?
You can do a mongodump from the server to your laptop and then run mongorestore on your laptop to sync the data. How frequently you want to do this depends on how big your dataset is and if you have the bandwidth for it. You could try the following steps and see if they work for you.
To backup:
From your laptop's Terminal, assuming you have Linux or Mac OSX, run the following commands (the Windows commands are very similar):
$ mongodump --host <HOST> --authenticationDatabase admin -u <USER> --password <PW>
If you don't have authentication set up, then you can omit the authentication and credentials information. So the command would look something like this:
$ mongodump --host <HOST>
To restore:
This will create a dump folder in your current directory containing the backup. Then when you want to restore you would run the following. You can omit the --drop flag if you don't want to drop the databases before restoring.
$ mongorestore --drop
Try these commands and if they work for you, you can write up a shell script which you can run daily in order to sync your database. Also, if your dataset is big enough and you want to be super precise about syncing your data, you can run the mongodump with the --oplog option. This means that the dump will include a partial oplog containing operations from the duration of the mongodump operation. Then to restore you run mongorestore with the --oplogReplay flag. This will ensure that any operations that occurred during the dump will also be backed up.
For more information, you should check out the mongodump and mongorestore pages in the documentation.
Related
I am trying to host my database online and based on what I have found online, I need to use mongodump to export my database first.
The way to use mongodump if am not wrong is:
mongodump -d <db-name> -o <directory>
when I use the above command, I get the following error in the terminal:
Failed: error connecting to db server: no reachable servers
I tried to add --host=127.0.0.1 after mongodump as follows:
mongodump --host=127.0.0.1 -d <db-name> -o <directory>
But I still get the same result. What am I doing wrong here?
Update:
I managed to overcome the error by starting the MongoDB service with the following command:
brew services start mongodb
Now when I run mongodump, it seems to be working but I can't find it when I navigate to the directory where it supposed to be located!
NOTE: I am using Meteor technology, and I am accessing my database with meteor mongo command
If you are doing mongodump from a remote server it can happen that versions are incompatible. This results in no documents being dump without any warnings. (At least from my tests, mongodump 2.6.10 won't be able to dump from mongod 3.2.13)
Also, make sure bash special characters are not breaking up your query.
Example:
mongodump --db DB_name --collection colname --query "{$or: [something1, something]}"
The previous query won't work as you need to escape the $ with \.
mongodump --db DB_name --collection colname --query "{\$or: [something1, something]}"
You could try:
Run mongodump --db <database> --port 3001 from the directory you want the output files to be created (it will create a dump dir with the files)
The port is 3001, as it seems meteor doesn't use Mongodb default port
Also, if nothing appears, try running with the -v flag for verbose mode, this will help you to find out why your files are not being created. Also, be sure yout database name is correct.
I was using MongoDB on localhost of my Ubuntu (just for education purpose), but now, I have a new computer. So, I would like to have this collection on my new PC. What files do I need to copy to do this after installing MongoDB on a new machine ?
Thank you.
First you must read mongo backup and restore this documentation explain how to backup your data base and restore.
Now you should follow this steps :
1> From your old Ubuntu systems takes back up of your DB using following command
mongodump --host DB name --port 27017 --out /path to save your files
this command write data in BSON format, if you want to take only some collections from your DB then use mongodump --collection your collection name --db DB name
2> Now copy all above BSON files to your new PC and use following command to restore your old Ubuntu systems DB.
mongorestore --port <port number> <path to the backup>
before running this command you must install mongoDB.
I have a MongoDB database that resides on a remote server machine whose IP address is 192.168.1.20 on a local network. For development and testing purposes, and since I am not allowed to modify or delete the database on the server for security purposes, I want to copy the database on my local machine for my personal use.
Can anyone please tell me, how do I achieve this?
I do this by creating a dump of the remote db to my local machine, which I then restore:
Make sure you have a mongo instance up and running (eg. run mongod.exe from your bin folder in a terminal window. On my windows computer that's C:\mongodb\bin)
Make a dump from remote db: Open a new terminal window, move to the bin folder again, run:
mongodump -h example.host.com --port 21018 -d dbname --username username --password yourpass
(Change the parameters to suit your own situation.)
Restore the dumped database: Once the dump has been made, run the following command so that you have a local db:
mongorestore -d theNameYouWantForYourLocalDB dump\nameOfRemoteDB
(replace nameOfRemoteDB with the name of the remote db, the same as in previous command, and replace theNameYouWantForYourLocalDB with the name that you want your new local db to have)
There is copy database command which I guess should be good fit for your need.
db.copyDatabase("DATABASENAME", "DATABASENAME", "localhost:27018");
Alternatively, you can just stop MongoDb, copy the database files to another server and run an instance of MongoDb there.
EDIT 2020-04-25
Quote from MongoDB documentation
MongoDB 4.0 deprecates the copydb and the clone commands and their mongo shell helpers db.copyDatabase() and db.cloneDatabase().
As alternatives, users can use mongodump and mongorestore (with the mongorestore options --nsFrom and --nsTo) or write a script using the drivers.
Reference here
This should be a comment to the answer of #malla, but I don't have enough reputation to comment so I'm posting it here for other's reference.
In step 2, When you are trying to dump file from a remote server, remember to add out option so that you can restore locally later: (in my first try, I didn't add it and it failed, saying dump\db_name was not found).I'm not sure whether my way efficient or not. But it worked for me.
Step 2:
mongodump -h example.host.com --port 21018 -d dbname --username username --password yourpass --out <path_you_want_to_dump>
Step 3:
mongorestore -d theNameYouWantForYourLocalDB \<path_you_want_to_dump> + nameOfRemoteDB
The mongoexport command:
http://docs.mongodb.org/manual/core/import-export/
Or, mongodump command:
http://docs.mongodb.org/manual/reference/program/mongodump/
mongodb has commandline tools for importing and exporting. Take a look at mongodump --collection collection --db test and mongorestore --collection people --db accounts dump/accounts/
http://docs.mongodb.org/v2.2/reference/mongodump/
http://docs.mongodb.org/v2.2/reference/mongorestore/
this even works over the network
You can use the mongoexport command to copy the database to your local machine.
I have dumped a standalone mongo database (14Gb big) running on the default port as follows:
mongodump --username <username> --password <password> --db <dbname>
and I am trying to restore it onto a replica set with the following:
mongorestore --host myReplSet/127.0.0.1:27020 --drop --username <username> --password <password>
The restore seems to work (it takes a minute or so and the output looks good) the database is created and all the collections are created but there is no data in the collections, they are all empty. What am I doing wrong?
For what it is worth, the local database is showing that it is 10 Gb big (I am not sure how big it was before the restore)
If anybody is facing the same issue. I used a replica set, and tried to do the Op's command. i.e
mongorestore --host myReplSet/mongo0.example.com:27020,mongo1.example.com:27012 --db <dbname> <folder_location>
Instead just go to the primary server and do this
mongorestore --port <if running on port other than 27017> --db <dbname> <location of dump>
This works for me, A possible explanation could be that whenever I write into the primary database, it is automatically copied over to my secondary databases. Hence no need to give any replica set address. This might work if you write in a secondary, I am not sure, haven't tried it. Maybe somebody else can throw a light on it.
Make sure the mongorestore version is the same version as the mongodb.
mongorestore --version
mongo --version
Same thing goes for the mongodump.
mongodump --version
They need to be the same or close enough versions.
These are issues that you will run into on a distributed system.
It will create the collections, but they will be empty if versions don't match.
Perhaps I have a complete misunderstanding of how mongodump is supposed to work, but I can't seem to get it to do anything besides returning a JavaScript execution failed: SyntaxError: Unexpected identifier error.
Here's what I'm doing:
Mongod is running
I want to backup a database called "mydb"
I'm inside the mongo shell
I tried the command mongodump --db mydb and get the above error
I've tried both mongodump and mongoexport, both have the same issue
What am I doing wrong here?
Try the following it will work
i.Open the terminal
ii. Enter mongodump --collection collectionname --db dbname (Don't go inside mongo shell);
iii.If default port is different(other than 27017) then go for the following command
mongodump --host mongodb1.example.net --port 37017 --username user --password pass --out /opt/backup/mongodump-2011-10-24
mongodump,mongorestore is not commands of mongodb shell. It is separate mongodb utlity. You can find it under mongodb bin folder.
Usually you will need to add all mongodb utilities to the system Path variable and after this easy backup/restore databases from any place in the command line or in the terminal.
Your command looks mongodump --db mydb good if your databases in on default port(27017).
I faced the problem in taking mongo dump and I also wanted to store the dump to S3. Finally I ended up with a bash script to take mongo dump and store it to S3. I used mongodump to take backup.
mongodump -h $MONGO_HOST:$MONGO_PORT -d $MONGO_DATABASE
Where $MONGO_HOST,$MONGO_PORT and $MONGO_DATABASE are bash variables for host, port and database-name respectively.
You can also use --username user --password pass option for mongodump command if you have username and password setup on the database.
Here is the script to take mongodb dump and store it to S3 with a cron.