I am using mongodb Atlas (M0), which does not provide db backup feature. I am creating a nodejs script that will export all the collections automatically periodically.
So far I have done the following:
created a programmatic api from my login.
whitelisted my ip
installed mongocli
created a default profile using public and private api keys.
And command looks like this:
mongoexport --uri mongodb+srv://${mongoUser}:${mongoPass}#mongoprojectname.lok0l.mongodb.net/${database} --collection ${collection} --type ${fileType} --out ${fileName} --profile default
I am not able to figure out how to run this command. If I run this command in my terminal I get this error: /bin/sh: mongoexport: command not found\n
Related
I have a mongodb database in a dump folder which contain some bson and json files. I want to restore it using mongorestore, but I can't connect to the cluster through mongo shell. So I use mongo compass to connect but I have no clue how to restore it in the mongo compass.
You run mongorestore runs from the system command line, not mongo shell. Use the Command Prompt in Windows or Terminal in OS X / Linux to enter the command and you should be good to go. The connection string will be the same one you used to connect through Compass.
To restore you have to follow these steps carefully:-
1: First check dump type if it is a zip file then extract it.
2: open your cmd prompt and go to the folder where your database XYZ (folder) exists.
mi#sujeet-mishra:~/Downloads/dump$
Inside the dump, there is a folder called XYZ (folder of XYZ database)
Now run the command: mongorestore -db XYZ
mi#sujeet-mishra:~/Downloads/dump$ mongorestore -db XYZ
that's it, now refresh your MongoDB and you will get a DB named XYZ.
To create a database dump and then restore it, you need to have (have been installed) tools for working with the database on your system - a set of utilities (executable files) of the command line for working with MongoDB,
such as: mongodump, mongorestore, bsondump, mongoimport, mongoexport and other...
The set of utilities supports all popular platforms. For example, system support by the mongorestore utility can be viewed here.
To install a set of utilities into your system, you need to use the installation instructions for your operating system: Linux, MacOS, Windows.
Since problems can usually arise when using utilities on Windows, let's look at an example of installing and using utilities specifically for this system. We will be using Windows 10.
So...
Open MongoDB Download Center.
Select the desired operating system and package type on the page in
the form. Downloading the archive!
Unpack the contents of the downloaded archive, for example, to the
C:\utilities folder.
Further, in order for the system to see our utilities and avoid an
error the utility
command is not an internal or external command,
executable program or batch file
, you must add a new entry to environment variables (read more about it).
* once environment variables are set, MongoDB utility commands can be executed from any directory of any console (simple console, PowerShell, IDE code editor console)
* You can`t use environment variables, but in this case, keep in mind that running utility commands will only be possible from the directory in which you placed the executable file. In other words, whenever you need to execute a console command of one of the utilities, you will need to change to the directory where the executable file is located!
Now it remains only to run the command to restore the database on your server, it will look like this - in the latest version of MongoDB, everything is as simple as impossible:
mongorestore C:\backups\db
// or, for example, specifying the host and port like this
mongorestore --host localhost --port 27017 C:\backups\db
// or like this
mongorestore --host localhost:27018 C:\backups\db
Most Important! Since the --db option for mongorestore is deprecated , please note that we do not specify the name of the database to restore the database to!
The name of the database where the data will be imported is the name of the directory in the db folder, in which, in turn, you should have dump files in .bson and .json formats.
If you specify a specific directory in which you have dumps, for example: C:\backups\db\my_app, then running the command will result in an error -
... don't know what to do with file....
Also keep in mind that running the mongorestore command from the C:\backups\db directory will restore the dumps of all applications located in this directory (if any)!
Before 4 version of MongoDB, the mongorestore command could look like this (deprecated):
mongorestore --host localhost --port 27017 --db name_your_db C:\backups\db\my_app // a
mongorestore --host localhost --port 27017 -d name_your_db C:\backups\db\my_app // b
// or short records:
mongorestore -h localhost --port 27017 -d name_your_db C:\backups\db\my_app // a
mongorestore -h localhost:27017 -d name_your_db C:\backups\db\my_app // b
mongorestore --uri="mongodb://localhost:27017/name_your_db" C:\backups\db\my_app // c
In fact, there may be more options for using the command, with various options, using authorization on the server. In this case, only a few examples are given.
You can see the full list of options on the official website.
To create a dump, use the command:
mongodump --out=C:\backups\db // <- to make dump all databases
mongodump -h localhost:27017 -d my_app --out=C:\backups\db // <- to create a dump of a specific single database
You can see the full list of options of mongodump on the official website.
* for Linux, commands are executed in a similar way
I'm using documentdb with the password being automatically rotated via aws secret manager. I want to script up a fast way to connect to the database via command line. Since the password changes frequently that means having a command that will load the password from aws secerts and pass it to the mongo connect string. I had come up with this hideous one liner to connect to mongo:
mongo --ssl --host *my_host*:27017 --sslCAFile rds-combined-ca-bundle.pem --username admin --password `aws secretsmanager get-secret-value --secret-id documentDB_login | jq .SecretString | jq fromjson | jq .password`
I had already run aws configure and secretmanager does return my password correctly.
I swear this worked at first, but it is now failing, saying authentication failed when it tries to connect to my mongo instance. If I echo the above ugly one liner, so that i can see the result of the aws secretmanager call, and then copy and paste the echo response into my commandline it connects correctly, so not sure why the functionally equivalent command does not.
How can I script up an easy way to connect to to documentdb via commandline? Is there a cleaner approach (preferably one that doesn't require yum install of jp) to do this?
I can not say for sure why your command was working before but does not now, but it looks like it is not stripping double quotes. Also the AWS cli can do JSON parsing for you, but unfortunately, it can not parse the nested JSON in the secret itself. For that you will still need jq. However, combining the CLI JSON parsing with jq you could simplify it a bit (though it is no shorter):
mongo --ssl --host *my_host*:27017 --sslCAFile rds-combined-ca-bundle.pem --username admin --password `aws secretsmanager get-secret-value --secret-id documentDB_login --query 'SecretString' --output text | jq -r .password`
I am a newbie to mongodb. I am trying to take a backup of my database using mongodump. But whenever I use this command I get the below error
Referenceerror: mongodump is not defined
I tried creating a new user with all the roles but still I get the same error. Should I add a specific role to take a backup? Or am I doing something wrong?
'mongodump' is a command/tool which is included in the 'mongodb-tools' package. If you don't have this package installed on your machine, it makes sense that it is not defined. The mongodb-tools also provide several other tools used for importing and exporting DBs (like mongorestore).
That being said, 'mongodump' is not a mongo-shell command, you shouldn't be using it in mongo-shell. It's a whole different command that you would be executing just like you execute 'mongod' or 'mongo' etc.
One simple way is to right-click mongodump.exe and "Run as administrator". It will create dump folder in bin of your mongodb home containing all database backups.
If you want to go with commands, open cmd as administrator and go to bin of your mongodb where you'll be able to fire commands such as mongorestore, mongodump etc. with intended parameters e.g for specific db or interact with remote mongodb.
tested on (v4.2.3)
Note: many folks try to execute these commands in mongod where you can execute queries which is wrong, these tools needs to be used separately.
Here ar e 2 simple exemples for a full backup using authentication and without
mongodump -h hostname -v -u sys_account -p ys_password --authenticationDatabase admin --out folder_location_for_backup
if no authentication
mongodump -h hostname -v --out folder_location_for_backup
here are the mongorestore commands as well
mongorestore -h hostname -v -u admin_user -p admin_password --authenticationDatabase admin --dir folder_location_where_backup_is_located
if no authentication
mongorestore -h hostname -v --dir folder_location_where_backup_is_located
for windows, you need to start executable file by running following command
"C:\Program Files\MongoDB\Server\3.4\bin\mongodump.exe" --db your_database_name
Above command will export your database to dump folder. This folder will be located where you have kept your "data" folder. If you use default "data/db" folder then it will be there but if you use different location then it will be kept over there.
This command must be run in normal command prompt, not in mongo shell. It is executable not mongo shell command. Official docs link is here.
Download MongoDB Command Line Database Tools .The mongodump tool is part of the MongoDB Database Tools package.
There are great chances that mongodump is being performed under Mongo Shell
mongodump should be run directly on command line or terminal (NOT inside mongo shell)
For creating dump of database
mongo --db database_name
For creating dump of any collection of database
mongo --db databas_name --collection collection_name
I am starting my first meteor app, using MongoDB for my database.
I have copied my data in from a JSON file into mongodb collection and when hosting locally, the data appears as expected; the collection is in the meteor mongo database and I can interpret at will
When I deploy the app to xx.meteor.com, the meteor collection I need (named 'assets') does not get posted to the mongodb on the server. I can check this by using meteor mongo onemore.meteor.com. Other collections are posted though.
How do I deploy my mongoDB colletion ('assets') along with the app using meteor deploy?
Is this issue due to the way the mongodatabase was orginally imported? I used this method: https://github.com/awatson1978/meteor-cookbook/blob/master/cookbook/database-management.md
I copied the collection ('assets) from a staging database using db.copyDatabase('staging3','meteor','localhost')
Using mongodump and mongorestore also works:
Dump data from existing mongodb (mongodb url: mongodb://USER:PASSWORD#DBHOST/DBNAME)
mongodump -h DBHOST -d DBNAME -u USER -p PASSWORD
This will create a "dump" directory, with all the data going to dump/DBNAME.
Get the mongodb url for the deployed meteor app (i.e. www.mymeteorapp.com)
meteor mongo --url METEOR_APP_URL
This will return the following:
mongodb://#USERNAME#:#PASSWORD###HOSTNAMEANDPORT#/#YOURAPPLICATION#
Note: the PASSWORD expires every min.
Upload the db dump data to the meteor app (using an example meteor db url)
mongorestore -u #USERNAME# -p #PASSWORD# -h #HOSTNAMEANDPORT# -d www_mymeteorapp_com dump/DBNAME/
All the data should get transferred!
This answer is basically a modified version of Davidd8's answer here, but because that was never accepted I reposted it here.
None of the above worked for me. Here is what did the trick for me:
1) How do I get my localhost dump of my meteor db?
tip 1: make sure mongo is running (type mongod to run it)
Which port was my meteor running on?
$ mongodump --host localhost:3002 (nope)
$ mongodump --host localhost:3000 (nope)
$ mongodump --host localhost:3001 (yes!) this worked
Now where the heck did it put my dump?
$ ls (shows me the dump is in my current directory and meteor is inside the dump directory)
I need to somehow get that meteor data onto my live site on meteor.com
Now the next problem is that meteor.com will give me the information I need but I only have 1 minute before their information expires.
This information is what I need to connect to my remote meteor site and import my local meteor data.
I type this in the terminal:
$ meteor mongo --url iamcool.meteor.com
Then it spits back something like this:
mongodb://client-11f3014w:f1c52f68-8506-b682-c880-b5db03a9510e#production-db-a2.meteor.io:27017/iamcool_meteor_com
I used info from above that told me what data to extract from the above code and where to put it in the mongorestore code
mongorestore -u #USERNAME# -p #PASSWORD# -h #HOSTNAMEANDPORT# -db www_mymeteorapp_com dump/DBNAME/
This is where you have to be a super fast typer. I recommend opening a blank file in your IDE (Sublime Text or Atom or whatever you use) and create this template:
mongorestore -u XXX -p XXX -h production-db-a2.meteor.io:27017 -db iamcool_meteor_com dump/meteor
So I took the data meteor.com gave me and I plugged it in below:
mongorestore -u client-4bf9be36 -p d1cdef86-6c71-3b11-ef4c-221fbdcf07d0 -h production-db-a2.meteor.io:27017 -db iamcool_meteor_com dump/meteor
And then the errors came. And boy did they come.
Sometimes I got this error:
too many positional arguments
Sometimes I got this error:
Failed: error connecting to db server: auth fails
But this error tip gave me the info that saved the day:
try 'mongorestore --help' for more information
I did and I found out that the flag to pass for database wasn't -db but rather -d
I made the following change and typed it fast using my trusty template (see above)
mongorestore -u client-4bf9be36 -p d1cdef86-6c71-3b11-ef4c-221fbdcf07d0 -h production-db-a2.meteor.io:27017 -d iamcool_meteor_com dump/meteor
And then it worked.
It took me two hours to figure out. Hope it saves you the time I lost.
This post is about backing up your meteor database
I am trying to back up my meteor database and I understand what this post is telling me to do, but I must not be in the right directory when I run the mongodump command, b/c I keep getting 'Command not found'. Or do I need to export a path?
[EDIT]
OK, now I have the binaries installed but when I run 'mongodump', I get:
couldn't connect to [127.0.0.1] couldn't connect to server 127.0.0.1:27017
... and when I run 'mongodump --host localhost:3002', I get:
couldn't connect to [localhost:3002] couldn't connect to server localhost:3002
Now what?
OK, thanks to #David Weldon, I can provide a fairly complete answer to this issue:
Backing up and restoring your local MongoDB for Meteor users (OSX)
Backup:
1) Your app must be running, so start up your Meteor server.
2) In a terminal window (NOT in the meteor mongo shell), enter: mongodump -h 127.0.0.1 --port 3001 -d meteor
This will create a 'dump' directory inside your home folder (your name under Users).
3) If you get a 'command not found' message, you probably just installed Mongo as a part of Meteor, meaning you don't have the mongo command line tools. Use a package like Homebrew to reinstall Mongo and you will have the command line tools. This will also add the correct PATH information to your system, so that it can find the tools.
Restoring:
1) From MiniMongo shell (run ‘meteor mongo’ inside your Meteor project dir), enter:
db.[collectionName].drop(); //repeat for all collections you wish to restore
2) Then, from a terminal window, enter:
mongorestore -h 127.0.0.1 --port 3001 -d meteor dump/meteor
Caveats:
The individual documents will not necessarily be in the same order after they are restored. So you need some way to sort documents that need to be presented in a certain order.
Caveats:
The individual documents will not necessarily be in the same order after they are restored. So you need some way to sort documents that need to be presented in a certain order.
There is a flag for this
mongorestore --maintainInsertionOrder
If you are restoring to a mup'ed meteor app, the command would be:
mongorestore -h 127.0.0.1 --port 3001 -d dump/meteor
If you forgot the name of your app, you can see it by going into the mongo shell and listing all databases.