I am using UBUNTU, from /etc/mongod.conf, I found that /var/lib/mongdb is the path for data.
I have found some files like collectionname.0, .1, .ns in that directory. but when I run a very big loop(lakhs), I am able to get them back using mongo shell, but that mongodb directory size is not increasing, so there must be someother place where this data is being stored
What is that place?
There is no another place. As indicated by #itsbruce, in Ubuntu it's /var/lib/mongodb.
On a non-packaged installation (on Linux), i.e. without a /etc/mongodb.conf file, the default is /data/db/ (unless otherwise specified).
You can modify the switch "--dbpath" on the command line to designate a different directory for the mongod instance to store its data. Typical locations include: /srv/mongodb, /var/lib/mongodb or /opt/mongodb.
(Windows systems use the \data\db directory.) If you installed using a package management system.
I recommend using the db.collection.stats command as outlined here to monitor the size of your collection as you insert data. The link also explains what each field (in the output) means.
That is the correct data location for MongoDB on Ubuntu. MongoDB pre-allocates filespace. Are you sure you have generated more data than would fit into the initial pre-allocated files? Try blowing away any existing data files and restarting Mongo with the --noprealloc flag. Then add data.
Related
I am running Mongo on Windows 10. Data access is made trugh Pymongo (Python). All my data is stored in an old large HDD, but I would like to have some smaller collection stored on a much faster SSD? When I create a new collection either trough Compass or Pymongo there is no option to specify the path on disk.
Is it possible to learn such power?
If you want to have databases in different disks under the same dbPath , this is the option:
Add the option --directoryperdb in the mongod config file or at startup.
db.createDatabase("newDatabase")
You will see inside the dbPath folder for the new database like:
\dbPath\newDatabase
Stop the mongodb process.
Copy the content from \dbPath\neWDatabase to your SSD let say:
ssd:\newData
make link to the folder with:
mklink /d \newData \dbPath\newDatabase
or follow this tutotial
Start the mongodb process again.
Note:
As suggested by #Wermfried in the comment it is safe to have the option --directoryperdb set initially in the member before to get populated ...
I've been using a MongoDB instance in my docker compose script. I want to set it up so I can keep my database from PC to PC but have all the same data.
There seems to be quite a bit of files in a MongoDB docker installation, .lock, .turtle, .wt, .bson diagnostic.data, journal etc.
Is there a rule of thumb of what I should store and would I should ignore in my repo? It's been unclear to me, I don't want to store anyfiles that could effect booting on another docker container.
Best is to preserve everything under the mongod dbPath folder , but some files/folders can be removed afcourse like diagnostic.data - the folder contain collected metrics during operation necessary for performance analysis at later stage , but in general the already collected stats are not necessary for the mongod process to be executed.
This is more of a general question about how MongoDB works.
But I've been using MongoDB for a while and everything seems to be working for me. The part that currently confuses me, though, is that when I visit the directory where the MongoDB data is saved (I'm using the default data/db) the directory is empty. The data is being persisted, I'm just confused - why does the directory appears empty on my computer?
I'm on Windows, if that's worth anything.
You can execute :
db.serverCmdLineOpts().parsed.storage.dbPath
from inside the mongo process and find what the dbPath startup parameter show to find where your data is saved
or alternatively check from the config file in windows:
<install directory>/bin/mongod.cfg
dbPath can be confusing.
When you run mongod without --dbpath option or unset it in config file then it defaults to \data\db
However, when you use the default config files which comes along the installation then the dpPath is different (I don't remember by heart which). So you should really check path with db.serverCmdLineOpts() as suggested by R2D2
I know, for Linux the default dbPath is /data/db but the pre-installed config file /etc/mongod.conf has set /var/lib/mongo
How can I restore an old MongoDB backup that I believe was made by copying the raw db files? (They did not use a dump command)
It was delivered compressed in a .7z format, which decompressed to the following files:
mydb_2014.1
mydb_2014.2
mydb_2014.3
mydb_2014.4
mydb_2014.5
mydb_2014.ns
I've tried mongorestore but it generates errors saying "don't know what to do with file ..., skipping"
It should be possible to just start up a mongod instance while specifying the path to the files in question:
mongodb --dbpath /path/to/files
You should try to use a version of MongoDB that matches to the version that was in use when the backup was taken. It looks like these files are from the MMAPv1 storage engine, so you may also need to specify --storageEngine mmapv1 if you're using MongoDB 3.2 or later.
I have 2 computers in different places (so it's impossible to use the same wifi network).
One contains about 50GBs of data (MongoDB files) that I want to move to the second one which has much more computation power for analysis. But how can I make MongoDB on the second machine recognize that folder?
When you start mongodprocess you provide an argument to it --dbpath /directory which is how it knows where the data folder is.
All you need to do is:
stop the mongod process on the old computer. wait till it exits.
copy the entire /data/db directory to the new computer
start mongod process on the new computer giving it --dbpath /newdirectory argument.
The mongod on the new machine will use the folder you indicate with --dbpath. There is no need to "recognize" as there is nothing machine specific in that folder, it's just data.
I did this myself recently, and I wanted to provide some extra considerations to be aware of, in case readers (like me) run into issues.
The following information is specific to *nix systems, but it may be applicable with very heavy modification to Windows.
If the source data is in a mongo server that you can still run (preferred)
Look into and make use of mongodump and mongorestore. That is probably safer, and it's the official way to migrate your database.
If you never made a dump and can't anymore
Yes, the data directory can be directly copied; however, you also need to make sure that the mongodb user has complete access to the directory after you copy it.
My steps are as follows. On the machine you want to transfer an old database to:
Edit /etc/mongod.conf and change the dbPath field to the desired location.
Use the following script as a reference, or tailor it and run it on your system, at your own risk.
I do not guarantee this works on every system --> please verify it manually.
I also cannot guarantee it works perfectly in every case.
WARNING: will delete everything in the target data directory you specify.
I can say, however, that it worked on my system, and that it passes shellcheck.
The important part is simply copying over the old database directory, and giving mongodb access to it through chown.
#!/bin/bash
TARGET_DATA_DIRECTORY=/path/to/target/data/directory # modify this
SOURCE_DATA_DIRECTORY=/path/to/old/data/directory # modify this too
echo shutting down mongod...
sudo systemctl stop mongod
if test "$TARGET_DATA_DIRECTORY"; then
echo removing existing data directory...
sudo rm -rf "$TARGET_DATA_DIRECTORY"
fi
echo copying backed up data directory...
sudo cp -r "$SOURCE_DATA_DIRECTORY" "$TARGET_DATA_DIRECTORY"
sudo chown -R mongodb "$TARGET_DATA_DIRECTORY"
echo starting mongod back up...
sudo systemctl start mongod
sudo systemctl status mongod # for verification
quite easy for windows, just move the data folder to the target location
run cmd
"C:\your\mongodb\bin-path\mongod.exe" --dbpath="c:\what\ever\path\data\db"
In case of Windows in case you need just to configure new path for data, all you need to create new folder, for example D:\dev\mongoDb-data, open C:\Program Files\MongoDB\Server\6.0\bin\mongod.cfg and change there path :
Then, restart your PC. Check folder - it should contains new files/folders with data.
Maybe what you didn't do was export or dump the database.
Databases aren't portable therefore must be exported or created as a dumpfile.
Here is another question where the answer is further explained