a quick question about how docker and mongo coexist.
When I deploy my app to docker hub, does it include db records?
When docker removes mongo records. When I stop container, or only when I remove it?
The answer is depends...
You could create a image with your records, but that would increase your image size, and if someone mount a volume to the path /data/db they would lose your database. So I do not recommend to upload a image with a loaded database, instead use a custom entrypoint script to init your database.
About when the records are destroyed, it will happen when you remove the container, but only if you did not mount a volume to the folder /data/db in the container, then the database will be persisted even if you remove the container.
You can see more info about how to use the image at: https://hub.docker.com/_/mongo/
Related
I have running MongoDB as Docker container. Now I want to create one more MongoDB container but read-only and read data from the existing one.
What should I do? I don't use Docker Swarm mode!
I want to have 2 MongoDB container run, the existing is keep running, the new one is read-only and read data from the existing container.
Thanks for reading!
I have an application which, for local development, has multiple Docker containers (organized under Docker Compose). One of those containers is a Postgres 10 instance, based on the official postgres:10 image. That instance has its data directory mounted as a Docker volume, which persists data across container runs. All fine so far.
As part of testing the creation and initialization of the postgres cluster, it is frequently the case that I need to remove the Docker volume that holds the data. (The official postgres image runs cluster init if-and-only-if the data directory is found to be empty at container start.) This is also fine.
However! I now have a situation where in order to test and use a third party Postgres extension, I need to load around 6GB of (entirely static) geocoding lookup data into a database on the cluster, from Postgres backup dump files. It's certainly possible to load the data from a local mount point at container start, and the resulting (very large) tables would persist across container restarts in the volume that holds the entire cluster.
Unfortunately, they won't survive the removal of the docker volume which, again, needs to happen with some frequency. I am looking for a way to speed up or avoid the rebuilding of the single database which holds the geocoding data.
Approaches I have been or currently am considering:
Using a separate Docker volume on the same container to create persistent storage for a separate Postgres tablespace that holds only the geocoder database. This appears to be unworkable because while I can definitely set it up, the official PG docs say that tablespaces and clusters are inextricably linked such that the loss of the rest of the cluster would render the additional tablespace unusable. I would love to be wrong about this, since it seems like the simplest solution.
Creating an entirely separate container running Postgres, which mounts a volume to hold a separate cluster containing only the geocoding data. Presumably I would then need to do something kludgy with foreign data wrappers (or some more arcane postgres admin trickery that I don't know of at this point) to make the data seamlessly accessible from the application code.
So, my question: Does anyone know of a way to persist a single database from a dockerized Postgres cluster, without resorting to a dump and reload strategy?
If you want to speed up then you could convert your database dump to a data directory (import your dump to a clean postgres container, stop it and create a tarball of the data directory, then upload it somewhere). Now when you need to create a new postgres container use use a init script to stop the database, download and unpack your tarball to the data directory and start the database again, this way you skip the whole db restore process.
Note: The data tarball has to match the postgres major version so the container has no problem to start from it.
If you want to speed up things even more then create a custom postgres image with the tarball and init script bundled so everytime it starts then it will wipe the empty cluster and copy your own.
You could even change the entrypoint to use your custom script and load the database data, then call docker-entrypoint.sh so there is no need to delete a possible empty cluster.
This will only work if you are OK with replacing the whole cluster everytime you want to run your tests, else you are stuck with importing the database dump.
I'm running a PostgreSQL db via docker postgres.
I have populated the db with lots of data and would like to share it with others.
Is there a way to 'save' this database with all the data as a new image and publish it to a Docker registry so it can be easily pulled and used?
You can use docker container commit https://docs.docker.com/engine/reference/commandline/commit/ to create an image from a container.
Then you can publish that image to a docker registry for use by others.
Hello everone I am new to AWS. I have created and Instance and installed MongoDB on It. But I learned from EC2 docs that the data stored on EC2 machine destroyed once they stop or terminated. So I have to use EBS for the data storage.
On the 4th step of creating EC2-Instance I selected the volume
But I don't know how to use it. Is it automatically used by EC2-Instance to store the data in it or we have to manually do it.
I know that for changing the database storage we have to change the file named mongod.conf and modify the dbpath there but I don't know what path to give there. and after that also if I destroyed that instance when I will get the EBS storage back.
thanks in advance for the help.
In your screenshot the EBS volume is the root volume for the instance. So unless you add other volumes to the instance, everything persisted to disk on that instance is going to be stored on the EBS volume. You can pick (or create) any location to store your database.
After you have mounted the EBS volume correctly as per instructions here, you can use the following steps to store your data to EBS volume.
Edit mongod.conf file.
sudo nano /etc/mongod.conf
Change the line dbpath=/var/lib/mongodb to the new path storage location, e.g. dbpath=/data/mongodb
Update the permissions of your chosen path to allow the MongoDB user to write to it, e.g. chown $USER -R /home/user/data/mongodb
In most cases you user is mongodb. So you can replace $USER with mongodb:mongodb
Restart the service
sudo service mongod stop then sudo service mongod start
Note, please be careful that if you have any data in the old location and you want to keep, you'll need to stop the MongoDB service first, manually move the files and then start the service again.
full disclosure: I am a complete n00b to mongodb and am just getting my feet wet with using mongo on AWS (but have 2 decades working in IT so not a total n00b :P)
I setup an EBS volume and installed mongo on a EC2 instance.
My problem is that I provisioned too small an EBS volume initially.
When I realized this I:
created a new larger EBS volume
mounted it on the server
stopped mongo ( $ sudo service mongod stop)
copied all my /data/db files into the new volume
updated conf files and fstab (dbpath, logpath, pidfilepath and mount point for new volume respectively)
restarted mongod
When I execute: $ sudo service mongod start
- everything runs fine.
- I can futz about in the admin and local databases.
However, when I run the mongos command: > show databases
- I only see the admin and local.
- the database I copied into the new volume (named encompass) is not listed.
I still have a working local copy of the database so my data is not lost, just not sure how best to move mongo data around other than:
A) start all over importing the data to the db on the AWS server (not what I would like since it is already loaded in my local db)
B) copy the local db to the new EBS volume again (also not preferred but better that importing all the data from scratch again!).
NOTE: originally I secure copied the data into the EBS volume with this command:
$ scp -r -i / / ec2-user#:/
then when I copied between volumes I used a vanilla cp command.
Did I miss something here?
The best I could find on SO and the web was this process (How to scale MongoDB?), but perhaps I missed a switch in a command or a nuance to the process that rendered my database files inert/useless?
Any idea how I can get mongo to see my other database files and collections?
Or did I make a irreversible error somewhere along the way?
Thanks for any help!!
Are you sure you conf file is being loaded? You can, for a test, load mongod.exe and specify the path directly to your db for a test, i.e.:
mongod --dbpath c:\mongo\data\db (unix syntax may vary a bit, this is windows)
run this from the command line and see what, if anything, mongo complains about.
A database has a very finicky algorithm that is easy to damage. Before copying from one database to another you should probably seed the database, a few dummy entries will tell you the database is working.