I'm running an ubuntu 16.04 LTS server with some docker container. One of these containers is a mongoDB container, where my data is stored.
Now I'm trying to make a backup by mongodump.
The problem for me is, that mongoDb is running as a docker container, and the backup should be stored outside of the docker container.
I think the syntax for this is something like this:
docker run \
--rm \
-it \
--link DOCKER_CONTAINER_NAME:mongo_alias \
-v /backup:/backup \
mongo mongodump \
--host mongo_alias \
--out /backup/
But I'm not sure for the parameters I have to use...
This is what I get for my mongoDb container via docker ps:
7bee41bfa08a mongo:3.4 "docker-entrypoint..." 4 months ago Up 2 months 27017/tcp mongo_db
And this is my docker-compose file:
version: "3"
services:
mongo_db:
container_name: mongo_db
image: 'mongo:3.4'
restart: 'always'
volumes:
- '/opt/mongo/project/live:/data/db'
So it should look like this?
docker run \
--rm \
-it \
--link mongo_db:mongo_alias \ # mongo_alias can be choosen freely?
-v /backup:/backup \ # Don't understand /backup:/backup
mongo mongodump \
--host mongo_alias \
--out /backup/ # This is in the root of the server?
Define the backup to run via compose as well. This will create the new container on the same network as the main mongo container. If you have any compose network definitions you will need to duplicate them in each compose file.
Create a second compose file for the backup command: docker-compose-backup.yml
version: "3"
services:
mongo_db_backup:
image: 'mongo:3.4'
volumes:
- '/opt/mongo/project/live_backup:/backup'
command: |
mongodump --host mongo_db --out /backup/
Then run the backup
docker-compose -f docker-compose-backup.yml run mongo_db_backup
You can also do this without docker-composer, directly from your host
Backup all databases
docker exec -t your-db-container mongodump --host mongo_db --out /backup/
Restore all databases
Move your backup folder to your host volume folder
docker exec -t your-db-container mongorestore /backup/
Related
i use
docker run --name db -d mongo:4.0 --smallfiles --replSet rs0 --oplogSize 128
docker exec -ti db mongo --eval "printjson(rs.initiate())"
Then start Rocket.Chat linked to this mongo instance:
docker run --name rocketchat -p 80:3000 --link db --env ROOT_URL=http://localhost --env MONGO_OPLOG_URL=mongodb://db:27017/local -d rocket.chat
now mongo 4.0 is depricated.
how can i upgrade mongo in docker?
You can do it this way:
Create mongo dump
docker exec -i db /usr/bin/mongodump --username <username> --password <password> --authenticationDatabase admin --db <database_name> --out dump
Run new mongo container
docker run -d --name dbnew -i mongo:4.4 --smallfiles --replSet rs0 --oplogSize 128
Restore dump to new container
docker cp dbnew:/dump dump
docker exec -i dbnew /usr/bin/mongorestore --username <username> --password <password> --authenticationDatabase admin --db <database_name> /dump/<database_name>
Check new db works fine and remove old container
Rename new container
docker rename dbnew db
If you want your new container to be persistent (which I think you would) you need to use docker volumes.
I have a dump.sql file that I would like to load with docker-compose.
docker-compose.yml:
services:
postgres:
environment:
POSTGRES_DB: my_db_name
POSTGRES_USER: my_name
POSTGRES_PASSWORD: my_password
build:
context: .
dockerfile: ./devops/db/Dockerfile.db
My Dockerfile.db is really simple at the moment:
FROM postgres
MAINTAINER me <me#me.me>
COPY ./devops/db ./devops/db
WORKDIR ./devops/db
I would like to run a command like psql my_db_name < dump.sql at some point. If I run a script like this from the Dockerfile.db, the issue is that the script is run after build but before docker-compose up, and the database is not running yet.
Any idea how to do this ?
Reading https://hub.docker.com/_/postgres/, the section 'Extend this image' explains that any .sql in /docker-entrypoint-initdb.d will be executed after build.
I just needed to change my Dockerfile.db to:
FROM postgres
ADD ./devops/db/dummy_dump.sql /docker-entrypoint-initdb.d
And it works!
Another option that doesn't require a Dockerfile would be to mount your sql file into the docker-entrypoint-initdb.d folder using the volumes attribute of docker-compose.
The official postgres image https://hub.docker.com/_/postgres/ will import and execute all SQL files placed in that folder. So something like
services:
postgres:
environment:
POSTGRES_DB: my_db_name
POSTGRES_USER: my_name
POSTGRES_PASSWORD: my_password
volumes:
- ./devops/db/dummy_dump.sql:/docker-entrypoint-initdb.d/dummy_dump.sql
This will automatically populate the specified POSTGRES_DB for you.
sudo docker exec postgres psql -U postgres my_db_name < dump.sql
You can use pg_restore inside the container:
cat ${BACKUP_SQL_File} | docker exec -i ${CONTAINER_NAME} pg_restore \
--verbose \
--clean \
--no-acl \
--no-owner \
-U ${USER} \
-d ${DATABASE}
After the docker-compose up, do docker ps it will give you a list of active docker containers. From that, you can get the container ID.
Then,
docker exec -i ${CONTAINER_ID} psql -U ${USER} < ${SQL_FILE}
CONTAINER_NAME="postgres"
DB_USER=postgres
LOCAL_DUMP_PATH="..."
docker run --name "${CONTAINER_NAME}" postgres
docker exec -i "${CONTAINER_NAME}" psql -U "${DB_USER}" < "${LOCAL_DUMP_PATH}"
In order to restore from I dump I use an sh to restore the database.
If you use a dump with docker-entrypoint-initdb.d it gives the error "The input is a PostgreSQL custom-format dump. Use the pg_restore command-line client to restore this dump to a database."
docker-compose.yaml
version: "3.9"
services:
db:
container_name: postgis_my_db_name
image: postgis/postgis:14-3.3
ports:
- "5430:5432"
# restart: always
volumes:
- ./my_db_name.sql:/my_db_name.sql
- ./restore.sh:/docker-entrypoint-initdb.d/restore.sh
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: my_password
POSTGRES_DB: my_db_name
restore.sh
pg_restore -d my_db_name my_db_name.sql
You can also do it without a dockerfile :
# start first container
docker compose start $db_container
# dump intial database
docker compose exec db pg_dump -U $user -Fc $database > $dump_file
# start container db
docker compose start $db_container
# get container id
docker ps
# copy to container
docker cp $dump_file $container_id:/var
# delete database container / Can't use
docker compose exec $db_container dropdb -U $user $database
# user pg_restore
docker compose exec $db_container pg_restore -U $user -C -d postgres /var/$dump_file
I use this docker image : https://hub.docker.com/_/rocket.chat/
So here is the code i used :
docker run --name db -d mongo:3.0 --smallfiles
docker run --name rocketchat --link db -d rocket.chat
I tried several things, but I can't find a way to have a clean backup/restore system.
Any advice ?
For the posterity : Backing up Rocket.chat on SERVER 1 and Restore it on SERVER 2, based on the official docker image :
SERVER 1
cd /backups
docker run -it --rm --link db -v /backups:/backups mongo:3.0 mongodump -h db -o /backups/mongoBACKUP
tar czf mongoBACKUP.tar.gz mongoBACKUP/
Then send mongoBACKUP.tar.gz on SERVER 2 in /backups.
SERVER 2 (+ test on :3000)
docker run --name db -d mongo:3.0 --smallfiles
cd /backups
tar xzf mongoBACKUP.tar.gz
docker run -it --rm --name mongorestore -v /backups/mongoBACKUP:/var/dump --link db:db mongo mongorestore --host db /var/dump
docker run -p 3000:3000 --name rocket --env ROOT_URL=http://yourwebsite.test --expose 3000 --link db -d rocket.chat
i'm running my app using docker-compose with the below yml file
postgres:
container_name: postgres
image: postgres:${POSTGRES_VERSION}
volumes:
- postgresdata:/var/lib/postgresql/data
expose:
- "5432"
environment:
- POSTGRES_DB=42EXP
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
node:
container_name: node
links:
- postgres:postgres
depends_on:
- postgres
volumes:
postgresdata:
As you can see here ,i'm using a named volume to manage postgres state.
According to the official docs, i can backup a volume like the below
docker run --rm --volumes postgresdata:/var/lib/postgresql/data -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /dbdata
Some other tutorials suggested i use the pg-dump function provided by postgres for backups.
pg_dump -Fc database_name_here > database.bak
I guess i would have to go inside the postgres container to perform this function and mount the backup directory to the host.
Is one approach better/preferable than the other?
To run pg_dump you can use docker exec command:
To backup:
docker exec -u <your_postgres_user> <postgres_container_name> pg_dump -Fc <database_name_here> > db.dump
To drop db (Don't do it on production, for test purpose only!!!):
docker exec -u <your_postgres_user> <postgres_container_name> psql -c 'DROP DATABASE <your_db_name>'
To restore:
docker exec -i -u <your_postgres_user> <postgres_container_name> pg_restore -C -d postgres < db.dump
Also you can use docker-compose analog of exec. In that case you can use short services name (postgres) instead of full container name (composeproject_postgres).
docker exec
docker-compose exec
pg_restore
Since you have
expose:
- "5432"
you can run
pg_dump -U <user> -h localhost -Fc <db_name> > 1.dump
pg_dump connects to 5432 port to make dump since it is listened by postgres in container you will dump db from container
You can also run
docker-compose exec -T postgres sh -c 'pg_dump -cU $POSTGRES_USER $POSTGRES_DB' | gzip > netbox.sql.gz
where netbox.sql.gz is the name of the backup file.
restoring would be
gunzip -c netbox.sql.gz | docker-compose exec -T postgres sh -c 'psql -U $POSTGRES_USER $POSTGRES_DB'
I have a dump.sql file that I would like to load with docker-compose.
docker-compose.yml:
services:
postgres:
environment:
POSTGRES_DB: my_db_name
POSTGRES_USER: my_name
POSTGRES_PASSWORD: my_password
build:
context: .
dockerfile: ./devops/db/Dockerfile.db
My Dockerfile.db is really simple at the moment:
FROM postgres
MAINTAINER me <me#me.me>
COPY ./devops/db ./devops/db
WORKDIR ./devops/db
I would like to run a command like psql my_db_name < dump.sql at some point. If I run a script like this from the Dockerfile.db, the issue is that the script is run after build but before docker-compose up, and the database is not running yet.
Any idea how to do this ?
Reading https://hub.docker.com/_/postgres/, the section 'Extend this image' explains that any .sql in /docker-entrypoint-initdb.d will be executed after build.
I just needed to change my Dockerfile.db to:
FROM postgres
ADD ./devops/db/dummy_dump.sql /docker-entrypoint-initdb.d
And it works!
Another option that doesn't require a Dockerfile would be to mount your sql file into the docker-entrypoint-initdb.d folder using the volumes attribute of docker-compose.
The official postgres image https://hub.docker.com/_/postgres/ will import and execute all SQL files placed in that folder. So something like
services:
postgres:
environment:
POSTGRES_DB: my_db_name
POSTGRES_USER: my_name
POSTGRES_PASSWORD: my_password
volumes:
- ./devops/db/dummy_dump.sql:/docker-entrypoint-initdb.d/dummy_dump.sql
This will automatically populate the specified POSTGRES_DB for you.
sudo docker exec postgres psql -U postgres my_db_name < dump.sql
You can use pg_restore inside the container:
cat ${BACKUP_SQL_File} | docker exec -i ${CONTAINER_NAME} pg_restore \
--verbose \
--clean \
--no-acl \
--no-owner \
-U ${USER} \
-d ${DATABASE}
After the docker-compose up, do docker ps it will give you a list of active docker containers. From that, you can get the container ID.
Then,
docker exec -i ${CONTAINER_ID} psql -U ${USER} < ${SQL_FILE}
CONTAINER_NAME="postgres"
DB_USER=postgres
LOCAL_DUMP_PATH="..."
docker run --name "${CONTAINER_NAME}" postgres
docker exec -i "${CONTAINER_NAME}" psql -U "${DB_USER}" < "${LOCAL_DUMP_PATH}"
In order to restore from I dump I use an sh to restore the database.
If you use a dump with docker-entrypoint-initdb.d it gives the error "The input is a PostgreSQL custom-format dump. Use the pg_restore command-line client to restore this dump to a database."
docker-compose.yaml
version: "3.9"
services:
db:
container_name: postgis_my_db_name
image: postgis/postgis:14-3.3
ports:
- "5430:5432"
# restart: always
volumes:
- ./my_db_name.sql:/my_db_name.sql
- ./restore.sh:/docker-entrypoint-initdb.d/restore.sh
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: my_password
POSTGRES_DB: my_db_name
restore.sh
pg_restore -d my_db_name my_db_name.sql
You can also do it without a dockerfile :
# start first container
docker compose start $db_container
# dump intial database
docker compose exec db pg_dump -U $user -Fc $database > $dump_file
# start container db
docker compose start $db_container
# get container id
docker ps
# copy to container
docker cp $dump_file $container_id:/var
# delete database container / Can't use
docker compose exec $db_container dropdb -U $user $database
# user pg_restore
docker compose exec $db_container pg_restore -U $user -C -d postgres /var/$dump_file