I am using docker-compose to spin up a spring api with a postgres database . I am new to docker and I am trying to persist my database using a named volume I created with
docker volume create employeedata
I add this volume inside my docker-compose.yml but the db does not persist if I stop or remove my containers .
version: '3.8'
services:
app:
container_name: springboot-postgresql
image: springboot-postgresql
build: ./
ports:
- "8080:8080"
depends_on:
- postgresqldb
postgresqldb:
image: postgres
ports:
- "5432:5432"
volumes:
- employeedata:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=postgres
- POSTGRES_USER=postgres
- POSTGRES_DB=employeedb
volumes:
employeedata:
I tried doing docker inspect employeedata and got the result below
It seems fishy to me that the docker-compose version is 2 and not 3 plus I don't understand how the mountpoint is related to the volume path I specify in my docker-compose.yml above
I would appreciate your help
docker compose created all objects in a project namespace. This namespace is usually the folder name of the docker-compose.yml file but you can set it by passing --project-name to (all) your calls to docker compose.
As docker does not have first class namespaces, the project name is simply used as a prefix for all objects defined in the compose file, so in this case, assuming your project was in a folder called "project" then compose would have created project_app as the container and project_employeedata as the volume.
To override this, you specify an explict container-name as you have done. But you really shouldn't as it means that any two deployments of compose files with this name will now conflict.
And to override it for volumes - tell docker compose that the volume is externally created and provide the external name. Otherwise compose will try to use the namespaced name.
volumes:
employeedata:
external: true
name: employeedata
Again, letting compose manage the volume name is probably the better option. Simply ensure the directory hosting the compose file has a unique name that is suitable - or ensure a suitable unique name is passed via --project-name, and then manage the volume using whatever_employeedata.
nb. Docker compose does not remove compose managed volumes unless -v / --volumes is passed to docker compose down so your data will persist here.
/var/lib/docker/volumes is simply the (default) location that docker will manage volumes.
The 2.0 refers to the version of compose, not the version in your compose.yml file.
Related
Is it possible to "combine" naming volumes and defining the path between containers? (docker compose)
I got a docker compose file with multiple services which need to share volumes. At the same time I want to specify the path of these volumes for backup purposes and to control on which drive the data resides.
Currently I am using an .env file combined with the docker-compose file to solve this as follows.
File .env:
VOLUME_SHARED=/home/docker_user/data_service
File docker-compose.yml
version: '3.8'
services:
service1:
image: service1
volumes:
- ${VOLUME_SHARED}:/data
service2:
image: service2
volumes:
- ${VOLUME_SHARED}:/data
I know there are named volumes, but I couldnt find a way to define their path. Is there a way to locate all the information within the docker-compose file?
So to make it clear, is something like the following possible?
File docker-compose.yml
version: '3.8'
services:
service1:
image: service1
volumes:
- shared_data:/data
service2:
image: service2
volumes:
- shared_data:/data
volumes:
shared_data=/home/docker_user/data_service
There is no such feature neither in compose file reference nor in docker volume create command manual. Thus you can't define storage location when you create a volume.
You can either change location for all docker files (see this post) or use bind mounts (as you do now).
There is also a hack which can do what you want but I don't recommend it. On Linux you can replace an existing volume with a soft link pointing somewhere else.
# move the volume to a new location
mv /var/lib/docker/volumes/postgres_volume/ /root
# replace it with a soft link
ln -sv /root/postgres_volume/ /var/lib/docker/volumes/postgres_volume
I can't seem to get docker/Maria to use my named docker volume. The host docker volumes directory is empty. But, there is a new container id right next to my named volume that looks like it has all of the MariaDB parts in it. The question is why?
My docker compose file:
version: "3.7"
#
# [Volumes]
#
volumes:
data-mysql:
#
# [Services]
#
services:
mariadb:
volumes:
- data-mysql:/var/lib/mysql
image: linuxserver/mariadb
container_name: mariadb
environment:
- PUID=1000
- GUID=1000
- MYSQL_ROOT_PASSWORD=<snipped>
- TZ=Etc/UTC
ports:
- 3306:3306
restart: unless-stopped
I've tried moving the volume part before and after the services line with no difference. When I do a docker-compose up, it does say it's creating the volume: mariadb_data-mysql, but when I shut down docker, there is nothing in the folder.
Thanks for any insight!
Nick
The data folder for MARIADB image you are using (linuxserver/mariadb) is /config/databases/ and not /var/lib/mysql. Replace this in your docker-compose.yml and it will work.
Also, the order in your docker-compose.yml does not matter: docker-compose will compile it and order everything alphabetically anyway before processing.
I am trying to get docker volumes working.
I have defined a volume in my Dockerfile as follows:
version: "3"
services:
redis:
image: redis:alpine
ports:
- "6379:6379"
db:
image: postgres:9.4
#container_name: db
volumes:
- "db-data:/var/lib/postgresql/data"
volumes:
db-data:
Now my question is, when I do a docker-compose up I don't see my data persisted on my local laptop (or server).
I just want to test/verify that my data is saving to the host's filesystem, so if I start/stop docker when it restarts it reads from the database file on the host.
That construct saves data in a named volume; if you look under /var/lib/docker/volumes as root you should be able to see it there (though mucking around in /var/lib/docker generally isn't advisable; and I believe this is one of the things where Docker Compose will change the name to try to make it unique).
If you want the data to be saved in a host directory, change the volume declaration to explicitly have a relative or absolute path. You won't need the explicit volume declaration, and for this you can remove the volumes block entirely. That would leave you with a docker-compose.yml that looks like:
version: "3"
services:
db:
image: postgres:9.4
volumes:
- "./db-data:/var/lib/postgresql/data"
New to Docker and I'm trying to set Postgres and pgadmin4 to run as a single service on docker for Mac inside a virtual machine. Everything works but as soon as I stop the service my data is gone. I'm using a named volume to persist data but probably doing something wrong. What is it?
Here's my setup:
# create my VM
docker-machine create dbvm
# set the right environment
eval $(docker-machine env dbvm)
Here's my docker-compose.yaml file:
version: '3'
services:
db:
image: postgres
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=my_db
volumes:
- pgdata:/pgdata
ports:
- 5432:5432
pgadmin:
image: fenglc/pgadmin4
ports:
- 5050:5050
volumes:
- pgadmindata:/pgadmindata
volumes:
pgdata:
pgadmindata:
With docker-compose.yaml, I run:
docker stack deploy -c docker-compose.yaml dbstack
I can do everything on this setup, but if I run docker stack rm dbstack the data is gone after this, but the volumes still exist.
$ docker volume ls
DRIVER VOLUME NAME
local 0c15b0b22c6b850e8768c14045da166253424dda4df8d2e13df75fd54d833412
local 22bab81d9d1de0e07de97363596b096f944752eba617ff574a0ab525239227f5
local 6da6e29fb98ad0f66d7da6a75dc76066ce014b26ea43567c55ed318fda707105
local dbstack_pgadmindata
local dbstack_pgdata
What am I missing?
Unless you have it in some config not shown, I believe you need to map to the default data location inside the container e.g., pgdata:/var/lib/postgresql/data
#Idg is partially correct. postgres data lives at /var/lib/postgresql/data per the Docker Hub readme.
But for it to work in your named volume, you can't use a path on the left side, so correct value would be:
volumes:
- pgdata:/var/lib/postgresql/data
Then the postgres data will stay in that named volume, on the node it was created on.
I'm looking into how to mount volumes with docker-compose for data persistence but I'm having trouble understanding all the examples I read.
https://www.linux.com/learn/docker-volumes-and-networks-compose
version: '2'
services:
mysql:
image: mysql
container_name: mysql
volumes:
- mysql:/var/lib/mysql
...
volumes:
mysql:
Ok so this defines a volume named mysql at the bottom and it references this volume in
- mysql:/var/lib/mysql
How will the mysql image know to look in this volume named mysql? Is it just designed to look in all the volumes it has to store data or something?
Then in other examples I see the following:
services:
nginx:
image: nginx
depends_on:
- ghost
volumes:
- ./default.conf:/etc/nginx/conf.d/default.conf
ports:
- "80:80"
networks:
- proxy
This example doesn't need to define a volume, why is that?
your MySQL data will be stored in the named volume mysql which is created by:
volumes:
mysql:
You can list the docker volumes using docker volume ls and the 'path' will be something like: /var/lib/docker/volumes/mysql/date. When you cd in this folder you will see the same data as the data which is in your mysql container on path: /var/lib/mysql. If you exec inside your container you will see the same data.
How does it know how to use this path?
Well check the Dockerfile of mysql. Here is:
VOLUME /var/lib/mysql
In short: all the data of your mysql is stored in /var/lib/mysql inside your container and mounted to your named docker volume mysql on your host, which path is something like /var/lib/docker/volumes/mysql/data/.
The next part is mounting ./default.conf (on your host, relative path) on the path /etc/nginx/conf.d/default.conf inside your nginx container.
Nginx and ghost don't need a named volume in this case because they don't need to keep specific data. When you create your environment you will add data using Ghost (write blogs), but the data itself will be stored in the mysql database. Not in the Ghost container.
Remark (if your second example has nothing to do with the mysql example): the default image of ghost is working with the sqlite3 db which is inside the same container (=! microservice for each container so this is fine to develop, not in production). But if you would use this setup you need to create a named volume for your sqlite which is in the same container as ghost. Take a look to the dockerfile of ghost.
If you want to use mysql you probably need to mount a config file to your ghost container to tell the container: use mysql, you will not need a named docker volume for ghost then, because data won't be stored in the ghost container but in the mysql container.
To keep your last example persistent without using mysql with a named volume you have to add a volume for the sqlite db which is inside the ghost container for this path: /var/lib/ghost/content. Check the Dockerfile again to see this path.
This blog post explains how to setup ghost with mysql in docker-compose