Docker compose fiware WireCloud data persistance not loaded from volume - postgresql

I am using a docker container for my FiWare WireCloud. It is working properly but when I stop my container with docker compose down and restart it with docker compose up all my data are erased even if I specified a volume for the postgresql database and I have the following error:
ERROR: relation "wirecloud_workspace" does not exist at character 370
If I want to make it work again, I have to recreate the whole database from scratch (initdb & createsuperuser)
What I would like to do is to be able to save my wirecloud data inside a volume and be able te backup it and reload it. Here is my current docker-compose.yml file in version 3:
version: '3.3'
services:
iot-mongo:
image: mongo:3.2
ports:
- "27017:27017"
volumes:
- ./data/mongo:/data/db
orion:
image: fiware/orion:1.9.0
links:
- iot-mongo
ports:
- "1026:1026"
command: -dbhost iot-mongo
nginx:
restart: always
image: nginx:1.13
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./nginx/sites-available:/etc/nginx/sites-available
- ./letsencrypt/well-known:/www/letsencrypt
- /etc/letsencrypt/:/etc/letsencrypt/
- wirecloudwww:/var/www/static
- wirecloudinstance:/opt/wirecloud_instance
links:
- wirecloud:wirecloud
- orion:orion
postgres:
restart: always
image: postgres:latest
ports:
- "5432:5432"
volumes:
- postgresdata:/var/lib/postgresql
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD:
POSTGRES_DB: postgres
PGDATA: /tmp
wirecloud:
restart: always
image: fiware/wirecloud:1.0-composable
links:
- postgres:postgres
volumes:
- wirecloudwww:/var/www/static
- wirecloudinstance:/opt/wirecloud_instance
volumes:
wirecloudwww: {}
wirecloudinstance: {}
postgresdata: {}
I also tried with docker-compose v1 like they show in the documentation but the result is the same.

The problem is the definition of the postgres volume and the PGDATA environment variable. The PGDATA environment is telling PostgreSQL to store data in /tmp, so it is not going to store data inside de volume (you can create a volume on /tmp, but this seems a bit strange). If you remove the PGDATA environment variable, postgres will store data into /var/lib/postgresql/data. Using this definition for the postgres service should do the trick:
postgres:
restart: always
image: postgres:latest
ports:
- "5432:5432"
volumes:
- postgresdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD:
POSTGRES_DB: postgres

Related

I don't manage to configure volumes in docker-compose to have a folder reachable by dockerized pgAdmin, to restore a database

I have setup from a tutorial a pgadmin4 container and a PostgreSQL container with the following docker-compose:
version: "2"
services:
db:
image: postgres:14
restart: always
environment:
POSTGRES_DB: postgres
POSTGRES_USER: admin
POSTGRES_PASSWORD: secret
PGDATA: /var/lib/postgresql/data
volumes:
- db-data:/opt/docker/pgadmin4/postgresql/data
- /home/laurent/Documents/Informatique/Docker/sample-database:/opt
ports:
- "5432:5432"
pgadmin:
image: dpage/pgadmin4:4
restart: always
environment:
PGADMIN_DEFAULT_EMAIL: admin#linuxhint.com
PGADMIN_DEFAULT_PASSWORD: secret
PGADMIN_LISTEN_PORT: 80
ports:
- "8080:80"
volumes:
- pgadmin-data:/var/lib/pgadmin
- /home/laurent/Documents/Informatique/Docker/sample-database:/opt
links:
- "db:pgsql-server"
volumes:
db-data:
pgadmin-data:
In both conainers, I have mounted the path /home/laurent/Documents/Informatique/Docker/sample-database where I have put a sample database.
After having created the dvdrental database as explained in the tutorial, I try to import it with restore, but I cannot find my /opt/dvdrental.tar as expected.
When I Select file, I get this:
If I add opt in the path and refresh, nothing more.
But if I open a console in both containers, I have /opt/dvdrental.tar.
What do I miss?

Accessing the same database in different docker containers

I have been using docker for a postgres database as I work on my project. I used this docker-compose file to spin it up
version: '3'
services:
postgres:
image: postgres
ports:
- "4001:5432"
environment:
- POSTGRES_DB=4x4-db
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- pgdata-4x4:/var/lib/postgresql/data
volumes:
pgdata-4x4: {}
I now want to containerise my back and front ends together with the database. I made this docker-compose file to do so
version: '3.8'
services:
frontend:
build: ./4x4
ports:
- "3000:3000"
backend:
build: ./server
ports:
- "8000:8000"
db:
image: postgres
ports:
- "4001:5432"
environment:
- POSTGRES_DB=4x4-db
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- pgdata-4x4:/var/lib/postgresql/data
volumes:
pgdata-4x4:
external: true
However, when I execute the command docker-compose up on the second file, I do not access the same data as the first one -- the database is blank. If I spin up the first one again, I return to the old data (i.e. nothing is overwritten).
I presumed that the same postgres database would be connected to
I would appreciate any elucidation

docker-compose.yml for Postgres works either way, but not the expected one

I have the following docker-compose.yml for running Postgres with Docker:
version: '3.8'
services:
postgres:
image: postgres:14.2-alpine
environment:
POSTGRES_USER: ${POSTGRES_USER:-postgres}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-postgres}
POSTGRES_DB: mydatabasename
PGDATA: /data/mydatabasename
volumes:
- postgres:/data/postgres
ports:
- '5432:5432'
networks:
- postgres
restart: unless-stopped
pgadmin:
... placeholder
networks:
postgres:
driver: bridge
volumes:
postgres:
pgadmin:
It works. What I don't understand, is that these two combinations work:
PGDATA: /data/mydatabasename
volumes:
- postgres:/data/postgres
and
PGDATA: /data/postgres
volumes:
- postgres:/data/mydatabasename
But this does not work:
PGDATA: /data/mydatabasename
volumes:
- postgres:/data/mydatabasename
I would just get: error: database "mydatabasename" does not exist.
The latter was my first attempt connecting everything though. So I am wondering, why do both fields not map to the actual database name? Thanks for your help!
Docker Compose File
version: '3'
services:
postgres:
image: postgres:alpine
container_name: postgres
restart: always
posts:
- 5432:5432
volumes:
- ./data_backup:/var/lib/postgresql/data #mount the data locally
env_file:
- .env
networks:
- postgres-net
pgadmin:
image: dpage/pgadmin4:6
restart: always
ports:
- 8080:80
volumes:
- pgadmin:/var/lib/pgadmin
env_file:
- .env
depends_on:
- postgres
netwroks:
- postgres-net
networks:
postgres-net:
name: postgres-net
.env File
#postgres
POSTGRES_PASSWORD="secure_password"
POSTGRES_DB=your_db_name
POSTGRES_USER=db_user_name
#pgadmin
PGADMIN_DEFAULT_EMAIL=ariful#firora.com
PGADMIN_DEFAULT_PASSWORD="secure_password"
PGADMIN_LISTEN_PORT=80
Since you may try to bind the database folder which was never created. Alternatively you can bind /var/lib/postgresql/data this path to store whatever database you created or about create in future.

How can i create a persistent Docker volume?

I am trying to create a persistent volume for postgresql so that when i do 'docker-compose.yml down' i don't lose the database. Currently i have a volume for postgresql but it's empty and the data is not going in it, can anyone help ?
volumes:
dependency-track:
postgresql:
services:
postgresql10:
image: postgres:10
environment:
- POSTGRES_DB=dtrack
- POSTGRES_USER=dtrack
- POSTGRES_PASSWORD=mypassword
ports:
- "5432:5432"
volumes:
- postgresql:/var/lib/postgresql
restart: unless-stopped
dtrack-apiserver:
image: dependencytrack/apiserver
depends_on:
- postgresql10
etc ...
You can create a volume for a postgres container, inside of a docker-compose file, with something like:
dbpostgres:
image: postgres
volumes:
- /your/path/here:/var/lib/postgresql
It needs an extra data at the end of the service volume path.
volumes:
dependency-track:
postgresql:
services:
postgresql10:
image: postgres:10
environment:
- POSTGRES_DB=dtrack
- POSTGRES_USER=dtrack
- POSTGRES_PASSWORD=mypassword
ports:
- "5432:5432"
volumes:
- postgresql:/var/lib/postgresql/data
restart: unless-stopped

Save Postgres Data to Directory in Docker Named Volume

Problem
I have an application with postgres. I want to be able to back up the initial database data so that I don't have to re enter it each deployment. However, despite having a named volume set up in my compose file.
What I'm not sure of is how to have postgres save its data into the directory associated with the volume. I'm also not sure exactly how to associate a directory with the named volume. What I want is for the docker host server to be able to see the postgress data in the named volume's associated directory.
Could someone please provide an explanation/some examples of how to handle this? Right now even though the volume is associated with the docker service in the compose file, it doesn't write any data to the database_volume/ directory. This is what I would like to address.
Code
Here's my Dockerfile:
FROM python:3.6
ARG requirements=requirements/production.txt
ENV DJANGO_SETTINGS_MODULE=sasite.settings.production_test
WORKDIR /app
COPY manage.py /app/
COPY requirements/ /app/requirements/
RUN pip install -r $requirements
COPY config config
COPY sasite sasite
COPY templates templates
COPY logs logs
ADD /scripts/docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod a+x /docker-entrypoint.sh
EXPOSE 8001
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["/usr/local/bin/gunicorn", "--config", "config/gunicorn.conf", "--log-config", "config/logging.conf", "-e", "DJANGO_SETTINGS_MODULE=sasite.settings.production_test", "-w", "4", "-b", "0.0.0.0:8001", "sasite.wsgi:application"]
And my docker-compose.yml:
version: "3.2"
services:
app:
restart: always
build:
context: .
dockerfile: Dockerfile.prodtest
args:
requirements: requirements/production.txt
container_name: dj01
environment:
- DJANGO_SETTINGS_MODULE=sasite.settings.production_test
- PYTHONDONTWRITEBYTECODE=1
volumes:
- ./:/app
- /static:/static
- /media:/media
networks:
- main
depends_on:
- db
db:
restart: always
image: postgres:10.1-alpine
container_name: ps01
environment:
POSTGRES_DB: sasite_db
POSTGRES_USER: pguser
POSTGRES_PASSWORD: pguser123
ports:
- "5432:5432"
volumes:
- database_volume:/var/lib/postgresql/data
networks:
- main
nginx:
restart: always
image: nginx
container_name: ng01
volumes:
- ./config/nginx-prodtest.conf:/etc/nginx/conf.d/default.conf:ro
- ./static:/usr/share/nginx/sasite/static
- ./media:/usr/share/nginx/sasite/media
ports:
- "80:80"
- "443:443"
networks:
- main
depends_on:
- app
networks:
main:
volumes:
database_volume:
driver_opts:
type: none
device: ./database_volume
o: bind