I have a docker compose file that looks like:
version: "3"
services:
redis:
image: 'redis:3.2.7'
# command: redis-server --requirepass redispass
postgres:
image: postgres:9.6
ports:
- "5432:5432"
environment:
- POSTGRES_USER=airflow
- POSTGRES_PASSWORD=airflow
- POSTGRES_DB=airflow
# - PGDATA=/var/lib/postgresql/data
webserver:
image: airflow:develop
depends_on:
- postgres
ports:
- "8080:8080"
command:
- webserver
After I run docker-compose up I see all the services started and seemingly working well. My webserver service connects to postgres with the following sqlalchemy connection string: postgresql+psycopg2://airflow:airflow#postgres/airflow
Whenever I kill the composition (with ctrl-c or docker-compose stop) and then restart it, the data in postgres seems to persist. The reason I believe it persists is because my webserver starts with data from the previous session.
I read the docs for the postgres docker image and found the PGDATA environment variable. I tried to force set it to the default (as seen in the commented line in my docker compose file, but that didn't help. I'm not sure how else to debug why data seems to be persisting between container starts.
How can I force my postgres container to start fresh with each new initialization?
Your data was persisted because you did not destroy postgres container. You used docker-compose stop which only hibernate containers. Use docker-compose down instead. It completely destroys containers (but not images).
Related
I am using a docker container to run postgres for testing purposes, it should NOT persist data between different runs.
This is the dockerfile:
FROM postgres:alpine
ENV POSTGRES_PASSWORD=1234
EXPOSE 5432
And this is my compose file:
version: "3.9"
services:
web:
build:
context: ../../.
dockerfile: ./services/web/Dockerfile
ports:
- "3000:3000"
db:
build: ../db
ports:
- "5438:5432"
graphql:
build:
context: ../../.
dockerfile: ./services/graphql/Dockerfile
ports:
- "4000:4000"
indexer:
build:
context: ../../.
dockerfile: ./services/indexer-ts/Dockerfile
volumes:
- ~/.aws/:/root/.aws:ro
However, I find that between sessions all data is being persisted and I have no clue why. This is totally messing my tests and is not expected to happen.
Even after running docker system prune, all data still persists, meaning that the container is probably using a volume somehow
Does anyone know why this is happening and how to not persist the data?
When your stop your docker-compose environment by typing CTRL-C or similar, next time you run docker-compose up it will restart the same container if the configuration hasn't changed. So even absent volumes, any data that was there previously will continue to be there.
To ensure you're starting with fresh containers, always run:
docker-compose down
If you have explicit volumes defined in your configuration, adding -v will also delete those volumes:
docker-compose down -v
(That's not necessary in this situation.)
Unrelated to your question, but why are you building a custom postgres image? You could just set things up in your docker-compose.yaml file:
db:
image: postgres:alpine
environment:
POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}"
ports:
- "5438:5432"
(And then set POSTGRES_PASSWORD in your .env file.)
You are correct, it is using a volume.
You can use the -v switch to clean up:
docker-compose rm -v db
This is my docker-compose.yml
version: '3.8'
services:
db:
container_name: "postgres"
image: postgres:14
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
- POSTGRES_DB:database
ports:
- "127.0.0.1:5434:5432"
volumes:
- ./db:/var/lib/postgresql/data
Docker don't create database for this container, what is wrong?
IF IT IS NOT AN ISSUE WITH TYPO here - POSTGRES_DB:database instead of - POSTGRES_DB=database
Did you ever start the Postgres container without specifying the user and database?
This could happen when the docker volume had already stored data from previous runs, where you didn't set the user and DB, then later runs with different env variables will not change the users. That is only done on the first initialization.
So you need to run
docker volume prune
There is already a closed issue on GitHub about that, see: https://github.com/docker-library/postgres/issues/453
I'm set up docker compose for my project with 2 services: spring-boot and postgresql. I created Dockerfile and docker-compose,yml as below:
Dockerfile :
FROM openjdk:8-jdk-alpine
MAINTAINER linhan.com
COPY target/LinhAn-0.0.1-SNAPSHOT.jar linhan-server-1.0.0.jar
ENTRYPOINT ["java","-jar","/linhan-server-1.0.0.jar"]
docker-compose.yml:
version: '2'
services:
spring_boot:
image: 'linhan'
build: .
container_name: api
ports:
- "8080:8080"
depends_on:
- postgres
environment:
- SPRING_DATASOURCE_URL=jdbc:jdbc:postgresql://localhost:5432/test_db
- SPRING_DATASOURCE_USERNAME=user
- SPRING_DATASOURCE_PASSWORD=123456
- SPRING_JPA_HIBERNATE_DDL_AUTO=update
postgres:
image: 'postgres:13.1-alpine'
container_name: db
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=123456
Then, when I type docker-compose up in terminal, postgres ran only, spring boot still not.
I searched google for solution but seems no hope. Please help me, thanks a lot!!!!!
I think you need to change the SPRING_DATASOURCE_URL to reference your service name instead of localhost. The service name is resolved automatically to your service since all services are part of the default_network by default in docker-compose.
- SPRING_DATASOURCE_URL=jdbc:jdbc:postgresql://postgres:5432/test_db
Also, for clarity I would suggest you add the port to your docker-compose postgres service, so it is clear which port is being used, even if it is the default:
postgres:
image: 'postgres:13.1-alpine'
container_name: db
ports:
- "5432"
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=123456
Also, another suggestion would be to try and use a healthcheck to see if your database service becomes available instead of a simple depends_on. The short version will mark the dependency fulfilled as soon as the container is Running, regardless of the availability of the database.
Either that, or you can add application logic to retry database connection in case of failure.
I would like to access my Postgres database (docker container) from PhpStorm.
docker-compose.yml
# Run docker-compose build
# Run docker-compose up
# Live long and prosper
version: '3.1'
services:
apache:
build: .docker/apache
container_name: sf-apache
ports:
- 82:80
volumes:
- .docker/config/vhosts:/etc/apache2/sites-enabled
- ${SYMFONY_APP}:/home/wwwroot/sf3
depends_on:
- php
postgres:
container_name: postgres
restart: always
image: 'postgres:12.6'
ports:
- "5432:5432"
environment:
- "POSTGRES_USER=${PGSQL_ADMIN_USER}"
- "POSTGRES_PASSWORD=${PGSQL_ADMIN_PASSWORD}"
volumes:
- ./API/var/postgres:/var/lib/postgresql/data
- .docker/postgresql/init-database.sh:/docker-entrypoint-initdb.d/init-database.sh
My PhpStorm config :
I can access to my database via docker exec -it postgres bash
If php storm is on the same host then you need to use localhost. If both phpstorm and pg is part of the same compose file , then you would use the service name since both would be in the same virtual network
I found the solution.
I have a Postgres local and a Postgres with docker. My Postgres local get the upper hand on my docker Postgres. I have killed the service and put the container service name on it.
Works perfectly.
Thanks for you help,
I have a multiple container application, that is using the postgres image in docker-compose.yml file. Postgres container has volume on host machine for persistent storage.
When I run docker-compose up at first time all is fine, postgres creates db files in my host folder.
After it I need to shut down application temporarily with docker-compose down if I'll change code of web container.
When I run docker-compose up second time, postgres overwriting all db files, but I need that data not changes. How can I solve this issue?
My docker-compose.yml
version: '2'
services:
web:
build: ./web
command: python3 main.py
volumes:
- ./web:/app
ports:
- "80:80"
depends_on:
- db
- redis
links:
- db:db
- redis:redis
db:
image: postgres
ports:
- "5432:5432"
environment:
- POSTGRES_PASSWORD:0000
volumes:
- ./pgdb:/var/lib/postgresql/data
redis:
image: redis
ports:
- "6379:6379"
command: redis-server --appendonly yes
volumes:
- ./redisdb:/data
I solve this problem. It occurs probably because I changed permissions for pgdb directory with host root user. By default I couldn't open pgdb in host machine because owner is postgres user. I could be wrong but after I stopped to change the resolutions the problem was gone.