I have been using docker for a postgres database as I work on my project. I used this docker-compose file to spin it up
version: '3'
services:
postgres:
image: postgres
ports:
- "4001:5432"
environment:
- POSTGRES_DB=4x4-db
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- pgdata-4x4:/var/lib/postgresql/data
volumes:
pgdata-4x4: {}
I now want to containerise my back and front ends together with the database. I made this docker-compose file to do so
version: '3.8'
services:
frontend:
build: ./4x4
ports:
- "3000:3000"
backend:
build: ./server
ports:
- "8000:8000"
db:
image: postgres
ports:
- "4001:5432"
environment:
- POSTGRES_DB=4x4-db
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- pgdata-4x4:/var/lib/postgresql/data
volumes:
pgdata-4x4:
external: true
However, when I execute the command docker-compose up on the second file, I do not access the same data as the first one -- the database is blank. If I spin up the first one again, I return to the old data (i.e. nothing is overwritten).
I presumed that the same postgres database would be connected to
I would appreciate any elucidation
Related
There are multiple parts of Compose that deal with environment variables in one sense or another. So how do I pass Environment variables in Compose ( docker-compose )
According to the documentation If you have multiple environment variables, you can substitute them by adding them to a default environment variable file named .env or by providing a path to your environment variables file using the --env-file command line option.
version: '3.9'
services:
nginx:
image: nginx:stable-alpine
container_name: nginx
ports:
- 80:80
- 443:443
restart: always
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
postgres:
container_name: postgres
image: postgres:13-alpine
environment:
- POSTGRES_USER=${DB_USER}
- POSTGRES_PASSWORD=${DB_PASSWORD}
- POSTGRES_DB=${DB_DATABASE}
volumes:
- ./pgdata:/var/lib/postgresql/data
- ./database/app.sql:/docker-entrypoint-initdb.d/app.sql
restart: always
ports:
- "35000:5432"
networks:
- app_network
app-api:
container_name: app-api
build:
dockerfile: Dockerfile
context: ./app-api
target: production
environment:
- DB_TYPE=${DATABASE_TYPE}
- POSTGRES_HOST=${DB_HOST}
- POSTGRES_USER=${DB_USER}
- POSTGRES_PASS=${DB_PASSWORD}
- POSTGRES_DB=${DB_DATABASE}
- POSTGRES_PORT=${DB_PORT}
- APP_PORT=${SERVER_PORT}
- NODE_ENV:production
## AWS
- AWS_S3_ACCESS_KEY=${AWS_S3_ACCESS_KEY}
- AWS_S3_SECRET_ACCESS_KEY=${AWS_S3_SECRET_ACCESS_KEY}
- AWS_S3_BUCKET=${AWS_S3_BUCKET}
- AWS_S3_REGION=${AWS_S3_REGION}
ports:
- "5050:80"
volumes:
- ./pgadmin-data:/var/lib/pgadmin
depends_on:
- postgres
links:
- postgres
networks:
- app_network
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4
restart: always
environment:
- PGADMIN_DEFAULT_EMAIL=${PGADMIN_DEFAULT_EMAIL}
- PGADMIN_DEFAULT_PASSWORD=${PGADMIN_DEFAULT_PASSWORD}
- PGADMIN_LISTEN_PORT=${PGADMIN_LISTEN_PORT}
restart: always
ports:
- "5400:5400"
depends_on:
- postgres
links:
- postgres
networks:
- app_network
you can use it like this. you have to pass value in each environment variable.
server:
environment:
- AWS_S3_ACCESS_KEY=ABCJQHWEQJHWQ
- AWS_S3_SECRET_ACCESS_KEY=ASKJHDAKJHNAWKLHEN
- AWS_S3_BUCKET=abc-text
- AWS_S3_REGION=eu-west-1
ports:
- "5050:80"
volumes:
- ./pgadmin-data:/var/lib/pgadmin
depends_on:
- postgres
links:
- postgres
networks:
- app_network
When you run docker-compose up, the web service defined above uses the image from the defined Dockerfile. You can verify this with the convert command, which prints your resolved application config to the terminal:
You can use this command to verify if you are pathing the proper environment variables
$ docker compose convert
https://docs.docker.com/compose/environment-variables/
I am trying to run my postgres server and nestjs project with docker script and it does fire up the server and database. While firing up it does run migrations too but when I open pgAdmin I see no database there and if i try to create new server i get fatal pasword incorrect error. Also my server crashes too with error saying Password authentication failed for user "postgres". It was running fine yesterday but today its not running at all. I tried pruning everything and made fresh build and then compose up but nothing. Here is docker script
version: "3.5"
services:
dev-api:
container_name: xxxxxxxx-api
build:
context: .
dockerfile: Dockerfile
depends_on:
- dev-db
environment:
DATABASE_URL: postgresql://postgres:postgres#dev-db:5432/xxxxxxx_api
APP_ENV: development
PORT: 3030
WAIT_HOSTS: dev-db:5432
ports:
- "3030:3030"
- "9229:9229"
volumes:
- .:/usr/api/
dev-db:
container_name: xxxxxxxx-postgres
image: postgres:13.5-alpine
restart: always
ports:
- "5432:5432"
volumes:
- ./pg-data:/var/lib/postgresql/data
- ./src/db/docker/init.sql:/docker-entrypoint-initdb.d/dbinit.sql
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=xxxxxxx_api
expose:
- "5432"
pgadmin:
container_name: xxxxxxx-pgadmin
image: dpage/pgadmin4:6.2
ports:
- 8080:80
volumes:
- pgadmin-data:/var/lib/pgadmin
environment:
- PGADMIN_DEFAULT_EMAIL=user#postgres.com
- PGADMIN_DEFAULT_PASSWORD=postgres
- PGADMIN_LISTEN_PORT=80
depends_on:
- dev-db
volumes:
pgadmin-data:
I have a requirement with docker/docker-compose to run 2 different instances of postgres but i need their data to be completely separate as both applications control the database server completely, not just a single database
Here is the docker file there is one in each project directory
FROM postgres:10-alpine
COPY data/resources.sql.gz /docker-entrypoint-initdb.d/resources.sql.gz
ENV POSTGRES_USER=postgres
ENV POSTGRES_PASSWORD=123456
Here is the section from each docker compose, there is one in each project directory
Project 1
db_test:
image: postgres:10-alpine
container_name: postgres_test
restart: always
expose:
- '5432'
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=123456
- PGDATA=/db
volumes:
- ./db:/db
networks:
backend:
ipv4_address: 172.16.1.6
Project 2
db:
image: postgres:10-alpine
container_name: postgres
restart: always
expose:
- '5432'
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=123456
- PGDATA=/db
volumes:
- ./db:/db
networks:
backend:
ipv4_address: 172.16.0.6
I should also note that the resources.sql.gz is unique to each project
The problem i am having is that i build project 1, then stop the docker
then i build project 2 and for some reason its inheriting the databases from project 1
what i need is to completely seperate both so that i could run them side by side with different ports (if required)
One option is to create two distinct docker volumes:
docker-compose.yml
version: "3.7"
volumes:
db1-pgdata-volume:
name: db1-postgres-data
db2-pgdata-volume:
name: db2-postgres-data
services:
db_test:
image: postgres:10-alpine
container_name: postgres_test
restart: always
expose:
- '5432'
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=123456
- PGDATA=/db
volumes:
- db1-pgdata-volume:/var/lib/postgresql/data
networks:
backend:
ipv4_address: 172.16.1.6
db:
image: postgres:10-alpine
container_name: postgres
restart: always
expose:
- '5432'
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=123456
- PGDATA=/db
volumes:
- db2-pgdata-volume:/var/lib/postgresql/data
networks:
backend:
ipv4_address: 172.16.0.6
You are using the same database storage for both services. It's like you're using a single database with two access points.
Change the volumes section to this :
in db service :
volumes:
- ./db:/var/lib/postgresql/data
in db_test service:
volumes:
- ./db_test:/var/lib/postgresql/data
Remove the following from the environment section :
- PGDATA=/db
I am using a docker container for my FiWare WireCloud. It is working properly but when I stop my container with docker compose down and restart it with docker compose up all my data are erased even if I specified a volume for the postgresql database and I have the following error:
ERROR: relation "wirecloud_workspace" does not exist at character 370
If I want to make it work again, I have to recreate the whole database from scratch (initdb & createsuperuser)
What I would like to do is to be able to save my wirecloud data inside a volume and be able te backup it and reload it. Here is my current docker-compose.yml file in version 3:
version: '3.3'
services:
iot-mongo:
image: mongo:3.2
ports:
- "27017:27017"
volumes:
- ./data/mongo:/data/db
orion:
image: fiware/orion:1.9.0
links:
- iot-mongo
ports:
- "1026:1026"
command: -dbhost iot-mongo
nginx:
restart: always
image: nginx:1.13
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./nginx/sites-available:/etc/nginx/sites-available
- ./letsencrypt/well-known:/www/letsencrypt
- /etc/letsencrypt/:/etc/letsencrypt/
- wirecloudwww:/var/www/static
- wirecloudinstance:/opt/wirecloud_instance
links:
- wirecloud:wirecloud
- orion:orion
postgres:
restart: always
image: postgres:latest
ports:
- "5432:5432"
volumes:
- postgresdata:/var/lib/postgresql
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD:
POSTGRES_DB: postgres
PGDATA: /tmp
wirecloud:
restart: always
image: fiware/wirecloud:1.0-composable
links:
- postgres:postgres
volumes:
- wirecloudwww:/var/www/static
- wirecloudinstance:/opt/wirecloud_instance
volumes:
wirecloudwww: {}
wirecloudinstance: {}
postgresdata: {}
I also tried with docker-compose v1 like they show in the documentation but the result is the same.
The problem is the definition of the postgres volume and the PGDATA environment variable. The PGDATA environment is telling PostgreSQL to store data in /tmp, so it is not going to store data inside de volume (you can create a volume on /tmp, but this seems a bit strange). If you remove the PGDATA environment variable, postgres will store data into /var/lib/postgresql/data. Using this definition for the postgres service should do the trick:
postgres:
restart: always
image: postgres:latest
ports:
- "5432:5432"
volumes:
- postgresdata:/var/lib/postgresql/data
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD:
POSTGRES_DB: postgres
I'm configured my project to docker. I have database that have been used in non-docker period and now I want to connect my docker-compose db service to it. But when I write docker-compose up - existing database not used - new one created instead (I suspect, docker container simply doesn't see the database). If I do nonsense please let me know. Maybe I shoud migrate my server db into container.
Here is my docker-compose.yml:
services:
db:
restart: always
image: postgres:latest
environment:
- POSTGRES_DB=mydb
- POSTGRES_PASSWORD=p#ssw0rd
- POSTGRES_USER=root
ports:
- "5432:5432"
volumes:
# We'll mount the 'postgres-data' volume into the location Postgres stores it's data:
- postgres-data:/var/lib/postgresql/data
web:
build: .
command: bash -c "python manage.py collectstatic --noinput && ./manage.py migrate && ./run_gunicorn.sh"
volumes:
- .:/code
- /static:/static
ports:
- 443:443
depends_on:
- db
nginx:
restart: always
image: nginx:latest
ports:
- 80:80
volumes:
- ./misc/nginx.conf:/etc/nginx/conf.d/default.conf
- /static:/static
depends_on:
- web
I think, the canonic approach is to have your DB engine running in container while storing the data on the persistent storage (map the volume to your hard disk).
So I would use the Postgres in docker as ServerDB, as you suggested.
If you only want your application connect to the external database, declare it as an external host:
version: '2'
services:
web:
build: .
command: bash -c "python manage.py collectstatic --noinput && ./manage.py migrate && ./run_gunicorn.sh"
volumes:
- .:/code
- /static:/static
ports:
- 443:443
extra_hosts:
- "db:192.168.1.2"
nginx:
restart: always
image: nginx:latest
ports:
- 80:80
volumes:
- ./misc/nginx.conf:/etc/nginx/conf.d/default.conf
- /static:/static
depends_on:
- web
Just be sure your application reference the database as db and replace the ip I put there with your host ip.
Regards