Here is my docker-compose file:
version: "3.8"
services:
postgres:
image: postgres:alpine
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- 5432:5432
restart: always
server:
build: .
depends_on:
- postgres
ports:
- "2302:2302"
- "80:80"
restart: always
For some reason docker-compose doesn't wait for postgresat all, as I don't see the postgres initialization at all after docker-compose up:
Building server
Step 1/9 : FROM golang:alpine
alpine: Pulling from library/golang
df20fa9351a1: Already exists
ed8968b2872e: Pull complete
a92cc7c5fd73: Pull complete
e871e8e8d7a9: Pull complete
e73272ec9a57: Pull complete
Digest: sha256:4d8abd16b03209b30b48f69a2e10347aacf7ce65d8f9f685e8c3e20a512234d9
Status: Downloaded newer image for golang:alpine
...
What might be the issue?
It seems that the postgres image was already running so that was the reason behind it not showing up in the logs.
Related
I have been using docker for a postgres database as I work on my project. I used this docker-compose file to spin it up
version: '3'
services:
postgres:
image: postgres
ports:
- "4001:5432"
environment:
- POSTGRES_DB=4x4-db
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- pgdata-4x4:/var/lib/postgresql/data
volumes:
pgdata-4x4: {}
I now want to containerise my back and front ends together with the database. I made this docker-compose file to do so
version: '3.8'
services:
frontend:
build: ./4x4
ports:
- "3000:3000"
backend:
build: ./server
ports:
- "8000:8000"
db:
image: postgres
ports:
- "4001:5432"
environment:
- POSTGRES_DB=4x4-db
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- pgdata-4x4:/var/lib/postgresql/data
volumes:
pgdata-4x4:
external: true
However, when I execute the command docker-compose up on the second file, I do not access the same data as the first one -- the database is blank. If I spin up the first one again, I return to the old data (i.e. nothing is overwritten).
I presumed that the same postgres database would be connected to
I would appreciate any elucidation
I have created a program and tested that works just fine. I decided to dockerize it, and it seems after maybe some hours or few days the data of mongoDB container get all deleted. The docker-compose.yml file:
version: '3'
services:
node:
restart: always
build: ./nodeServer
container_name: nodeserver
ports:
- 5000:5000
depends_on:
- database
networks:
twitter_articles:
ipv4_address: 172.24.0.2
environment:
- TZ=Europe/Athens
database:
restart: always
build: ./mongoDump/database
container_name: mongodb
ports:
- 27017:27017
networks:
twitter_articles:
ipv4_address: 172.24.0.4
volumes:
- ./data:/data/db
environment:
- TZ=Europe/Athens
pythonscript:
restart: always
build: ./python
container_name: pythonscript
depends_on:
- database
networks:
twitter_articles:
ipv4_address: 172.24.0.3
environment:
- TZ=Europe/Athens
networks:
twitter_articles:
ipam:
config:
- subnet: 172.24.0.0/24
And the three Dockerfile's that they are builded:
nodeserver:
FROM node:14.16.1
COPY package*.json ./
RUN npm install
COPY . ./
CMD [ "npm", "start"]
mongodb:
FROM mongo:5.0.3
CMD docker-entrypoint.sh mongod
pythonscript
FROM python:3.9
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . ./
CMD [ "python", "-u", "./init2.py" ]
As mentioned before without Docker the app works just fine and there isn't that kind of behaviour of database getting wiped out. I have tried also internal Docker storage which also does the same thing. I have tried to check the logs and I saw that there is an error happening in pythonscript container each time database wipes out. I know that an error should happen in pythonscript but there is no such a code anywhere in the app to perform deletion of collections or databases (also without Docker this error still happens but nothing gets deleted).
Any ideas?
You can create an external volume and add the data of the mongoDB into it. That way your data doesn't get wiped even when you turn off your docker-compose.
version: '3'
services:
node:
restart: always
build: ./nodeServer
container_name: nodeserver
ports:
- 5000:5000
depends_on:
- database
networks:
twitter_articles:
ipv4_address: 172.24.0.2
environment:
- TZ=Europe/Athens
database:
restart: always
build: ./mongoDump/database
container_name: mongodb
ports:
- 27017:27017
networks:
twitter_articles:
ipv4_address: 172.24.0.4
volumes:
- mongo_data:/data/db
environment:
- TZ=Europe/Athens
pythonscript:
restart: always
build: ./python
container_name: pythonscript
depends_on:
- database
networks:
twitter_articles:
ipv4_address: 172.24.0.3
environment:
- TZ=Europe/Athens
networks:
twitter_articles:
ipam:
config:
- subnet: 172.24.0.0/24
volumes:
mongo_data:
external: true
now you have to create a volume in your docker using
docker volume create --name=mongo_data
then
docker-compose down
and
docker-compose up --build -d
I have been advised that it is always better idea to save data outside of docker container in separate volume. Look for this tutorial volumes.
You need to make an persistant volume for your database, because as you noted on your docker-compose.yml file you got:
restart: always
so everytime your python script got an error, it's stopped and it's depending on Mariadb, so it's restarted and data got wiped.
Make sure the data is stored outside the docker container because are treated like cattles and not pets. New containers are created freshly with no data from previous version.
I'd ensure that container user has a pre-configured ID with write access to the host folder targeted for db data persistence.
I'd use an absolute path on the host side too when mapping persistent data folders in Docker.
Referring to:
volumes:
- ./data:/data/db
How to access postgres-docker container other docker container without ip address?
I want to store data in postgres by using myweb. in jar given host like localhost:5432/db..
Here my compose file:
version: "3"
services:
myweb:
build: ./myweb
container_name: app
ports:
- "8080:8080"
- "9090:9090"
networks:
- front-tier
- back-tier
depends_on:
- "postgresdb"
postgresdb:
build: ./mydb
image: ppk:postgres9.5
volumes:
- dbdata:/var/lib/postgresql
ports:
- "5432:5432"
networks:
- back-tier
volumes:
dbdata: {}
networks:
front-tier:
back-tier:
Instead of localhost:5432/db.. use postgresdb:5432/db.. connection string.
By default the container has the same hostname as the service name.
Here is my minimal working example, which is connecting a java client (boxfuse/flyway) with postgres server. The most important part is the heath check, which is delaying the start of the myweb container to the time when postgres is ready to accept connections.
Note that this can be directly executed by docker-compose up, it dosen't have any other dependencies. Both the images are from docker hub.
version: '2.1'
services:
myweb:
image: boxfuse/flyway
command: -url=jdbc:postgresql://postgresdb/postgres -user=postgres -password=123 info
depends_on:
postgresdb:
condition: service_healthy
postgresdb:
image: postgres
environment:
- POSTGRES_PASSWORD=123
healthcheck:
test: "pg_isready -q -U postgres"
That is the Docker Networking problem. The solution is to use postgresdb:5432/db in place of localhost:5432/db because the two service is in the same network named back-tier and docker deamon will use name service like a DNS name to make communication between the two container. I think that my solution will help you so.
I have my docker installed in Windows. I am trying to install this application. It has given me the following docker-compose.yml file:
version: '2'
services:
web:
build:
context: .
dockerfile: Dockerfile-nginx
ports:
- "8085:80"
networks:
- attendizenet
volumes:
- .:/usr/share/nginx/html/attendize
depends_on:
- php
php:
build:
context: .
dockerfile: Dockerfile-php
depends_on:
- db
- maildev
- redis
volumes:
- .:/usr/share/nginx/html/attendize
networks:
- attendizenet
php-worker:
build:
context: .
dockerfile: Dockerfile-php
depends_on:
- db
- maildev
- redis
volumes:
- .:/usr/share/nginx/html/attendize
command: php artisan queue:work --daemon
networks:
- attendizenet
db:
image: postgres
environment:
- POSTGRES_USER=attendize
- POSTGRES_PASSWORD=attendize
- POSTGRES_DB=attendize
ports:
- "5433:5432"
volumes:
- ./docker/pgdata:/var/lib/postgresql/data
networks:
- attendizenet
maildev:
image: djfarrelly/maildev
ports:
- "1080:80"
networks:
- attendizenet
redis:
image: redis
networks:
- attendizenet
networks:
attendizenet:
driver: bridge
All the installation goes well, but the PostgreSQL container stops after starting for a moment giving following error.
2018-03-07 08:24:47.927 UTC [1] FATAL: data directory "/var/lib/postgresql/data" has wrong ownership
2018-03-07 08:24:47.927 UTC [1] HINT: The server must be started by the user that owns the data directory
A simple PostgreSQL container from Docker Hub works smoothly, but the error occurs when we try to attach a volume to the container.
I am new to docker, so please ignore usage of terms wrongly.
This is a documented problem with the Postgres Docker image on Windows [1][2][3][4]. Currently, there doesn't appear to be a way to correctly mount Windows directories as volumes. You could instead use a persistent Docker volume, for example:
db:
image: postgres
environment:
- POSTGRES_USER=attendize
- POSTGRES_PASSWORD=attendize
- POSTGRES_DB=attendize
ports:
- "5433:5432"
volumes:
- pgdata:/var/lib/postgresql/data
networks:
- attendizenet
volumes:
pgdata:
Other things that didn't work:
Set PGDATA to a subdirectory (See PGDATA Setting)
environment:
- PGDATA=/var/lib/postgresql/data/mnt
volumes:
- ./pgdata:/var/lib/postgresql/data
Use a Bind Mount (docker-compose 3.2)
volumes:
- type: bind
source: ./pgdata
target: /var/lib/postgresql/data
Running as POSTGRES_USER=root
More Information:
GitHub
data directory "/var/lib/postgresql/data" has wrong ownership
Docker Forums
postgresql-data-pgdata-has-wrong-ownership
postgres-to-work-on-persistent-windows-mount
Please refer reinierkors' answer from here. The answer is as follows copied as is from the link here for reader's convenience and works for me
I solved this by mapping my local volume one directory below the one Postgres needs:
version: '3'
services:
postgres:
image: postgres
restart: on-failure
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
- PGDATA=/var/lib/postgresql/data/pgdata
- POSTGRES_DB=postgres
volumes:
- ./postgres_data:/var/lib/postgresql
ports:
- 5432:5432
I was having the same issue after downgrading my Docker from WSL 2 to WSL 1 and what Thomas Taylor's pertaining, I solved the issue by using named volume.
version: '3.8'
services:
postgres:
image: timescale/timescaledb:latest-pg12
...
volumes:
- pgdata:/var/lib/postgresql/data
...
volumes:
pgdata:
Map the local volume (e.g. C:\docker\pgdata) to one level (one directory) above what PostgreSQL needs. You can also do it from command line when starting the docker:
docker run -itd -e POSTGRES_USER=pguser -e POSTGRES_PASSWORD=pgpasswd \
-e PGDATA=/var/lib/postgresql/data/pgdata -p 5432:5432 \
-v c:\docker\pgdata:/var/lib/postgresql --name postgresql postgres
I met this issue when re-installed docker and used wsl-1 backend.
solution: switch docker to wsl-2 backend.
Even i had the problem i had to copy the data dir at regular intervals.
docker cp <container-name>:/var/lib/postgresql/data C:/docker/volumes/postgres
Owner for the data folder in postgres inside the container is Postgres user. Your current user may not have access privilege in the mounted folder. You need to give all permissions according to the requirements by given command below :
chmod 777 ./docker/pgdata
If this command is not helping to resolve this issue please refer the following link to do the user mapping from inside the container to outside the container.
https://docs.docker.com/engine/security/userns-remap/#prerequisites
version: '3'
services:
db:
image: postgres
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Why I don't lose data when running docker-compose build --force-em --no-cache. If this is normal, why do we need to create volume for data folder ?
When running the command docker-compose build --force-em --no-cache, this will only build the web Docker image from the Dockerfile which in your case is in the same directory.
This command will not stop the containers that you have previously started using this compose file, thus you want lose any data when running this command.
However, as soon as you remove the containers using docker-compose down or when containers are stopped docker-compose rm, you won't find the postgres data when you restart the container.
If you want to persist the data, and make the container pick it up when it is recreated, you need to give the postgres data volume a name as such.
version: '3'
services:
db:
image: postgres
volumes:
- pgdata:/var/lib/postgresql/data
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Now the postgres data won't be lost when the containers are recreated.