Docker container shuts down giving 'data directory has wrong ownership' error when executed in windows 10 - postgresql

I have my docker installed in Windows. I am trying to install this application. It has given me the following docker-compose.yml file:
version: '2'
services:
web:
build:
context: .
dockerfile: Dockerfile-nginx
ports:
- "8085:80"
networks:
- attendizenet
volumes:
- .:/usr/share/nginx/html/attendize
depends_on:
- php
php:
build:
context: .
dockerfile: Dockerfile-php
depends_on:
- db
- maildev
- redis
volumes:
- .:/usr/share/nginx/html/attendize
networks:
- attendizenet
php-worker:
build:
context: .
dockerfile: Dockerfile-php
depends_on:
- db
- maildev
- redis
volumes:
- .:/usr/share/nginx/html/attendize
command: php artisan queue:work --daemon
networks:
- attendizenet
db:
image: postgres
environment:
- POSTGRES_USER=attendize
- POSTGRES_PASSWORD=attendize
- POSTGRES_DB=attendize
ports:
- "5433:5432"
volumes:
- ./docker/pgdata:/var/lib/postgresql/data
networks:
- attendizenet
maildev:
image: djfarrelly/maildev
ports:
- "1080:80"
networks:
- attendizenet
redis:
image: redis
networks:
- attendizenet
networks:
attendizenet:
driver: bridge
All the installation goes well, but the PostgreSQL container stops after starting for a moment giving following error.
2018-03-07 08:24:47.927 UTC [1] FATAL: data directory "/var/lib/postgresql/data" has wrong ownership
2018-03-07 08:24:47.927 UTC [1] HINT: The server must be started by the user that owns the data directory
A simple PostgreSQL container from Docker Hub works smoothly, but the error occurs when we try to attach a volume to the container.
I am new to docker, so please ignore usage of terms wrongly.

This is a documented problem with the Postgres Docker image on Windows [1][2][3][4]. Currently, there doesn't appear to be a way to correctly mount Windows directories as volumes. You could instead use a persistent Docker volume, for example:
db:
image: postgres
environment:
- POSTGRES_USER=attendize
- POSTGRES_PASSWORD=attendize
- POSTGRES_DB=attendize
ports:
- "5433:5432"
volumes:
- pgdata:/var/lib/postgresql/data
networks:
- attendizenet
volumes:
pgdata:
Other things that didn't work:
Set PGDATA to a subdirectory (See PGDATA Setting)
environment:
- PGDATA=/var/lib/postgresql/data/mnt
volumes:
- ./pgdata:/var/lib/postgresql/data
Use a Bind Mount (docker-compose 3.2)
volumes:
- type: bind
source: ./pgdata
target: /var/lib/postgresql/data
Running as POSTGRES_USER=root
More Information:
GitHub
data directory "/var/lib/postgresql/data" has wrong ownership
Docker Forums
postgresql-data-pgdata-has-wrong-ownership
postgres-to-work-on-persistent-windows-mount

Please refer reinierkors' answer from here. The answer is as follows copied as is from the link here for reader's convenience and works for me
I solved this by mapping my local volume one directory below the one Postgres needs:
version: '3'
services:
postgres:
image: postgres
restart: on-failure
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
- PGDATA=/var/lib/postgresql/data/pgdata
- POSTGRES_DB=postgres
volumes:
- ./postgres_data:/var/lib/postgresql
ports:
- 5432:5432

I was having the same issue after downgrading my Docker from WSL 2 to WSL 1 and what Thomas Taylor's pertaining, I solved the issue by using named volume.
version: '3.8'
services:
postgres:
image: timescale/timescaledb:latest-pg12
...
volumes:
- pgdata:/var/lib/postgresql/data
...
volumes:
pgdata:

Map the local volume (e.g. C:\docker\pgdata) to one level (one directory) above what PostgreSQL needs. You can also do it from command line when starting the docker:
docker run -itd -e POSTGRES_USER=pguser -e POSTGRES_PASSWORD=pgpasswd \
-e PGDATA=/var/lib/postgresql/data/pgdata -p 5432:5432 \
-v c:\docker\pgdata:/var/lib/postgresql --name postgresql postgres

I met this issue when re-installed docker and used wsl-1 backend.
solution: switch docker to wsl-2 backend.

Even i had the problem i had to copy the data dir at regular intervals.
docker cp <container-name>:/var/lib/postgresql/data C:/docker/volumes/postgres

Owner for the data folder in postgres inside the container is Postgres user. Your current user may not have access privilege in the mounted folder. You need to give all permissions according to the requirements by given command below :
chmod 777 ./docker/pgdata
If this command is not helping to resolve this issue please refer the following link to do the user mapping from inside the container to outside the container.
https://docs.docker.com/engine/security/userns-remap/#prerequisites

Related

Docker wipes out mongoDB container data

I have created a program and tested that works just fine. I decided to dockerize it, and it seems after maybe some hours or few days the data of mongoDB container get all deleted. The docker-compose.yml file:
version: '3'
services:
node:
restart: always
build: ./nodeServer
container_name: nodeserver
ports:
- 5000:5000
depends_on:
- database
networks:
twitter_articles:
ipv4_address: 172.24.0.2
environment:
- TZ=Europe/Athens
database:
restart: always
build: ./mongoDump/database
container_name: mongodb
ports:
- 27017:27017
networks:
twitter_articles:
ipv4_address: 172.24.0.4
volumes:
- ./data:/data/db
environment:
- TZ=Europe/Athens
pythonscript:
restart: always
build: ./python
container_name: pythonscript
depends_on:
- database
networks:
twitter_articles:
ipv4_address: 172.24.0.3
environment:
- TZ=Europe/Athens
networks:
twitter_articles:
ipam:
config:
- subnet: 172.24.0.0/24
And the three Dockerfile's that they are builded:
nodeserver:
FROM node:14.16.1
COPY package*.json ./
RUN npm install
COPY . ./
CMD [ "npm", "start"]
mongodb:
FROM mongo:5.0.3
CMD docker-entrypoint.sh mongod
pythonscript
FROM python:3.9
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . ./
CMD [ "python", "-u", "./init2.py" ]
As mentioned before without Docker the app works just fine and there isn't that kind of behaviour of database getting wiped out. I have tried also internal Docker storage which also does the same thing. I have tried to check the logs and I saw that there is an error happening in pythonscript container each time database wipes out. I know that an error should happen in pythonscript but there is no such a code anywhere in the app to perform deletion of collections or databases (also without Docker this error still happens but nothing gets deleted).
Any ideas?
You can create an external volume and add the data of the mongoDB into it. That way your data doesn't get wiped even when you turn off your docker-compose.
version: '3'
services:
node:
restart: always
build: ./nodeServer
container_name: nodeserver
ports:
- 5000:5000
depends_on:
- database
networks:
twitter_articles:
ipv4_address: 172.24.0.2
environment:
- TZ=Europe/Athens
database:
restart: always
build: ./mongoDump/database
container_name: mongodb
ports:
- 27017:27017
networks:
twitter_articles:
ipv4_address: 172.24.0.4
volumes:
- mongo_data:/data/db
environment:
- TZ=Europe/Athens
pythonscript:
restart: always
build: ./python
container_name: pythonscript
depends_on:
- database
networks:
twitter_articles:
ipv4_address: 172.24.0.3
environment:
- TZ=Europe/Athens
networks:
twitter_articles:
ipam:
config:
- subnet: 172.24.0.0/24
volumes:
mongo_data:
external: true
now you have to create a volume in your docker using
docker volume create --name=mongo_data
then
docker-compose down
and
docker-compose up --build -d
I have been advised that it is always better idea to save data outside of docker container in separate volume. Look for this tutorial volumes.
You need to make an persistant volume for your database, because as you noted on your docker-compose.yml file you got:
restart: always
so everytime your python script got an error, it's stopped and it's depending on Mariadb, so it's restarted and data got wiped.
Make sure the data is stored outside the docker container because are treated like cattles and not pets. New containers are created freshly with no data from previous version.
I'd ensure that container user has a pre-configured ID with write access to the host folder targeted for db data persistence.
I'd use an absolute path on the host side too when mapping persistent data folders in Docker.
Referring to:
volumes:
- ./data:/data/db

Docker ownership of '/var/lib/postgresql/data'

Docker-compose is not running and I don't know why. There's this question of
chown: changing ownership of '/var/lib/postgresql/data': Operation not permitted
At a suggestion of a member on the docker community slack channel I installed the homebrew for Docker, but that hasn't managed to solve the problem. There was another stackoverflow post that suggested cc'ing inside the container and changing the permissions, but that doesn't make sense to me - the /var/lib/postgresql/data is created on startup.
Here is the docker-compose file -
version: "3.9"
services:
db:
restart: always
image: postgres
volumes:
- ./docker-entrypoint-initdb.d/init.sql:/docker-entrypoint-initdb.d/init.sql
- ./data/db:/var/lib/postgresql/data
environment:
- POSTGRES_NAME=dev-postgres
- POSTGRES_USER=pixel
- POSTGRES_DATABASE=lightchan
- POSTGRES_PASSWORD=stardust
web:
build: .
restart: always
volumes:
- .:/code
command: sh -c "./waitfor.sh db:5432 -- python3 manage.py runserver"
ports:
- "8001:8001"
environment:
- POSTGRES_NAME=dev-postgres
- POSTGRES_USER=pixel
- POSTGRES_DATABASE=lightchan
- POSTGRES_PASSWORD=stardust
- POSTGRES_HOST=db

Starting Tryton server with docker-compose file

I am trying to link an external postgres to tryton/tryton from docker hub.
docker-compose.yaml
version: '3.7'
services:
tryton-postgres:
image: postgres
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_DB=tryton
restart: always
gnuserver:
image: tryton/tryton:4.6
links:
- tryton-postgres:postgres
ports:
- 8000:8000
depends_on:
- tryton-postgres
entrypoint: /entrypoint.sh trytond
when i ssh into the container and run trytond-admin --all -d tryton it seems to be looking for sqlite file instead of the connected postgres database. Are there some env variagbles i must set? What am i missing in my docker compose file?
Instead of changing the configuration file, with Docker it is simpler to set environment variable like:
DB_USER=
DB_PASSWORD=
DB_HOSTNAME=tryton-postgres
DB_PORT=5432
you need to edit /etc/tryton/trytond.conf to look at postgresql:
uri = postgresql://USERNAME:PASSWORD#tryton-postgres:5432/
see the Docs

How to connect to localhost postgres database from docker container?

I'm configured my project to docker. I have database that have been used in non-docker period and now I want to connect my docker-compose db service to it. But when I write docker-compose up - existing database not used - new one created instead (I suspect, docker container simply doesn't see the database). If I do nonsense please let me know. Maybe I shoud migrate my server db into container.
Here is my docker-compose.yml:
services:
db:
restart: always
image: postgres:latest
environment:
- POSTGRES_DB=mydb
- POSTGRES_PASSWORD=p#ssw0rd
- POSTGRES_USER=root
ports:
- "5432:5432"
volumes:
# We'll mount the 'postgres-data' volume into the location Postgres stores it's data:
- postgres-data:/var/lib/postgresql/data
web:
build: .
command: bash -c "python manage.py collectstatic --noinput && ./manage.py migrate && ./run_gunicorn.sh"
volumes:
- .:/code
- /static:/static
ports:
- 443:443
depends_on:
- db
nginx:
restart: always
image: nginx:latest
ports:
- 80:80
volumes:
- ./misc/nginx.conf:/etc/nginx/conf.d/default.conf
- /static:/static
depends_on:
- web
I think, the canonic approach is to have your DB engine running in container while storing the data on the persistent storage (map the volume to your hard disk).
So I would use the Postgres in docker as ServerDB, as you suggested.
If you only want your application connect to the external database, declare it as an external host:
version: '2'
services:
web:
build: .
command: bash -c "python manage.py collectstatic --noinput && ./manage.py migrate && ./run_gunicorn.sh"
volumes:
- .:/code
- /static:/static
ports:
- 443:443
extra_hosts:
- "db:192.168.1.2"
nginx:
restart: always
image: nginx:latest
ports:
- 80:80
volumes:
- ./misc/nginx.conf:/etc/nginx/conf.d/default.conf
- /static:/static
depends_on:
- web
Just be sure your application reference the database as db and replace the ip I put there with your host ip.
Regards

How to make persistent storage with docker-compose up-down-up?

I have a multiple container application, that is using the postgres image in docker-compose.yml file. Postgres container has volume on host machine for persistent storage.
When I run docker-compose up at first time all is fine, postgres creates db files in my host folder.
After it I need to shut down application temporarily with docker-compose down if I'll change code of web container.
When I run docker-compose up second time, postgres overwriting all db files, but I need that data not changes. How can I solve this issue?
My docker-compose.yml
version: '2'
services:
web:
build: ./web
command: python3 main.py
volumes:
- ./web:/app
ports:
- "80:80"
depends_on:
- db
- redis
links:
- db:db
- redis:redis
db:
image: postgres
ports:
- "5432:5432"
environment:
- POSTGRES_PASSWORD:0000
volumes:
- ./pgdb:/var/lib/postgresql/data
redis:
image: redis
ports:
- "6379:6379"
command: redis-server --appendonly yes
volumes:
- ./redisdb:/data
I solve this problem. It occurs probably because I changed permissions for pgdb directory with host root user. By default I couldn't open pgdb in host machine because owner is postgres user. I could be wrong but after I stopped to change the resolutions the problem was gone.