How to move a docker-compose environment to other computer - postgresql

I am developing a service using docker-compose and I deploy the the containers to a remote host using this commands:
eval $(docker-machine env digitaloceanserver)
docker-compose build && docker-compose stop && docker-compose rm -f && docker-compose up -d
My problem is that I'm changing laptop and I exported the docker-machines to the new laptop and I can activate them.
But when I try to deploy new changes it raises these errors:
Creating postgres ... error Creating redis ...ERROR: for postgres
Cannot create container for service postgres: b'Conflict. The
container name "/postgres" is already in use by container
"612f3887544224aeCreating redis ... errorERROR: for redis Cannot
create container for service redis: b'Conflict. The container name
"/redis" is already in use by container
"01875947f0ce7ba3978238525923e54e0c800fa0a4b419dd2a28cc07c285eb78".
You have to remove (or rename) that container to be able to reuse that
name.'ERROR: for postgres Cannot create container for service
postgres: b'Conflict. The container name "/postgres" is already in use
by container
"612f3887544224ae79f67e29552b4d97e246104b8a057b3a03d39f6546dbbd38".
You have to remove (or rename) that container to be able to reuse that
name.'ERROR: for redis Cannot create container for service redis:
b'Conflict. The container name "/redis" is already in use by container
"01875947f0ce7ba3978238525923e54e0c800fa0a4b419dd2a28cc07c285eb78".
You have to remove (or rename) that container to be able to reuse that
name.' ERROR: Encountered errors while bringing up the project.
My docker-compose.yml is this:
services:
nginx:
build: './docks/nginx/.'
ports:
- '80:80'
- "443:443"
volumes:
- letsencrypt_certs:/etc/nginx/certs
- letsencrypt_www:/var/www/letsencrypt
volumes_from:
- web:ro
depends_on:
- web
letsencrypt:
build: './docks/certbot/.'
command: /bin/true
volumes:
- letsencrypt_certs:/etc/letsencrypt
- letsencrypt_www:/var/www/letsencrypt
web:
build: './sources/.'
image: 'websource'
ports:
- '127.0.0.1:8000:8000'
env_file: '.env'
command: 'gunicorn cuidum.wsgi:application -w 2 -b :8000 --reload --capture-output --enable-stdio-inheritance --log-level=debug --access-logfile=- --log-file=-'
volumes:
- 'cachedata:/cache'
- 'mediadata:/media'
depends_on:
- postgres
- redis
celery_worker:
image: 'websource'
env_file: '.env'
command: 'python -m celery -A cuidum worker -l debug'
volumes_from:
- web
depends_on:
- web
celery_beat:
container_name: 'celery_beat'
image: 'websource'
env_file: '.env'
command: 'python -m celery -A cuidum beat --pidfile= -l debug'
volumes_from:
- web
depends_on:
- web
postgres:
container_name: 'postgres'
image: 'mdillon/postgis'
ports:
- '127.0.0.1:5432:5432'
volumes:
- 'pgdata:/var/lib/postgresql/data/'
redis:
container_name: 'redis'
image: 'redis:3.2.0'
ports:
- '127.0.0.1:6379:6379'
volumes:
- 'redisdata:/data'
volumes:
pgdata:
redisdata:
cachedata:
mediadata:
staticdata:
letsencrypt_certs:
letsencrypt_www:

You’re seeing those errors because you’re explicitly setting container_name:, and those same container names are used elsewhere. Remove those explicit settings. (You don’t need them even for inter-container DNS; Docker Compose automatically creates an alias for you using the name of the service block.)
There are still potential issues from port conflicts. If your other PostgreSQL container is listening on the same (default) host port 5432 then the one you declare in this docker-compose.yml file will conflict with it. You might be able to just not expose your database container ports, or you might need to change the port numbers in this file.

Related

Docker swarm stack ignores postgres password env variable

I'm trying to deploy postgres and pgadmin as a swarm stack via docker stack deploy with this compose file
version: '3.7'
services:
postgres:
image: postgres
ports:
- "5432:5432"
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=87654321
pgadmin:
image: dpage/pgadmin4
ports:
- "5433:80"
environment:
- PGADMIN_DEFAULT_EMAIL=developer#happycode.io
- PGADMIN_DEFAULT_PASSWORD=12345678
depends_on:
- postgres
volumes:
postgres-data:
With docker stack deploy - POSTGRES_PASSWORD is never applied to postgres, I can echo env variable inside the container and it contains correct value 87654321 but postgres still uses the default one. However if I use the same compose file with docker-compose everything works fine
I think the volume postgres-data has already all the data required for postgres.
try to delete it first and re-deploy the stack.
docker-compose down --remove-orphans --volumes
or stop the stack and run:
docker volume rm postgres-data

How to access postgres-docker container other docker container without ip address

How to access postgres-docker container other docker container without ip address?
I want to store data in postgres by using myweb. in jar given host like localhost:5432/db..
Here my compose file:
version: "3"
services:
myweb:
build: ./myweb
container_name: app
ports:
- "8080:8080"
- "9090:9090"
networks:
- front-tier
- back-tier
depends_on:
- "postgresdb"
postgresdb:
build: ./mydb
image: ppk:postgres9.5
volumes:
- dbdata:/var/lib/postgresql
ports:
- "5432:5432"
networks:
- back-tier
volumes:
dbdata: {}
networks:
front-tier:
back-tier:
Instead of localhost:5432/db.. use postgresdb:5432/db.. connection string.
By default the container has the same hostname as the service name.
Here is my minimal working example, which is connecting a java client (boxfuse/flyway) with postgres server. The most important part is the heath check, which is delaying the start of the myweb container to the time when postgres is ready to accept connections.
Note that this can be directly executed by docker-compose up, it dosen't have any other dependencies. Both the images are from docker hub.
version: '2.1'
services:
myweb:
image: boxfuse/flyway
command: -url=jdbc:postgresql://postgresdb/postgres -user=postgres -password=123 info
depends_on:
postgresdb:
condition: service_healthy
postgresdb:
image: postgres
environment:
- POSTGRES_PASSWORD=123
healthcheck:
test: "pg_isready -q -U postgres"
That is the Docker Networking problem. The solution is to use postgresdb:5432/db in place of localhost:5432/db because the two service is in the same network named back-tier and docker deamon will use name service like a DNS name to make communication between the two container. I think that my solution will help you so.

Docker container shuts down giving 'data directory has wrong ownership' error when executed in windows 10

I have my docker installed in Windows. I am trying to install this application. It has given me the following docker-compose.yml file:
version: '2'
services:
web:
build:
context: .
dockerfile: Dockerfile-nginx
ports:
- "8085:80"
networks:
- attendizenet
volumes:
- .:/usr/share/nginx/html/attendize
depends_on:
- php
php:
build:
context: .
dockerfile: Dockerfile-php
depends_on:
- db
- maildev
- redis
volumes:
- .:/usr/share/nginx/html/attendize
networks:
- attendizenet
php-worker:
build:
context: .
dockerfile: Dockerfile-php
depends_on:
- db
- maildev
- redis
volumes:
- .:/usr/share/nginx/html/attendize
command: php artisan queue:work --daemon
networks:
- attendizenet
db:
image: postgres
environment:
- POSTGRES_USER=attendize
- POSTGRES_PASSWORD=attendize
- POSTGRES_DB=attendize
ports:
- "5433:5432"
volumes:
- ./docker/pgdata:/var/lib/postgresql/data
networks:
- attendizenet
maildev:
image: djfarrelly/maildev
ports:
- "1080:80"
networks:
- attendizenet
redis:
image: redis
networks:
- attendizenet
networks:
attendizenet:
driver: bridge
All the installation goes well, but the PostgreSQL container stops after starting for a moment giving following error.
2018-03-07 08:24:47.927 UTC [1] FATAL: data directory "/var/lib/postgresql/data" has wrong ownership
2018-03-07 08:24:47.927 UTC [1] HINT: The server must be started by the user that owns the data directory
A simple PostgreSQL container from Docker Hub works smoothly, but the error occurs when we try to attach a volume to the container.
I am new to docker, so please ignore usage of terms wrongly.
This is a documented problem with the Postgres Docker image on Windows [1][2][3][4]. Currently, there doesn't appear to be a way to correctly mount Windows directories as volumes. You could instead use a persistent Docker volume, for example:
db:
image: postgres
environment:
- POSTGRES_USER=attendize
- POSTGRES_PASSWORD=attendize
- POSTGRES_DB=attendize
ports:
- "5433:5432"
volumes:
- pgdata:/var/lib/postgresql/data
networks:
- attendizenet
volumes:
pgdata:
Other things that didn't work:
Set PGDATA to a subdirectory (See PGDATA Setting)
environment:
- PGDATA=/var/lib/postgresql/data/mnt
volumes:
- ./pgdata:/var/lib/postgresql/data
Use a Bind Mount (docker-compose 3.2)
volumes:
- type: bind
source: ./pgdata
target: /var/lib/postgresql/data
Running as POSTGRES_USER=root
More Information:
GitHub
data directory "/var/lib/postgresql/data" has wrong ownership
Docker Forums
postgresql-data-pgdata-has-wrong-ownership
postgres-to-work-on-persistent-windows-mount
Please refer reinierkors' answer from here. The answer is as follows copied as is from the link here for reader's convenience and works for me
I solved this by mapping my local volume one directory below the one Postgres needs:
version: '3'
services:
postgres:
image: postgres
restart: on-failure
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
- PGDATA=/var/lib/postgresql/data/pgdata
- POSTGRES_DB=postgres
volumes:
- ./postgres_data:/var/lib/postgresql
ports:
- 5432:5432
I was having the same issue after downgrading my Docker from WSL 2 to WSL 1 and what Thomas Taylor's pertaining, I solved the issue by using named volume.
version: '3.8'
services:
postgres:
image: timescale/timescaledb:latest-pg12
...
volumes:
- pgdata:/var/lib/postgresql/data
...
volumes:
pgdata:
Map the local volume (e.g. C:\docker\pgdata) to one level (one directory) above what PostgreSQL needs. You can also do it from command line when starting the docker:
docker run -itd -e POSTGRES_USER=pguser -e POSTGRES_PASSWORD=pgpasswd \
-e PGDATA=/var/lib/postgresql/data/pgdata -p 5432:5432 \
-v c:\docker\pgdata:/var/lib/postgresql --name postgresql postgres
I met this issue when re-installed docker and used wsl-1 backend.
solution: switch docker to wsl-2 backend.
Even i had the problem i had to copy the data dir at regular intervals.
docker cp <container-name>:/var/lib/postgresql/data C:/docker/volumes/postgres
Owner for the data folder in postgres inside the container is Postgres user. Your current user may not have access privilege in the mounted folder. You need to give all permissions according to the requirements by given command below :
chmod 777 ./docker/pgdata
If this command is not helping to resolve this issue please refer the following link to do the user mapping from inside the container to outside the container.
https://docs.docker.com/engine/security/userns-remap/#prerequisites

Why I don't lose postgresql data when rebuild docker image?

version: '3'
services:
db:
image: postgres
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Why I don't lose data when running docker-compose build --force-em --no-cache. If this is normal, why do we need to create volume for data folder ?
When running the command docker-compose build --force-em --no-cache, this will only build the web Docker image from the Dockerfile which in your case is in the same directory.
This command will not stop the containers that you have previously started using this compose file, thus you want lose any data when running this command.
However, as soon as you remove the containers using docker-compose down or when containers are stopped docker-compose rm, you won't find the postgres data when you restart the container.
If you want to persist the data, and make the container pick it up when it is recreated, you need to give the postgres data volume a name as such.
version: '3'
services:
db:
image: postgres
volumes:
- pgdata:/var/lib/postgresql/data
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Now the postgres data won't be lost when the containers are recreated.

How to make persistent storage with docker-compose up-down-up?

I have a multiple container application, that is using the postgres image in docker-compose.yml file. Postgres container has volume on host machine for persistent storage.
When I run docker-compose up at first time all is fine, postgres creates db files in my host folder.
After it I need to shut down application temporarily with docker-compose down if I'll change code of web container.
When I run docker-compose up second time, postgres overwriting all db files, but I need that data not changes. How can I solve this issue?
My docker-compose.yml
version: '2'
services:
web:
build: ./web
command: python3 main.py
volumes:
- ./web:/app
ports:
- "80:80"
depends_on:
- db
- redis
links:
- db:db
- redis:redis
db:
image: postgres
ports:
- "5432:5432"
environment:
- POSTGRES_PASSWORD:0000
volumes:
- ./pgdb:/var/lib/postgresql/data
redis:
image: redis
ports:
- "6379:6379"
command: redis-server --appendonly yes
volumes:
- ./redisdb:/data
I solve this problem. It occurs probably because I changed permissions for pgdb directory with host root user. By default I couldn't open pgdb in host machine because owner is postgres user. I could be wrong but after I stopped to change the resolutions the problem was gone.