It is really difficult to explain. I am not expert in docker. I will try it.
I have 3 components:
1 container with Azurite
1 container with a queue function
Microsoft Azure Storage Explorer in the host.
I am using docker-compose.yml to reach my goal.
After two day I could configure correctly the Azurite container to communicate correctly with the Storage Explorer in the host. Here the portion of code in my composer file:
services:
storage:
image: mcr.microsoft.com/azure-storage/azurite
container_name: storage
restart: always
ports:
- 7777:10000
- 8888:10001
- 9999:10002
environment:
- "AZURITE_ACCOUNTS=devnotificationsstorage:Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
command: "azurite -l /workspace -d /workspace/debug.log --blobPort 10000 --blobHost 0.0.0.0 --queuePort 10001 --queueHost 0.0.0.0 --tablePort 10002 --tableHost 0.0.0.0"
volumes:
- "./datadir/azurite:/workspace"
I mapped the port 7777 in my host with the port 10000 of the container with Azurite. I cannot use the port 10000 in my host because it is already used by Storage Explorer with another local container. However I find strange that everything works setting port 8888 on the host. Here a screen of Sotrage Explorer:
Surely I haven'y understood something.
In any case, now I can correctly insert the message in the containerized queue from the Storage Explorer.
But when I insert the message in the queue I expect to receive the message on the queue container. I let you see the whole compose file:
version: '3.4'
services:
storage:
image: mcr.microsoft.com/azure-storage/azurite
container_name: storage
restart: always
ports:
- 7777:10000
- 8888:10001
- 9999:10002
environment:
- "AZURITE_ACCOUNTS=devnotificationsstorage:Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
command: "azurite -l /workspace -d /workspace/debug.log --blobPort 10000 --blobHost 0.0.0.0 --queuePort 10001 --queueHost 0.0.0.0 --tablePort 10002 --tableHost 0.0.0.0"
volumes:
- "./datadir/azurite:/workspace"
networks:
- shared_network
xxx.notifications.azurefunction.emails:
image: ${DOCKER_REGISTRY-}xxxgonotificationsazurefunctionemails
container_name: notifications_email
build:
context: .
dockerfile: XXX.Notifications.AzureFunction.Emails/Dockerfile
depends_on:
- storage
networks:
- shared_network
networks:
shared_network:
And here the localhost.settings.json file:
{
"IsEncrypted": false,
"Values": {
"FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated",
"StorageConnectionString": "UseDevelopmentStorage=true"
}
}
Please, note that if I send a message from local queue on port 10000 (the non-dockerized queue), I receive the message on the queue triggered function.
But if I send the queue from the dockerized queue nothing happend.
I also thought that perhpas, the connection string is wrong. So I changed the StorageConnectionString with the value I see in the Storage explorer. That is the new localhost.settings.json file:
{
"IsEncrypted": false,
"Values": {
"FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated",
"StorageConnectionString": "AccountName=devnotificationsstorage;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;DefaultEndpointsProtocol=http;BlobEndpoint=http://127.0.0.1:7777/devnotificationsstorage;QueueEndpoint=http://127.0.0.1:8888/devnotificationsstorage;TableEndpoint=http://127.0.0.1:9999/devnotificationsstorage;"
}
}
In this case, when I write a message on the dockerized queue (and just when I write the message on the queue), I get this error:
Azure.Core: Connection refused (127.0.0.1:8888). System.Net.Http: Connection refused (127.0.0.1:8888). System.Net.Sockets: Connection refused.
How do I have to configure my containers?
Thank you.
This connection string is wrong:
"StorageConnectionString": "AccountName=devnotificationsstorage;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;DefaultEndpointsProtocol=http;BlobEndpoint=http://127.0.0.1:7777/devnotificationsstorage;QueueEndpoint=http://127.0.0.1:8888/devnotificationsstorage;TableEndpoint=http://127.0.0.1:9999/devnotificationsstorage;"
I must replace the host 127.0.0.1 and the ports with the ones I defined for my container.
First of all, the host. We can use the same name we used to define the container in the compose. In my case storage (second line):
services:
storage: <-- This is thehost name
image: mcr.microsoft.com/azure-storage/azurite
container_name: storage
restart: always
ports:
- 7777:10000
- 8888:10001
- 9999:10002
environment:
- "AZURITE_ACCOUNTS=devnotificationsstorage:Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
command: "azurite -l /workspace -d /workspace/debug.log --blobPort 10000 --blobHost 0.0.0.0 --queuePort 10001 --queueHost 0.0.0.0 --tablePort 10002 --tableHost 0.0.0.0"
volumes:
- "./datadir/azurite:/workspace"
networks:
- shared_network
Second I mapped the ports of my service in this way:
ports:
- 7777:10000
- 8888:10001
- 9999:10002
That means host:container. So every Http message sent on port 8888 of the host (in the context of Docker) will be received on port 10001 in my container.
So if I have to check if there are messages in the queue of my container I must change the connection string in this way:
"StorageConnectionString": "AccountName=devnotificationsstorage;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;DefaultEndpointsProtocol=http;BlobEndpoint=http://storage:10000/devnotificationsstorage;QueueEndpoint=http://storage:10001/devnotificationsstorage;TableEndpoint=http://storage:10002/devnotificationsstorage;"
Related
I've been beating my head against this for a few days now and I'm finally asking for help after trying to find the solution myself from all over.
I have a docker-compose file that looks like this:
services:
db:
image: ...
container_name: db
ports:
- "8095:5432"
networks:
- mynetwork
springservice:
image: ...
container_name: springservice
depends_on:
- db
ports:
- "8090:8090"
networks:
- mynetwork
environment:
- SPRING_DATASOURCE_URL: jdbc:postgresql://db:8095/dbname
- SPRING_DATASOURCE_USER: user
- SPRING_DATASOURCE_PASSWORD: password
networks:
mynetwork:
driver: bridge
name: mynetwork
Postgres has to be put to another port because we've got 3 postgres containers in that compose, so each get their own port.
Postgres's listen_address is set to "*".
pg_hba is set with "host all 0.0.0.0/0 md5"
Both containers come up, but when I curl from the service container to http://db:8095/ , I get connection refused.
What am I missing here?
Your port mapping is meaningless inside the docker network. This is only a mapping to the host system. Inside the network, the container is always available on its native port.
- SPRING_DATASOURCE_URL: jdbc:postgresql://db:5432/dbname
Also note that you don't need to publish the port to access it from inside the network. Doing so for a database can impose security risks. If you can, you should not publish it. That way, it will be only accessible from inside the docker network.
I'm developing a project based on the Github template dunglas/symfony-docker to which I want to add a postgres database..
It seems that my docker compose.yml file is incorrectly configured because the communication between PHP and postgres is malfunctioning.
Indeed when I try to perform a symfony migration, doctrine returns me the following error :
password authentication failed for user "postgres"
When I inspect the PHP logs I notice that PHP is waiting after the database
php_1 | Still waiting for db to be ready... Or maybe the db is not reachable.
My docker-compose.yml :
version: "3.4"
services:
php:
links:
- database
build:
context: .
target: symfony_php
args:
SYMFONY_VERSION: ${SYMFONY_VERSION:-}
SKELETON: ${SKELETON:-symfony/skeleton}
STABILITY: ${STABILITY:-stable}
restart: unless-stopped
volumes:
- php_socket:/var/run/php
healthcheck:
interval: 10s
timeout: 3s
retries: 3
start_period: 30s
environment:
# Run "composer require symfony/orm-pack" to install and configure Doctrine ORM
DATABASE_URL: ${DATABASE_URL}
# Run "composer require symfony/mercure-bundle" to install and configure the Mercure integration
MERCURE_URL: ${CADDY_MERCURE_URL:-http://caddy/.well-known/mercure}
MERCURE_PUBLIC_URL: https://${SERVER_NAME:-localhost}/.well-known/mercure
MERCURE_JWT_SECRET: ${CADDY_MERCURE_JWT_SECRET:-!ChangeMe!}
caddy:
build:
context: .
target: symfony_caddy
depends_on:
- php
environment:
SERVER_NAME: ${SERVER_NAME:-localhost, caddy:80}
MERCURE_PUBLISHER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET:-!ChangeMe!}
MERCURE_SUBSCRIBER_JWT_KEY: ${CADDY_MERCURE_JWT_SECRET:-!ChangeMe!}
restart: unless-stopped
volumes:
- php_socket:/var/run/php
- caddy_data:/data
- caddy_config:/config
ports:
# HTTP
- target: 80
published: 80
protocol: tcp
# HTTPS
- target: 443
published: 443
protocol: tcp
# HTTP/3
- target: 443
published: 443
protocol: udp
###> doctrine/doctrine-bundle ###
database:
image: postgres:${POSTGRES_VERSION:-13}-alpine
environment:
POSTGRES_DB: ${POSTGRES_DB:-app}
# You should definitely change the password in production
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-ChangeMe}
POSTGRES_USER: ${POSTGRES_USER:-symfony}
volumes:
- db-data:/var/lib/postgresql/data:rw
# You may use a bind-mounted host directory instead, so that it is harder to accidentally remove the volume and lose all your data!
# - ./docker/db/data:/var/lib/postgresql/data:rw
###< doctrine/doctrine-bundle ###
volumes:
php_socket:
caddy_data:
caddy_config:
###> doctrine/doctrine-bundle ###
db-data:
###< doctrine/doctrine-bundle ###
extract of my .env file :
POSTGRES_DB=proximityNL
POSTGRES_PASSWORD=postgres
POSTGRES_USER=postgres
DATABASE_URL="postgresql://postgres:postgres#database:5432/proximityNL?serverVersion=13&charset=utf8"
Can you help me ?
Best regards ..
UPDATE :
Indeed I understood on Saturday that it was just necessary to remove orphan ..
docker-compose down --remove-orphans --volumes
When running in a container, 127.0.0.1 refers to the container itself. Docker compose creates a virtual network where each container has its own IP address. You can address the containers by their service names.
So your connection string should point to database:5432 instead of 127.0.0.1:5432 like this
DATABASE_URL="postgresql://postgres:postgres#database:5432/proximityNL?serverVersion=13&charset=utf8"
You use database because that's the service name of your postgresql container in your docker compose file.
In docker you can call images via the name of it.
So try to use the name of the docker image for your config
DATABASE_URL="postgresql://postgres:postgres#database:5432/proximityNL?serverVersion=13&charset=utf8"
and maybe add an link between your php and database image
services:
php:
links:
- database
This is the way how i am connect a java app with an mysql db.
Docker should map DNS resolution from the Docker Host into your containers.
Networking in Compose link
Because of that, you DB URL should look like:
"postgresql://postgres:postgres#database:5432/..."
I understood on Saturday that it was just necessary to remove orphan
docker-compose down --remove-orphans --volumes
My Postgres DB is running in a Docker container. When container is started, it says it's ready to listen on 5432.
My application container is set to depend on it.
container_name: my_postgres_db
image: library/postgres:latest
network_mode: bridge
expose:
- 5432
ports:
- 5432:5432
environment:
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=admin
- POSTGRES_DB=localdb
restart: always
The config for app:
container_name: my_test_app
depends_on:
- db
build:
context: ./
dockerfile: Dockerfile
image: my_test_app
ports:
- 8080:8080
Based on solutions to the similar questions, I changed the localhost in the DB URL to:
spring.datasource.url=jdbc:postgresql://db:5432/localdb
It causes another error = "Unknown host exception". Even if I manage to build app this way, it still doesn't work. Logs say,
Connection to localhost:5432 refused
What else am I missing?
Why is it still listening to localhost:5432 when I obviously changed it to db:5432 and gradlew clean/build it?
just change the network_mode in postgres and app to host
network_mode: host
note that this will ignore the expose option and will use the host network an containers network
I'm trying to set up a docker-compose where one container has the database and the other one has the application. To my best knowledge, I need to connect two containers with a network.
version: "3"
services:
psqldb:
image: "postgres"
container_name: "psqldb"
environment:
- POSTGRES_USER=usr
- POSTGRES_PASSWORD=pwd
- POSTGRES_DB=mydb
ports:
- "5432:5432"
expose:
- "5432"
volumes:
- $HOME/docker/volumes/postgres/:/var/lib/postgresql/data
networks:
- backend
sbapp:
image: "dsb"
container_name: "dsb-cont"
ports:
- "8080:8080"
expose:
- "8080"
depends_on:
- psqldb
networks:
- backend
networks:
backend:
I also tried it with setting network driver to bridge (which didn't change the end result)
networks:
backend:
driver: bridge
I'm using the following url to connect to the database
spring.datasource.url=jdbc:postgresql://psqldb:5432/mydb
I also tried to expose the port and connect using localhost
spring.datasource.url=jdbc:postgresql://localhost:5432/mydb
I also tried to use port 5433 instead of 5432.
No matter what combination I used, I got the following error
Connection to psqldb:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
in my app container. However, my database container remains up and I can connect to it fine from host with the url
spring.datasource.url=jdbc:postgresql://localhost:5432/mydb
I can also connect to the database from host if I remove psqldb container entirely from docker-compose.yml (and use the latter connection url).
If it makes any difference, I'm using Spring
Boot for application with the Dockerfile
FROM openjdk:8-jdk-alpine
VOLUME /tmp
ARG JAR_FILE
COPY ${JAR_FILE} app.jar
EXPOSE 8080
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom", "-jar", "app.jar"]
and I'm building the image with the command
docker build --build-arg JAR_FILE=target/*.jar -t dsb .
What am I missing in the two container setup?
The issue I had was that docker depends_on only starts the containers in the defined order but doesn't wait for them to be actually ready.
https://docs.docker.com/compose/startup-order/
I'm having trouble accessing a database created from a docker-compose file.
Given the following compose file, I should be able to connect to it from java using something like:
jdbc:postgresql://eprase:eprase#database:7000/eprase
However, the connection is rejected. I can't even use PGAdmin to connect it using the same details to create a new server.
I've entered the database container and ran psql commands to verify that the eprase user and database have been created according to postgres Docker documentation, everything seems fine. I can't tell if the problem is within the database container or something I need to change in the compose network.
The client & server services can largely be ignored, the server is a java based web API and the client is an Angular app.
Compose file:
version: "3"
services:
client:
image: eprase/client:latest
build: ./client/eprase-app
networks:
api:
ports:
- "5000:80"
tty: true
depends_on:
- server
server:
image: eprase/server:latest
build: ./server
networks:
api:
ports:
- "6000:8080"
depends_on:
- database
database:
image: postgres:9
volumes:
- "./database/data:/var/lib/postgresql/data"
environment:
- "POSTGRES_USER=eprase"
- "POSTGRES_PASSWORD=eprase"
- "POSTGRES_DB=eprase"
networks:
api:
ports:
- "7000:5432"
restart: unless-stopped
pgadmin:
image: dpage/pgadmin4:latest
environment:
- "PGADMIN_DEFAULT_EMAIL=admin#eprase.com"
- "PGADMIN_DEFAULT_PASSWORD=eprase"
networks:
api:
ports:
- "8000:80"
depends_on:
- database
networks:
api:
The PostgreSQL database is listening on container port 5432. The 7000:5432 line is mapping host port 7000 to container port 5432. That allows you to connect to the database on port 7000. But, your services on a common network (api) should communicate with each other via the container ports.
So, from the perspective of the containers for the client and server services, the connection string should be:
jdbc:postgresql://eprase:eprase#database:5432/eprase