I am trying to connect a postgres DB with Node api, with docker compose like this:
version: '3.8'
services:
db:
image: postgres
environment:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: biobox
volumes:
- ./db-data:/var/lib/postgresql/data
ports:
- 5432:5432
restart: always
api:
build: .
depends_on:
- db
ports:
- 3333:3333
command: yarn dev:server
env_file:
- .env
volumes:
- .:/usr/src/app
working_dir: /usr/src/app
env
DATABASE_USER=postgres
DATABASE_PASSWORD=postgres
DATABASE=biobox
DATABASE_HOST=db
DATABASE_PORT=5432
dockerfile
FROM node:12.16.2-alpine
COPY package*.json ./
RUN yarn
COPY . .
COPY .env .
EXPOSE 3333
My api and db are online on container and the logs show no error.
because when i try make a query return this:
Using ts-node version 8.8.2, typescript version 3.8.3
api_1 | Create a new user User {
api_1 | completName: 'Name',
api_1 | email: 'Name#gmail.com',
api_1 | cpf: '23425',
api_1 | dateNasc: '10/10/2020',
api_1 | number: 'Namefd222',
api_1 | password: '1Name5234'
api_1 | }
db_1 | 2020-05-31 16:37:00.199 UTC [74] LOG: unexpected EOF on client connection with an open transaction
I don't understand why this happens. My environment variables are in accordance with the docker-compose create, in this case the database, USER, PASS.
Related
I have a Golang server alongside Postgres instance running inside docker compose. For some reason the Postgres is refusing connection. From all of my previous searches, usually the problem is typo, not exposing the port, having SSL and so on, but I don't have anything like that going on and still having this issue
version: "3.2"
services:
ingress:
image: jwilder/nginx-proxy
ports:
- "3000:80"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
auth-service:
depends_on:
- rabbitmq
- auth-db
- ingress
build: ./auth
container_name: "auth-service"
ports:
- 3001:3000
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_HOST=auth-db
- POSTGRES_DB=auth-dev
- POSTGRES_PORT=5435
- PORT=3000
- RABBITMQ_USER=guest
- RABBITMQ_PASSWORD=guest
- RABBITMQ_HOST=rabbitmq
- RABBITMQ_PORT=5672
- VIRTUAL_HOST=api.twitchy.dev
- VIRTUAL_PATH=/v1/auth/
deploy:
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
# networks:
# - rabbitmq_net
# - default
rabbitmq:
image: rabbitmq:3-management-alpine
container_name: "rabbitmq"
ports:
- 5672:5672
- 15672:15672
volumes:
- rabbitmq_data:/var/lib/rabbitmq/
- rabbitmq_log:/var/log/rabbitmq/
# networks:
# - rabbitmq_net
auth-db:
image: postgres:14.1-alpine
restart: always
container_name: "auth-db"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=auth-dev
ports:
- "5435:5432"
volumes:
- db:/var/lib/postgresql/data
chat-db:
image: postgres:14.1-alpine
restart: always
container_name: "chat-db"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=chat-dev
ports:
- "5434:5432"
volumes:
- db:/var/lib/postgresql/data
# networks:
# rabbitmq_net:
# driver: bridge
volumes:
db:
driver: local
rabbitmq_data:
rabbitmq_log:
This is the error I am getting
auth-service | Retrying connection to database...
auth-service | failed to connect to `host=auth-db user=postgres database=auth-dev`: dial error (dial tcp 172.23.0.3:5435: connect: connection refused)
And my Golang code used to connect to the DB (using pgx)
dbUrl := fmt.Sprintf("postgres://%s:%s#%s:%s/%s?sslmode=disable",
os.Getenv("POSTGRES_USER"),
os.Getenv("POSTGRES_PASSWORD"),
os.Getenv("POSTGRES_HOST"),
os.Getenv("POSTGRES_PORT"),
os.Getenv("POSTGRES_DB"))
This is why I am confused
The ports match up, I expose 5435 from postgres, and I connect to 5435
The host should be correct as I am referencing the auth-db service name and they are on the same default network so that should be fine
The password and username match up
The POSTGRES_DB also match up, the default database should be auth-dev
POSTGRES_DB
This optional environment variable can be used to define a different name for the default database that is created when the image is first started. If it is not specified, then the value of POSTGRES_USER will be used.
I have sslmode disable as well
Is there anything else that can cause the connection to be refused?
Tried changing db to template1 and postgres as they are created by default but both aren't working either
54511e50369c postgres:14.1-alpine "docker-entrypoint.s…" 16 minutes ago Up 16 seconds 0.0.0.0:5435->5432/tcp, :::5435->5432/tcp auth-db
docker exec -it 54511e50369c psql -U postgres
psql (14.1)
Type "help" for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
The database is ready when I am trying to connect (I am retrying 20 times so, and restarting the service if it crashes, so it should be available)
When you map ports in docker-compose, say like "5435:5432", you are mapping port 5435 on the HOST machine to 5432 on the CONTAINER. However, your db url in the auth-service definition is using the name of the service, auth-db, so you are actually hitting the db container directly, not going through the host machine. Because the db container does not expose 5435, you are unable to connect using port 5435.
If you were to try to connect to the database from your host machine for example, you would probably be successful using port 5435 and localhost.
When I run my docker-compose, I face this error:
Error message
app | 2021-09-23 11:52:51.860 ERROR 1 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization.
app |
app | org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
app | at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:280) ~[postgresql-42.2.5.jar!/:42.2.5]
app | at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) ~[postgresql-42.2.5.jar!/:42.2.5]
app | at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195) ~[postgresql-42.2.5.jar!/:42.2.5]
I tried to change the url in my application.properties file and the error is the same.
This is my application.properties file:
application.properties
spring.datasource.url=jdbc:postgresql://postgres:5432/employees
spring.datasource.username=postgres
spring.datasource.password=root
spring.jpa.show-sql=true
server.port=8086
## Hibernate Properties
# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
# Hibernate ddl auto (create, create-drop, validate, update)
spring.jpa.hibernate.ddl-auto = update
`
This is my Dockerfile:
Dockerfile
from openjdk:8
copy ./target/springboot2-postgresql-jpa-hibernate-crud-example-0.0.1-SNAPSHOT.jar employee-jdbc-0.0.1-SNAPSHOT.jar
CMD ["java","-jar","employee-jdbc-0.0.1-SNAPSHOT.jar"]
And this is my docker-compose:
docker-compose
version: "3"
services:
postgres:
image: postgres:latest
network_mode: bridge
container_name: postgres
volumes:
- postgres-data:/var/lib/postgresql/data
expose:
- 5430
ports:
- 5430:5430
environment:
- POSTGRES_PASSWORD=root
- POSTGRES_USER=postgres
- POSTGRES_DB=employees
# APP*****************************************
app:
image: app
network_mode: bridge
container_name: app
expose:
- 8081
ports:
- 8081:8081
depends_on:
- postgres
links:
- postgres
environment:
- POSTGRES_PASSWORD=root
- POSTGRES_USER=postgres
- POSTGRES_DB=employees
- POSTGRES_URL=jdbc:postgresql://postgres:5432/employees
volumes:
postgres-data:
Can you run this command to check
sudo lsof -n -u postgres |grep LISTEN or sudo netstat -ltnp | grep postgres
should show the TCP/IP addresses and ports PostgreSQL is listening on
It can be postgres is already running and your image is not running using docker. Maybe killing the process will help you if it is already running.
I have the following docker-compose.yml file
version: '3'
services:
postgres:
image: 'postgres:latest'
redis:
image: 'redis:latest'
nginx:
restart: always
build:
dockerfile: Dockerfile.dev
context: ./nginx
ports:
- '3050:80'
api:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
client:
build:
dockerfile: Dockerfile.dev
context: ./client
volumes:
- /app/node_modules
- ./client:/app
worker:
build:
dockerfile: Dockerfile.dev
context: ./worker
volumes:
- /app/node_modules
- ./worker:/app
When I run docker-compose up --build, everything works fine apart from postgres, which provides me with the error below. I've tried applying the accepted answer for this question but the error message won't go away; not sure what I'm missing here.
postgres_1 | Error: Database is uninitialized and superuser password is not specified.
postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
postgres_1 |
postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
postgres_1 | connections without a password. This is *not* recommended.
postgres_1 |
postgres_1 | See PostgreSQL documentation about "trust":
postgres_1 | https://www.postgresql.org/docs/current/auth-trust.html
I've got this keys file inside of my server folder, which I think was used on the course I'm following to provide the environment parameters of the api in the docker-compose file:
module.exports = {
redisHost: process.env.REDIS_HOST,
redisPort: process.env.REDIS_PORT,
pgUser: process.env.PGUSER,
pgHost: process.env.PGHOST,
pgDatabase: process.env.PGDATABASE,
pgPassword: process.env.PGPASSWORD,
pgPort: process.env.PGPORT,
}
As per documentation, the postgres image needs the environment variable POSTGRES_PASSWORD to be set.
Something like this:
postgres:
image: 'postgres:latest'
environment:
POSTGRES_PASSWORD: example
There's not much more to it. You can try to create a docker-compose.yml with just that code and see if it starts correctly with docker compose up (it should).
If it still doesn't there might be something wrong with the cached postgres:latest image.
Might wanna try a docker image rm postgres and then calling the docker compose command again.
Warning: make sure you didn't have important data in the db before doing that.
I try to mapping folder on the host to postgres container in order to save my data even if container destroy.
Firstly I created new directory pg-data
Secondly I describe container in docker-compose.yml
version: '3.7'
networks:
backend-network:
driver: bridge
frontend-network:
driver: bridge
volumes:
redis-data:
pg-data:
services:
&app-service app: &app-service-template
container_name: k4fntr_app
build:
context: ./docker/php-fpm
args:
UID: ${UID?Use your user ID}
GID: ${GID?Use your group ID}
USER: ${USER?Use your user name}
user: "${UID}:${GID}"
hostname: *app-service
volumes:
- /etc/passwd/:/etc/passwd:ro
- /etc/group/:/etc/group:ro
- ./:/var/www/k4fntr
environment:
APP_ENV: "${APP_ENV}"
CONTAINER_ROLE: app
FPM_PORT: &php-fpm-port 9000
FPM_USER: "${UID:-1000}"
FPM_GROUP: "${GID:-1000}"
depends_on:
- redis
- database
networks:
- backend-network
&queue-service queue:
<<: *app-service-template
container_name: k4fntr_queue
restart: always
hostname: *queue-service
depends_on:
- app
environment:
CONTAINER_ROLE: queue
&schedule-service schedule:
<<: *app-service-template
container_name: k4fntr_schedule
restart: always
hostname: *schedule-service
depends_on:
- app
environment:
CONTAINER_ROLE: scheduler
&php-fpm-service php-fpm:
<<: *app-service-template
container_name: k4fntr_php-fpm
user: 'root:root'
restart: always
hostname: *php-fpm-service
ports: [*php-fpm-port]
entrypoint: /fpm-entrypoint.sh
command: php-fpm --nodaemonize -d "opcache.enable=0" -d "display_startup_errors=On" -d "display_errors=On" -d "error_reporting=E_ALL"
networks:
- backend-network
- frontend-network
mail:
container_name: k4fntr_mail
image: mailhog/mailhog
ports:
- "1025:1025"
- "8025:8025"
- backend-network
database:
container_name: k4fntr_database
build: ./docker/postgres
restart: always
environment:
ENV: ${APP_ENV}
TESTING_DB: ${DB_DATABASE_TESTING}
POSTGRES_DB: ${DB_DATABASE}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_PASSWORD: ${DB_PASSWORD}
ports:
- "15432:5432"
volumes:
- ./docker/postgres/prod/:/prod
- ./docker/postgres/pg-data:/var/lib/postgresql/data:Z
networks:
- backend-network
- backend-network
database:
container_name: k4fntr_database
build: ./docker/postgres
restart: always
environment:
ENV: ${APP_ENV}
TESTING_DB: ${DB_DATABASE_TESTING}
POSTGRES_DB: ${DB_DATABASE}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_PASSWORD: ${DB_PASSWORD}
ports:
- "15432:5432"
volumes:
- ./docker/postgres/prod/:/prod
- ./docker/postgres/pg-data:/var/lib/postgresql/data:rw
networks:
- backend-network
- backend-network
nginx:
container_name: k4fntr_nginx
image: nginx
volumes:
- ./docker/nginx/config:/etc/nginx/conf.d
- ./:/var/www/k4fntr
depends_on:
- *php-fpm-service
ports:
- "8084:80"
networks:
- frontend-network
redis:
container_name: k4fntr_redis
image: redis
restart: always
command: redis-server
volumes:
- ./docker/redis/config/redis.conf:/usr/local/etc/redis/redis.conf
- ./docker/redis/redis-data:/data:rw
ports:
- "16379:6379"
networks:
- backend-network
database:
container_name: k4fntr_database
build: ./docker/postgres
restart: always
environment:
ENV: ${APP_ENV}
TESTING_DB: ${DB_DATABASE_TESTING}
POSTGRES_DB: ${DB_DATABASE}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_PASSWORD: ${DB_PASSWORD}
ports:
- "15432:5432"
volumes:
- ./docker/postgres/prod/:/prod
- ./docker/postgres/pg-data:/var/lib/postgresql/data:Z
networks:
- backend-network
I also have Dockerfile in docker/postgres/ which is contains the next code
FROM postgres:10.5-alpine
COPY /docker-entrypoint-initdb.d/ /docker-entrypoint-initdb.d/
RUN apk add openssh-client
RUN chmod a+r /docker-entrypoint-initdb.d/*
RUN mkdir /prod
RUN chmod a+r /prod
Then I build my image with docker-compose exec -d --build
After it my user changes on 70:root
and I can't doing any with this folder without sudo. Also when I try to rebuild the container I got an error
/usr/local/bin/docker-compose up -d --build
Building database
Traceback (most recent call last):
File "bin/docker-compose", line 6, in <module>
File "compose/cli/main.py", line 72, in main
File "compose/cli/main.py", line 128, in perform_command
File "compose/cli/main.py", line 1077, in up
File "compose/cli/main.py", line 1073, in up
File "compose/project.py", line 548, in up
File "compose/service.py", line 351, in ensure_image_exists
File "compose/service.py", line 1106, in build
File "site-packages/docker/api/build.py", line 160, in build
File "site-packages/docker/utils/build.py", line 30, in tar
File "site-packages/docker/utils/build.py", line 49, in exclude_paths
File "site-packages/docker/utils/build.py", line 214, in rec_walk
File "site-packages/docker/utils/build.py", line 184, in rec_walk
PermissionError: [Errno 13] Permission denied: '/home/ubuntu/PhpstormProjects/fntr/docker/postgres/pg-data'
[16332] Failed to execute script docker-compose
And I need to remove this folder (with sudo) and then do rebuild.
What I tried:
1 - I have read the instruction from https://hub.docker.com/_/postgres (Arbitrary --user Notes) and tried to volume /etc/passwd inside the container and user: "${UID}:${GID}", but got an error
initdb: could not access directory "/var/lib/postgresql/data": permission denied
2 - use PGDATA into another place and got the same error with rebuild
3 - create /var/lib/postgresql/data in Dockerfile with 777 permissions, but postgres entrypoints from image just removed this folder and recreated it
I really need help with this issue because I spent 3 days without effect
This is what have worked for me on manjaro linux.
First I need to manually create pgdata folder. If I omit this step, docker will create pgdata with the wrong permission.
mkdir -p /some/path/pddata
Next I create the docker container and map my user with the --user argument:
docker container run -d --name=pg -p 5432:5432 --user $(id -u):$(id -g) -e POSTGRES_PASSWORD=postgres -e PGDATA=/pgdata -v /host/path/to/pgdata:/pgdata postgres:latest
The following works for me, perhaps you need to start with a simpler set up that works and find out what causes a problem as you move towards your complete config:
version: '3.7'
volumes:
pg-data:
services:
database:
image: postgres:latest
restart: always
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
PGDATA: /pg-data
ports:
- "5432:5432"
volumes:
- pg-data:/pg-data
So ...
########################################
# start db
########################################
▶ docker-compose up -d
Creating docker-compose-pg_database_1 ... done
########################################
# note pg-data is created
########################################
▶ ls
docker-compose.yml pg-data
########################################
# create some data
########################################
▶ docker-compose exec database psql -U postgres
psql (12.2 (Debian 12.2-2.pgdg100+1))
Type "help" for help.
postgres=# create database foo;
CREATE DATABASE
postgres=# \q
########################################
# remove the container and any non-mounted data
########################################
▶ docker-compose rm -sf
Stopping docker-compose-pg_database_1 ... done
Going to remove docker-compose-pg_database_1
Removing docker-compose-pg_database_1 ... done
########################################
# start the container again
########################################
▶ docker-compose up -d
Creating docker-compose-pg_database_1 ... done
########################################
# see data from before
########################################
▶ docker-compose exec database psql -U postgres
psql (12.2 (Debian 12.2-2.pgdg100+1))
Type "help" for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
foo | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
Want to use docker-compose to run api application and postgresql database together.
docker-compose file:
version: '3'
volumes:
database_data:
driver: local
services:
db:
image: postgres:latest
volumes:
- database_data:/var/lib/postgresql/data
api:
build: ./api
expose:
- 8080
ports:
- 8080:8080
volumes:
- ./api:/usr/src/app/
links:
- db
environment:
- PGHOST=db
- PGDATABASE=postgres
- PGUSER=postgres
Api main.go file:
func main() {
db, err = gorm.Open("postgres", "host=db port=5432 user=postgres dbname=postgres")
// ...
}
When run the services, got message from log:
api_1 | [GIN] 2018/06/22 - 07:31:10 | 404 | 1.4404ms | 172.20.0.1 | GET /posts
api_1 |
api_1 | (sql: database is closed)
api_1 | [2018-06-22 07:31:10]
api_1 |
api_1 | (sql: database is closed)
api_1 | [2018-06-22 07:31:10]
api_1 | [GIN] 2018/06/22 - 07:32:14 | 403 | 15.6µs | 172.20.0.1 | GET /posts
db_1 | 2018-06-22 07:34:27.296 UTC [81] FATAL: role "root" does not exist
db_1 | 2018-06-22 07:34:36.897 UTC [90] FATAL: role "root" does not exist
Does this way not good? host=db in the connection string? Since db is the docker compose service name.
Add
It can work:
https://docs.docker.com/samples/library/postgres/#-or-via-psql
In the link you provided there are configuration settings that you did not added
restart: always
environment:
POSTGRES_PASSWORD: example
You should try this
version: '3'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
environment:
POSTGRES_PASSWORD: 'postgres'
volumes:
- database_data:/var/lib/postgresql/data
api:
build: ./api
expose:
- 8080
ports:
- 8080:8080
volumes:
- ./api:/usr/src/app/
links:
- db
environment:
- PGHOST: 'db'
- PGDATABASE: 'postgres'
- PGUSER: 'postgres'
- PGPASSWORD: 'postgres'
volumes:
database_data:
driver: local
and
db, err := gorm.Open("postgres", "host=db port=5432 user=postgres dbname=postgres password=postgres")