How to connect to PostgreSQL using docker-compose? - postgresql

Want to use docker-compose to run api application and postgresql database together.
docker-compose file:
version: '3'
volumes:
database_data:
driver: local
services:
db:
image: postgres:latest
volumes:
- database_data:/var/lib/postgresql/data
api:
build: ./api
expose:
- 8080
ports:
- 8080:8080
volumes:
- ./api:/usr/src/app/
links:
- db
environment:
- PGHOST=db
- PGDATABASE=postgres
- PGUSER=postgres
Api main.go file:
func main() {
db, err = gorm.Open("postgres", "host=db port=5432 user=postgres dbname=postgres")
// ...
}
When run the services, got message from log:
api_1 | [GIN] 2018/06/22 - 07:31:10 | 404 | 1.4404ms | 172.20.0.1 | GET /posts
api_1 |
api_1 | (sql: database is closed)
api_1 | [2018-06-22 07:31:10]
api_1 |
api_1 | (sql: database is closed)
api_1 | [2018-06-22 07:31:10]
api_1 | [GIN] 2018/06/22 - 07:32:14 | 403 | 15.6µs | 172.20.0.1 | GET /posts
db_1 | 2018-06-22 07:34:27.296 UTC [81] FATAL: role "root" does not exist
db_1 | 2018-06-22 07:34:36.897 UTC [90] FATAL: role "root" does not exist
Does this way not good? host=db in the connection string? Since db is the docker compose service name.
Add
It can work:
https://docs.docker.com/samples/library/postgres/#-or-via-psql

In the link you provided there are configuration settings that you did not added
restart: always
environment:
POSTGRES_PASSWORD: example
You should try this
version: '3'
services:
db:
image: postgres:latest
restart: always
ports:
- 5432:5432
environment:
POSTGRES_PASSWORD: 'postgres'
volumes:
- database_data:/var/lib/postgresql/data
api:
build: ./api
expose:
- 8080
ports:
- 8080:8080
volumes:
- ./api:/usr/src/app/
links:
- db
environment:
- PGHOST: 'db'
- PGDATABASE: 'postgres'
- PGUSER: 'postgres'
- PGPASSWORD: 'postgres'
volumes:
database_data:
driver: local
and
db, err := gorm.Open("postgres", "host=db port=5432 user=postgres dbname=postgres password=postgres")

Related

Postgres inside docker compose refusing connection from my server

I have a Golang server alongside Postgres instance running inside docker compose. For some reason the Postgres is refusing connection. From all of my previous searches, usually the problem is typo, not exposing the port, having SSL and so on, but I don't have anything like that going on and still having this issue
version: "3.2"
services:
ingress:
image: jwilder/nginx-proxy
ports:
- "3000:80"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
auth-service:
depends_on:
- rabbitmq
- auth-db
- ingress
build: ./auth
container_name: "auth-service"
ports:
- 3001:3000
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_HOST=auth-db
- POSTGRES_DB=auth-dev
- POSTGRES_PORT=5435
- PORT=3000
- RABBITMQ_USER=guest
- RABBITMQ_PASSWORD=guest
- RABBITMQ_HOST=rabbitmq
- RABBITMQ_PORT=5672
- VIRTUAL_HOST=api.twitchy.dev
- VIRTUAL_PATH=/v1/auth/
deploy:
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
# networks:
# - rabbitmq_net
# - default
rabbitmq:
image: rabbitmq:3-management-alpine
container_name: "rabbitmq"
ports:
- 5672:5672
- 15672:15672
volumes:
- rabbitmq_data:/var/lib/rabbitmq/
- rabbitmq_log:/var/log/rabbitmq/
# networks:
# - rabbitmq_net
auth-db:
image: postgres:14.1-alpine
restart: always
container_name: "auth-db"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=auth-dev
ports:
- "5435:5432"
volumes:
- db:/var/lib/postgresql/data
chat-db:
image: postgres:14.1-alpine
restart: always
container_name: "chat-db"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=chat-dev
ports:
- "5434:5432"
volumes:
- db:/var/lib/postgresql/data
# networks:
# rabbitmq_net:
# driver: bridge
volumes:
db:
driver: local
rabbitmq_data:
rabbitmq_log:
This is the error I am getting
auth-service | Retrying connection to database...
auth-service | failed to connect to `host=auth-db user=postgres database=auth-dev`: dial error (dial tcp 172.23.0.3:5435: connect: connection refused)
And my Golang code used to connect to the DB (using pgx)
dbUrl := fmt.Sprintf("postgres://%s:%s#%s:%s/%s?sslmode=disable",
os.Getenv("POSTGRES_USER"),
os.Getenv("POSTGRES_PASSWORD"),
os.Getenv("POSTGRES_HOST"),
os.Getenv("POSTGRES_PORT"),
os.Getenv("POSTGRES_DB"))
This is why I am confused
The ports match up, I expose 5435 from postgres, and I connect to 5435
The host should be correct as I am referencing the auth-db service name and they are on the same default network so that should be fine
The password and username match up
The POSTGRES_DB also match up, the default database should be auth-dev
POSTGRES_DB
This optional environment variable can be used to define a different name for the default database that is created when the image is first started. If it is not specified, then the value of POSTGRES_USER will be used.
I have sslmode disable as well
Is there anything else that can cause the connection to be refused?
Tried changing db to template1 and postgres as they are created by default but both aren't working either
54511e50369c postgres:14.1-alpine "docker-entrypoint.s…" 16 minutes ago Up 16 seconds 0.0.0.0:5435->5432/tcp, :::5435->5432/tcp auth-db
docker exec -it 54511e50369c psql -U postgres
psql (14.1)
Type "help" for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
The database is ready when I am trying to connect (I am retrying 20 times so, and restarting the service if it crashes, so it should be available)
When you map ports in docker-compose, say like "5435:5432", you are mapping port 5435 on the HOST machine to 5432 on the CONTAINER. However, your db url in the auth-service definition is using the name of the service, auth-db, so you are actually hitting the db container directly, not going through the host machine. Because the db container does not expose 5435, you are unable to connect using port 5435.
If you were to try to connect to the database from your host machine for example, you would probably be successful using port 5435 and localhost.

airflow runs connections check against metadata db before init db

I am running airflow locally based on dockerfile, .env, docker-compose.yaml and entrypoint.sh
as "docker-compose -f docker-compose.yaml up"
And just after "airflow init db" in entrypoint.sh I am getting the following error:
[after it all is cool, I can run airflow. But this drives me crazy. Can anyone help me to resolve it, please?]
what's strange is that service queries the tables even before the db has been initiated
airflow_webserver | initiating db
airflow_webserver | DB: postgresql://airflow:***#airflow_metadb:5432/airflow
airflow_webserver | [2022-02-22 13:52:26,318] {db.py:929} INFO - Dropping tables that exist
airflow_webserver | [2022-02-22 13:52:26,570] {migration.py:201} INFO - Context impl PostgresqlImpl.
airflow_webserver | [2022-02-22 13:52:26,570] {migration.py:204} INFO - Will assume transactional DDL.
airflow_metadb | 2022-02-22 13:52:26.712 UTC [71] ERROR: relation "connection" does not exist at character 55
airflow_metadb | 2022-02-22 13:52:26.712 UTC [71] STATEMENT: SELECT connection.conn_id AS connection_conn_id
airflow_metadb | FROM connection GROUP BY connection.conn_id
airflow_metadb | HAVING count(*) > 1
airflow_metadb | 2022-02-22 13:52:26.714 UTC [72] ERROR: relation "connection" does not exist at character 55
airflow_metadb | 2022-02-22 13:52:26.714 UTC [72] STATEMENT: SELECT connection.conn_id AS connection_conn_id
airflow_metadb | FROM connection
airflow_metadb | WHERE connection.conn_type IS NULL
airflow_webserver | [2022-02-22 13:52:26,733] {db.py:921} INFO - Creating tables
airflow 2.2.3
postgres 13
in dockerfile:
ENTRYPOINT ["/entrypoint.sh"]
in docker-compose.yaml:
webserver:
env_file: ./.env
image: airflow
container_name: airflow_webserver
restart: always
depends_on:
- postgres
environment:
<<: *env_common
AIRFLOW__CORE__LOAD_EXAMPLES: ${AIRFLOW__CORE__LOAD_EXAMPLES}
AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: ${AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION}
EXECUTOR: ${EXECUTOR}
_AIRFLOW_DB_UPGRADE: ${_AIRFLOW_DB_UPGRADE}
_AIRFLOW_WWW_USER_CREATE: ${_AIRFLOW_WWW_USER_CREATE}
_AIRFLOW_WWW_USER_USERNAME: ${_AIRFLOW_WWW_USER_USERNAME}
_AIRFLOW_WWW_USER_PASSWORD: ${_AIRFLOW_WWW_USER_PASSWORD}
_AIRFLOW_WWW_USER_ROLE: ${_AIRFLOW_WWW_USER_ROLE}
_AIRFLOW_WWW_USER_EMAIL: ${_AIRFLOW_WWW_USER_EMAIL}
logging:
options:
max-size: 10m
max-file: "3"
volumes:
- ./dags:bla-bla
- ./logs:bla-bla
ports:
- ${AIRFLOW_WEBSERVER_PORT}:${AIRFLOW_WEBSERVER_PORT}
command: webserver
healthcheck:
test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
interval: 30s
timeout: 30s
retries: 3

docker compose error "Connection to localhost:5432 refused."

When I run my docker-compose, I face this error:
Error message
app | 2021-09-23 11:52:51.860 ERROR 1 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization.
app |
app | org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
app | at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:280) ~[postgresql-42.2.5.jar!/:42.2.5]
app | at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) ~[postgresql-42.2.5.jar!/:42.2.5]
app | at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195) ~[postgresql-42.2.5.jar!/:42.2.5]
I tried to change the url in my application.properties file and the error is the same.
This is my application.properties file:
application.properties
spring.datasource.url=jdbc:postgresql://postgres:5432/employees
spring.datasource.username=postgres
spring.datasource.password=root
spring.jpa.show-sql=true
server.port=8086
## Hibernate Properties
# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
# Hibernate ddl auto (create, create-drop, validate, update)
spring.jpa.hibernate.ddl-auto = update
`
This is my Dockerfile:
Dockerfile
from openjdk:8
copy ./target/springboot2-postgresql-jpa-hibernate-crud-example-0.0.1-SNAPSHOT.jar employee-jdbc-0.0.1-SNAPSHOT.jar
CMD ["java","-jar","employee-jdbc-0.0.1-SNAPSHOT.jar"]
And this is my docker-compose:
docker-compose
version: "3"
services:
postgres:
image: postgres:latest
network_mode: bridge
container_name: postgres
volumes:
- postgres-data:/var/lib/postgresql/data
expose:
- 5430
ports:
- 5430:5430
environment:
- POSTGRES_PASSWORD=root
- POSTGRES_USER=postgres
- POSTGRES_DB=employees
# APP*****************************************
app:
image: app
network_mode: bridge
container_name: app
expose:
- 8081
ports:
- 8081:8081
depends_on:
- postgres
links:
- postgres
environment:
- POSTGRES_PASSWORD=root
- POSTGRES_USER=postgres
- POSTGRES_DB=employees
- POSTGRES_URL=jdbc:postgresql://postgres:5432/employees
volumes:
postgres-data:
Can you run this command to check
sudo lsof -n -u postgres |grep LISTEN or sudo netstat -ltnp | grep postgres
should show the TCP/IP addresses and ports PostgreSQL is listening on
It can be postgres is already running and your image is not running using docker. Maybe killing the process will help you if it is already running.

Pgadmin container kills postgres when using docker-compose

I run postgres using docker-compose -f postgres.yml up. When I try to run pgadmin4 using docker-compose -f pgadmin4.yml up, postgres container is automatically killed.
I am new to docker and doesn't understand why is this issue happening?
postgres.yml
version: "3.7"
services:
postgres:
container_name: "postgres"
image: "postgres"
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: ****
volumes:
- maple:/var/lib/postgresql/data
volumes:
maple:
name: maple
external: true
pgadmin.yml
version: "3.7"
services:
postgres:
container_name: "pgadmin4"
image: "dpage/pgadmin4"
ports:
- "5433:80"
environment:
PGADMIN_DEFAULT_EMAIL: ******
PGADMIN_DEFAULT_PASSWORD: *****
volumes:
- pgadmin4-vol:/var/lib/pgadmin
volumes:
pgadmin4-vol:
name: pgadmin4-vol
external: true
$ docker-compose -f psql.yml up
Starting postgres ... done
Attaching to postgres
postgres |
postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres |
postgres | 2020-11-27 10:18:05.635 UTC 1 LOG: starting PostgreSQL 13.0 (Debian 13.0-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres | 2020-11-27 10:18:05.636 UTC 1 LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres | 2020-11-27 10:18:05.636 UTC 1 LOG: listening on IPv6 address "::", port 5432
postgres | 2020-11-27 10:18:05.639 UTC 1 LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
postgres | 2020-11-27 10:18:05.643 UTC [24] LOG: database system was shut down at 2020-11-27 10:17:59 UTC
postgres | 2020-11-27 10:18:05.647 UTC 1 LOG: database system is ready to accept connections
postgres | 2020-11-27 10:18:11.089 UTC 1 LOG: received fast shutdown request
postgres | 2020-11-27 10:18:11.092 UTC 1 LOG: aborting any active transactions
postgres | 2020-11-27 10:18:11.099 UTC 1 LOG: background worker "logical replication launcher" (PID 30) exited with exit code 1
postgres | 2020-11-27 10:18:11.108 UTC [25] LOG: shutting down
postgres | 2020-11-27 10:18:11.127 UTC 1 LOG: database system is shut down
postgres exited with code 0
$ docker-compose -f pgadmin.yml up
Recreating postgres ... done
Attaching to pgadmin4
pgadmin4 | sudo: setrlimit(RLIMIT_CORE): Operation not permitted
pgadmin4 | [2020-11-27 10:18:13 +0000] 1 [INFO] Starting gunicorn 19.9.0
pgadmin4 | [2020-11-27 10:18:13 +0000] 1 [INFO] Listening at: http://[::]:80 (1)
pgadmin4 | [2020-11-27 10:18:13 +0000] 1 [INFO] Using worker: threads
pgadmin4 | /usr/local/lib/python3.8/os.py:1023: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
pgadmin4 | return io.open(fd, *args, **kwargs)
pgadmin4 | [2020-11-27 10:18:13 +0000] [87] [INFO] Booting worker with pid: 87
docker inpsect 69438bebc3f9 here
your services are both named postgres
so the pgadmin.yml recreates the service postgres, wich is your first database container.. as the logs are clearly showing
'Recreating postgres ... done'
take another name for the pgadmin service then postgres and it should work.
version: "3.7"
services:
pgadmin:
also docker-compose is for building stacks, if you would have done this in one compose file you would have seen the error from begin:
docker-compose.failing.yml:
version: "3.7"
services:
postgres:
container_name: "postgres"
image: "postgres"
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: test
volumes:
- maple:/var/lib/postgresql/data
postgres:
container_name: "pgadmin4"
image: "dpage/pgadmin4"
ports:
- "5433:80"
environment:
PGADMIN_DEFAULT_EMAIL: test
PGADMIN_DEFAULT_PASSWORD: test
volumes:
- pgadmin4-vol:/var/lib/pgadmin
volumes:
maple:
name: maple
external: true
pgadmin4-vol:
name: pgadmin4-vol
external: true
docker-compose.working.yml:
version: "3.7"
services:
postgres:
container_name: "postgres"
image: "postgres"
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: test
volumes:
- maple:/var/lib/postgresql/data
pgadmin:
container_name: "pgadmin4"
image: "dpage/pgadmin4"
ports:
- "5433:80"
environment:
PGADMIN_DEFAULT_EMAIL: test
PGADMIN_DEFAULT_PASSWORD: test
volumes:
- pgadmin4-vol:/var/lib/pgadmin
volumes:
maple:
name: maple
external: true
pgadmin4-vol:
name: pgadmin4-vol
external: true

Docker: no connection between postgres with node

I am trying to connect a postgres DB with Node api, with docker compose like this:
version: '3.8'
services:
db:
image: postgres
environment:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: biobox
volumes:
- ./db-data:/var/lib/postgresql/data
ports:
- 5432:5432
restart: always
api:
build: .
depends_on:
- db
ports:
- 3333:3333
command: yarn dev:server
env_file:
- .env
volumes:
- .:/usr/src/app
working_dir: /usr/src/app
env
DATABASE_USER=postgres
DATABASE_PASSWORD=postgres
DATABASE=biobox
DATABASE_HOST=db
DATABASE_PORT=5432
dockerfile
FROM node:12.16.2-alpine
COPY package*.json ./
RUN yarn
COPY . .
COPY .env .
EXPOSE 3333
My api and db are online on container and the logs show no error.
because when i try make a query return this:
Using ts-node version 8.8.2, typescript version 3.8.3
api_1 | Create a new user User {
api_1 | completName: 'Name',
api_1 | email: 'Name#gmail.com',
api_1 | cpf: '23425',
api_1 | dateNasc: '10/10/2020',
api_1 | number: 'Namefd222',
api_1 | password: '1Name5234'
api_1 | }
db_1 | 2020-05-31 16:37:00.199 UTC [74] LOG: unexpected EOF on client connection with an open transaction
I don't understand why this happens. My environment variables are in accordance with the docker-compose create, in this case the database, USER, PASS.