How do I run Percona PMM2 with docker-compose?
I can run PMM1 just fine. But PMM2 has absolutely zero documentation available and I can't seem to figure it out.
Sample docker-compose.yml file
db:
image: mariadb:10.4.13
ports:
- ${DB_PORT}:3306
volumes:
- db_data:/var/lib/mysql
tmpfs:
- /tmp/mysql-tmp
restart: always
environment:
MYSQL_ROOT_PASSWORD: "${DB_ROOT_PASS}"
MYSQL_USER: "${DB_USER}"
MYSQL_PASSWORD: "${DB_PASS}"
pmm-server:
image: percona/pmm-server:2.7
ports:
- 8100:80
environment:
SERVER_USER: "${PMM_USER}"
SERVER_PASSWORD: "${PMM_PASS}"
restart: always
volumes:
- pmm_data:/srv
pmm-client:
image: perconalab/pmm-client:2.7
environment:
PMM_AGENT_SERVER_ADDRESS: pmm-server:443
PMM_AGENT_SERVER_USERNAME: "${PMM_USER}"
PMM_AGENT_SERVER_PASSWORD: "${PMM_PASS}"
PMM_AGENT_SERVER_INSECURE_TLS: 1
DB_TYPE: mysql
DB_HOST: "${DB_HOST}"
DB_PORT: 3306
DB_USER: root
DB_PASSWORD: "${DB_ROOT_PASS}"
restart: always
depends_on:
- db
volumes:
db_data:
pmm_data:
DB_* ENV variables are form PMM1 config. I have no idea how to set DB configuration for PMM2's Docker image. But the pmm-client seems to be failing before that. With above config I get following log for pmm-client. I can't understand why am I getting this error: "Agent ID is not provided, halting"
pmm-client | Starting pmm-agent ...
pmm-client | INFO[2020-06-09T20:28:20.963+00:00] Using /usr/local/percona/pmm2/exporters/node_exporter component=main
pmm-client | INFO[2020-06-09T20:28:20.963+00:00] Using /usr/local/percona/pmm2/exporters/mysqld_exporter component=main
pmm-client | INFO[2020-06-09T20:28:20.963+00:00] Using /usr/local/percona/pmm2/exporters/mongodb_exporter component=main
pmm-client | INFO[2020-06-09T20:28:20.963+00:00] Using /usr/local/percona/pmm2/exporters/postgres_exporter component=main
pmm-client | INFO[2020-06-09T20:28:20.963+00:00] Using /usr/local/percona/pmm2/exporters/proxysql_exporter component=main
pmm-client | INFO[2020-06-09T20:28:20.964+00:00] Using /usr/local/percona/pmm2/exporters/rds_exporter component=main
pmm-client | INFO[2020-06-09T20:28:20.964+00:00] Starting... component=client
pmm-client | INFO[2020-06-09T20:28:20.964+00:00] Starting local API server on http://127.0.0.1:7777/ ... component=local-server/JSON
pmm-client | ERRO[2020-06-09T20:28:20.964+00:00] Agent ID is not provided, halting. component=client
pmm-client | INFO[2020-06-09T20:28:20.966+00:00] Started. component=local-server/JSON
Can anyone push me to the right direction with this?
I can't understand why there isn't any documentation available for it neither. For PMM1 docker image I found some but for PMM2 there's absolutely nothing.
They are releasing this image publicly but no info on how to use it.
Related
I have a Golang server alongside Postgres instance running inside docker compose. For some reason the Postgres is refusing connection. From all of my previous searches, usually the problem is typo, not exposing the port, having SSL and so on, but I don't have anything like that going on and still having this issue
version: "3.2"
services:
ingress:
image: jwilder/nginx-proxy
ports:
- "3000:80"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
auth-service:
depends_on:
- rabbitmq
- auth-db
- ingress
build: ./auth
container_name: "auth-service"
ports:
- 3001:3000
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_HOST=auth-db
- POSTGRES_DB=auth-dev
- POSTGRES_PORT=5435
- PORT=3000
- RABBITMQ_USER=guest
- RABBITMQ_PASSWORD=guest
- RABBITMQ_HOST=rabbitmq
- RABBITMQ_PORT=5672
- VIRTUAL_HOST=api.twitchy.dev
- VIRTUAL_PATH=/v1/auth/
deploy:
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
# networks:
# - rabbitmq_net
# - default
rabbitmq:
image: rabbitmq:3-management-alpine
container_name: "rabbitmq"
ports:
- 5672:5672
- 15672:15672
volumes:
- rabbitmq_data:/var/lib/rabbitmq/
- rabbitmq_log:/var/log/rabbitmq/
# networks:
# - rabbitmq_net
auth-db:
image: postgres:14.1-alpine
restart: always
container_name: "auth-db"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=auth-dev
ports:
- "5435:5432"
volumes:
- db:/var/lib/postgresql/data
chat-db:
image: postgres:14.1-alpine
restart: always
container_name: "chat-db"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=chat-dev
ports:
- "5434:5432"
volumes:
- db:/var/lib/postgresql/data
# networks:
# rabbitmq_net:
# driver: bridge
volumes:
db:
driver: local
rabbitmq_data:
rabbitmq_log:
This is the error I am getting
auth-service | Retrying connection to database...
auth-service | failed to connect to `host=auth-db user=postgres database=auth-dev`: dial error (dial tcp 172.23.0.3:5435: connect: connection refused)
And my Golang code used to connect to the DB (using pgx)
dbUrl := fmt.Sprintf("postgres://%s:%s#%s:%s/%s?sslmode=disable",
os.Getenv("POSTGRES_USER"),
os.Getenv("POSTGRES_PASSWORD"),
os.Getenv("POSTGRES_HOST"),
os.Getenv("POSTGRES_PORT"),
os.Getenv("POSTGRES_DB"))
This is why I am confused
The ports match up, I expose 5435 from postgres, and I connect to 5435
The host should be correct as I am referencing the auth-db service name and they are on the same default network so that should be fine
The password and username match up
The POSTGRES_DB also match up, the default database should be auth-dev
POSTGRES_DB
This optional environment variable can be used to define a different name for the default database that is created when the image is first started. If it is not specified, then the value of POSTGRES_USER will be used.
I have sslmode disable as well
Is there anything else that can cause the connection to be refused?
Tried changing db to template1 and postgres as they are created by default but both aren't working either
54511e50369c postgres:14.1-alpine "docker-entrypoint.s…" 16 minutes ago Up 16 seconds 0.0.0.0:5435->5432/tcp, :::5435->5432/tcp auth-db
docker exec -it 54511e50369c psql -U postgres
psql (14.1)
Type "help" for help.
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
The database is ready when I am trying to connect (I am retrying 20 times so, and restarting the service if it crashes, so it should be available)
When you map ports in docker-compose, say like "5435:5432", you are mapping port 5435 on the HOST machine to 5432 on the CONTAINER. However, your db url in the auth-service definition is using the name of the service, auth-db, so you are actually hitting the db container directly, not going through the host machine. Because the db container does not expose 5435, you are unable to connect using port 5435.
If you were to try to connect to the database from your host machine for example, you would probably be successful using port 5435 and localhost.
When I run my docker-compose, I face this error:
Error message
app | 2021-09-23 11:52:51.860 ERROR 1 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization.
app |
app | org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
app | at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:280) ~[postgresql-42.2.5.jar!/:42.2.5]
app | at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) ~[postgresql-42.2.5.jar!/:42.2.5]
app | at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195) ~[postgresql-42.2.5.jar!/:42.2.5]
I tried to change the url in my application.properties file and the error is the same.
This is my application.properties file:
application.properties
spring.datasource.url=jdbc:postgresql://postgres:5432/employees
spring.datasource.username=postgres
spring.datasource.password=root
spring.jpa.show-sql=true
server.port=8086
## Hibernate Properties
# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
# Hibernate ddl auto (create, create-drop, validate, update)
spring.jpa.hibernate.ddl-auto = update
`
This is my Dockerfile:
Dockerfile
from openjdk:8
copy ./target/springboot2-postgresql-jpa-hibernate-crud-example-0.0.1-SNAPSHOT.jar employee-jdbc-0.0.1-SNAPSHOT.jar
CMD ["java","-jar","employee-jdbc-0.0.1-SNAPSHOT.jar"]
And this is my docker-compose:
docker-compose
version: "3"
services:
postgres:
image: postgres:latest
network_mode: bridge
container_name: postgres
volumes:
- postgres-data:/var/lib/postgresql/data
expose:
- 5430
ports:
- 5430:5430
environment:
- POSTGRES_PASSWORD=root
- POSTGRES_USER=postgres
- POSTGRES_DB=employees
# APP*****************************************
app:
image: app
network_mode: bridge
container_name: app
expose:
- 8081
ports:
- 8081:8081
depends_on:
- postgres
links:
- postgres
environment:
- POSTGRES_PASSWORD=root
- POSTGRES_USER=postgres
- POSTGRES_DB=employees
- POSTGRES_URL=jdbc:postgresql://postgres:5432/employees
volumes:
postgres-data:
Can you run this command to check
sudo lsof -n -u postgres |grep LISTEN or sudo netstat -ltnp | grep postgres
should show the TCP/IP addresses and ports PostgreSQL is listening on
It can be postgres is already running and your image is not running using docker. Maybe killing the process will help you if it is already running.
I have the following docker-compose.yml file
version: '3'
services:
postgres:
image: 'postgres:latest'
redis:
image: 'redis:latest'
nginx:
restart: always
build:
dockerfile: Dockerfile.dev
context: ./nginx
ports:
- '3050:80'
api:
build:
dockerfile: Dockerfile.dev
context: ./server
volumes:
- /app/node_modules
- ./server:/app
environment:
- REDIS_HOST=redis
- REDIS_PORT=6379
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
client:
build:
dockerfile: Dockerfile.dev
context: ./client
volumes:
- /app/node_modules
- ./client:/app
worker:
build:
dockerfile: Dockerfile.dev
context: ./worker
volumes:
- /app/node_modules
- ./worker:/app
When I run docker-compose up --build, everything works fine apart from postgres, which provides me with the error below. I've tried applying the accepted answer for this question but the error message won't go away; not sure what I'm missing here.
postgres_1 | Error: Database is uninitialized and superuser password is not specified.
postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
postgres_1 |
postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
postgres_1 | connections without a password. This is *not* recommended.
postgres_1 |
postgres_1 | See PostgreSQL documentation about "trust":
postgres_1 | https://www.postgresql.org/docs/current/auth-trust.html
I've got this keys file inside of my server folder, which I think was used on the course I'm following to provide the environment parameters of the api in the docker-compose file:
module.exports = {
redisHost: process.env.REDIS_HOST,
redisPort: process.env.REDIS_PORT,
pgUser: process.env.PGUSER,
pgHost: process.env.PGHOST,
pgDatabase: process.env.PGDATABASE,
pgPassword: process.env.PGPASSWORD,
pgPort: process.env.PGPORT,
}
As per documentation, the postgres image needs the environment variable POSTGRES_PASSWORD to be set.
Something like this:
postgres:
image: 'postgres:latest'
environment:
POSTGRES_PASSWORD: example
There's not much more to it. You can try to create a docker-compose.yml with just that code and see if it starts correctly with docker compose up (it should).
If it still doesn't there might be something wrong with the cached postgres:latest image.
Might wanna try a docker image rm postgres and then calling the docker compose command again.
Warning: make sure you didn't have important data in the db before doing that.
Trying to start up a Spring Boot application that connects to two PostgreSQL databases using docker-compose and I am getting a connection refused when Spring tries to connect to either of these databases.
My configuration is as follows:
docker-compose.yml
version: '3.2'
services:
mydb-1:
container_name: mydb-1
image: mydb-1
ports:
- '5432:5432'
environment:
- POSTGRES_PASSWORD=postgres
- POSTGRES_USER=postgres
- POSTGRES_DB=testdb1
mydb-2:
container_name: mydb-2
image: mydb-2
ports:
- '5433:5432'
environment:
- POSTGRES_PASSWORD=postgres
- POSTGRES_USER=postgres
- POSTGRES_DB=testdb2
my-server:
image: my-server-spring
restart: on-failure
depends_on:
- mydb-1
- mydb-2
environment:
- SPRING_DB1-DATASOURCE_JDBC-URL=jdbc:postgresql://mydb-1:5432/testdb1
- SPRING_DB2-DATASOURCE_JDBC-URL=jdbc:postgresql://mydb-2:5433/testdb2
expose:
- '8080'
ports:
- '8080:8080'
Spring application.properties
spring.db1-datasource.jdbc-url= jdbc:postgresql://localhost:5432/testdb1
spring.db1-datasource.username= postgres
spring.db1-datasource.password= postgres
spring.db1-datasource.driverClassName= org.postgresql.Driver
spring.db2-datasource.jdbc-url= jdbc:postgresql://localhost:5433/testdb2
spring.db2-datasource.username= postgres
spring.db2-datasource.password= postgres
spring.db2-datasource.driverClassName= org.postgresql.Driver
Stacktrace (Part of)
org.postgresql.util.PSQLException: Connection to testdb2:5433 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
advidi-server_1 | at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:285) ~[postgresql-42.2.14.jar!/:42.2.14]
advidi-server_1 | at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49) ~[postgresql-42.2.14.jar!/:42.2.14]
advidi-server_1 | at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:217) ~[postgresql-42.2.14.jar!/:42.2.14]
advidi-server_1 | at org.postgresql.Driver.makeConnection(Driver.java:458) ~[postgresql-42.2.14.jar!/:42.2.14]
advidi-server_1 | at org.postgresql.Driver.connect(Driver.java:260) ~[postgresql-42.2.14.jar!/:42.2.14]
advidi-server_1 | at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138) ~[HikariCP-3.4.5.jar!/:na]
advidi-server_1 | at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:358) ~[HikariCP-3.4.5.jar!/:na]
advidi-server_1 | at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206) ~[HikariCP-3.4.5.jar!/:na]
advidi-server_1 | at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:477) ~[HikariCP-3.4.5.jar!/:na]
advidi-server_1 | at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:560) ~[HikariCP-3.4.5.jar!/:na]
advidi-server_1 | at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115) ~[HikariCP-3.4.5.jar!/:na]
advidi-server_1 | at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) ~[HikariCP-3.4.5.jar!/:na]
advidi-server_1 | at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122) ~[hibernate-core-5.4.18.Final.jar!/:5.4.18.Final]
advidi-server_1 | at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator$ConnectionProviderJdbcConnectionAccess.obtainConnection(JdbcEnvironmentInitiator.java:180) ~[hibernate-core-5.4.18.Final.jar!/:5.4.18.Final]
Initially I thought that the problem was that the Spring application was trying to connect to the dbs before their bootstrapping is complete, but that doesn't seem to be the case since with restart: on-failure it should at some point manage to connect.
The default localhost values should also not be a problem since these are replaced by the environment variables in my docker-compose file.
Any ideas?
You're not showing the mydb-1 and mydb-2 image configurations, so I'm practically guessing here.
No need to map the ports to the host. Remove the ports: entries.
The spring container should use port 5432 in order to connect to both DB containers. Each DB container has its own IP so there's no problem with that. Don't connect to localhost because that's the container itself.
No need to specify :5432 in the connection strings because that's the default.
Seems like you have connection settings both in application.properties and the environment: entry in the server configuration. Which is the application using? Anyway, your connection strings should end with mydb-1/testdb1 and mydb-2/testdb2 - nothing more complicated than that.
You're passing SPRING_DB1-DATASOURCE_JDBC-URL in your environment, and not using it in the application.properties. So the variable are not overriding.
You need to either in application.properties use something like
spring.db1-datasource.jdbc-url=$SPRING_DB1-DATASOURCE_JDBC-URL
OR
in environment set the exact name of the variables in your application. properties
- spring.db1-datasource.jdbc-url=jdbc:postgresql://mydb-1:5432/testdb1
I would suggest the second option, which will still allow you to use the application.properties as default values for running locally.
I am trying to connect a postgres DB with Node api, with docker compose like this:
version: '3.8'
services:
db:
image: postgres
environment:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: biobox
volumes:
- ./db-data:/var/lib/postgresql/data
ports:
- 5432:5432
restart: always
api:
build: .
depends_on:
- db
ports:
- 3333:3333
command: yarn dev:server
env_file:
- .env
volumes:
- .:/usr/src/app
working_dir: /usr/src/app
env
DATABASE_USER=postgres
DATABASE_PASSWORD=postgres
DATABASE=biobox
DATABASE_HOST=db
DATABASE_PORT=5432
dockerfile
FROM node:12.16.2-alpine
COPY package*.json ./
RUN yarn
COPY . .
COPY .env .
EXPOSE 3333
My api and db are online on container and the logs show no error.
because when i try make a query return this:
Using ts-node version 8.8.2, typescript version 3.8.3
api_1 | Create a new user User {
api_1 | completName: 'Name',
api_1 | email: 'Name#gmail.com',
api_1 | cpf: '23425',
api_1 | dateNasc: '10/10/2020',
api_1 | number: 'Namefd222',
api_1 | password: '1Name5234'
api_1 | }
db_1 | 2020-05-31 16:37:00.199 UTC [74] LOG: unexpected EOF on client connection with an open transaction
I don't understand why this happens. My environment variables are in accordance with the docker-compose create, in this case the database, USER, PASS.