User and Database does not create automatically when docker starts - postgresql

I have Spring Boot App with Postgres which I want to run with Docker and user and database should create automatically with environmental variables.
docker-compose.yml
version: '3'
services:
microservice:
build: ./
image: someimage
ports:
- "8080:8080"
environment:
MEMORY_OPTS: "-Xmx512M"
PG_DB_HOST: postgres
PG_DB_PORT: 5432
PG_DB_PASSWORD: postgres
depends_on:
- postgres
postgres:
image: postgres:10.4-alpine
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=db
ports:
- 5432:5432
application.yml
spring:
config:
activate:
on-profile: local
jpa:
database-platform: org.hibernate.dialect.PostgreSQLDialect
hibernate:
ddl-auto: update
show-sql: true
datasource:
url: jdbc:postgresql://postgres:5432/db
username: postgres
password: postgres
liquibase:
change-log: classpath:/db/changelog/db.changelog-master.yaml
db/changelog/db.changelog-master.yaml
databaseChangeLog:
- logicalFilePath: db/changelog/db.changelog-master.yaml
- preConditions:
- runningAs:
username: postgres
- changeSet:
id: 1
author: postgres
changes:
- createTable:
tableName: employees
columns:
- column:
name: EMPLOYEE_ID
type: int
autoIncrement: true
constraints:
primaryKey: true
nullable: false
- column:
name: FIRST_NAME
type: varchar(50)
- column:
name: LAST_NAME
type: varchar(50)
constraints:
nullable: false
- column:
name: LOCATION
type: varchar(50)
And when I run docker compose up, I always get
Caused by: org.postgresql.util.PSQLException: FATAL: role "postgres" does not exist
or Caused by: org.postgresql.util.PSQLException: FATAL: database "db" does not exist
If I create manually user and database, it works, but as you understand, I need to do it automatically, I suppose some problems with env. variables.

Related

Asp.net Core : Docker-Compose.yml configuration vs Docker Section configuration in the LaunchSettings.json

I have an Asp.net Core application with Docker-Compose.yml configuration and Docker Section configuration in the LaunchSettings.json
Connection string is configured like mentioned below in the Docker-Compose.yml
PersistenceAccess__ConnectionString= Server=db;Port=5432;Database=DemoDatabase;User Id=postgres;Password=postgres;
version: '3.4'
services:
Demo.api:
image: ${DOCKER_REGISTRY-}demoapi
build:
context: .
dockerfile: Sources/Code/Demo.Api/Dockerfile
environment:
- ASPNETCORE_ENVIRONMENT=Development
- PersistenceAccess__ConnectionString= Server=db;Port=5432;Database=DemoDatabase;User Id=postgres;Password=postgres;
ports:
- '8081:80'
depends_on:
- db
db:
image: postgres
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
logging:
options:
max-size: 10m
max-file: "3"
ports:
- '5438:5432'
volumes:
- ./postgres-data:/var/lib/postgresql/data
# copy the sql script to create tables
- ./sql/create_tables.sql:/docker-entrypoint-initdb.d/create_tables.sql
# copy the sql script to fill tables
- ./sql/fill_tables.sql:/docker-entrypoint-initdb.d/fill_tables.sql
Same Connection string is configured in the LaunchSettings.json like mentioned below
"PersistenceAccess__ConnectionString": "Server=host.docker.internal;Port=5432;Database=DemoDatabaseNew;User Id=postgres;Password=postgres;"
{
"iisSettings": {
..
},
"profiles": {
"IIS Express": {
...
},
"Docker": {
"commandName": "Docker",
"launchBrowser": true,
"launchUrl": "{Scheme}://{ServiceHost}:{ServicePort}",
"environmentVariables": {
"PersistenceAccess__ConnectionString": "Server=host.docker.internal;Port=5432;Database=DemoDatabaseNew;User Id=postgres;Password=postgres;"
},
"DockerfileRunArguments": "--add-host host.docker.internal:host-gateway",
"publishAllPorts": true,
"useSSL": false
}
}
}
Which configuration will be used while I run the application using
docker-compose -f docker-compose.yml up
Does the above command create a Database? If so, when it will create the database and what would be the name of the database? Also, when it would create the tables and seed the data?
Please suggest.

Docker-compose Postgres connection refused

I'm running Postgres DB with pg-admin and GO on the docker-compose.
Problem: I can connect from pg-admin to Postgres. But cannot establish a connection from Go.
I tried different combinations of authentication string but it does not work. String format same as here https://github.com/karlkeefer/pngr - but different container name - database
(ERROR) Connection URl:
backend_1 | 2021/08/08 14:24:40 DB connection: database://main:fugZwypczB94m0LP7CcH#postgres:5432/temp_db?sslmode=disable
backend_1 | 2021/08/08 14:24:40 Unalble to open DB connection: dial tcp 127.0.0.1:5432: connect: connection refused
(URI generation same as here https://github.com/karlkeefer/pngr)
Docker:
version: '3.8'
services:
backend:
restart: always
build:
context: backend
target: dev
volumes:
- ./backend:/root
ports:
- "5000:5000"
env_file: .env
depends_on:
- database
database:
build: database
restart: always
environment:
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
PGDATA: /var/lib/postgresql/data
volumes:
- ./database/data:/var/lib/postgresql/data
- ./logs/databse:/var/log/postgresql
- ./database/migrations:/docker-entrypoint-initdb.d/migrations
ports:
- "5432:5432"
database-admin:
image: dpage/pgadmin4:5.5
restart: always
environment:
PGADMIN_DEFAULT_EMAIL: ${PG_ADMIN_EMAIL}
PGADMIN_DEFAULT_PASSWORD: ${PG_ADMIN_PASSWORD}
PGADMIN_LISTEN_PORT: 80
ports:
- "8080:80"
volumes:
- ./database/admin:/var/lib/pgadmin
links:
- "database:pgsql-server"
depends_on:
- database
volumes:
database:
database-admin:
Environment:
POSTGRES_HOST=postgres
POSTGRES_PORT=5432
POSTGRES_DB=temp_db
POSTGRES_USER=main
POSTGRES_PASSWORD=fugZwypczB94m0LP7CcH
PG_ADMIN_EMAIL=admin#temp.com
PG_ADMIN_PASSWORD=ayzi2ta8f1TnX3vKQSN1
PG_ADMIN_PORT=80
GO Code:
db, err = sqlx.Open("postgres", str)
str
func buildConnectionString() string {
user := os.Getenv("POSTGRES_USER")
pass := os.Getenv("POSTGRES_PASSWORD")
if user == "" || pass == "" {
log.Fatalln("You must include POSTGRES_USER and POSTGRES_PASSWORD environment variables")
}
host := os.Getenv("POSTGRES_HOST")
port := os.Getenv("POSTGRES_PORT")
dbname := os.Getenv("POSTGRES_DB")
if host == "" || port == "" || dbname == "" {
log.Fatalln("You must include POSTGRES_HOST, POSTGRES_PORT, and POSTGRES_DB environment variables")
}
str := fmt.Sprintf("database://%s:%s#%s:%s/%s?sslmode=disable", user, pass, host, port, dbname)
log.Println("DB connection: " + str)
return str
}
Thanks in advance!
You reference the database hostname as postgres (POSTGRES_HOST=postgres) which is fine, but the container/service name is database.
Either change the name in your compose.yaml from database to postgres or add an explicit hostname field:
database:
build: database
restart: always
hostname: postgres # <- add this
You may also want to add a dedicated network for multiple container services to talk to one another (or prevent others from). To do this, add this to each service your want to use a specific network e.g.
database:
# ...
networks:
- mynet
backend:
# ...
networks:
- mynet
and define the network at the end of your compose.yaml
networks:
mynet:
name: my-shared-db-network

Knex & postgres connection error - error: role "admin" does not exist

I am trying to connect to postgres db image using knex. But I am getting error:-
error: role "admin" does not exist
My knexfile is ass below:
module.exports = {
development: {
debug: true,
client: 'pg',
connection: {
database: process.env.POSTGRES_DB,
user: process.env.POSTGRES_USER,
password: process.env.POSTGRES_PASSWORD
},
migrations: {
directory: __dirname + '/src/migrations'
},
seeds: {
directory: __dirname + '/src/seeds'
}
}
};
And my docker-compose file is:-
version: '3.1'
services:
db:
image: postgres
restart: always
volumes:
- ./docker-data/db-data:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRESS_USER: ${POSTGRES_USER}
POSTGRESS_DB: ${POSTGRES_DB}
ports:
- 5432:5432
pgadmin:
depends_on:
- db
image: dpage/pgadmin4
restart: always
environment:
PGADMIN_DEFAULT_EMAIL: ${PGADMIN_DEFAULT_EMAIL}
PGADMIN_DEFAULT_PASSWORD: ${PGADMIN_DEFAULT_PASSWORD}
volumes:
- ./docker-data/pgadmin-data:/root/.pgadmin
ports:
- 8080:80
Environment variables are straight forward
POSTGRES_USER=admin
POSTGRES_PASSWORD=admin
POSTGRES_DB=configs
PGADMIN_DEFAULT_EMAIL=admin#admin.com
PGADMIN_DEFAULT_PASSWORD=admin
Packages details:
"knex": "^0.21.1",
"pg": "^8.3.0"
I ran command docker-compose up and on completion i created migrations and post that i ran npx knex migrate:latest. This is resulting in error
error: role "admin" does not exist
at Parser.parseErrorMessage (/Users/test/packages/config-server/node_modules/pg-protocol/dist/parser.js:278:15)
I am not able to debug and fix. Please help.

SequelizeConnectionRefusedError: connect ECONNREFUSED 127.0.0.1:5432 when using docker to use sequelize

I'm getting the following error when dockerizing a node postgres database using sequelize as an orm backend
Unhandled rejection SequelizeConnectionRefusedError: connect
ECONNREFUSED 127.0.0.1:5432 app_1 | at
connection.connect.err
(/home/app/node_modules/sequelize/lib/dialects/postgres/connection-manager.js:170:24)
These lines of code seems to be the culprit, docker should not be connecting these credentials as this is for my local.
if (process.env.NODE_ENV === "production") {
var sequelize = new Sequelize(process.env.DATABASE_URL);
} else {
// docker is looking at these credentials..... when it should not
var sequelize = new Sequelize("elifullstack", "eli", "", {
host: "127.0.0.1",
dialect: "postgres",
pool: {
max: 100,
min: 0,
idle: 200000,
// #note https://github.com/sequelize/sequelize/issues/8133#issuecomment-359993057
acquire: 1000000,
},
});
}
docker-compose.yml
# docker-compose.yml
version: "3"
services:
app:
build: ./server
depends_on:
- database
ports:
- 5000:5000
environment:
# database refers to the database server at the bottom called "database"
- PSQL_HOST=database
- PSQL_USER=postgres
- PORT=5000
- PSQL_NAME=elitypescript
command: npm run server
client:
build: ./client
image: react_client
links:
- app
working_dir: /home/node/app/client
volumes:
- ./:/home/node/app
ports:
- 3001:3001
command: npm run start
env_file:
- ./client/.env
database:
image: postgres:9.6.8-alpine
volumes:
- database:/var/lib/postgresql/data
ports:
- 3030:5432
volumes:
database:
./server/dockerFile
FROM node:10.6.0
COPY . /home/app
WORKDIR /home/app
COPY package.json ./
RUN npm install
EXPOSE 5000
I looked at other similar questions like the following, but it ultimately did not help solve the issue.
Docker - SequelizeConnectionRefusedError: connect ECONNREFUSED 127.0.0.1:3306
SequelizeConnectionRefusedError: connect ECONNREFUSED 127.0.0.1:3306
I solved it...
What i did was change this
host: "127.0.0.1",
to this
let sequelize;
if (process.env.NODE_ENV === "production") {
sequelize = new Sequelize(process.env.DATABASE_URL);
} else {
sequelize = new Sequelize(
process.env.POSTGRES_DB || "elitypescript",
process.env.POSTGRES_USER || "eli",
"",
{
host: process.env.PSQL_HOST || "localhost",
dialect: "postgres",
pool: {
max: 100,
min: 0,
idle: 200000,
// #note https://github.com/sequelize/sequelize/issues/8133#issuecomment-359993057
acquire: 1000000,
},
}
);
}
that way the host would be set to docker environment variable like this
PSQL_HOST: database
and that connects to
database:
image: postgres:9.6.8-alpine
volumes:
- database:/var/lib/postgresql/data
ports:
- 3030:5432
Edit
# docker-compose.yml
version: "3"
services:
app:
build: ./server
depends_on:
- database
ports:
- 5000:5000
environment:
PSQL_HOST: database
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password}
POSTGRES_USER: ${POSTGRES_USER:-postgres}
POSTGRES_DB: ${POSTGRES_DB:-elitypescript}
command: npm run server
client:
build: ./client
image: react_client
links:
- app
working_dir: /home/node/app/client
volumes:
- ./:/home/node/app
ports:
- 3001:3001
command: npm run start
env_file:
- ./client/.env
database:
image: postgres:9.6.8-alpine
volumes:
- database:/var/lib/postgresql/data
ports:
- 3030:5432
volumes:
database:

Authentication Error while running Konga in docker-compose

I have a docker-compose file starting kong and konga services. Kong starts correctly but konga fails with the error below:
debug: Preparing database...
Using postgres DB Adapter.
Failed to connect to DB { error: password authentication failed for user "kong"
at Connection.parseE (/app/node_modules/sails-postgresql/node_modules/pg/lib/connection.js:539:11)
at Connection.parseMessage (/app/node_modules/sails-postgresql/node_modules/pg/lib/connection.js:366:17)
at Socket.<anonymous> (/app/node_modules/sails-postgresql/node_modules/pg/lib/connection.js:105:22)
at Socket.emit (events.js:198:13)
at addChunk (_stream_readable.js:288:12)
at readableAddChunk (_stream_readable.js:269:11)
at Socket.Readable.push (_stream_readable.js:224:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
name: 'error',
length: 100,
severity: 'FATAL',
code: '28P01',
detail: undefined,
hint: undefined,
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: undefined,
table: undefined,
column: undefined,
dataType: undefined,
constraint: undefined,
file: 'auth.c',
line: '333',
routine: 'auth_failed' }
In my docker-compose, Kong service is from their github page and runs fine.. I believe, the problem is with Konga. Konga service is below:
version: '2.1'
volumes:
kong_data: {}
services:
db:
image: postgres:latest
environment:
POSTGRES_DB: ${KONG_PG_DATABASE:-kong}
POSTGRES_PASSWORD: ${KONG_PG_PASSWORD:-kong}
POSTGRES_USER: ${KONG_PG_USER:-kong}
healthcheck:
test: ["CMD", "pg_isready", "-U", "${KONG_PG_USER:-kong}"]
interval: 30s
timeout: 30s
retries: 3
restart: on-failure
stdin_open: true
tty: true
networks:
- kong_net
volumes:
- kong_data:/var/lib/postgresql/data
konga-prepare:
image: pantsel/konga:latest
command: “-c prepare -a postgres -u postgresql://kong#db:5432/konga_db”
networks:
- kong_net
restart: on-failure
links:
- db
depends_on:
- db
konga:
image: pantsel/konga:latest
restart: always
networks:
- kong_net
environment:
DB_ADAPTER: postgres
DB_HOST: db
DB_PORT: 5432
DB_USER: konga
DB_PASSWORD:
TOKEN_SECRET: km1GUr4RkcQD7DewhJPNXrCuZwcKmqjb
DB_DATABASE: konga_db
NODE_ENV: production
depends_on:
- db
ports:
- "1337:1337"
How do I solve the authentication error?
I had the same issue, I solved it by adding password to konga-prepare command as:
docker run --rm --network=kong-net pantsel/konga:latest -c prepare -a postgres -u postgresql://kong:kong#kong-database:5432/konga_database