Docker Postgres psql connection: FATAL: role "..." does not exist - postgresql

I am using docker-compose to build a node express & Postgres application.
Problem
I cannot enter the container with psql to inspect the tables but my initial migration is logging that there was a successful migration.
--- EDIT ---
Codebase
--- END EDIT ---
Running Containers
note: the db is immediately shutting down and this might be the reason my psql is failing while the migration (which is creating a new connection) is working??
## -- LOG FROM RUNNING DB CONTAINER -- ##
2021-12-04 22:08:27.683 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-12-04 22:08:27.683 UTC [1] LOG: listening on IPv6 address "::", port 5432
2021-12-04 22:08:27.688 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-12-04 22:08:27.723 UTC [76] LOG: database system was shut down at 2021-12-04 22:08:27 UTC
2021-12-04 22:08:27.734 UTC [1] LOG: database system is ready to accept connections
✅ Successful Migration Command
docker exec reactquery_demo-backend-1 npm run migrate
❌ Failed psql Command(s)
trying to connect to db w/ docker user: psql -p 5432 -d reactquery_demo_db -U docker
trying to connect to db container w/ docker user:psql -p 5432 -d reactquery_demo-db-1 -U docker
THE ERROR:
psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: FATAL: role "docker" does not exist
I have seen posts about this being caused by the fact that a POSTGRES_USER is not added in the docker-compose but I have that in mine (as shown below)
I have also tried:
uninstalling / reinstalling / upgrading postgresql w/ homebrew
running rm /usr/local/var/postgres/postmaster.pid and then restarting Postgres w/ homebrew
stopping the homebrew Postgres processes completely so that is does not conflict with docker Postgres??
when I do this I get the following error: psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory Is the server running locally and accepting connections on that socket?
Repo Structure
Package.json
{
"name": "ReactQueryDemo",
"version": "0.0.1",
"description": "Simple Overview of React Query for Weekly Dev Meeting",
"main": "index.js",
"scripts": {
"dev": "node index.js",
"dev:watch": "nodemon index.js",
"migrate": "node db/scripts/migrate.js"
},
"dependencies": {
"express": "^4.17.1",
"knex": "^0.95.14",
"pg": "^8.7.1"
}
}
Docker Compose
version: '3.8'
services:
backend:
build: ./backend
ports:
- '8080:8080'
depends_on:
- db
db:
image: 'postgres'
ports:
- '5432:5432'
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_USER=docker
- POSTGRES_DB=react_query_demo_db
volumes:
- data:/var/lib/postgresql/data
volumes:
data:
Dockerfile
FROM node:17
WORKDIR /backend
# copy everything from local backend dir
# into your current working directory in
# the container: /backend
COPY . .
# install dependencies in the container
# the .dockerignore does not copy the
# node_modules || Dockerfile itself
RUN npm i
CMD ["npm", "run", "dev"]
Backend Index
// server
const server = require("./api/server.js");
// port
const PORT = process.env.PORT || 8080;
// listening
server.listen(PORT, () => {
console.log(`*** \n\tServer listening on http://0.0.0.0:${PORT}\n***`)
});
Server
// import packages
const express = require('express')
// import routes
// create server
const server = express()
// configure server
server.use(express.json())
// configure routing
// base route
server.get('/', async(req,res) => {
console.log('React Query Demo Server Running')
res.json({ status: "SUCCESS", message: "React Query Demo Server Running" })
})
// exports
module.exports = server
Database Index
// imports
const knex = require('knex')
module.exports = knex({
client: 'postgres',
connection: {
host: 'db', // this is the name of the service in the docker-compose
user: 'docker',
password: 'password',
database: 'react_query_demo_db'
}
})

Related

Unable to connect to postgres database running in a docker container using psycopg2

I am running a postgres database inside a docker container on a Windows 10 machine using Docker desktop. I would like to run postgresql queries on the database using psycopg2 but I am unable to establish connection to the database. I used the following command to run the database
docker run --name thedatabase -p 5432:5432 -e POSTGRES_PASSWORD=postgres -e POSTGRES_HOST_AUTH_METHOD=md5 postgres:13.5
I used docker inspect to get the IP address for the database. I used the following code to establish a connection with the database using psycopg2.
import psycopg2
from psycopg2 import OperationalError
def create_connection(db_name, db_user, db_password, db_host, db_port):
connection = None
try:
connection = psycopg2.connect(
database=db_name,
user=db_user,
password=db_password,
host=db_host,
port=db_port,
)
print("Connection to PostgreSQL DB successful")
except OperationalError as e:
print(f"The error '{e}' occurred")
return connection
if __name__=="__main__":
connection = create_connection(
"thedatabase", "postgres", "postgres", "172.17.0.2", "5432"
)
I am getting the following error
The error connection to server at 172.17.0.2, port 5432 failed: Connection timed out (0x0000274C/10060) Is the server running on that host and accepting TCP/IP connections?
occurred
Any help would be appreciated. Thanks.
Try to replace docker ip by localhost if your python script is not executed in docker.
if __name__=="__main__":
connection = create_connection(
"thedatabase", "postgres", "postgres", "localhost", "5432"
)

PostgreSQL error: FATAL: password authentication failed for user "pgadmin"

I am making a Telegram bot with a PostgreSQL database. Everything works correctly on the local computer, but when I upload the code to the server using docker-compose, the following errors appear in the console from time to time:
2021-10-10 02:08:58.865 UTC [3458] FATAL: password authentication failed for user "pgadmin"
2021-10-10 02:08:58.865 UTC [3458] DETAIL: Role "pgadmin" does not exist.
Connection matched pg_hba.conf line 94: "host all all 0.0.0.0/0 md5"
2021-10-09 16:27:22.198 UTC [2259] FATAL: password authentication failed for user "pgsql"
2021-10-09 16:27:22.198 UTC [2259] DETAIL: Role "pgsql" does not exist.
Connection matched pg_hba.conf line 94: "host all all 0.0.0.0/0 md5"
2021-10-09 16:25:16.113 UTC [2250] FATAL: password authentication failed for user "postgres"
2021-10-09 16:25:18.992 UTC [2252] DETAIL: Password does not match for user "postgres".
Connection matched pg_hba.conf line 94: "host all all 0.0.0.0/0 md5"
2021-10-09 21:06:32.169 UTC [2831] FATAL: expected password response, got message type 0
2021-10-09 21:07:39.610 UTC [2834] FATAL: unsupported frontend protocol 0.0: server supports 2.0 to 3.0
2021-10-09 21:07:39.819 UTC [2835] FATAL: unsupported frontend protocol 255.255: server supports 2.0 to 3.0
2021-10-09 21:07:40.029 UTC [2836] FATAL: no PostgreSQL user name specified in startup packet
And I don't use these users. It seems to me that someone else is trying to access the database. If so, is it possible to somehow specify the IP addresses from which traffic will be allowed? But my bot parses sites, so you need to limit the IP so that it would not affect its work. Thanks!
UPD:
As I was prompted in the comments, maybe the contents of the docker files will help
Dockerfile:
FROM python:3.9.5
WORKDIR /src
COPY requirements.txt /src
RUN pip install -r requirements.txt
COPY . /src
docker-compose.yml:
version: "3.1"
services:
steamtrader_db:
container_name: steamtrader_db
image: sameersbn/postgresql:10-2
environment:
PG_PASSWORD: $PGPASSWORD
DB_USER: $PGUSER
DB_PASS: $PGPASSWORD
DB_NAME: $DATABASE
restart: always
ports:
- 5432:5432
networks:
- steamtrader_botnet
volumes:
- ./pgdata:/var/lib/postgresql
steamtrader_bot:
container_name: steamtrader
build:
context: .
command: python bot.py
restart: always
networks:
- steamtrader_botnet
env_file:
- ".env"
volumes:
- .:/src
depends_on:
- steamtrader_db
networks:
steamtrader_botnet:
driver: bridge

Gitlab CI services are stopped after bizzar psql call

I am trying to use services in gitlab ci, namely, postgres. Anyway, postgres service doesn't seem to be running, although I just copied what is there in Gitlab CI docs. In logs, after service logs that it started, some psql command (I don't know where it came from) gives an name resolution error. If I am doing something wrong here, what is the way to run postgres service in gitlab-ci?
Below are .gitlab-ci.yml file and logs:
.gitlab-ci.yml
image: ubuntu
services:
- name: postgres:12.2-alpine
alias: postgres
variables:
POSTGRES_DB: badr
POSTGRES_USER: badr
POSTGRES_PASSWORD: badr
PGHOST: postgres
POSTGRES_HOST_AUTH_METHOD: trust
stages:
- test
test db:
stage: test
before_script:
- until (echo > /dev/tcp/postgres/5432) >/dev/null 2>&1;do >&2 echo "service not ready...sleeping";sleep 5;done
script:
- echo "connected to...$PGHOST"
- sleep 10
logs
Running with gitlab-runner 13.2.0-rc2 (45f2b4ec)
on docker-auto-scale fa6cab46
Preparing the "docker+machine" executor
00:55
Using Docker executor with image ubuntu ...
Starting service postgres:12.2-alpine ...
Pulling docker image postgres:12.2-alpine ...
Using docker image sha256:ae192c4d3adaebbbf2f023e1e50eaadfabccb6b08c855ac13d6ce2232381a58a for postgres:12.2-alpine ...
WARNING: Service postgres:12.2-alpine is already created. Ignoring.
Waiting for services to be up and running...
*** WARNING: Service runner-fa6cab46-project-14794655-concurrent-0-f52b350b86ad38db-postgres-0 probably didn't start properly.
Health check error:
service "runner-fa6cab46-project-14794655-concurrent-0-f52b350b86ad38db-postgres-0-wait-for-service" timeout
Health check container logs:
Service container logs:
2020-07-31T09:15:00.377204181Z ********************************************************************************
2020-07-31T09:15:00.377254629Z WARNING: POSTGRES_HOST_AUTH_METHOD has been set to "trust". This will allow
2020-07-31T09:15:00.377259167Z anyone with access to the Postgres port to access your database without
2020-07-31T09:15:00.377262471Z a password, even if POSTGRES_PASSWORD is set. See PostgreSQL
2020-07-31T09:15:00.377265670Z documentation about "trust":
2020-07-31T09:15:00.377269181Z https://www.postgresql.org/docs/current/auth-trust.html
2020-07-31T09:15:00.377272282Z In Docker's default configuration, this is effectively any other
2020-07-31T09:15:00.377276152Z container on the same system.
2020-07-31T09:15:00.377295876Z
2020-07-31T09:15:00.377299453Z It is not recommended to use POSTGRES_HOST_AUTH_METHOD=trust. Replace
2020-07-31T09:15:00.377302412Z it with "-e POSTGRES_PASSWORD=password" instead to set a password in
2020-07-31T09:15:00.377305641Z "docker run".
2020-07-31T09:15:00.377308656Z ********************************************************************************
2020-07-31T09:15:00.404620899Z The files belonging to this database system will be owned by user "postgres".
2020-07-31T09:15:00.406021814Z This user must also own the server process.
2020-07-31T09:15:00.406074886Z
2020-07-31T09:15:00.406083517Z The database cluster will be initialized with locale "en_US.utf8".
2020-07-31T09:15:00.406087263Z The default database encoding has accordingly been set to "UTF8".
2020-07-31T09:15:00.406090884Z The default text search configuration will be set to "english".
2020-07-31T09:15:00.406094281Z
2020-07-31T09:15:00.406097490Z Data page checksums are disabled.
2020-07-31T09:15:00.406101511Z
2020-07-31T09:15:00.406197662Z fixing permissions on existing directory /var/lib/postgresql/data ... ok
2020-07-31T09:15:00.406858429Z creating subdirectories ... ok
2020-07-31T09:15:00.407274720Z selecting dynamic shared memory implementation ... posix
2020-07-31T09:15:00.428414929Z selecting default max_connections ... 100
2020-07-31T09:15:00.506801199Z selecting default shared_buffers ... 128MB
2020-07-31T09:15:00.689382376Z selecting default time zone ... UTC
2020-07-31T09:15:00.695744690Z creating configuration files ... ok
2020-07-31T09:15:01.009439741Z running bootstrap script ... ok
2020-07-31T09:15:01.355673765Z sh: locale: not found
2020-07-31T09:15:01.355836607Z 2020-07-31 09:15:01.355 UTC [30] WARNING: no usable system locales were found
2020-07-31T09:15:01.784080826Z performing post-bootstrap initialization ... ok
2020-07-31T09:15:02.416545146Z syncing data to disk ... ok
2020-07-31T09:15:02.416652656Z
2020-07-31T09:15:02.416854775Z initdb: warning: enabling "trust" authentication for local connections
2020-07-31T09:15:02.416911707Z You can change this by editing pg_hba.conf or using the option -A, or
2020-07-31T09:15:02.416917642Z --auth-local and --auth-host, the next time you run initdb.
2020-07-31T09:15:02.416962149Z
2020-07-31T09:15:02.416967325Z Success. You can now start the database server using:
2020-07-31T09:15:02.416970415Z
2020-07-31T09:15:02.416990907Z pg_ctl -D /var/lib/postgresql/data -l logfile start
2020-07-31T09:15:02.416995097Z
2020-07-31T09:15:02.440378884Z waiting for server to start....2020-07-31 09:15:02.440 UTC [35] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.2.0) 9.2.0, 64-bit
2020-07-31T09:15:02.442773414Z 2020-07-31 09:15:02.442 UTC [35] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2020-07-31T09:15:02.461804500Z 2020-07-31 09:15:02.461 UTC [36] LOG: database system was shut down at 2020-07-31 09:15:01 UTC
2020-07-31T09:15:02.465323529Z 2020-07-31 09:15:02.465 UTC [35] LOG: database system is ready to accept connections
2020-07-31T09:15:02.524643142Z done
2020-07-31T09:15:02.524766601Z server started
2020-07-31T09:15:02.537508874Z psql: error: could not connect to server: could not translate host name "postgres" to address: Name does not resolve
*********
Pulling docker image ubuntu ...
Using docker image sha256:1e4467b07108685c38297025797890f0492c4ec509212e2e4b4822d367fe6bc8 for ubuntu ...
Preparing environment
00:02
Getting source from Git repository
00:01
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/badrmoh/cicdtest/.git/
Created fresh repository.
Checking out 604433de as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ until (echo > /dev/tcp/postgres/5432) >/dev/null 2>&1;do >&2 echo "service not ready...sleeping";sleep 5;done
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
service not ready...sleeping
gitlab-runner
The problem was in PGHOST variable. It seems it is used internally by postgres container that is why it fails to start.
The solution is to set PGHOST in script directive:
image: ubuntu
services:
- name: postgres:9
alias: postgres
variables:
POSTGRES_DB: badr
POSTGRES_USER: badr
POSTGRES_PASSWORD: badr
POSTGRES_HOST_AUTH_METHOD: trust
stages:
- test
test db:
stage: test
before_script:
- export PGHOST=postgres
- until (echo > /dev/tcp/$PGHOST/5432) >/dev/null 2>&1;do >&2 echo "service $PGHOST not ready...sleeping";sleep 5;done
script:
- echo "connected to...$PGHOST"
- sleep 10
Note: You can't use variables directive within jobs in this case since it seems to be populated before even starting services itself.

Gitlab ci, symfony + postgres Error Connection refused

I have an error when I create the schema with doctine
$ php bin/console doctrine:schema:create --env=test
In AbstractPostgreSQLDriver.php line 73:
An exception occurred in driver: SQLSTATE[08006] [7] could not connect to s
erver: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
with my gitlab-ci.yml
database:
stage: building
services:
- postgres:10
variables:
POSTGRES_DB: back
POSTGRES_USER: postgres
POSTGRES_PASSWORD: ''
POSTGRES_HOST: postgres
POSTGRES_PORT: 5432
before_script:
# Install pdo_pgsql driver
- apt-get update
- apt-get install -y libpq-dev
- docker-php-ext-install pdo_pgsql
script:
- php bin/console doctrine:schema:create --env=test
- php -d memory_limit=2048M bin/console --env=test cache:warmup --no-debug
And my gitlab-parameters.yml (who is cp on app/config/parameters.yml)
parameters:
database_host: postgres
database_port: 5432
database_name: back
database_user: postgres
database_password: ''
Other information :
when I try to create database, I have the good message from gitlab-ci
$ php bin/console doctrine:database:create --env=test
Could not create database "back" for connection named default
An exception occurred while executing 'CREATE DATABASE "back"':
SQLSTATE[42P04]: Duplicate database: 7 ERROR: database "back" already exists
Version :
php 7.2.19
smyfony 3.4
and my config_text.yml
imports:
- { resource: config_dev.yml }
framework:
test: ~
session:
storage_id: session.storage.mock_file
profiler:
collect: false
web_profiler:
toolbar: false
intercept_redirects: false
swiftmailer:
disable_delivery: true
How fix it?
EDIT :
postgres is not running :
netstat -plntu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN -

FATAL: password authentication failed for user "postgres" using simple docker

I was having issues connecting to my PG database in my compose file. I finally eliminated all other factors and made this bare-bones compose file:
version: '2.1'
services:
database:
image: postgres:9.5.6
environment:
POSTGRES_PASSWORD: secretpass
ports:
- 5432:5432
I then started it up with this command:
docker-compose --file docker-compose.pg.yml up --build
and it seemed to start ok:
Starting app_database_1 ...
Starting app_database_1 ... done
Attaching to app_database_1
database_1 | LOG: database system was shut down at 2017-09-22 18:48:55 UTC
database_1 | LOG: MultiXact member wraparound protections are now enabled
database_1 | LOG: database system is ready to accept connections
database_1 | LOG: autovacuum launcher started
Then from another shell, tried to connect with this command:
psql postgresql://postgres:secretpass#0.0.0.0
And got this output on my first shell where the compose was run:
database_1 | FATAL: password authentication failed for user "postgres"
database_1 | DETAIL: Connection matched pg_hba.conf line 95: "host all all 0.0.0.0/0 md5"
I have also tried to connect from another container running
Rails defined in the compose file and get the same error. I always end up with an authentication error. What am I missing?