How to use SCRAM-SHA-256 based authentication for user with TLS config in POSTGRESQL server? - postgresql

I have created a PostgreSQL 13.1 cluster with 1` where I have 2 standby and 1 primary.
Here I have used scram-sha-256as authentication mode for client where my cluster is config with TLS.
When I started my PostgreSQL server, I got a continuous fatal log.
the log is:
UTC [181] FATAL: expected SASL response, got message type 88
postgresql.conf
wal_level = replica
max_wal_senders = 90
wal_keep_size = 64
wal_log_hints = on
hot_standby = on
password_encryption = scram-sha-256
recovery_target_timeline = 'latest'
primary_conninfo = 'application_name=<application_name> host=<host_name> password=<Postgres_password sslmode=verify-full sslrootcert=<path_to_ca_cert>/ca.crt'
promote_trigger_file = '/run_scripts/tmp/pg-failover-trigger'
...
<some other basic config>
...
pg_hba.conf
#TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 trust
# IPv6 local connections:
host all all ::1/128 scram-sha-256
local replication all scram-sha-256
host replication all 127.0.0.1/32 scram-sha-256
host replication all ::1/128 scram-sha-256
host all all 0.0.0.0/0 scram-sha-256
host replication postgres 0.0.0.0/0 scram-sha-256
host all all ::/0 scram-sha-256
host replication postgres ::/0 scram-sha-256
Is it possible to add scram-sha-256 for tls config cluster? If yes, then how can I do it?
Here is how I have tried to connect as a client.
$ export PGPASSWORD=<POSTGRES_PASSWORD>
$ export PGUSER=<POSTGRES_USER>
$ psql -w -h <hostname>
psql (13.1)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
postgres=#
I don't see any error from my client site but get continuous fatal log.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/pv/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
sh: locale: not found
2021-03-03 07:43:56.823 UTC [35] WARNING: no usable system locales were found
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
pg_ctl -D /var/pv/data -l logfile start
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2021-03-03 07:44:03.835 GMT [42] LOG: skipping missing configuration file "/etc/config/user.conf"
2021-03-03 07:44:03.836 GMT [42] LOG: skipping missing configuration file "/etc/config/user.conf"
2021-03-03 07:44:03.996 UTC [42] LOG: starting PostgreSQL 13.1 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.2.1_pre1) 10.2.1 20201203, 64-bit
2021-03-03 07:44:03.996 UTC [42] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-03-03 07:44:03.996 UTC [42] LOG: listening on IPv6 address "::", port 5432
2021-03-03 07:44:04.073 UTC [42] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-03-03 07:44:04.187 UTC [43] LOG: database system was shut down at 2021-03-03 07:43:59 UTC
2021-03-03 07:44:04.237 UTC [44] FATAL: the database system is starting up
2021-03-03 07:44:04.281 UTC [42] LOG: database system is ready to accept connections
done
server started
ALTER ROLE
/run_scripts/role/start.sh: ignoring /var/initdb/*
waiting for server to shut down...2021-03-03 07:44:04.387 UTC [42] LOG: received fast shutdown request
.2021-03-03 07:44:04.478 UTC [42] LOG: aborting any active transactions
2021-03-03 07:44:04.480 UTC [42] LOG: background worker "logical replication launcher" (PID 51) exited with exit code 1
2021-03-03 07:44:04.482 UTC [45] LOG: shutting down
2021-03-03 07:44:05.169 UTC [56] FATAL: the database system is shutting down
.2021-03-03 07:44:05.702 UTC [57] FATAL: the database system is shutting down
2021-03-03 07:44:05.806 UTC [58] FATAL: the database system is shutting down
2021-03-03 07:44:06.217 UTC [59] FATAL: the database system is shutting down
..2021-03-03 07:44:07.648 UTC [60] FATAL: the database system is shutting down
2021-03-03 07:44:07.853 UTC [61] FATAL: the database system is shutting down
2021-03-03 07:44:08.163 UTC [42] LOG: database system is shut down
done
server stopped
2021-03-03 07:44:08.208 GMT [25] LOG: skipping missing configuration file "/etc/config/user.conf"
2021-03-03 07:44:08.208 GMT [25] LOG: skipping missing configuration file "/etc/config/user.conf"
2021-03-03 07:44:08.487 UTC [25] LOG: starting PostgreSQL 13.1 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.2.1_pre1) 10.2.1 20201203, 64-bit
2021-03-03 07:44:08.487 UTC [25] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-03-03 07:44:08.487 UTC [25] LOG: listening on IPv6 address "::", port 5432
2021-03-03 07:44:08.688 UTC [25] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-03-03 07:44:08.895 UTC [67] LOG: database system was shut down at 2021-03-03 07:44:07 UTC
2021-03-03 07:44:08.905 UTC [68] FATAL: the database system is starting up
2021-03-03 07:44:08.945 UTC [69] FATAL: the database system is starting up
2021-03-03 07:44:08.946 UTC [70] FATAL: the database system is starting up
2021-03-03 07:44:08.991 UTC [25] LOG: database system is ready to accept connections
2021-03-03 07:44:15.990 UTC [94] FATAL: expected SASL response, got message type 88
2021-03-03 07:44:16.011 UTC [95] FATAL: expected SASL response, got message type 88
2021-03-03 07:44:30.197 UTC [108] FATAL: expected SASL response, got message type 88
2021-03-03 07:44:31.324 UTC [110] FATAL: expected SASL response, got message type 88
2021-03-03 07:44:57.571 UTC [114] FATAL: expected SASL response, got message type 88
2021-03-03 07:44:57.729 UTC [115] FATAL: expected SASL response, got message type 88
2021-03-03 07:45:07.982 UTC [127] FATAL: expected SASL response, got message type 88
2021-03-03 07:45:08.020 UTC [128] FATAL: expected SASL response, got message type 88
2021-03-03 07:45:18.135 UTC [140] FATAL: expected SASL response, got message type 88
2021-03-03 07:45:18.173 UTC [141] FATAL: expected SASL response, got message type 88

The problem was with the go-client i was trying to use to connect my server. The problem was solved when i tried https://github.com/lib/pq client instead of https://github.com/go-pg/pg.

Related

invalid length of startup packet docker postgressql

PostgreSQL init process complete; ready for start up.
2022-05-16 19:28:51.674 UTC [1] LOG: starting PostgreSQL 14.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
2022-05-16 19:28:51.674 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2022-05-16 19:28:51.674 UTC [1] LOG: listening on IPv6 address "::", port 5432
2022-05-16 19:28:51.677 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-05-16 19:28:51.681 UTC [51] LOG: database system was shut down at 2022-05-16 19:28:51 UTC
2022-05-16 19:28:51.684 UTC [1] LOG: database system is ready to accept connections
2022-05-16 19:29:07.610 UTC [59] LOG: invalid length of startup packet
2022-05-16 19:29:07.611 UTC [58] LOG: invalid length of startup packet
2022-05-16 19:29:07.612 UTC [60] LOG: invalid length of startup packet
2022-05-16 19:29:08.647 UTC [62] LOG: invalid length of startup packet
2022-05-16 19:29:08.650 UTC [61] LOG: invalid length of startup packet
2022-05-16 19:29:08.652 UTC [63] LOG: invalid length of startup packet
Any help would be appreciated I couldn't figure out
Docker composer file
version: '3.9'
services:
postgres:
image: postgres:14-alpine
ports:
- 5432:5432
volumes:
- ~/apps/postgres:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=S3cret
- POSTGRES_USER=citizix_user
- POSTGRES_DB=citizix_db
Using ubuntu 20.4
Something, probably a monitoring system, is establishing TCP connections to the database server, then closing them without sending a correct PostgreSQL connection packet.
You can include %h in your log_line_prefix to see the client address of these connections, which may help identify the culprit.
The solution is to get the monitoring system to establish a database connection rather than only a TCP connection. Not only will that get rid of the message, but it will make the test much more useful.

postgresql log about no existing database

I don't understand this postgresql log :
2022-03-27 08:00:19.441 UTC [584262] postgres#boutique2 FATAL: database "boutique2" does not exist
2022-03-27 08:00:19.704 UTC [584264] postgres#boutique2 FATAL: database "boutique2" does not exist
2022-03-27 08:01:54.770 UTC [781] LOG: received fast shutdown request
2022-03-27 08:01:54.773 UTC [781] LOG: aborting any active transactions
2022-03-27 08:01:54.779 UTC [781] LOG: background worker "logical replication launcher" (PID 800) exited with exit code 1
2022-03-27 08:01:54.780 UTC [795] LOG: shutting down
2022-03-27 08:01:54.797 UTC [781] LOG: database system is shut down
2022-03-27 08:02:16.254 UTC [770] LOG: starting PostgreSQL 13.5 (Debian 13.5-0+deb11u1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-03-27 08:02:16.255 UTC [770] LOG: listening on IPv4 address "127.0.0.1", port 5432
2022-03-27 08:02:16.256 UTC [770] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-03-27 08:02:16.271 UTC [772] LOG: database system was shut down at 2022-03-27 08:01:54 UTC
2022-03-27 08:02:16.285 UTC [770] LOG: database system is ready to accept connections
2022-03-27 08:02:17.243 UTC [891] postgres#boutique2 FATAL: database "boutique2" does not exist
2022-03-27 08:02:17.640 UTC [1044] postgres#boutique2 FATAL: database "boutique2" does not exist
I dropped this database, which is not present in my mojolicious scripts, the only ones on this server.
root#perso:/etc/postgresql/13/main# grep postgresql.conf -e 'boutique2'
root#perso:/etc/postgresql/13/main# grep pg_hba.conf -e 'boutique2'
root#perso:/etc/postgresql/13/main#
Has please someone an idea about ?

postgresql running in docker compose doesn't create role

I am trying to run postgreSQL via docker-compose and I am getting the issue that user/password is not created when I started the service.
version: "3"
services:
db:
image: postgres:latest
container_name: postgres
#volumes:
#- postgres-data:/var/lib/postgresql/data
ports:
- 5432:5432
environment:
- POSTGRES_PASSWORD=postgrespassword
- POSTGRES_USER=postgres
- POSTGRES_DB=random_db_name
restart: always
I have this block of code in my docker-compose.yml and I run the following command:
docker-compose up -d (this allow me to start the service in background)
and when I check the logs I got:
docker logs -f 0e1731f95396
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Success. You can now start the database server using:
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
pg_ctl -D /var/lib/postgresql/data -l logfile start
waiting for server to start....2021-04-27 16:20:44.592 UTC [49] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-04-27 16:20:44.594 UTC [49] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-04-27 16:20:44.603 UTC [50] LOG: database system was shut down at 2021-04-27 16:20:44 UTC
2021-04-27 16:20:44.609 UTC [49] LOG: database system is ready to accept connections
done
server started
CREATE DATABASE
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
waiting for server to shut down...2021-04-27 16:20:44.889 UTC [49] LOG: received fast shutdown request
.2021-04-27 16:20:44.891 UTC [49] LOG: aborting any active transactions
2021-04-27 16:20:44.892 UTC [49] LOG: background worker "logical replication launcher" (PID 56) exited with exit code 1
2021-04-27 16:20:44.892 UTC [51] LOG: shutting down
2021-04-27 16:20:44.907 UTC [49] LOG: database system is shut down
done
server stopped
PostgreSQL init process complete; ready for start up.
2021-04-27 16:20:45.018 UTC [1] LOG: starting PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-04-27 16:20:45.019 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2021-04-27 16:20:45.019 UTC [1] LOG: listening on IPv6 address "::", port 5432
2021-04-27 16:20:45.023 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-04-27 16:20:45.029 UTC [77] LOG: database system was shut down at 2021-04-27 16:20:44 UTC
2021-04-27 16:20:45.034 UTC [1] LOG: database system is ready to accept connections
But when I try to connect to this database locally I get the message: "FATAL: role "postgres" does not exist"
Do you have any input about how to solve this problem?
I already made a few attempts after reading a few comments from different places but I got always the same problem.
I was expecting to run locally postgreSQL and setup already a user/password and a Database with that name in the docker-compose

PostgreSQL connection issue after service restart

I have edited my pg_hba file and copied it to server and restarted the services by "sudo service postgresql restart" but after that the server is not connecting.
Showing the below error, Your database returned: "Connection to 138.2xx.1xx.xx:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections."
The Jenkins job and data visualization tools are failing which was working fine previously. What could be the reason.
Getting this in PostgreSQL Log
2019-10-23 07:21:25.829 CEST [11761] LOG: received fast shutdown request
2019-10-23 07:21:25.829 CEST [11761] LOG: aborting any active transactions
2019-10-23 07:21:25.829 CEST [11766] LOG: autovacuum launcher shutting down
2019-10-23 07:21:25.832 CEST [11763] LOG: shutting down
2019-10-23 07:21:25.919 CEST [11761] LOG: database system is shut down
2019-10-23 07:21:27.068 CEST [22633] LOG: database system was shut down at 2019-10-23 07:21:25 CEST
2019-10-23 07:21:27.073 CEST [22633] LOG: MultiXact member wraparound protections are now enabled
2019-10-23 07:21:27.075 CEST [22631] LOG: database system is ready to accept connections
2019-10-23 07:21:27.075 CEST [22637] LOG: autovacuum launcher started
2019-10-23 07:21:27.390 CEST [22639] [unknown]#[unknown] LOG: incomplete startup packet
Below shows no response.
root#Ubuntu-1604-xenial-64-minimal ~ # pg_isready -h localhost -p 5432
localhost:5432 - no response
Below was already added to the postgresql.config file.
listen_addresses = '*'
Do i need to restart the entire server?
Can anyone please help me to resolve this.

Postgres:10 in docker swarm cluster. Database system is shut down

I use postgres:10 (https://hub.docker.com/_/postgres/) image for DB.
It is deployed in docker swarm cluster.
After running DB replica I got database system is shut down in DB's log.
2018-05-11 10:26:53.073 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432,
2018-05-11 10:26:53.073 UTC [1] LOG: listening on IPv6 address "::", port 5432,
2018-05-11 10:26:53.077 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432",
2018-05-11 10:26:53.092 UTC [20] LOG: database system was shut down at 2018-05-11 10:26:20 UTC,
2018-05-11 10:26:53.100 UTC [1] LOG: database system is ready to accept connections,
The files belonging to this database system will be owned by user "postgres".,
This user must also own the server process.,
,
The database cluster will be initialized with locale "en_US.utf8".,
The default database encoding has accordingly been set to "UTF8".,
The default text search configuration will be set to "english".,
,
Data page checksums are disabled.,
,
fixing permissions on existing directory /var/lib/postgresql/data ... ok,
creating subdirectories ... ok,
selecting default max_connections ... 100,
selecting default shared_buffers ... 128MB,
selecting dynamic shared memory implementation ... posix,
creating configuration files ... ok,
running bootstrap script ... ok,
performing post-bootstrap initialization ... ok,
,
WARNING: enabling "trust" authentication for local connections,
You can change this by editing pg_hba.conf or using the option -A, or,
--auth-local and --auth-host, the next time you run initdb.,
syncing data to disk ... ok,
,
Success. You can now start the database server using:,
,
pg_ctl -D /var/lib/postgresql/data -l logfile start,
,
waiting for server to start....2018-05-11 09:39:21.129 UTC [37] LOG: listening on IPv4 address "127.0.0.1", port 5432,
2018-05-11 09:39:21.130 UTC [37] LOG: could not bind IPv6 address "::1": Cannot assign requested address,
2018-05-11 09:39:21.130 UTC [37] HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.,
2018-05-11 09:39:21.133 UTC [37] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432",
2018-05-11 09:39:21.147 UTC [38] LOG: database system was shut down at 2018-05-11 09:39:20 UTC,
2018-05-11 09:39:21.152 UTC [37] LOG: database system is ready to accept connections,
done,
server started,
CREATE DATABASE,
,
CREATE ROLE,
,
,
/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*,
,
2018-05-11 09:39:21.595 UTC [37] LOG: received fast shutdown request,
waiting for server to shut down....2018-05-11 09:39:21.596 UTC [37] LOG: aborting any active transactions,
2018-05-11 09:39:21.598 UTC [37] LOG: worker process: logical replication launcher (PID 44) exited with exit code 1,
2018-05-11 09:39:21.599 UTC [39] LOG: shutting down,
2018-05-11 09:39:21.613 UTC [37] LOG: database system is shut down,
done,
server stopped,
,
PostgreSQL init process complete; ready for start up.,
,
2018-05-11 09:39:21.706 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432,
2018-05-11 09:39:21.706 UTC [1] LOG: listening on IPv6 address "::", port 5432,
2018-05-11 09:39:21.709 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432",
2018-05-11 09:39:21.724 UTC [64] LOG: database system was shut down at 2018-05-11 09:39:21 UTC,
2018-05-11 09:39:21.729 UTC [1] LOG: database system is ready to accept connections,
2018-05-11 10:26:20.444 UTC [1] LOG: received smart shutdown request,
2018-05-11 10:26:20.449 UTC [1] LOG: worker process: logical replication launcher (PID 70) exited with exit code 1,
2018-05-11 10:26:20.449 UTC [65] LOG: shutting down,
2018-05-11 10:26:20.460 UTC [1] LOG: database system is shut down,
Image :
FROM postgres:10
COPY healthcheck /usr/local/bin/
RUN chmod +x /usr/local/bin/healthcheck
HEALTHCHECK --interval=30s --timeout=30s --retries=3 \
CMD healthcheck
Snippet from docker-compose :
db_jackrabbit:
build: ./images/pgsql_jackrabbit
container_name: db_jackrabbit
environment:
- POSTGRES_DB=${JACK_POSTGRES_DB}
- POSTGRES_USER=${JACK_POSTGRES_USER}
- POSTGRES_PASSWORD=${JACK_POSTGRES_PASSWORD}
volumes:
- pgsql_jackrabbit_local:/var/lib/postgresql/data
ports:
- ${PORT_DB_JACKRABBIT}:5432
healthcheck:
#!/bin/bash
set -eo pipefail
host="$(hostname -i || echo '127.0.0.1')"
user="${POSTGRES_USER:-postgres}"
db="${POSTGRES_DB:-$POSTGRES_USER}"
export PGPASSWORD="${POSTGRES_PASSWORD:-}"
args=(
# force postgres to not use the local unix socket (test "external" connectibility)
--host "$host"
--username "$user"
--dbname "$db"
--quiet --no-align --tuples-only
)
if select="$(echo 'SELECT 1' | psql "${args[#]}")" && [ "$select" = '1' ]; then
exit 0
fi
exit 1
But DB still alive. It is shutdown periodically and accept connections again (
What it the problem?
Thanks in advance!
Ok, so I solved my issue. This issue helped me.
It seems like Postgres initialization process does stop the initialization process once done, and it is another process which follows up and accepts connexion.
Hence I had:
postgres:
deploy:
restart_policy:
condition: on-failure
window: 15m
and apparently docker received a end-of-process status code, so it stopped without going to the next process so never accepting connexions.
My interpretation may be incorrect, but at least if you face the issue, try removing restart_policy key to see if it fixes it.
I haven't tried to restore healthcheck yet, as it may also have undesired side effects.