Hardware and software used:
RAM size: 512 GB
SSD: 1TB
CPU(s): 256(0-255)
PostgreSQL version: 13.3
Operating system: RHEL8.4
Example Perf data for 24vu(TPCC)
14.99% postgres postgres [.] LWLockAcquire
7.09% postgres postgres [.] _bt_compare
8.66% postgres postgres [.] LWLockRelease
2.28% postgres postgres [.] GetSnapshotData
2.25% postgres postgres [.] hash_search_with_hash_value
2.11% postgres postgres [.] XLogInsertRecord
1.98% postgres postgres [.] PinBuffer
Postgres.conf
shared_buffers = 64000MB
huge_pages = on
temp_buffers = 4000MB
work_mem = 4000MB
maintenance_work_mem = 512MB
autovacuum_work_mem = -1
max_stack_depth = 7MB
dynamic_shared_memory_type = posix
max_files_per_process = 4000
effective_io_concurrency = 32
wal_level = minimal
synchronous_commit = off
wal_buffers = 512MB
#checkpoint_segments = 256
checkpoint_timeout = 1h
checkpoint_completion_target = 1
checkpoint_warning = 0
log_min_messages = error
log_min_error_statement = error
log_timezone = ‘GB’
autovacuum = off
datestyle = ‘iso, dmy’
timezone = ‘GB’
lc_messages = ‘en_GB.UTF-8’
lc_monetary = ‘en_GB.UTF-8’
lc_numeric = ‘en_GB.UTF-8’
lc_time = ‘en_GB.UTF-8’
default_text_search_config = ‘pg_catalog.english’
max_locks_per_transaction = 64
max_pred_locks_per_transaction = 64
LWLockAcquire(Lock contention) is high in GCC/Clang while running TPCC(HammerDB bench environment)
Is there a way to tune the lock contention?
Any recommendations to tune/reduce the lock contention?
I am aware that you can show the duration and log queries using the configuration below in postgresql.conf
------------------------------------------------------------------------------
CUSTOMIZED OPTIONS
------------------------------------------------------------------------------
log_statement = 'all'
log_duration = on
log_line_prefix = '{"Time":[%t], Host:%h} '
And then returns logs like
{"Time":[2018-08-13 16:24:20 +08], Host:172.18.0.2} LOG: statement: SELECT "auth_user"."id", "auth_user"."password", "auth_user"."last_login", "auth_user"."is_superuser", "auth_user"."username", "auth_user"."first_name", "auth_user"."last_name", "auth_user"."email", "auth_user"."is_staff", "auth_user"."is_active", "auth_user"."date_joined" FROM "auth_user" WHERE "auth_user"."id" = 1
{"Time":[2018-08-13 16:24:20 +08], Host:172.18.0.2} LOG: duration: 7.694 ms
But can I combine the duration and statement in a single line like?
LOG: { statement: ..., duration: 7.694 ms}
The way you are logging, the statement is logged when the server starts processing it, but the duration is only known at the end of the execution.
This is why it has to be logged as two different messages.
If you use log_min_duration_statement = 0 instead, the statement is logged at the end of execution together with the duration.
pgbouncer version 1.7.2
psql (9.5.6)
I try use auth_hba_file (/var/lib/pgsql/9.5/data/pg_hba.conf) in PgBouncer.
Config pgbouncer.ini
postgres = host=localhost port=5432 dbname=postgres user=postgres
test = host=localhost port=5432 dbname=test user=test
[pgbouncer]
logfile = /var/log/pgbouncer/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid
listen_addr = *
listen_port = 6432
auth_type = hba
auth_hba_file = /var/lib/pgsql/9.5/data/pg_hba.conf
admin_users = postgres
stats_users = stats, postgres
pool_mode = session
server_reset_query = DISCARD ALL
max_client_conn = 100
default_pool_size = 20
cat pg_hba.conf | grep -v "#" | grep -v "^$"
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
host test test 10.255.4.0/24 md5
psql -h 10.233.4.16 -p 5432 -U test
Password for user test:
psql (9.5.6)
Type "help" for help.
test=> \q
psql -h 10.233.4.16 -p 6432 -U test
psql: ERROR: No such user: test
tail -fn10 /var/log/pgbouncer/pgbouncer.log
LOG C-0x78f7e0: (nodb)/(nouser)#10.255.4.245:8963 closing because: No such user: test (age=0)
WARNING C-0x78f7e0: (nodb)/(nouser)#10.255.4.245:8963 Pooler Error: No such user: test
LOG C-0x78f7e0: (nodb)/(nouser)#10.255.4.245:8963 login failed: db=test user=test
But i cannot connect to postgresql (using PgBouncer) using pg_hba.conf
Can somebody help?
May you have example for use auth_hba_file.
Thanks
I changed config:
[root#dev-metrics2 pgbouncer]# cat pgbouncer.ini | grep -v ";" | grep -v "^$" | grep -v "#"
[databases]
postgres = host=localhost port=5432 dbname=postgres user=postgres
test = host=localhost port=5432 dbname=test auth_user=test
[pgbouncer]
logfile = /var/log/pgbouncer/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid
listen_addr = *
listen_port = 6432
auth_query = SELECT usename, passwd FROM pg_shadow WHERE usename=$1
admin_users = postgres
stats_users = stats, postgres
pool_mode = session
server_reset_query = DISCARD ALL
max_client_conn = 100
default_pool_size = 20
Drop and Create user and DB
[local]:5432 postgres#postgres # DROP DATABASE test;
DROP DATABASE
[local]:5432 postgres#postgres # DROP USER test ;
DROP ROLE
[local]:5432 postgres#postgres # CREATE USER test with password 'test';
CREATE ROLE
[local]:5432 postgres#postgres # CREATE DATABASE test with owner test;
CREATE DATABASE
PGPASSWORD=test psql -h 10.233.4.16 -p 6432 -U test
Password for user test:
psql: ERROR: Auth failed
tail -fn1 /var/log/pgbouncer/pgbouncer.log
LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
LOG C-0x17b57a0: test/test#10.255.4.245:3069 login attempt: db=test user=test tls=no
LOG C-0x17b57a0: test/test#10.255.4.245:3069 closing because: client unexpected eof (age=0)
LOG C-0x17b57a0: test/test#10.255.4.245:3070 login attempt: db=test user=test tls=no
LOG C-0x17b57a0: test/test#10.255.4.245:3070 closing because: Auth failed (age=0)
WARNING C-0x17b57a0: test/test#10.255.4.245:3070 Pooler Error: Auth failed
Work config:
cat pgbouncer.ini | grep -v ";" | grep -v "^$" | grep -v "#"
[databases]
*= port=5432 auth_user=postgres
[pgbouncer]
logfile = /var/log/pgbouncer/pgbouncer.log
pidfile = /var/run/pgbouncer/pgbouncer.pid
listen_addr = *
listen_port = 6432
auth_query = SELECT usename, passwd FROM pg_shadow WHERE usename=$1
admin_users = postgres
stats_users = stats, postgres
pool_mode = session
server_reset_query = DISCARD ALL
max_client_conn = 100
default_pool_size = 20
Try put space
*= port=5432 auth_user=postgres # old string
* = port=5432 auth_user=postgres # new string
work for me
Every time I invoke a parameterized query I get too much output in the log file. For example, when inserting 3 users into a table I get the following log output:
2013-10-29 06:01:43 EDT LOG: duration: 0.000 ms parse <unnamed>: INSERT INTO users (login,role,password) VALUES
($1,$2,$3)
,($4,$5,$6)
,($7,$8,$9)
2013-10-29 06:01:43 EDT LOG: duration: 0.000 ms bind <unnamed>: INSERT INTO users (login,role,password) VALUES
($1,$2,$3)
,($4,$5,$6)
,($7,$8,$9)
2013-10-29 06:01:43 EDT DETAIL: parameters: $1 = 'guest', $2 = 'user', $3 = '123', $4 = 'admin', $5 = 'admin', $6 = '123', $7 = 'mark', $8 = 'power user', $9 = '123'
2013-10-29 06:01:43 EDT LOG: execute <unnamed>: INSERT INTO users (login,role,password) VALUES
($1,$2,$3)
,($4,$5,$6)
,($7,$8,$9)
2013-10-29 06:01:43 EDT DETAIL: parameters: $1 = 'guest', $2 = 'user', $3 = '123', $4 = 'admin', $5 = 'admin', $6 = '123', $7 = 'mark', $8 = 'power user', $9 = '123'
2013-10-29 06:01:43 EDT LOG: duration: 4.000 ms
Notice, that the whole query appears three times - for parse, for bind and for execute. And the complete set of parameters appears twice - for bind and for execute.
Note, that this extra verbosity is only present when running parameterized queries.
Here is my config:
C:\Program Files\PostgreSQL\9.2\data>findstr log_ postgresql.conf
# "postgres -c log_connections=on". Some parameters can be changed at run time
log_destination = 'stderr' # Valid values are combinations of
log_directory = 'pg_log' # directory where log files are written,
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # log file name pattern,
log_file_mode = 0600 # creation mode for log files,
#log_truncate_on_rotation = off # If on, an existing log file with the
#log_rotation_age = 1d # Automatic rotation of logfiles will
#log_rotation_size = 10MB # Automatic rotation of logfiles will
#syslog_facility = 'LOCAL0'
#syslog_ident = 'postgres'
log_min_messages = notice # values in order of decreasing detail:
log_min_error_statement = error # values in order of decreasing detail:
log_min_duration_statement = 0 # -1 is disabled, 0 logs all statements
#log_checkpoints = off
#log_connections = off
#log_disconnections = off
#log_duration = off
#log_error_verbosity = default # terse, default, or verbose messages
#log_hostname = off
log_line_prefix = '%t ' # special values:
#log_lock_waits = off # log lock waits >= deadlock_timeout
log_statement = 'all' # none, ddl, mod, all
#log_temp_files = -1 # log temporary files equal or larger
log_timezone = 'US/Eastern'
#log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off
#log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and
C:\Program Files\PostgreSQL\9.2\data>
So, my question is how can I reduce the verbosity of the log for parameterized queries without affecting the other queries? Ideally, I would like to have the query SQL and its parameters logged just once.
I don't think that's possible out of the box. You could write a logging hook and filter log entries.
The same issue I was facing before Above config which you have mentioned just change log_min_duration_statement=-1 ( disable) the query will be once only logged
BUT you have enabled the duration it will just log duration 3 times but not the query
Like
2013-10-29 06:01:43 EDT LOG: duration: 0.000 ms
2013-10-29 06:01:43 EDT LOG: duration: 0.000 ms
2013-10-29 06:01:43 EDT LOG: execute <unnamed>: INSERT INTO users (login,role,password) VALUES
($1,$2,$3)
,($4,$5,$6)
,($7,$8,$9)
2013-10-29 06:01:43 EDT DETAIL: parameters: $1 = 'guest', $2 = 'user', $3 = '123', $4 = 'admin', $5 = 'admin', $6 = '123', $7 = 'mark', $8 = 'power user', $9 = '123'
2013-10-29 06:01:43 EDT LOG: duration: 4.000 ms
I'm running a log (log_min_duration_statement = 200) to analyse some slow queries in PostgreSQL 9.0 but the statements for worst queries aren't being logged. Is there any way I can find out what the queries actually are?
(some values replaced with *** for brevity and privacy.)
2012-06-29 02:10:39 UTC LOG: duration: 266.658 ms statement: SELECT *** FROM "oauth_accesstoken" WHERE "oauth_accesstoken"."token" = E'***'
2012-06-29 02:10:40 UTC LOG: duration: 1797.400 ms statement:
2012-06-29 02:10:49 UTC LOG: duration: 1670.132 ms statement:
2012-06-29 02:10:50 UTC LOG: duration: 354.336 ms statement: SELECT *** FROM ***
...
There are some log file destination options in postgresql.conf, as shown below. I suggest to use csvlog.
log_destination = 'csvlog'
logging_collector = on
log_directory = '/var/applog/pg_log/1922/'
log_rotation_age = 1d
log_rotation_size = 10MB
log_statement = 'ddl' # none, ddl, mod, all
log_min_duration_statement = 200
After making any changes, you need to reload the postgresql.conf file.
It turns out because I was keeping an eye on the logs with tail -f path | grep 'duration .+ ms' any statement starting with a newline was not visible. I was mainly doing this to highlight the duration string.