pgbouncer can't connect to local postgresql database (timeout) - postgresql

Im trying to get pgbouncer to work on my standalone postgresql server, but when I try to connect to the database using the pgbouncer port, I get an timeout error.
Postgresql: 9.1.8
pgBouncer: 1.4.2
OS: Ubuntu 12.04.2 LTS (GNU/Linux 3.8.4-x86_64-linode31 x86_64)
Here is my conf file:
[databases]
postgres = host=127.0.0.1 port=5432 dbname=postgres
mydb = host=127.0.0.1 port=5432 dbname=mydb
[pgbouncer]
logfile = /home/username/pg_log/pgbouncer.log
pidfile = /tmp/pgbouncer.pid
listen_addr = xxx.xxx.xxx.xxx
listen_port = 6432
unix_socket_dir = /var/run/postgresql
auth_type = md5
auth_file = /etc/pgbouncer/userlist.txt
admin_users = postgres
stats_users = stats, root
user = postgres
pool_mode = session
server_reset_query = DISCARD ALL
max_client_conn = 1000
default_pool_size = 20
log_connections = 1
log_disconnections = 1
log_pooler_errors = 1
Im trying to test the connection like this:
psql -p 6432 -U postgres mydb
Here is my log file:
28396 LOG listening on xxx.xxx.xxx.xxx:6432
28396 LOG listening on unix:/var/run/postgresql/.s.PGSQL.6432
28396 LOG process up: pgbouncer 1.4.2, libevent 2.0.16-stable (epoll), adns: evdns2
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 login attempt: db=mydb user=postgres
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 closing because: client unexpected eof (age=0)
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 login attempt: db=mydb user=postgres
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 closing because: client_login_timeout (server down) (age=60)
28864 WARNING C-0x23f7890: mydb/postgres#unix:6432 Pooler Error: client_login_timeout (server down)
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
Is there something else I need to change to make pgbouncer work?

Change
listen_addresses = 'private_ip'
to
listen_addresses = '*'
In the postgresql.conf file.

Looks like you are trying to connect using a Unix socket. Depending on your Postgres installation, the socket may have been created in /tmp rather than /var/run/postgres. I've run into this same issue and it has been discussed here.
Try changing the pgbouncer.ini config to this:
unix_socket_dir = /tmp
You could also try just connecting to localhost by adding the -h as follows:
psql -p 6432 -h localhost -U postgres mydb
Hope this helps!

For me the issue seemed to be when I defined the database in the [databases] section of the config file; I was not able to use host=127.0.0.1 or host=localhost, but I had to use the actual IP, host=192.168.0.1. This could be due to a misconfigured postgresql pg_hba.conf file, but I couldn't find a way to make it work with local IPs.

Related

PgBouncer - Server DNS lookup failed

I am trying to test PgBouncer connection with pgbench on a PostgreSQL server. But I'm getting error as closing because: server DNS lookup failed and at pgbench getting this message:
pgbench -c 10 -t 10 -C -f C:\Users\Administrator\Downloads\query.sql -U postgres -p 6432 -n tags
Password:
pgbench: error: connection to server at "localhost" (::1), port 6432 failed: FATAL: client_login_timeout (server down)
This is observed in PgBouncer log:
LOG C-01715ff0: tags/postgres#[::1]:52768 login attempt: db=tags user=postgres tls=no
WARNING DNS lookup failed: localhost: result=11001
LOG S-0174d218: tags/postgres#(bad-af):0 closing because: server DNS lookup failed (age=0s)
I'm running PostgreSQL-14.5 on Windows Server 2019
PgBouncer Config:
[databases]
postgres = host=localhost port=5435
tags = dbname=postgres host=localhost port=5435 user=postgres password=Admin#!23 auth_user=postgres
[pgbouncer]
logfile = C:\Program Files (x86)\PgBouncer\log\pgbouncer.log
pidfile = C:\Program Files (x86)\PgBouncer\log\pgbouncer.pid
listen_addr = *
listen_port = 6432
auth_type = md5
auth_file = C:\Program Files (x86)\PgBouncer\etc\userlist.txt
admin_users = postgres
stats_users = postgres
pool_mode = session
max_client_conn = 100
default_pool_size = 20
I tried solutions suggested in this post, like, setting
listen_addresses = '*' in postgresql.conf
but could not get the issue resolved. I also searched over the internet for the solution & did not find any.
Am I missing anything or please suggest a solution to this issue. Thanks in advance.
I can connect to Postgres DB after setting [databases] configuration in pgbouncer.ini like this : tags = host=127.0.0.1 port=5435 auth_user=postgres dbname=postgres.
Turns out the issue was with localhost not added in pg_hba.conf.

Unable to setup Postgres in Nix-Shell

I'm working on a Phoenix/Elixir project which is managed using nix-shell. I'm having issues regarding postgres setup.
I'm starting server using pg_ctl -D $PGDATA start and it outputs as:
pg_ctl: another server might be running; trying to start server anyway
waiting for server to start....2021-09-01 19:19:00.780 PKT [72367] LOG: listening on IPv4 address "127.0.0.1", port 5432
2021-09-01 19:19:00.784 PKT [72367] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
2021-09-01 19:19:00.806 PKT [72369] LOG: database system was interrupted; last known up at 2021-09-01 19:14:57 PKT
2021-09-01 19:19:00.848 PKT [72369] LOG: database system was not properly shut down; automatic recovery in progress
2021-09-01 19:19:00.849 PKT [72369] LOG: redo starts at 0/16D84A0
2021-09-01 19:19:00.849 PKT [72369] LOG: invalid record length at 0/16D8580: wanted 24, got 0
2021-09-01 19:19:00.849 PKT [72369] LOG: redo done at 0/16D8548
2021-09-01 19:19:00.862 PKT [72367] LOG: database system is ready to accept connections
done
server started
when I execute createuser postgres --createdb --echo in nix-shell envoirnment it produces following error even though the run directory exists.
createuser: could not connect to database template1: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/run/postgresql/.s.PGSQL.5432"?
No other postgres instance is running.
In pg_hba.conf auth method is set to trust.
unix_socket_directories is set to /tmp.
shell.nix looks something like
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
inherit (lib) optional optionals;
elixir = beam.packages.erlangR22.elixir_1_10;
postgresql = postgresql_11;
in
mkShell {
buildInputs = [
ps
elixir
coreutils
which
git
postgresql
redis
doxygen
mongodb-tools
redis-dump
(python37.withPackages(ps: with ps; [ credstash awscli ]))
cmake
nix-prefetch-git
zlib
jq
teleport
]);
# Fix GLIBC Locale
LOCALE_ARCHIVE = lib.optionalString stdenv.isLinux
"${pkgs.glibcLocales}/lib/locale/locale-archive";
LANG = "en_US.UTF-8";
# Put the PostgreSQL and Redis databases in the project diretory.
PGDATA="./.db/postgres";
RDDATA="./.db/redis";
shellHook = ''
ERL_INCLUDE_PATH="${erlangR22}/lib/erlang/usr/include";
}
I totally a noob to Nix and have no idea how to get it working.
Any help would be appreciated, thanks.
You have to tell createuser to connect via the socket in /tmp:
createuser -h /tmp -U postgres --createdb --echo newuser
You've set a custom path in the postgres config, so you probably also need to specify that with pqsl / createuser. In other words createuser -h /tmp/ ....

pg_Ctl -data directory has wrong ownership

I am unable to start the Postgres server and whenever I use pg_ctl I am getting the following error - can some one help me to fix this. I changed the folder permissions using CHmod and tried running with Sudo -s also but still the problem exists.
one error I did was, I deleted the Postmaster.pid when the server was running- post this I am getting this issue when ever I try to start the server through pg_ctl and another error when I use the pgadmin.
Any suggestions here will be really helpful- thanks.
Using Macos Shell command :
'pg_ctl start -D /Library/PostgreSQL/12/data waiting for server to start....2020-05-05 11:40:04.838 IST [1216] FATAL: data directory "/Library/PostgreSQL/12/data" has wrong ownership 2020-05-05 11:40:04.838 IST [1216] HINT: The server must be started by the user that owns the data directory. stopped waiting pg_ctl: could not start server Examine the log output.'
Using pgadmin the error is as follows :
'could not connect to server: Connection refused Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5434? could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5434?'
p.s. : I modified the hba.conf and also the postgres.conf files to allow connection from the local ip
Error received on 5May
waiting for server to start....2020-05-05 19:54:13.029 IST [7274] LOG: starting PostgreSQL 12.2 on x86_64-apple-darwin, compiled by Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn), 64-bit
2020-05-05 19:54:13.030 IST [7274] LOG: listening on IPv6 address "::", port 5433
2020-05-05 19:54:13.030 IST [7274] LOG: listening on IPv4 address "0.0.0.0", port 5433
2020-05-05 19:54:13.030 IST [7274] LOG: listening on Unix socket "/tmp/.s.PGSQL.5433"
2020-05-05 19:54:13.039 IST [7274] LOG: redirecting log output to logging collector process... 2020-05-05 19:54:13.039 IST [7274] HINT: Future log output will appear in directory "log" stopped waiting .. pg_ctl: could not start server
Examine the log output.
Log file details
2020-05-05 21:29:30.748 IST [8853] LOG: invalid authentication method "0.0.0.0/0"
2020-05-05 21:29:30.748 IST [8853] CONTEXT: line 80 of configuration file "/Library/PostgreSQL/12/data/pg_hba.conf"
2020-05-05 21:29:30.748 IST [8853] FATAL: could not load pg_hba.conf
2020-05-05 21:29:30.749 IST [8853] LOG: database system is shut down
Details of my pg_HBA conf
# "local" is for Unix domain socket connections only
local all all 0.0.0.0/0 md5
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all md5
host replication all 127.0.0.1/32 md5
host replication all ::1/128 md5
host all all 0.0.0.0/0 md5
host all all ::/0 md5
latest log file
bash-3.2$ cat postgresql-2020-05-05_221328.log
2020-05-05 22:13:28.794 IST [9834] LOG: database system was interrupted; last known up at 2020-05-05 22:13:09 IST
2020-05-05 22:13:28.872 IST [9834] LOG: database system was not properly shut down; automatic recovery in progress
2020-05-05 22:13:28.874 IST [9834] LOG: redo starts at 0/17742C8
2020-05-05 22:13:28.874 IST [9834] LOG: invalid record length at 0/1774300: wanted 24, got 0
2020-05-05 22:13:28.874 IST [9834] LOG: redo done at 0/17742C8
2020-05-05 22:13:28.881 IST [9832] LOG: database system is ready to accept connections
......
also I found this error while staring the server and the PID is chaning everytime..
pg_ctl: another server might be running; trying to start server anyway
waiting for server to start....2020-05-05 22:09:21.941 IST [9746] FATAL: lock file "postmaster.pid" already exists
2020-05-05 22:09:21.941 IST [9746] HINT: Is another postmaster (PID 9735) running in data directory "/Library/PostgreSQL/12/data"?
stopped waiting
pg_ctl: could not start server
Examine the log output.
bash-3.2$ kill -9 9735
bash-3.2$ pg_ctl start -D /Library/PostgreSQL/12/data
pg_ctl: another server might be running; trying to start server anyway
waiting for server to start....2020-05-05 22:09:35.829 IST [9758] FATAL: lock file "postmaster.pid" already exists
2020-05-05 22:09:35.829 IST [9758] HINT: Is another postmaster (PID 9747) running in data directory "/Library/PostgreSQL/12/data"?
stopped waiting
pg_ctl: could not start server
Examine the log output.
502 9833 9832 0 10:13PM ?? 0:00.00 postgres: logger
502 9835 9832 0 10:13PM ?? 0:00.00 postgres: checkpointer
502 9836 9832 0 10:13PM ?? 0:00.04 postgres: background writer
502 9837 9832 0 10:13PM ?? 0:00.01 postgres: walwriter
502 9838 9832 0 10:13PM ?? 0:00.01 postgres: autovacuum launcher
502 9839 9832 0 10:13PM ?? 0:00.01 postgres: stats collector
502 9840 9832 0 10:13PM ?? 0:00.00 postgres: logical replication launcher
0 9641 9504 0 10:03PM ttys000 0:00.02 sudo -u postgres -s /bin/bash
502 9904 9642 0 10:37PM ttys000 0:00.00 grep postgres
The data directory should be owned by the postgres user and have user-only access (700 or u+rwx)
Does this match what you have set up?
Thom Brown
Disclosure: I am an EnterpriseDB employee.
Try running this code
pg_ctl -D /usr/local/var/postgres start

pgbouncer error: closing because: client unexpected eof (age=0)

My project is written with golang,I use postgresql Database and use pgbouncer as a database pool connection. I did a pressure test after I finished the basic function.
It runs well as test by the command(just 1 concurrency):
ab -n 1000 -c 1 -p '/home/mwh/postfile.txt' -T 'application/x-www-form-urlencoded' 'http://192.168.1.229:8090/hwSdk/tsk/get_t_k.json'
But when I
increase the concurrency:
ab -n 10000 -c 100 -p '/home/mwh/postfile.txt1 -T 'application/x-www-form-urlencoded' 'http://192.168.1.229:8090/hwSdk/tsk/get_t_k.json'
something happened error with database query in the app log:
[2017-06-01 12:04:00] (pq: bind message supplies 18 parameters, but prepared statement "" requires6)
[2017-06-01 12:04:00] (pq: bind message supplies 19 parameters, but prepared statement "" requires 3)
In the pgbouncer.log file
8035 2017-06-01 12:04:01.702 45619 LOG C-0xf9a4c0: cygnus/cygnus#192.168.1.229:40397 closing because: client close request (age=0)
8036 2017-06-01 12:04:01.703 45619 LOG C-0xf8b9f0: cygnus/cygnus#192.168.1.229:40406 closing because: client close request (age=0)
8037 2017-06-01 12:04:01.703 45619 LOG C-0xf8b248: cygnus/cygnus#192.168.1.229:40574 login attempt: db=cygnus user=cygnus tls=no
8038 2017-06-01 12:04:01.703 45619 LOG C-0xf8c198: cygnus/cygnus#192.168.1.229:40468 closing because: client close request (age=0
8039 2017-06-01 12:04:01.703 45619 LOG C-0xfac6f0: cygnus/cygnus#192.168.1.229:40504 closing because: client close request (age=0)
8040 2017-06-01 12:04:01.707 45619 LOG C-0xf9e698: cygnus/cygnus#192.168.1.229:40235 closing because: client unexpected eof (age=0)
my pgbouncer.ini settings:
pool_mode = Transaction
listen_port = 6543
listen_addr = *
auth_type = md5
auth_file = /etc/pgbouncer/userlist.txt
logfile = /tmp/pgbouncer.log
pidfile = /tmp/pgbouncer.pid
stats_users = readonly,cygnus
max_client_conn = 2000
default_pool_size = 400
ignore_startup_parameters = extra_float_digits
what's the problem?
If you're using transaction mode in PgBouncer you have to set binary_parameters=yes to your database dsn connection
postgres://user:pass#host:post/db_name?binary_parameters=yes

pgbouncer windows 10 - client_login_timeout(server_down) error

I am trying to setup pgbouncer on my local machine. I have standard (didnt change anything after installation) configuration file with entries:
postgres = host=127.0.0.1 port=5432
listen_addr = *
listen_port = 6432
auth_type = trust //also tested with md5 and plain
My postgresql (ver 9.4) is running on port 5432. When I execute
psql -U postgres -p 5432 -d postgres
i can successfully connect. Now i am trying to connect to pgbouncer
psql -U postgres -p 6432 -d postgres
after providing password pgbouncer cannot connect (it hangs for 60 sec) and then timeouts with error
psql: ERROR: client_login_timeout (server down)
Pgbouncer logs:
2017-05-05 00:17:27.084 14696 LOG File descriptor limit: -1 (H:-1), max_client_conn: 100, max fds possible: 130
2017-05-05 00:17:27.104 14696 LOG listening on ::/6432
2017-05-05 00:17:27.105 14696 LOG listening on 0.0.0.0:6432
2017-05-05 00:17:27.106 14696 LOG process up: pgbouncer 1.7.2, libevent 2.0.21-stable (win32), adns: evdns2, tls: OpenSSL 1.0.2k 26 Jan 2017
2017-05-05 00:18:27.104 14696 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
2017-05-05 00:18:51.852 14696 LOG C-009B8FE0: postgres/postgres#[::1]:55878 login attempt: db=postgres user=postgres tls=no
2017-05-05 00:18:51.854 14696 WARNING
2017-05-05 00:18:51.854 14696 LOG S-009EF248: postgres/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
2017-05-05 00:19:06.929 14696 WARNING
2017-05-05 00:19:06.929 14696 LOG S-009EF248: postgres/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
2017-05-05 00:19:21.949 14696 WARNING
2017-05-05 00:19:21.950 14696 LOG S-009EF248: postgres/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
2017-05-05 00:19:27.105 14696 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
2017-05-05 00:19:36.969 14696 WARNING
2017-05-05 00:19:36.970 14696 LOG S-009EF248: postgres/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
2017-05-05 00:19:51.990 14696 LOG C-009B8FE0: postgres/postgres#[::1]:55878 closing because: client_login_timeout (server down) (age=60)
2017-05-05 00:19:51.991 14696 WARNING C-009B8FE0: postgres/postgres#[::1]:55878 Pooler Error: client_login_timeout (server down)
2017-05-05 00:20:27.105 14696 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
hba.conf
host all all 127.0.0.1/32 md5
host all all ::1/128 md5
postgresql.conf
listen_addresses = '*'
What I am doing wrong?
EDIT 1:
Struggling to make this work I tried:
Connecting to other db versions: 9.5 and 9.6
Since I saw in logs that various ports are used I opened whole pgbouncer app in firewall, before that I opened only 6432 port
I thought maybe pgbouncer has problems connecting to localhost so I tried to connect to remote server
Even disabled antivirus
None of above worked. Always the same log shows up (port 5434 is 9.5 version):
2017-05-05 22:26:01.899 8008 LOG C-010C8FF0: postgres/postgres#[::1]:61687 login attempt: db=postgres user=postgres tls=no
2017-05-05 22:26:01.899 8008 LOG C-010C8FF0: postgres/postgres#[::1]:61687 closing because: client unexpected eof (age=0)
2017-05-05 22:26:04.753 8008 LOG C-010C8FF0: postgres/postgres#[::1]:61690 login attempt: db=postgres user=postgres tls=no
2017-05-05 22:26:04.753 8008 WARNING
2017-05-05 22:26:04.753 8008 LOG S-010FF258: postgres/postgres#127.0.0.1:5434 closing because: connect failed (age=0)
2017-05-05 22:26:19.803 8008 WARNING
2017-05-05 22:26:19.803 8008 LOG S-010FF258: postgres/postgres#127.0.0.1:5434 closing because: connect failed (age=0)
2017-05-05 22:26:35.086 8008 WARNING
2017-05-05 22:26:35.086 8008 LOG S-010FF258: postgres/postgres#127.0.0.1:5434 closing because: connect failed (age=0)
2017-05-05 22:26:41.581 8008 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
2017-05-05 22:26:50.359 8008 WARNING
2017-05-05 22:26:50.359 8008 LOG S-010FF258: postgres/postgres#127.0.0.1:5434 closing because: connect failed (age=0)
2017-05-05 22:27:04.961 8008 LOG C-010C8FF0: postgres/postgres#[::1]:61690 closing because: client_login_timeout (server down) (age=60)
2017-05-05 22:27:04.961 8008 WARNING C-010C8FF0: postgres/postgres#[::1]:61690 Pooler Error: client_login_timeout (server down)
Can someone explain why it tries to connect on port 61687? On this port it gets unexpected eof.
Here is the whole pgbouncer.ini (lines that are not commented out):
[databases]
postgres = host=127.0.0.1 port=5434 dbname=postgres
[pgbouncer]
logfile = C:\Program Files\PostgreSQL\PgBouncer\log\pgbouncer.log
pidfile = C:\Program Files\PostgreSQL\PgBouncer\log\pgbouncer.pid
listen_addr = *
listen_port = 6432
auth_type = md5
auth_file = C:\Program Files\PostgreSQL\PgBouncer\etc\userlist.txt
admin_users = postgres
stats_users = postgres
pool_mode = session
max_client_conn = 100
default_pool_size = 20
For me it was an issue of Port number in the [Database] section.
The default (if not set) is 5432.
On my machine, Ubuntu 22 and PostgreSQL 15 the port is 5433.
I set it explicitly in the [Database] section, now it works perfectly.