I am trying to setup pgbouncer on my local machine. I have standard (didnt change anything after installation) configuration file with entries:
postgres = host=127.0.0.1 port=5432
listen_addr = *
listen_port = 6432
auth_type = trust //also tested with md5 and plain
My postgresql (ver 9.4) is running on port 5432. When I execute
psql -U postgres -p 5432 -d postgres
i can successfully connect. Now i am trying to connect to pgbouncer
psql -U postgres -p 6432 -d postgres
after providing password pgbouncer cannot connect (it hangs for 60 sec) and then timeouts with error
psql: ERROR: client_login_timeout (server down)
Pgbouncer logs:
2017-05-05 00:17:27.084 14696 LOG File descriptor limit: -1 (H:-1), max_client_conn: 100, max fds possible: 130
2017-05-05 00:17:27.104 14696 LOG listening on ::/6432
2017-05-05 00:17:27.105 14696 LOG listening on 0.0.0.0:6432
2017-05-05 00:17:27.106 14696 LOG process up: pgbouncer 1.7.2, libevent 2.0.21-stable (win32), adns: evdns2, tls: OpenSSL 1.0.2k 26 Jan 2017
2017-05-05 00:18:27.104 14696 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
2017-05-05 00:18:51.852 14696 LOG C-009B8FE0: postgres/postgres#[::1]:55878 login attempt: db=postgres user=postgres tls=no
2017-05-05 00:18:51.854 14696 WARNING
2017-05-05 00:18:51.854 14696 LOG S-009EF248: postgres/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
2017-05-05 00:19:06.929 14696 WARNING
2017-05-05 00:19:06.929 14696 LOG S-009EF248: postgres/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
2017-05-05 00:19:21.949 14696 WARNING
2017-05-05 00:19:21.950 14696 LOG S-009EF248: postgres/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
2017-05-05 00:19:27.105 14696 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
2017-05-05 00:19:36.969 14696 WARNING
2017-05-05 00:19:36.970 14696 LOG S-009EF248: postgres/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
2017-05-05 00:19:51.990 14696 LOG C-009B8FE0: postgres/postgres#[::1]:55878 closing because: client_login_timeout (server down) (age=60)
2017-05-05 00:19:51.991 14696 WARNING C-009B8FE0: postgres/postgres#[::1]:55878 Pooler Error: client_login_timeout (server down)
2017-05-05 00:20:27.105 14696 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
hba.conf
host all all 127.0.0.1/32 md5
host all all ::1/128 md5
postgresql.conf
listen_addresses = '*'
What I am doing wrong?
EDIT 1:
Struggling to make this work I tried:
Connecting to other db versions: 9.5 and 9.6
Since I saw in logs that various ports are used I opened whole pgbouncer app in firewall, before that I opened only 6432 port
I thought maybe pgbouncer has problems connecting to localhost so I tried to connect to remote server
Even disabled antivirus
None of above worked. Always the same log shows up (port 5434 is 9.5 version):
2017-05-05 22:26:01.899 8008 LOG C-010C8FF0: postgres/postgres#[::1]:61687 login attempt: db=postgres user=postgres tls=no
2017-05-05 22:26:01.899 8008 LOG C-010C8FF0: postgres/postgres#[::1]:61687 closing because: client unexpected eof (age=0)
2017-05-05 22:26:04.753 8008 LOG C-010C8FF0: postgres/postgres#[::1]:61690 login attempt: db=postgres user=postgres tls=no
2017-05-05 22:26:04.753 8008 WARNING
2017-05-05 22:26:04.753 8008 LOG S-010FF258: postgres/postgres#127.0.0.1:5434 closing because: connect failed (age=0)
2017-05-05 22:26:19.803 8008 WARNING
2017-05-05 22:26:19.803 8008 LOG S-010FF258: postgres/postgres#127.0.0.1:5434 closing because: connect failed (age=0)
2017-05-05 22:26:35.086 8008 WARNING
2017-05-05 22:26:35.086 8008 LOG S-010FF258: postgres/postgres#127.0.0.1:5434 closing because: connect failed (age=0)
2017-05-05 22:26:41.581 8008 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
2017-05-05 22:26:50.359 8008 WARNING
2017-05-05 22:26:50.359 8008 LOG S-010FF258: postgres/postgres#127.0.0.1:5434 closing because: connect failed (age=0)
2017-05-05 22:27:04.961 8008 LOG C-010C8FF0: postgres/postgres#[::1]:61690 closing because: client_login_timeout (server down) (age=60)
2017-05-05 22:27:04.961 8008 WARNING C-010C8FF0: postgres/postgres#[::1]:61690 Pooler Error: client_login_timeout (server down)
Can someone explain why it tries to connect on port 61687? On this port it gets unexpected eof.
Here is the whole pgbouncer.ini (lines that are not commented out):
[databases]
postgres = host=127.0.0.1 port=5434 dbname=postgres
[pgbouncer]
logfile = C:\Program Files\PostgreSQL\PgBouncer\log\pgbouncer.log
pidfile = C:\Program Files\PostgreSQL\PgBouncer\log\pgbouncer.pid
listen_addr = *
listen_port = 6432
auth_type = md5
auth_file = C:\Program Files\PostgreSQL\PgBouncer\etc\userlist.txt
admin_users = postgres
stats_users = postgres
pool_mode = session
max_client_conn = 100
default_pool_size = 20
For me it was an issue of Port number in the [Database] section.
The default (if not set) is 5432.
On my machine, Ubuntu 22 and PostgreSQL 15 the port is 5433.
I set it explicitly in the [Database] section, now it works perfectly.
Related
So, as in title I cannot connect to database on docker. I created new container to see where is a problem. On new container everything is working just fine, on old one is a problem. Both of them are created in the same way.
d7289b0bb93f postgres "docker-entrypoint.s…" 24 minutes ago Up 14 minutes 0.0.0.0:5544->5432/tcp, :::5544->5432/tcp
bc4118c3356c postgres "docker-entrypoint.s…" 2 months ago Up 14 minutes 0.0.0.0:5437->5432/tcp, :::5437->5432/tcp
The one on 5437 just randomly stoped. I use psql -h localhost -p 5437 -U postgres and I get following error:
psql: connection to server at "localhost" (::1), port 5437 failed:
server closed the connection unexpectedly
This probably means the server terminated abnormally before or while
processing the request.
When I connect to newly created container psql -h localhost -p 5544 -U postgres everything works just fine.
Also the same with ports:
> root#:/var/lib/docker/containers# netstat -tunlp | grep 5437
tcp 0 0 0.0.0.0:5437 0.0.0.0:* LISTEN 16161/docker-proxy
tcp6 0 0 :::5437 :::* LISTEN 16167/docker-proxy
> root#:/var/lib/docker/containers# netstat -tunlp | grep 5544
tcp 0 0 0.0.0.0:5544 0.0.0.0:* LISTEN 16426/docker-proxy
tcp6 0 0 :::5544 :::* LISTEN 16434/docker-proxy
Found something strange in logs, I do not understand it tho.
PostgreSQL Database directory appears to contain a database; Skipping initialization
2022-05-30 13:08:27.890 GMT [1] LOG: starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-05-30 13:08:27.891 GMT [1] LOG: listening on IPv4 address "127.0.0.1", port 5432
2022-05-30 13:08:27.891 GMT [1] LOG: could not bind IPv6 address "::1": Cannot assign requested address
2022-05-30 13:08:27.893 GMT [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-05-30 13:08:27.899 GMT [26] LOG: database system was shut down at 2022-05-30 13:08:25 GMT
2022-05-30 13:08:27.904 GMT [1] LOG: database system is ready to accept connections
2022-05-30 13:12:43.197 GMT [1] LOG: received fast shutdown request
2022-05-30 13:12:43.198 GMT [1] LOG: aborting any active transactions
2022-05-30 13:12:43.200 GMT [1] LOG: background worker "logical replication launcher" (PID 32) exited with exit code 1
2022-05-30 13:12:43.200 GMT [27] LOG: shutting down
2022-05-30 13:12:43.210 GMT [1] LOG: database system is shut down
So... After I opend container and went to /var/lib/postgresql/data/postgresql.conf to my shock file was empty. I mean nothing inside... I dunno what the heck happend! Recoverd file, all is good as new.
I am unable to start the Postgres server and whenever I use pg_ctl I am getting the following error - can some one help me to fix this. I changed the folder permissions using CHmod and tried running with Sudo -s also but still the problem exists.
one error I did was, I deleted the Postmaster.pid when the server was running- post this I am getting this issue when ever I try to start the server through pg_ctl and another error when I use the pgadmin.
Any suggestions here will be really helpful- thanks.
Using Macos Shell command :
'pg_ctl start -D /Library/PostgreSQL/12/data waiting for server to start....2020-05-05 11:40:04.838 IST [1216] FATAL: data directory "/Library/PostgreSQL/12/data" has wrong ownership 2020-05-05 11:40:04.838 IST [1216] HINT: The server must be started by the user that owns the data directory. stopped waiting pg_ctl: could not start server Examine the log output.'
Using pgadmin the error is as follows :
'could not connect to server: Connection refused Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5434? could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5434?'
p.s. : I modified the hba.conf and also the postgres.conf files to allow connection from the local ip
Error received on 5May
waiting for server to start....2020-05-05 19:54:13.029 IST [7274] LOG: starting PostgreSQL 12.2 on x86_64-apple-darwin, compiled by Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn), 64-bit
2020-05-05 19:54:13.030 IST [7274] LOG: listening on IPv6 address "::", port 5433
2020-05-05 19:54:13.030 IST [7274] LOG: listening on IPv4 address "0.0.0.0", port 5433
2020-05-05 19:54:13.030 IST [7274] LOG: listening on Unix socket "/tmp/.s.PGSQL.5433"
2020-05-05 19:54:13.039 IST [7274] LOG: redirecting log output to logging collector process... 2020-05-05 19:54:13.039 IST [7274] HINT: Future log output will appear in directory "log" stopped waiting .. pg_ctl: could not start server
Examine the log output.
Log file details
2020-05-05 21:29:30.748 IST [8853] LOG: invalid authentication method "0.0.0.0/0"
2020-05-05 21:29:30.748 IST [8853] CONTEXT: line 80 of configuration file "/Library/PostgreSQL/12/data/pg_hba.conf"
2020-05-05 21:29:30.748 IST [8853] FATAL: could not load pg_hba.conf
2020-05-05 21:29:30.749 IST [8853] LOG: database system is shut down
Details of my pg_HBA conf
# "local" is for Unix domain socket connections only
local all all 0.0.0.0/0 md5
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all md5
host replication all 127.0.0.1/32 md5
host replication all ::1/128 md5
host all all 0.0.0.0/0 md5
host all all ::/0 md5
latest log file
bash-3.2$ cat postgresql-2020-05-05_221328.log
2020-05-05 22:13:28.794 IST [9834] LOG: database system was interrupted; last known up at 2020-05-05 22:13:09 IST
2020-05-05 22:13:28.872 IST [9834] LOG: database system was not properly shut down; automatic recovery in progress
2020-05-05 22:13:28.874 IST [9834] LOG: redo starts at 0/17742C8
2020-05-05 22:13:28.874 IST [9834] LOG: invalid record length at 0/1774300: wanted 24, got 0
2020-05-05 22:13:28.874 IST [9834] LOG: redo done at 0/17742C8
2020-05-05 22:13:28.881 IST [9832] LOG: database system is ready to accept connections
......
also I found this error while staring the server and the PID is chaning everytime..
pg_ctl: another server might be running; trying to start server anyway
waiting for server to start....2020-05-05 22:09:21.941 IST [9746] FATAL: lock file "postmaster.pid" already exists
2020-05-05 22:09:21.941 IST [9746] HINT: Is another postmaster (PID 9735) running in data directory "/Library/PostgreSQL/12/data"?
stopped waiting
pg_ctl: could not start server
Examine the log output.
bash-3.2$ kill -9 9735
bash-3.2$ pg_ctl start -D /Library/PostgreSQL/12/data
pg_ctl: another server might be running; trying to start server anyway
waiting for server to start....2020-05-05 22:09:35.829 IST [9758] FATAL: lock file "postmaster.pid" already exists
2020-05-05 22:09:35.829 IST [9758] HINT: Is another postmaster (PID 9747) running in data directory "/Library/PostgreSQL/12/data"?
stopped waiting
pg_ctl: could not start server
Examine the log output.
502 9833 9832 0 10:13PM ?? 0:00.00 postgres: logger
502 9835 9832 0 10:13PM ?? 0:00.00 postgres: checkpointer
502 9836 9832 0 10:13PM ?? 0:00.04 postgres: background writer
502 9837 9832 0 10:13PM ?? 0:00.01 postgres: walwriter
502 9838 9832 0 10:13PM ?? 0:00.01 postgres: autovacuum launcher
502 9839 9832 0 10:13PM ?? 0:00.01 postgres: stats collector
502 9840 9832 0 10:13PM ?? 0:00.00 postgres: logical replication launcher
0 9641 9504 0 10:03PM ttys000 0:00.02 sudo -u postgres -s /bin/bash
502 9904 9642 0 10:37PM ttys000 0:00.00 grep postgres
The data directory should be owned by the postgres user and have user-only access (700 or u+rwx)
Does this match what you have set up?
Thom Brown
Disclosure: I am an EnterpriseDB employee.
Try running this code
pg_ctl -D /usr/local/var/postgres start
I'm trying to do python manage.py syncdb on a Django installation, but I keep getting OperationalError: ERROR: pgbouncer cannot connect to server. pgbouncer.log contains lines such as:
2017-09-19 19:44:15.107 1128 LOG C-0x8a9930: mydb/myuser#unix:6432 closing because: pgbouncer cannot connect to server (age=0)
2017-09-19 19:44:15.107 1128 WARNING C-0x8a9930: mydb/myuser#unix:6432 Pooler Error: pgbouncer cannot connect to server
2017-09-19 19:44:15.107 1128 LOG S-0x8c72e0: mydb/myuser#35.154.149.188:5432 new connection to server
2017-09-19 19:44:15.107 1128 LOG C-0x8a9930: mydb/myuser#unix:6432 login failed: db=mydb user=myuser
2017-09-19 19:44:30.108 1128 LOG S-0x8c72e0: mydb/myuser#35.154.149.188:5432 closing because: connect failed (age=15)
In case needed, ps -aef | grep pgbouncer yields:
postgres 1128 1 0 18:38 ? 00:00:00 /usr/sbin/pgbouncer -d /etc/pgbouncer/pgbouncer.ini
myuser 1919 1533 0 19:45 pts/0 00:00:00 grep --color=auto pgbouncer
Moreover, grep port /etc/pgbouncer/pgbouncer.ini results in:
;; dbname= host= port= user= password=
mydb = host=xx.xxx.xxx.xxx port=5432 dbname=mydb
;forcedb = host=127.0.0.1 port=300 user=baz password=foo client_encoding=UNICODE datestyle=ISO connect_query='SELECT 1'
listen_port = 6432
Lastly, the relevant parts of settings.py contain:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'mydb',
'USER': 'myuser',
'PASSWORD': 'mypassword',
'HOST': '/var/run/postgresql',
'PORT': '6432',
}
I turned log_connections to on in postgresql.conf, restarted PG and tried again. Here's the relevant lines:
2017-09-20 07:50:59 UTC LOG: database system is ready to accept connections
2017-09-20 07:50:59 UTC LOG: autovacuum launcher started
2017-09-20 07:51:00 UTC LOG: connection received: host=[local]
2017-09-20 07:51:00 UTC LOG: incomplete startup packet
2017-09-20 07:51:00 UTC LOG: connection received: host=[local]
2017-09-20 07:51:00 UTC LOG: connection authorized: user=postgres database=postgres
2017-09-20 07:51:01 UTC LOG: connection received: host=[local]
2017-09-20 07:51:01 UTC LOG: connection authorized: user=postgres database=postgres
2017-09-20 07:51:01 UTC LOG: connection received: host=[local]
2017-09-20 07:51:01 UTC LOG: connection authorized: user=postgres database=postgres
It seems the connection is going through, but the user and database name is postgres. Those credentials aren't what I supplied in pgbouncer.ini.
However, explicitly adding myuser in the connection string described in pgbouncer.ini leads to:
2017-09-20 09:37:37 UTC FATAL: Peer authentication failed for user "myuser"
2017-09-20 09:37:37 UTC DETAIL: Connection matched pg_hba.conf line 90: "local all all peer"
Totally stumped.
It seems the mis-configuration emanated from this line in settings.py:
'PORT': '6432',
I commented it and pgbouncer started working.
Though I'm not sure 'why'.
Maybe there's a collision on this port; Pgbouncer and PG coexist on a single server in my case. I've set them up over different VMs in the past without a hitch (and without needing to comment 'PORT': '6432',)
My project is written with golang,I use postgresql Database and use pgbouncer as a database pool connection. I did a pressure test after I finished the basic function.
It runs well as test by the command(just 1 concurrency):
ab -n 1000 -c 1 -p '/home/mwh/postfile.txt' -T 'application/x-www-form-urlencoded' 'http://192.168.1.229:8090/hwSdk/tsk/get_t_k.json'
But when I
increase the concurrency:
ab -n 10000 -c 100 -p '/home/mwh/postfile.txt1 -T 'application/x-www-form-urlencoded' 'http://192.168.1.229:8090/hwSdk/tsk/get_t_k.json'
something happened error with database query in the app log:
[2017-06-01 12:04:00] (pq: bind message supplies 18 parameters, but prepared statement "" requires6)
[2017-06-01 12:04:00] (pq: bind message supplies 19 parameters, but prepared statement "" requires 3)
In the pgbouncer.log file
8035 2017-06-01 12:04:01.702 45619 LOG C-0xf9a4c0: cygnus/cygnus#192.168.1.229:40397 closing because: client close request (age=0)
8036 2017-06-01 12:04:01.703 45619 LOG C-0xf8b9f0: cygnus/cygnus#192.168.1.229:40406 closing because: client close request (age=0)
8037 2017-06-01 12:04:01.703 45619 LOG C-0xf8b248: cygnus/cygnus#192.168.1.229:40574 login attempt: db=cygnus user=cygnus tls=no
8038 2017-06-01 12:04:01.703 45619 LOG C-0xf8c198: cygnus/cygnus#192.168.1.229:40468 closing because: client close request (age=0
8039 2017-06-01 12:04:01.703 45619 LOG C-0xfac6f0: cygnus/cygnus#192.168.1.229:40504 closing because: client close request (age=0)
8040 2017-06-01 12:04:01.707 45619 LOG C-0xf9e698: cygnus/cygnus#192.168.1.229:40235 closing because: client unexpected eof (age=0)
my pgbouncer.ini settings:
pool_mode = Transaction
listen_port = 6543
listen_addr = *
auth_type = md5
auth_file = /etc/pgbouncer/userlist.txt
logfile = /tmp/pgbouncer.log
pidfile = /tmp/pgbouncer.pid
stats_users = readonly,cygnus
max_client_conn = 2000
default_pool_size = 400
ignore_startup_parameters = extra_float_digits
what's the problem?
If you're using transaction mode in PgBouncer you have to set binary_parameters=yes to your database dsn connection
postgres://user:pass#host:post/db_name?binary_parameters=yes
Im trying to get pgbouncer to work on my standalone postgresql server, but when I try to connect to the database using the pgbouncer port, I get an timeout error.
Postgresql: 9.1.8
pgBouncer: 1.4.2
OS: Ubuntu 12.04.2 LTS (GNU/Linux 3.8.4-x86_64-linode31 x86_64)
Here is my conf file:
[databases]
postgres = host=127.0.0.1 port=5432 dbname=postgres
mydb = host=127.0.0.1 port=5432 dbname=mydb
[pgbouncer]
logfile = /home/username/pg_log/pgbouncer.log
pidfile = /tmp/pgbouncer.pid
listen_addr = xxx.xxx.xxx.xxx
listen_port = 6432
unix_socket_dir = /var/run/postgresql
auth_type = md5
auth_file = /etc/pgbouncer/userlist.txt
admin_users = postgres
stats_users = stats, root
user = postgres
pool_mode = session
server_reset_query = DISCARD ALL
max_client_conn = 1000
default_pool_size = 20
log_connections = 1
log_disconnections = 1
log_pooler_errors = 1
Im trying to test the connection like this:
psql -p 6432 -U postgres mydb
Here is my log file:
28396 LOG listening on xxx.xxx.xxx.xxx:6432
28396 LOG listening on unix:/var/run/postgresql/.s.PGSQL.6432
28396 LOG process up: pgbouncer 1.4.2, libevent 2.0.16-stable (epoll), adns: evdns2
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 login attempt: db=mydb user=postgres
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 closing because: client unexpected eof (age=0)
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 login attempt: db=mydb user=postgres
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 closing because: client_login_timeout (server down) (age=60)
28864 WARNING C-0x23f7890: mydb/postgres#unix:6432 Pooler Error: client_login_timeout (server down)
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
Is there something else I need to change to make pgbouncer work?
Change
listen_addresses = 'private_ip'
to
listen_addresses = '*'
In the postgresql.conf file.
Looks like you are trying to connect using a Unix socket. Depending on your Postgres installation, the socket may have been created in /tmp rather than /var/run/postgres. I've run into this same issue and it has been discussed here.
Try changing the pgbouncer.ini config to this:
unix_socket_dir = /tmp
You could also try just connecting to localhost by adding the -h as follows:
psql -p 6432 -h localhost -U postgres mydb
Hope this helps!
For me the issue seemed to be when I defined the database in the [databases] section of the config file; I was not able to use host=127.0.0.1 or host=localhost, but I had to use the actual IP, host=192.168.0.1. This could be due to a misconfigured postgresql pg_hba.conf file, but I couldn't find a way to make it work with local IPs.