I am trying to test PgBouncer connection with pgbench on a PostgreSQL server. But I'm getting error as closing because: server DNS lookup failed and at pgbench getting this message:
pgbench -c 10 -t 10 -C -f C:\Users\Administrator\Downloads\query.sql -U postgres -p 6432 -n tags
Password:
pgbench: error: connection to server at "localhost" (::1), port 6432 failed: FATAL: client_login_timeout (server down)
This is observed in PgBouncer log:
LOG C-01715ff0: tags/postgres#[::1]:52768 login attempt: db=tags user=postgres tls=no
WARNING DNS lookup failed: localhost: result=11001
LOG S-0174d218: tags/postgres#(bad-af):0 closing because: server DNS lookup failed (age=0s)
I'm running PostgreSQL-14.5 on Windows Server 2019
PgBouncer Config:
[databases]
postgres = host=localhost port=5435
tags = dbname=postgres host=localhost port=5435 user=postgres password=Admin#!23 auth_user=postgres
[pgbouncer]
logfile = C:\Program Files (x86)\PgBouncer\log\pgbouncer.log
pidfile = C:\Program Files (x86)\PgBouncer\log\pgbouncer.pid
listen_addr = *
listen_port = 6432
auth_type = md5
auth_file = C:\Program Files (x86)\PgBouncer\etc\userlist.txt
admin_users = postgres
stats_users = postgres
pool_mode = session
max_client_conn = 100
default_pool_size = 20
I tried solutions suggested in this post, like, setting
listen_addresses = '*' in postgresql.conf
but could not get the issue resolved. I also searched over the internet for the solution & did not find any.
Am I missing anything or please suggest a solution to this issue. Thanks in advance.
I can connect to Postgres DB after setting [databases] configuration in pgbouncer.ini like this : tags = host=127.0.0.1 port=5435 auth_user=postgres dbname=postgres.
Turns out the issue was with localhost not added in pg_hba.conf.
Related
I followed these steps to set up QWC services https://github.com/qwc-services/qwc-services-core#quick-start and I can run the demo. But if load my own QGIS project, I receive the following error message:
qwc-qgis-server_1 | 07:50:07 WARNING Server[99]: <ServerException>Layer(s) not valid</ServerException>
qwc-qgis-server_1 |
qwc-qgis-server_1 | 07:50:07 WARNING ClearCapabilities[99]: Cached cleared : /data/MeasurementDemo.qgs
qwc-qgis-server_1 | 07:50:07 WARNING PostGIS[99]: Connection to database failed
qwc-qgis-server_1 | could not connect to server: No such file or directory
qwc-qgis-server_1 | Is the server running locally and accepting
qwc-qgis-server_1 | connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
qwc-qgis-server_1 |
qwc-qgis-server_1 | 07:50:07 CRITICAL Server[99]: Error, Layer(s) measurement_b46e976f_2d0f_4bf0_942a_9d9462b40c3e not valid in project /data/MeasurementDemo.qgs
qwc-qgis-server_1 | 07:50:07 WARNING Server[99]: <ServerException>Layer(s) not valid</ServerException>
qwc-qgis-server_1 |
qwc-config-service_1 | [2022-01-04 07:50:09,360] WARNING in config_generator: Skipping theme item '': Could not get capabilities for /ows/MeasurementDemo
qwc-config-service_1 | [2022-01-04 07:50:19,468] CRITICAL in config_generator: The generation of the configuration files resulted in a failure
qwc-config-service_1 | [2022-01-04 07:50:19,468] CRITICAL in config_generator: The configuration files were not updated!
qwc-config-service_1 | [2022-01-04 07:50:20,856] CRITICAL in config_generator: The generation of the permission files resulted in a failure.
qwc-config-service_1 | [2022-01-04 07:50:20,857] CRITICAL in config_generator: The permission files were not updated!
qwc-config-service_1 | [pid: 15|app: 0|req: 18/18] 172.18.0.11 () {30 vars in 408 bytes} [Tue Jan 4 07:50:05 2022] POST /generate_configs?tenant=default => generated 2881 bytes in 15083 msecs (HTTP/1.1 200) 2 headers in 81 bytes (1 switches on core 0)
As the error is quite similar to this question: PostgreSQL: Why psql can't connect to server?, I followed the answers but with no result.
ps -ef | grep postgres gives me the following result:
postgres 203911 1 0 07:35 ? 00:00:00 /usr/lib/postgresql/13/bin/postgres -D /var/lib/postgresql/13/main -c config_file=/etc/postgresql/13/main/postgresql.conf
Also I found the socket in
/var/run/postgresql/.s.PGSQL.5432
And I run the command
psql -h /var/run/postgresql/ GeoDB
But without result. After that I checked the ph_hba.conf File:
# "local" is for Unix domain socket connections only
local all all peer
Running the command pg_lsclusters gives me:
Ver Cluster Port Status Owner Data directory Log file
13 main 5432 online postgres /var/lib/postgresql/13/main /var/log/postgresql/postgresql-13-main.log
Also after restarting the pg_ctlcluster and PostgreSQL the error remained the same.
Edit 1
After the answer from cnaimi I checked the postgresql.confFile:
# - Connection Settings -
#listen_addresses = '*' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost'; use '*' for all
# (change requires restart)
port = 5432 # (change requires restart)
max_connections = 100 # (change requires restart)
#superuser_reserved_connections = 3 # (change requires restart)
unix_socket_directories = '/var/run/postgresql' # comma-separated list of directories
# (change requires restart)
#unix_socket_group = '*' # (change requires restart)
#unix_socket_permissions = 0777 # begin with 0 to use octal notation
# (change requires restart)
#bonjour = off # advertise server via Bonjour
# (change requires restart)
#bonjour_name = '' # defaults to the computer name
# (change requires restart)
But I can't find an error there as the port is 5432 and it listen to all adresses.
Edit 2
During my search I found several pg_service.conf Files:
./qwc-services/qwc-docker/wsgi-service/pg_service.conf
./qwc-services/qwc-docker/qgis-server/pg_service.conf
./qwc-services/qwc-docker/postgis/pg_service.conf
./qwc-services/qwc-docker/pg_service.conf
Each if them contain one or more credentials for databases like the one below:
[qwc_geodb]
host=qwc-postgis
port=5432
dbname=qwc_demo
user=qwc_service
password=qwc_service
sslmode=disable
The port is in all files correct, as far as I saw. But of course the db name and user/password are wrong. Does this could cause the error? Or does QWS get the credentials through the .qgs file?
Edit 3
Thanks to the hints from Devdatta Tengshe I set the host for PostgreSQL to 127.0.0.1. By using sudo docker-compose ps one can see the used container and their ports:
Name Command State Ports
-------------------------------------------------------------------------------------------------------------------------------
qwc-docker_qwc-admin-gui_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5031->9090/tcp
qwc-docker_qwc-api-gateway_1 /docker-entrypoint.sh ngin ... Up 0.0.0.0:8088->80/tcp,:::8088->80/tcp
qwc-docker_qwc-auth-service_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5017->9090/tcp
qwc-docker_qwc-config-service_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5010->9090/tcp
qwc-docker_qwc-data-service_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5012->9090/tcp
qwc-docker_qwc-elevation-service_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5002->9090/tcp
qwc-docker_qwc-fulltext-search-service_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5011->9090/tcp
qwc-docker_qwc-map-viewer_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5030->9090/tcp
qwc-docker_qwc-mapinfo-service_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5016->9090/tcp
qwc-docker_qwc-ogc-service_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5013->9090/tcp
qwc-docker_qwc-permalink-service_1 /bin/sh -c uwsgi --http-so ... Up 127.0.0.1:5001->9090/tcp
qwc-docker_qwc-postgis_1 docker-entrypoint.sh postgres Up (healthy) 127.0.0.1:5439->5432/tcp
qwc-docker_qwc-qgis-server_1 /sbin/my_init Up 127.0.0.1:8001->80/tcp
qwc-docker_qwc-solr_1 docker-entrypoint.sh solr- ... Up 127.0.0.1:8983->8983/tcp
Can you check the postgres.conf file located in
/etc/postgresql/13/main/postgresql.conf
specially the parameter listen_address
Maybe you have to specify from which host you are listening.
But if the demo example is working the database configuration should be ok.
You can also check the port for postgres on postgres.conf and validate it's 5432.
There are a couple of things that need to be fixed to get this working.
I'm assuming that you have the Postgres Server running on the host machine, and not within any Docker container.
When you configured your QGIS Map file, you probably connected to localhost, and this information got saved in the .qgs file.
This is why your first error message says that it trying to connect to localhost, and no server was found. This error was thrown within the qwc docker container.
This error is occuring, because QGIS server (within the docker container) is not able to connect to the postgres server which is running on the host, using 'localhost' as the hostname
To solve this, you need to do the following:
In QGIS, connect to the Postgres Server using 127.0.0.1 and not localhost.
Save your qgs file using this new connection.
When you run the docker container for qwc, use --network="host" as the commandline parameter.
See: From inside of a Docker container, how do I connect to the localhost of the machine?
After this, the qgis server (within docker container) should be able to connect to the Postgres Server running on your host, using 127.0.0.1 as IP address.
I have 2 ubuntu-20.04 VM on VMWARE with Postgres 12 installed on each
pgprimary on ip 192.168.1.131
pgbackup on ip 192.168.1.130
barman CLI tools are installed on pgprimary
barman is installed on pgbackup
I want to backup data from pgprimary on pgbackupsame 2 users as Postgress users
on each machine I created
2 Linux sudoist users
useradd barman
useradd streaming_barman
also created the same two user as Postgress users
createuser --superuser --replication -P barman
createuser --superuser --replication -P streaming_barman
here are relevant parts on the configuration files
On pgprimary
postgressql.conf
listen_addresses = '*' # what IP address(es) to listen on;
port = 5432
archive_mode = on
archive_command = 'cp %p /var/lib/postgresql/12/arc/%f'
wal_level = replica
restore_command = 'cp /var/lib/postgresql/12/arc/%f %p'
recovery_target_time = '2021-03-24 16:18:11.319298+05:30'
recovery_target_inclusive = false
pg_hba.conf
local all postgres peer
# TYPE DATABASE USER ADDRESS METHOD
local all all peer
host all all 127.0.0.1/32 md5
host all all ::1/128 md5
#local replication all peer
#host replication all 127.0.0.1/32 md5
#host replication all ::1/128 md5
# FOR TESTING
local replication all trust
host replication all 127.0.0.1/32 trust
host replication all ::1/128 trust
also did
firewall-cmd --permanent --add-port=5432/tcp
firewall-cmd --reload
========================
con
On pgbackup
sudo cat <<'EOF' >> /etc/barman.d/pgprimary.conf
[pgprimary]
description = "Example of PostgreSQL Database (Streaming-Only)"
conninfo = host=192.168.1.131 user=barman dbname=training
streaming_conninfo = host=192.168.1.131 user=streaming_barman dbname=training
backup_method = postgres
streaming_archiver = on
slot_name = barman
create_slot = auto
EOF
pg_hba.conf
cat <<'EOF' >>~/.pgpass
pgprimary:*:*:barman:barman
pgprimary:*:*:streaming_barman:barman
EOF
Then I did
barman cron
Output
Starting WAL archiving for server pgprimary
Starting streaming archiver for server pgprimary
barman check pgprimary
Then I get this error
[13643] barman.utils WARNING: Failed opening the requested log file. Using standard error instead.
Server pgprimary:
2021-10-30 21:39:15,982 [13643] barman.server ERROR: Check 'WAL archive' failed for server 'pgprimary'
WAL archive: FAILED (please make sure WAL shipping is setup)
2021-10-30 21:39:37,006 [13643] barman.postgres WARNING: Error retrieving PostgreSQL status: connection to server at "192.168.131" (192.168.0.131), port 5432 failed: Connection refused
2021-10-30 21:39:58,021 [13643] barman.server ERROR: Check 'check timeout' failed for server 'pgprimary'
check timeout: FAILED (barman check command timed out)
Why cannot connect barman to the server ?
UPDATE:
psql -h 192.168.1.131 -U barman -d training
Password for user barman:
psql (12.8 (Ubuntu 12.8-0ubuntu0.20.04.1))
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
I also can connect to server via netstat
When I put postgres -D /usr/local/var/postgres in terminal I get this back:
LOG: could not translate host name "localhost", service "5432" to address: nodename nor servname provided, or not known
WARNING: could not create listen socket for "localhost"
FATAL: could not create any TCP/IP sockets`
Can someone help me fix this?
Check your /etc/hosts file for missing line:
127.0.0.1 localhost.localdomain localhost
Regards
H
Im trying to get pgbouncer to work on my standalone postgresql server, but when I try to connect to the database using the pgbouncer port, I get an timeout error.
Postgresql: 9.1.8
pgBouncer: 1.4.2
OS: Ubuntu 12.04.2 LTS (GNU/Linux 3.8.4-x86_64-linode31 x86_64)
Here is my conf file:
[databases]
postgres = host=127.0.0.1 port=5432 dbname=postgres
mydb = host=127.0.0.1 port=5432 dbname=mydb
[pgbouncer]
logfile = /home/username/pg_log/pgbouncer.log
pidfile = /tmp/pgbouncer.pid
listen_addr = xxx.xxx.xxx.xxx
listen_port = 6432
unix_socket_dir = /var/run/postgresql
auth_type = md5
auth_file = /etc/pgbouncer/userlist.txt
admin_users = postgres
stats_users = stats, root
user = postgres
pool_mode = session
server_reset_query = DISCARD ALL
max_client_conn = 1000
default_pool_size = 20
log_connections = 1
log_disconnections = 1
log_pooler_errors = 1
Im trying to test the connection like this:
psql -p 6432 -U postgres mydb
Here is my log file:
28396 LOG listening on xxx.xxx.xxx.xxx:6432
28396 LOG listening on unix:/var/run/postgresql/.s.PGSQL.6432
28396 LOG process up: pgbouncer 1.4.2, libevent 2.0.16-stable (epoll), adns: evdns2
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 login attempt: db=mydb user=postgres
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 closing because: client unexpected eof (age=0)
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 login attempt: db=mydb user=postgres
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 new connection to server
28864 LOG S-0x2415240: mydb/postgres#127.0.0.1:5432 closing because: connect failed (age=0)
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
28864 LOG C-0x23f7890: mydb/postgres#unix:6432 closing because: client_login_timeout (server down) (age=60)
28864 WARNING C-0x23f7890: mydb/postgres#unix:6432 Pooler Error: client_login_timeout (server down)
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
28864 LOG Stats: 0 req/s, in 0 b/s, out 0 b/s,query 0 us
Is there something else I need to change to make pgbouncer work?
Change
listen_addresses = 'private_ip'
to
listen_addresses = '*'
In the postgresql.conf file.
Looks like you are trying to connect using a Unix socket. Depending on your Postgres installation, the socket may have been created in /tmp rather than /var/run/postgres. I've run into this same issue and it has been discussed here.
Try changing the pgbouncer.ini config to this:
unix_socket_dir = /tmp
You could also try just connecting to localhost by adding the -h as follows:
psql -p 6432 -h localhost -U postgres mydb
Hope this helps!
For me the issue seemed to be when I defined the database in the [databases] section of the config file; I was not able to use host=127.0.0.1 or host=localhost, but I had to use the actual IP, host=192.168.0.1. This could be due to a misconfigured postgresql pg_hba.conf file, but I couldn't find a way to make it work with local IPs.
I downloaded PostgreSQL from the official website and ran the .dmg installer. After that I downloaded pgadmin3 and I am indeed able to connect to the database.
when I run 'psql' I get the following error:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
After hours of googling I read about some $PATH issues, so I put this into my .bashrc:
export PATH=/Library/PostgreSQL/9.1/bin:$PATH
However, this doesn't solve the error above at all. After some more hours of googling I tried to run 'psql -l localhost -U postgres'. This gives another error:
psql: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (fe80::1) and accepting
TCP/IP connections on port 5432?
After some more googling I tried to edit /Library/PostgreSQL/9.1/data/pg_hba.conf and replaced all occurrences of 'md5' with 'trust'.
Then I changed user to postgres and executed 'pg_ctl stop' and 'pg_ctl start', switched back to my own user and tried to connect again, no luck.
Here are some more infos:
[~]$ which psql
/Library/PostgreSQL/9.1/bin/psql
ps aux | grep postgres
postgres 19022 0.0 0.0 2446096 484 ?? Ss 11:31PM 0:00.01 postgres: stats collector process
postgres 19021 0.0 0.0 2486532 1776 ?? Ss 11:31PM 0:00.01 postgres: autovacuum launcher process
postgres 19020 0.0 0.0 2486400 576 ?? Ss 11:31PM 0:00.03 postgres: wal writer process
postgres 19019 0.0 0.0 2486400 820 ?? Ss 11:31PM 0:00.05 postgres: writer process
postgres 19017 0.0 0.0 2446096 356 ?? Ss 11:31PM 0:00.01 postgres: logger process
postgres 19015 0.0 0.1 2486400 8216 s001 S 11:31PM 0:00.17 /Library/PostgreSQL/9.1/bin/postgres
sudo find / -name .s.PGSQL.5432
No file was found?!?
Update 1:
In /etc/sysctl.conf I added the values suggested by the installer's README:
kern.sysv.shmmax=1610612736
kern.sysv.shmall=393216
kern.sysv.shmmin=1
kern.sysv.shmmni=32
kern.sysv.shmseg=8
kern.maxprocperuid=512
Before these settings, the installer quits with an error, afterwards the wizard appears and installs postgres (and again, using pgadmin3 works, so I assume that the database is running fine).
Run lsof on the master process to all of this. In your case it's 19015 (shown with my PID):
> sudo lsof -p 286 | awk '$5 == "unix" && $NF ~ /\// { print $NF }'
/tmp/.s.PGSQL.5432
You can leave off the awk, but basically it's getting the UNIX socket on which postgres is listening. From there, you can use the -h option to psql (but only include the directory).
> psql -h /tmp template1
template1=# \q
If that doesn't work, you can check the lsof output to show you what TCP port it's actually listening on, and if it's not 5432, use the -p option to postgres
If you are able to connect to the database through pgadmin3, then your database is running.
(1) I'd verify by running netstat -an | grep 5432 on the command line, as a paranoia check. You are in business i.e. you actually installed Postgresql if you get this below:
tcp4 0 0 *.5432 . LISTEN
tcp6 0 0 *.5432 . LISTEN
ffffff80133bfed8 stream 0 0 ffffff8013be8000 0 0 0 /tmp/.s.PGSQL.5432
(2) Speaking for myself, I found that just running psql gets me your psql error message. I had to run
psql -h 127.0.0.1 -p 5432 [-d database] -U postgres
to connect with the database. In other words, I had to explicitly specify IP and port. Since you were able to run psql and get a non-connect error message, the OS understands that psql is a command and you don't have a pathing issue.