Shouldn't pgpool cache PG backend processes? After disconnecting and reconnecting pool_backendpidchanges.
Relevant parameters:
num_init_children = 1
max_pool = 1
child_life_time = 300
child_max_connections = 0
connection_life_time = 0
client_idle_limit = 0
connection_cache = on
Test:
postgres#node3:/etc/pgpool2$ psql -p 5433 -U postgres postgres
psql (9.6.10)
Type "help" for help.
postgres=# show pool_pools;
LOG: statement: show pool_pools;
pool_pid | start_time | pool_id | backend_id | database | username | create_time | majorversion | minorversion | pool_counter | pool_backendpid | pool_connected
----------+---------------------+---------+------------+----------+----------+---------------------+--------------+--------------+--------------+-----------------+----------------
3569 | 2018-09-13 20:18:22 | 0 | 0 | postgres | postgres | 2018-09-13 20:25:04 | 3 | 0 | 1 | 3631 | 1
(1 row)
postgres=# \q
postgres#node3:/etc/pgpool2$ psql -p 5433 -U postgres postgres
psql (9.6.10)
Type "help" for help.
postgres=# show pool_pools;
LOG: statement: show pool_pools;
pool_pid | start_time | pool_id | backend_id | database | username | create_time | majorversion | minorversion | pool_counter | pool_backendpid | pool_connected
----------+---------------------+---------+------------+----------+----------+---------------------+--------------+--------------+--------------+-----------------+----------------
3569 | 2018-09-13 20:18:22 | 0 | 0 | postgres | postgres | 2018-09-13 20:25:15 | 3 | 0 | 1 | 3640 | 1
(1 row)
Found out why:
connection_cache (boolean)
Caches connections to backends when set to on. Default is on. However,
connections to template0, template1, postgres and regression databases
are not cached even if connection_cache is on.
I was connecting to postgres database.
Related
I have a Postgres cluster with 3 nodes: ETCD+Patroni+Postgres13.
Now there was a problem of constantly growing pg_wal folder. It now contains 5127 files. After searching the internet, I found an article advising you to pay attention to the following database parameters (their meaning at the time of the case is this):
archive_mode off;
wal_level replica;
max_wal_size 1G;
SELECT * FROM pg_replication_slots;
postgres=# SELECT * FROM pg_replication_slots;
-[ RECORD 1 ]-------+------------
slot_name | db2
plugin |
slot_type | physical
datoid |
database |
temporary | f
active | t
active_pid | 2247228
xmin |
catalog_xmin |
restart_lsn | 2D/D0ADC308
confirmed_flush_lsn |
wal_status | reserved
safe_wal_size |
-[ RECORD 2 ]-------+------------
slot_name | db1
plugin |
slot_type | physical
datoid |
database |
temporary | f
active | t
active_pid | 2247227
xmin |
catalog_xmin |
restart_lsn | 2D/D0ADC308
confirmed_flush_lsn |
wal_status | reserved
safe_wal_size |
All other functionality of the Patroni cluster works (switchover, reinit, replication);
root#srvdb3:~# patronictl -c /etc/patroni/patroni.yml list
+ Cluster: mobile (7173650272103321745) --+----+-----------+
| Member | Host | Role | State | TL | Lag in MB |
+--------+------------+---------+---------+----+-----------+
| db1 | 10.01.1.01 | Replica | running | 17 | 0 |
| db2 | 10.01.1.02 | Replica | running | 17 | 0 |
| db3 | 10.01.1.03 | Leader | running | 17 | |
+--------+------------+---------+---------+----+-----------+
Patroni patroni-edit:
loop_wait: 10
maximum_lag_on_failover: 1048576
postgresql:
parameters:
checkpoint_timeout: 30
hot_standby: 'on'
max_connections: '1100'
max_replication_slots: 5
max_wal_senders: 5
shared_buffers: 2048MB
wal_keep_segments: 5120
wal_level: replica
use_pg_rewind: true
use_slots: true
retry_timeout: 10
ttl: 100
Help please, what could be the matter?
This is what I see in pg_stat_archiver:
postgres=# select * from pg_stat_archiver;
-[ RECORD 1 ]------+------------------------------
archived_count | 0
last_archived_wal |
last_archived_time |
failed_count | 0
last_failed_wal |
last_failed_time |
stats_reset | 2023-01-06 10:21:45.615312+00
If you have wal_keep_segments set to 5120, it is completely normal if you have 5127 WAL segments in pg_wal, because PostgreSQL will always retain at least 5120 old WAL segments. If that is too many for you, reduce the parameter. If you are using replication slots, the only disadvantage is that you might only be able to pg_rewind soon after a failover.
I am new in postgresqlDB configuration. I am trying to configure postgresDB in HA mode with the help of pgpool and Elastic IP. Full setup is in AWS RHEL 8 servers.
pgpool version : 4.1.2
postgres version - 12
Below links I have followed during the configuration:
https://www.pgpool.net/docs/pgpool-II-4.1.2/en/html/example-cluster.html#EXAMPLE-CLUSTER-STRUCTURE
https://www.pgpool.net/docs/42/en/html/example-aws.html
https://www.enterprisedb.com/docs/pgpool/latest/03_configuring_connection_pooling/
Currently the postgres and pgpool services are up in all 3 component nodes. But if I am stopping master postgres service/server whole setup is going down and standby node is not taking the place of master. Please find the status of the pool nodes when master is down:
node_id | hostname | port | status | lb_weight | role | select_cnt | load_balance_node | replication_delay | replication_state | replication_sync_state | last_status_change
---------+--------------+------+--------+-----------+---------+------------+-------------------+-------------------+-------------------+------------------------+---------------------
0 | server1 | 5432 | down | 0.333333 | standby | 0 | false | 0 | | | 2022-10-12 12:10:13
1 | server2 | 5432 | up | 0.333333 | standby | 0 | true | 0 | | | 2022-10-13 09:16:07
2 | server3 | 5432 | up | 0.333333 | standby | 0 | false | 0 | | | 2022-10-13 09:16:07
Any help would be appreciated. Thanks in advance.
When I create a new mysql db, slashdb's test connection fails.
Here is how I log into mysql:
$ mysql -u 7stud -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 15
Server version: 5.5.5-10.4.13-MariaDB Homebrew
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| chat |
| ectoing_repo |
| ejabberd |
| information_schema |
| mydb |
| mysql |
| performance_schema |
| test |
+--------------------+
8 rows in set (0.00 sec)
mysql> use mydb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+----------------+
| Tables_in_mydb |
+----------------+
| cheetos |
| greetings |
| mody |
| people |
+----------------+
4 rows in set (0.00 sec)
mysql> select * from people;
+----+--------+------+
| id | name | info |
+----+--------+------+
| 1 | 7stud | abc |
| 2 | Beth | xxx |
| 3 | Diane | xyz |
| 4 | Kathy | xyz |
| 5 | Kathy | xyz |
| 6 | Dave | efg |
| 7 | Tom | zzz |
| 8 | David | abc |
| 9 | Eloise | abc |
| 10 | Jess | xyz |
| 11 | Jeffsy | 2.0 |
| 12 | XXX | xxx |
| 13 | XXX | xxx |
+----+--------+------+
13 rows in set (0.00 sec)
In the slashdb form for creating a new database, here is the info I entered:
Hostname: 127.0.0.1
Port: 80
Database Login: 7stud
Database Password: **
Database Name: mydb
Then I hit the "Test Connection" button, whereupon I get a spinning wheel, which disappears after a few minutes, but no "Connection Successful" message. What am I doing wrong?
Now, I'm using port 3306:
mysql> SHOW GLOBAL VARIABLES LIKE 'PORT';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| port | 3306 |
+---------------+-------+
1 row in set (0.00 sec)
but when slashdb tries to connect, I get the error:
Host localhost:3306 is not accessible
Your port is wrong in the database connection. You said that your MySQL is configured on port 3306, but you also posted your SlashDB config for database with port 80. Please change that to 3306.
Also, not sure how if you don't need to enable remote access to MySQL. Even if your SlashDB is running on the same machine as the MySQL database, it uses TCP/IP to connect.
I wanted to create a graph in Datadog to display iddle connections per user.
Following this example: http://www.miketheman.net/tag/postgres/ I changed my postgres.yaml configuration to:
init_config:
instances:
- host: 127.0.0.1
port: 5432
username: datadog
password: '**************'
tags:
- environment:qa
- role:db
custom_metrics:
- # Postgres Connection state
descriptors:
- [datname, database]
- [usename, user]
- [state, state]
metrics:
COUNT(state): [postgresql.connection_state, GAUGE]
query: >
SELECT datname, usename, state, %s FROM pg_stat_activity
GROUP BY datname, usename, state HAVING COUNT(state) > 0;
relation: false
I can see the metric is appearing in Datadog, but I can just see one of the rows that should appear (has I have several databases in my PostgreSQL).
Here is the datadog connection graph
Am I missing any step? Is postgres.yaml missing any configuration?
Running that query in psql, I get this (modifying names but not data):
postgres=# SELECT datname, usename, state, COUNT(state) FROM pg_stat_activity GROUP BY datname, usename, state HAVING COUNT(state) > 0;
datname | usename | state | count
---------------------+-------------+---------------------+-------
compan_strawberr_qa | compan | idle in transaction | 1
compan_qa | compan | idle | 130
pineappplee_qa | compan | idle | 10
compan_strawberr_qa | compan | idle | 29
compan_qa | watermel | active | 1
pineappplee_qa | pineappplee | idle | 10
pear_qa | pear | idle | 6
postgres | postgres | active | 1
postgres | datadog | idle | 1
apple_qa | apple_qa | idle | 1
bblluebberrriiess | compan | idle | 3
compan_ser_qa | ser_qa | idle | 4
compan_service_qa | compan | idle | 26
(13 rows)
I found the solution: the user datadog didnt have permission to read connections that wasnt form him. So it was just getting a single row.
I gave permissions for that user to read pg_stat_activity
We usea time series extension prometheus storage adaptor along with postgres to store prometheus data to Postgres.
We have a Prometheus exporter to export Postgres metrics.
And it tries to get number of reads/writes to DB. Since values not showing up, I did following.
postgres=# select * from pg_stat_database;
datid | datname | numbackends | xact_commit | xact_rollback | blks_read | blks_hit | tup_returned | tup_fetched | tup_inserted | tup_updated | tup_deleted | conflicts | temp_files | temp_bytes | deadlocks | blk_read_time | blk_write_time | stats_reset
-------+-----------+-------------+-------------+---------------+-----------+----------+--------------+-------------+--------------+-------------+-------------+-----------+------------+------------+-----------+---------------+----------------+-------------
1 | template1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
12291 | template0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
12292 | postgres | 12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
(3 rows)
Specifically things like tup_fetched, tup_inserted etc shows zero values on 'postgres' database.
But I used following query to find current running queries on the DB.
postgres=# select pg_stat_activity.datname, pg_stat_activity.usename, pg_stat_activity.query from pg_stat_activity;
datname | usename | query
----------+-------------------+-----------------------------------------------------------------------------------------------------------
| postgres |
postgres | psqladmin | COMMIT
postgres | psqladmin | COMMIT
postgres | psqladmin | COMMIT
postgres | psqladmin | COMMIT
postgres | psqladmin | COMMIT
postgres | psqladmin | select * from pg_stat_database;
postgres | postgres_exporter | SELECT * FROM pg_stat_database;
postgres | psqladmin | COMMIT
postgres | psqladmin | COMMIT
postgres | postgres | select pg_stat_activity.datname, pg_stat_activity.usename, pg_stat_activity.query from pg_stat_activity;
postgres | psqladmin | COMMIT
postgres | psqladmin | COMMIT
| |
| |
| |
(16 rows)
Unfortunately I could not see any WRITE per say, but COMMIT as you can see. I assume those are inserts ? But anyway we can see some select queries. So why not tup_fetched or tup_returned non zero ?
Has anyone came across this situation. Any ideas to tackle this issue?