I wanted to create a graph in Datadog to display iddle connections per user.
Following this example: http://www.miketheman.net/tag/postgres/ I changed my postgres.yaml configuration to:
init_config:
instances:
- host: 127.0.0.1
port: 5432
username: datadog
password: '**************'
tags:
- environment:qa
- role:db
custom_metrics:
- # Postgres Connection state
descriptors:
- [datname, database]
- [usename, user]
- [state, state]
metrics:
COUNT(state): [postgresql.connection_state, GAUGE]
query: >
SELECT datname, usename, state, %s FROM pg_stat_activity
GROUP BY datname, usename, state HAVING COUNT(state) > 0;
relation: false
I can see the metric is appearing in Datadog, but I can just see one of the rows that should appear (has I have several databases in my PostgreSQL).
Here is the datadog connection graph
Am I missing any step? Is postgres.yaml missing any configuration?
Running that query in psql, I get this (modifying names but not data):
postgres=# SELECT datname, usename, state, COUNT(state) FROM pg_stat_activity GROUP BY datname, usename, state HAVING COUNT(state) > 0;
datname | usename | state | count
---------------------+-------------+---------------------+-------
compan_strawberr_qa | compan | idle in transaction | 1
compan_qa | compan | idle | 130
pineappplee_qa | compan | idle | 10
compan_strawberr_qa | compan | idle | 29
compan_qa | watermel | active | 1
pineappplee_qa | pineappplee | idle | 10
pear_qa | pear | idle | 6
postgres | postgres | active | 1
postgres | datadog | idle | 1
apple_qa | apple_qa | idle | 1
bblluebberrriiess | compan | idle | 3
compan_ser_qa | ser_qa | idle | 4
compan_service_qa | compan | idle | 26
(13 rows)
I found the solution: the user datadog didnt have permission to read connections that wasnt form him. So it was just getting a single row.
I gave permissions for that user to read pg_stat_activity
Related
I have a Postgres cluster with 3 nodes: ETCD+Patroni+Postgres13.
Now there was a problem of constantly growing pg_wal folder. It now contains 5127 files. After searching the internet, I found an article advising you to pay attention to the following database parameters (their meaning at the time of the case is this):
archive_mode off;
wal_level replica;
max_wal_size 1G;
SELECT * FROM pg_replication_slots;
postgres=# SELECT * FROM pg_replication_slots;
-[ RECORD 1 ]-------+------------
slot_name | db2
plugin |
slot_type | physical
datoid |
database |
temporary | f
active | t
active_pid | 2247228
xmin |
catalog_xmin |
restart_lsn | 2D/D0ADC308
confirmed_flush_lsn |
wal_status | reserved
safe_wal_size |
-[ RECORD 2 ]-------+------------
slot_name | db1
plugin |
slot_type | physical
datoid |
database |
temporary | f
active | t
active_pid | 2247227
xmin |
catalog_xmin |
restart_lsn | 2D/D0ADC308
confirmed_flush_lsn |
wal_status | reserved
safe_wal_size |
All other functionality of the Patroni cluster works (switchover, reinit, replication);
root#srvdb3:~# patronictl -c /etc/patroni/patroni.yml list
+ Cluster: mobile (7173650272103321745) --+----+-----------+
| Member | Host | Role | State | TL | Lag in MB |
+--------+------------+---------+---------+----+-----------+
| db1 | 10.01.1.01 | Replica | running | 17 | 0 |
| db2 | 10.01.1.02 | Replica | running | 17 | 0 |
| db3 | 10.01.1.03 | Leader | running | 17 | |
+--------+------------+---------+---------+----+-----------+
Patroni patroni-edit:
loop_wait: 10
maximum_lag_on_failover: 1048576
postgresql:
parameters:
checkpoint_timeout: 30
hot_standby: 'on'
max_connections: '1100'
max_replication_slots: 5
max_wal_senders: 5
shared_buffers: 2048MB
wal_keep_segments: 5120
wal_level: replica
use_pg_rewind: true
use_slots: true
retry_timeout: 10
ttl: 100
Help please, what could be the matter?
This is what I see in pg_stat_archiver:
postgres=# select * from pg_stat_archiver;
-[ RECORD 1 ]------+------------------------------
archived_count | 0
last_archived_wal |
last_archived_time |
failed_count | 0
last_failed_wal |
last_failed_time |
stats_reset | 2023-01-06 10:21:45.615312+00
If you have wal_keep_segments set to 5120, it is completely normal if you have 5127 WAL segments in pg_wal, because PostgreSQL will always retain at least 5120 old WAL segments. If that is too many for you, reduce the parameter. If you are using replication slots, the only disadvantage is that you might only be able to pg_rewind soon after a failover.
When I create a new mysql db, slashdb's test connection fails.
Here is how I log into mysql:
$ mysql -u 7stud -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 15
Server version: 5.5.5-10.4.13-MariaDB Homebrew
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| chat |
| ectoing_repo |
| ejabberd |
| information_schema |
| mydb |
| mysql |
| performance_schema |
| test |
+--------------------+
8 rows in set (0.00 sec)
mysql> use mydb;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+----------------+
| Tables_in_mydb |
+----------------+
| cheetos |
| greetings |
| mody |
| people |
+----------------+
4 rows in set (0.00 sec)
mysql> select * from people;
+----+--------+------+
| id | name | info |
+----+--------+------+
| 1 | 7stud | abc |
| 2 | Beth | xxx |
| 3 | Diane | xyz |
| 4 | Kathy | xyz |
| 5 | Kathy | xyz |
| 6 | Dave | efg |
| 7 | Tom | zzz |
| 8 | David | abc |
| 9 | Eloise | abc |
| 10 | Jess | xyz |
| 11 | Jeffsy | 2.0 |
| 12 | XXX | xxx |
| 13 | XXX | xxx |
+----+--------+------+
13 rows in set (0.00 sec)
In the slashdb form for creating a new database, here is the info I entered:
Hostname: 127.0.0.1
Port: 80
Database Login: 7stud
Database Password: **
Database Name: mydb
Then I hit the "Test Connection" button, whereupon I get a spinning wheel, which disappears after a few minutes, but no "Connection Successful" message. What am I doing wrong?
Now, I'm using port 3306:
mysql> SHOW GLOBAL VARIABLES LIKE 'PORT';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| port | 3306 |
+---------------+-------+
1 row in set (0.00 sec)
but when slashdb tries to connect, I get the error:
Host localhost:3306 is not accessible
Your port is wrong in the database connection. You said that your MySQL is configured on port 3306, but you also posted your SlashDB config for database with port 80. Please change that to 3306.
Also, not sure how if you don't need to enable remote access to MySQL. Even if your SlashDB is running on the same machine as the MySQL database, it uses TCP/IP to connect.
I cannot create a database with postgres 9.6.12, viewing pg_activity there's no blocking and waiting queries
this is my query:
-[ RECORD 1 ]----+------------------------------------
datid | 16390
datname | mydb
pid | 7275
usesysid | 10
usename | postgres96
application_name | pgAdmin III - Query Tool
client_addr | myip
client_hostname | mypc
client_port | 55202
backend_start | 2019-07-22 09:12:11.238705-04
xact_start | 2019-07-22 09:12:13.010278-04
query_start | 2019-07-22 09:12:13.010278-04
state_change | 2019-07-22 09:12:13.010282-04
wait_event_type |
wait_event |
state | active
backend_xid | 991367173
backend_xmin | 991367173
query | CREATE DATABASE mydb2\r +
| WITH OWNER = postgres96\r +
| ENCODING = 'UTF8'\r +
| TABLESPACE = system\r +
| LC_COLLATE = 'en_US.UTF-8'\r+
| LC_CTYPE = 'en_US.UTF-8'\r +
| CONNECTION LIMIT = -1;
why is tacking so long?
well... after dropping all subscriptions of pglogical and restart de service I could create the database (I couldn't after a simple restart)
Shouldn't pgpool cache PG backend processes? After disconnecting and reconnecting pool_backendpidchanges.
Relevant parameters:
num_init_children = 1
max_pool = 1
child_life_time = 300
child_max_connections = 0
connection_life_time = 0
client_idle_limit = 0
connection_cache = on
Test:
postgres#node3:/etc/pgpool2$ psql -p 5433 -U postgres postgres
psql (9.6.10)
Type "help" for help.
postgres=# show pool_pools;
LOG: statement: show pool_pools;
pool_pid | start_time | pool_id | backend_id | database | username | create_time | majorversion | minorversion | pool_counter | pool_backendpid | pool_connected
----------+---------------------+---------+------------+----------+----------+---------------------+--------------+--------------+--------------+-----------------+----------------
3569 | 2018-09-13 20:18:22 | 0 | 0 | postgres | postgres | 2018-09-13 20:25:04 | 3 | 0 | 1 | 3631 | 1
(1 row)
postgres=# \q
postgres#node3:/etc/pgpool2$ psql -p 5433 -U postgres postgres
psql (9.6.10)
Type "help" for help.
postgres=# show pool_pools;
LOG: statement: show pool_pools;
pool_pid | start_time | pool_id | backend_id | database | username | create_time | majorversion | minorversion | pool_counter | pool_backendpid | pool_connected
----------+---------------------+---------+------------+----------+----------+---------------------+--------------+--------------+--------------+-----------------+----------------
3569 | 2018-09-13 20:18:22 | 0 | 0 | postgres | postgres | 2018-09-13 20:25:15 | 3 | 0 | 1 | 3640 | 1
(1 row)
Found out why:
connection_cache (boolean)
Caches connections to backends when set to on. Default is on. However,
connections to template0, template1, postgres and regression databases
are not cached even if connection_cache is on.
I was connecting to postgres database.
Im trying to create index on large table:
datid | datname | pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start | xact_start | query_start | state_change | waiting | state | backend_xid | backend_xmin | query
-------+----------------------+-------+----------+-------------------------+--- --------------------------+--------------+-----------------+-------------+------ -------------------------+-------------------------------+---------------------- ---------+-------------------------------+---------+--------+-------------+----- ---------+---------------------------------------------------------------------- ---------------------
25439 | messengerdb | 30692 | 25438 | messengerdb_rw | pgAdmin III - Przegl??darka | 10.167.12.52 | | 50593 | 2016-08-11 05:27:12.101452+02 | 2016-08-11 05:28:01.535943+02 | 2016-08-11 05:28:01.535943+02 | 2016-08-11 05:28:01.535958+02 | t | active | | 1173740991 | CREATE INDEX CONCURRENTLY user_time_idx +
| | | | | | | | | | | | | | | | | ON core.conversations (user_id ASC NULLS LAST, last_message_timestamp ASC NULLS LAST);+
Is this query working? I'm worried about 'waiting' column === 't' does it mean that it is waiting for lock or sth?
Creating an index concurrently may take a long time since it does not lock the table from writes and it waits until other transactions are finished. However, it may wait forever if you have connections that stay idle in transactions (for example when a client or application keeps an open connection without rollback/commit).
Check if there are some idle connections in transactions (you should be able to see them in the processes list). You can also check PostgreSQL logs.
Section about creating index concurrently in PostgreSQL documentation can be helpful. There is also a nice article about concurrent indexes under this link.