setting up postgresql monitoring in zabbix, error - postgresql

Question

tell me please, I'm setting up postgresql monitoring via zabbix 4.2. I am using the standard built-in postgresql template. All data is displayed correctly, except for metrics from the pgsql.query.time.sql query, data from pgsql.query.time.sql is not displayed
I try to manually execute this request, I get an error:
psql -qtAX -h "$1" -p "$2" -U "$3" -d "$4" -v tmax=$5 -f "/var/lib/zabbix/postgresql/pgsql.query.time.sql"
psql:/var/lib/zabbix/postgresql/pgsql.query.time.sql:31: ERROR: syntax error (near position: ")")
STRING 22: ...'epoch' FROM (clock_timestamp() - query_start)) > )::integer...
Here is the query itself from /var/lib/zabbix/postgresql/pgsql.query.time.sql
WITH T AS
(SELECT db.datname,
coalesce(T.query_time_max, 0) query_time_max,
coalesce(T.tx_time_max, 0) tx_time_max,
coalesce(T.mro_time_max, 0) mro_time_max,
coalesce(T.query_time_sum, 0) query_time_sum,
coalesce(T.tx_time_sum, 0) tx_time_sum,
coalesce(T.mro_time_sum, 0) mro_time_sum,
coalesce(T.query_slow_count, 0) query_slow_count,
coalesce(T.tx_slow_count, 0) tx_slow_count,
coalesce(T.mro_slow_count, 0) mro_slow_count
FROM pg_database db NATURAL
LEFT JOIN (
SELECT datname,
extract(epoch FROM now())::integer ts,
coalesce(max(extract('epoch' FROM (clock_timestamp() - query_start))::integer * (state NOT IN ('idle', 'idle in transaction', 'idle in transaction (aborted)') AND query !~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) query_time_max,
coalesce(max(extract('epoch' FROM (clock_timestamp() - query_start))::integer * (state NOT IN ('idle') AND query !~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) tx_time_max,
coalesce(max(extract('epoch' FROM (clock_timestamp() - query_start))::integer * (state NOT IN ('idle') AND query ~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) mro_time_max,
coalesce(sum(extract('epoch' FROM (clock_timestamp() - query_start))::integer * (state NOT IN ('idle', 'idle in transaction', 'idle in transaction (aborted)') AND query !~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) query_time_sum,
coalesce(sum(extract('epoch' FROM (clock_timestamp() - query_start))::integer * (state NOT IN ('idle') AND query !~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) tx_time_sum,
coalesce(sum(extract('epoch' FROM (clock_timestamp() - query_start))::integer * (state NOT IN ('idle') AND query ~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) mro_time_sum,
coalesce(sum((extract('epoch' FROM (clock_timestamp() - query_start)) > :tmax)::integer * (state NOT IN ('idle', 'idle in transaction', 'idle in transaction (aborted)') AND query !~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) query_slow_count,
coalesce(sum((extract('epoch' FROM (clock_timestamp() - query_start)) > :tmax)::integer * (state NOT IN ('idle') AND query !~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) tx_slow_count,
coalesce(sum((extract('epoch' FROM (clock_timestamp() - query_start)) > :tmax)::integer * (state NOT IN ('idle') AND query ~* E'^(\\s*(--[^\\n]*\\n|/\\*.*\\*/|\\n))*(autovacuum|VACUUM|ANALYZE|REINDEX|CLUSTER|CREATE|ALTER|TRUNCATE|DROP)')::integer), 0) mro_slow_count
FROM pg_stat_activity
WHERE pid <> pg_backend_pid()
GROUP BY 1) T
WHERE NOT db.datistemplate )
SELECT json_object_agg(datname, row_to_json(T))
FROM T

Answer 1

The documentation says:
-c command
[...]
command must be either a command string that is completely parsable by the server (i.e., it contains no psql-specific features), or a single backslash command.
So you cannot use variables that way.
You could use a "here document":
psql <<EOF
\set x $5
SELECT :x
EOF
Or you use the shell variable directly in the statement:
psql -c "SELECT $5"

Answer 2

The issue here is that you are passing an empty value to tmax=, wich is indicated by an error
psql:/var/lib/zabbix/postgresql/pgsql.query.time.sql:31: ERROR: syntax error (near position: ")")
STRING 22: ...'epoch' FROM (clock_timestamp() - query_start)) > )::integer...
^-- missing value here
I believe {$PG.SLOW_QUERIES.MAX.WARN} macro is the culprit here. Make sure that:
It exists in template macros;
It is not being overwritten to an empty value by a host macro.
Now, this may not be an actual reason why monitoring doesn't work. To be sure what exactly goes wrong we need to see zabbix agent logs, you can find those in /var/log/zabbix/zabbix_agentd.log. You may need to change DebugLevel in agent config to 3 or even 4 (be weary level 4 produces a lot of output).

Answer 3

I enabled debug level = 4 and in the log of the zabbix agent I saw the correct command now:
psql -qtAX -h "127.0.0.1" -p "5432" -U "zbx_monitor" -d "test2" -v tmax=30 -f "/var/lib/zabbix/postgresql/pgsql.query.time.sql"
But in the output of the command, I get 0 values for all metrics for all databases, although I ran a parallel query to the test2 database, which takes 14 seconds (i have run this query many times). Why is that?
Here is the output of the command execution result
psql -qtAX -h "127.0.0.1" -p "5432" -U "zbx_monitor" -d "test2" -v tmax=30 -f "/var/lib/zabbix/postgresql/pgsql.query.time.sql"
{ "postgres" : {"datname":"postgres","query_time_max":0,"tx_time_max":0,"mro_time_max":0,"query_time_sum":0,"tx_time_sum":0,"mro_time_sum":0,"query_slow_count":0,"tx_slow_count":0,"mro_slow_count":0}, "test" : {"datname":"test","query_time_max":0,"tx_time_max":0,"mro_time_max":0,"query_time_sum":0,"tx_time_sum":0,"mro_time_sum":0,"query_slow_count":0,"tx_slow_count":0,"mro_slow_count":0}, "test2" : {"datname":"test2","query_time_max":0,"tx_time_max":0,"mro_time_max":0,"query_time_sum":0,"tx_time_sum":0,"mro_time_sum":0,"query_slow_count":0,"tx_slow_count":0,"mro_slow_count":0} }