how to properly create my own custom items and triggers for zabbix 4 - postgresql

Question

:)
I have Zabbix 4.4.1 installed on Ubuntu 19.10.
I have a postgresql plugin configured and working properly so it checks my database metrics.
I have a a table that I want to check a timestamp column for the last inserted row. column name is insert_time.
if the last inserted row have a insert time of more then 5 minutes to product warning and 10 minutes to product error.
I'm new to zabbix.. all I did so far is for googling, not sure if that's the way to go.. it's probably not cause it's not working :)
ok so first thing I did is created a bash files at /etc/zabbix/mytools, get-last-insert-time.sh.
I perform the query and send the output to zabbix_sender with the following template:
#!/bin/bash
PGPASSWORD=<PASSWORD> -U <USER> <DB> -t -c "<RELEVANT QUERY>" | awk '{$1=$1};1' | tr -d "\n" | xargs -I {} /usr/bin/zabbix_sender -z $ZABBIXSERVER -p $ZABBIXPORT -s $ZABBIXAGENT -k "my.pgsql.cdr.last_insert_time" -o {}
is there a way to test this step? how can I make sure that zabbix_sender receives that information? is there some kind of.. zabbix_sender_sniffer of some sort ? :)
next.. I created a configuration files at /etc/zabbix/zabbix_agentd.d called get-last-insert-time.conf with the following template:
UserParameter=my_pgsql_cdr_last_insert_time,/etc/zabbix/mytools/get-last-insert-time.sh;echo $?
here the key is my_pgsql_cdr_last_insert_time while the key in zabbix_sender is my.pgsql.cdr.last_insert_time. as far as I understand these should be two different keys.
why?!
then I created a template and attached it to the relevant host and I created 2 items for it:
item for insert time with the key my.pgsql.cdr.last_insert_time and of type Zabbix Trapper
a Zabbix Agent item with the key my_pgsql_cdr_last_insert_time Type of information: text.
is that the type of information for timestamp ?
now on Overview -> latest data I see:
CDR last insert time with no data
and Run my database trappers insert time that is... the text is disabled? it's in gray.. and there is also no data.
so before I begin to create an alert. what did I do wrong ?
any information regarding this issue would be greatly appreciated.
update
thanks Jan Garaj for this valuable information.
I was expecting that creating such a trigger should be easier then what I found on google, glad to see I was correct.
I edited my bash scripts to return seconds since epoch, since it's from postgresql it returns float, so I configured the items as float. I do see in latest data that the items receive the proper values.
I created triggers, I made sure that the warning trigger depends on the critical trigger so they won't both appear on the same time.
for example I created this trigger {cdrs:pgsql.cdr.last_insert_time.fuzzytime(300)}=0 so if the last insert time is greater then 5 minutes to return a critical error. the problem is that it returns a critical error.. always! even when it shouldn't. I couldn't find a way to debug this. so besides actually getting the triggers to work properly everything else is well configured.
any ideas ?
update 2
when I configured the script to return a timestamp, I changed it to a different timezone instead of living it as it is, which actually compared the data with current time + 2 hours in the future :)
I found that out while going to latest data, checking the timestamp and converting it to actual time. so everything works now thanks a lot!

Answer 1

It looks over complicated, because you are mixing sender with agent approach. Simpler approach - agent only:
UserParameter=pgsql.cdr.last_insert_time,/etc/zabbix/mytools/get-last-insert-time.sh
Script /etc/zabbix/mytools/get-last-insert-time.sh returns last insert Unix timestamp only, e.g.1574111464 (no new line and don't use zabbix_sender in the script). Keep in mind, that zabbix agent uses zabbix user usually, so you need to configure proper script (exec) permissions, eventually env variables.
Test it with zabbix-get from the Zabbix server, e.g.:
zabbix_get -s <HOST IP> -p 10050 -k "pgsql.cdr.last_insert_time"
For any issue on the agent side: increase log agent level and watch agent logs
When you sort agent part, then create template with item key pgsql.cdr.last_insert_time and Numeric (unsigned) type. Trigger can use fuzzytime(60) function.