Postgresql Monitoring using Grafana - postgresql

I am very new in Grafana. We have setup grafana and now we want to monitor our postgresql DB using it.
We have created the postgresql datasource and provide all the required detail of our psql DB machine and connection showing ok in Grafana. Now we have imported the Grafana dashboard with id 9948 Dashboard is imported but didn't showing any stats over it.
We are getting the below error:
Error pq: relation "collectd" does not exist
Error Templating init failed
pq: relation "identifiers" does not exist
We have installed the collectd also using apt install collectd on DB machine but didn't find any configurable option in collectd.conf file.
Can you please help me.
Thanks.

You need to install the timescaleDB and create the PG extension.
\c database
CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;
Once done, follow the instructions from following git project.
https://github.com/petercelentanojr/timescaledb_collectd_resources/blob/master/collectd_timescaledb_schema/collectd_timescaledb_bootstrap.sql

Related

Database Migration Service - Aurora PostgreSql -> CloudSQL fails with confusing error( Unable to drop table postgres)

Attempting to migrate from AWS Aurora PostgreSQL 13.4 to Google Cloud SQL PostgreSQL 13.
Migration job gives this error:
finished setup replication with errors: failed to drop database "postgres": generic::unknown: retry budget exhausted (10 attempts): pq: database "postgres" is being accessed by other users
The user the DMS is using only has SELECT permissions on the source database(Aurora)
I'm very confused as to why it is trying to drop the "postgres" database at all. Not sure if it is trying to drop the database in the source or destination. Not sure what I'm missing.
I've installed necessary extensions in the destination DB(pg_cron). No difference.
User in source database has SELECT on all tables/schemas outlined in the docs(including pglogical schema)
I've tried various PostgreSQL versions in the destination cluster( 13.x, 14.x). No difference.
The "Test connection" tool when creating the migration job, shows no errors. (There is a warning about a few tables not having Primary keys, but that's it.)

You have specified a database that is not empty, please specify an empty database

I'm trying to connect to an RDS cluster in AWS that's an Aurora PostgreSQL database. It's a brand-new database that I created along with the instances that I have Jira deployed to. However, when I try to connect to the instance from the Jira configuration screen I get this error
You have specified a database that is not empty, please specify an empty database.
I haven't touched this database at all, why is it giving me this error? I have one read and one write database in my cluster and the "hostname" is the endpoint for my write database, which is what the docs say. Could this be an issue with the Jira version I'm using?
This is the download link I'm using in my user-data script to install Jira. I'm also using PostgreSQL version 12.11
https://www.atlassian.com/software/jira/downloads/binary/atlassian-servicedesk-4.19.1-x64.bin
I switched to a different PostgreSQL version and now it's working.
PostgreSQL version 12.11 was giving me the error and switching to version 13.7 works as expected.

Installation of pg_cron on Azure Flexible PostgeSQL

I am trying to install pg-cron extension for Azure PostgreSQL Flexible server.
According to documentation found here:
https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions#postgres-13-extensions
pg_cron is available extension, but when I am trying to install it:
create schema cron_pg;
CREATE EXTENSION pg_cron SCHEMA cron_pg;
What I get is:
SQL Error [0A000]: ERROR: extension "pg_cron" is not allow-listed for "azure_pg_admin" users in Azure Database for PostgreSQL
Hint: to see the full allow list of extensions, please run: "show azure.extensions;"
When executing:
show azure.extensions;
pg_cron is missing:
address_standardizer,address_standardizer_data_us,amcheck,bloom,btree_gin,btree_gist,citext,cube,dblink,dict_int,dict_xsyn,earthdistance,fuzzystrmatch,hstore,intagg,intarray,isn,lo,ltree,pageinspect,pg_buffercache,pg_freespacemap,pg_partman,pg_prewarm,pg_stat_statements,pg_trgm,pg_visibility,pgaudit,pgcrypto,pgrowlocks,pglogical,pgstattuple,plpgsql,postgis,postgis_sfcgal,postgis_tiger_geocoder,postgis_topology,postgres_fdw,sslinfo,tablefunc,tsm_system_rows,tsm_system_time,unaccent,uuid-ossp,lo,postgis_raster
What am I doing wrong?
You can tell pg_cron to run jobs in another database by updating the database column job in the jobs table.
For example:
UPDATE cron.job SET database = 'wordpress' WHERE jobname = 'wordpress-job';
Pretty late but this issue showed up when I was searching for same problem but with pg_trgm extension. After some looking around eventually realised you just need to update the database settings.
Go to Database in Azure Portal, then to Server parameters and search for azure.extensions parameter. You can then click on the list and enable/disable desired extensions (PG_CRON is available), the server will restart on save and then you will be able to enable the extensions in database.
Seems that the pg_cron extension is already enabled, by default, in the default 'postgres' database.
The reason why I was not seeing this is because I am not using the default 'postgres' database. I have created my own DB which I was connected to.
This actually does not resolve my problem, because I can't execute jobs from pg_cron across databases...

pg_cron install: No cron tables created

I am running ThingsBoard (Internet-of-Things) on an Ubuntu 18.04 LTS VPS, using a PostgreSQL DB. Since the IoT devices send a lot of data to the DB, I need to clean regularly the DB.
For this, I would like to use pg_cron.
I followed the steps described at https://github.com/citusdata/pg_cron:
I installed postgresql-10-cron
I modified postgresql.conf:
shared_preload_libraries = 'pg_cron'
cron.database_name = 'postgres'
I restarted the DB by
service postgresql restart
I logged into my postgres DB with Adminer and executed:
CREATE EXTENSION pg_cron;
=> Success
However, I cannot see any tables in postgres that I could configure...
I tried step 4 again, with the result
ERROR: extension "pg_cron" already exists
Is there anything I don't understand?
The metadata tables are created in the cron schema:
\dt cron.*
To find all objects that belong to the extension, run
\dx+ pg_cron

SymmetricDS Postgres target gives "Failed to read table" for all sym_* tables

I'm trying to setup a simple replication from MySQL to Postgres. Identical schemas. After following the steps in the Demo Tutorial with a slight change (using MySQL and Postgres drivers) I am still unable to get the replication working.
A few changes were needed based on complaints after running bin/sym
SET GLOBAL show_compatibility_56 = ON needed to be set in the MySQL DB
For Postgres I needed to use protocolVersion=3 instead of 2 which was set in the example.
The weird thing is that SymmetricDS is able to create the sym_* tables, but complains about not being able to read them. I have verified that the tables do not exist before bin/sym is run, but do exist after. Here is an excerpt from the log
// Successful creation of table
[store-001] - PostgreSqlSymmetricDialect - DDL applied: CREATE TABLE "sym_notification"(
"notification_id" VARCHAR(128) NOT NULL,
...
PRIMARY KEY ("notification_id")
)
...
// Unable to read from created table
[store-001] - PostgreSqlDdlReader - Failed to read table: sym_notification
[store-001] - PostgreSqlDdlReader - Failed to read table: sym_notification
[store-001] - AbstractDatabaseWriter - Did not find the sym_notification table in the target database
[store-001] - PostgreSqlDdlReader - Failed to read table: sym_monitor
[store-001] - PostgreSqlDdlReader - Failed to read table: sym_monitor
[store-001] - AbstractDatabaseWriter - Did not find the sym_monitor table in the target database
The same error apply for all the sym_* tables.
The databases are running in Docker, but since SymmetricDS is not complaining about being unable to connect, and is able to create the tables, I assume it is not related to Docker.
The database in the Postgres DB is created by the same user as specified in engines/store-001.properties. Could this still have something to do with roles and access privileges?
If you upgrade to the latest JDBC driver from Postgres it will work.
Replace the existing Postgres driver from the lib directory from the latest from here: https://jdbc.postgresql.org/download.html
Try to connect to the postgres database with the same username/password used by symmetric-ds from some DB navigator, for example Jetbrain's Datagrip and then try inserting, updating, selecting something from sym_* tables. Assign access rights to the user if necessary.
When using Postgres 9.6.1 (current latest release) the following error is logged on the server when running bin/sym
ERROR: column am.amcanorder does not exist at character 427
The problem was resolved by using Postgres 9.5.5 instead thanks to Nick Barnes pointing this out in a comment.