I'm trying to setup some custom monitoring for postgres and one of the metrics I want to see is the index usage from pg_stat_user_indexes. I have multiple databases, each of them have indexes and I want a single metrics role that has access to all of them.
The problem is that when I select * from pg_stat_user_indexes using the metrics role I see no results. But if I select with the individual database role I see the stats just fine.
The metrics role is a superuser with Bypass RLS. I tried looking at pg_stat_all_indexes but I can see only pg_* stuff.
https://www.postgresql.org/docs/current/static/monitoring-stats.html#pg-stat-all-indexes-view
The pg_stat_all_indexes view will contain one row for each index in
the current database, showing statistics about accesses to that
specific index. The pg_stat_user_indexes and pg_stat_sys_indexes views
contain the same information, but filtered to only show user and
system indexes respectively.
emphasis mine
Seems like you need to be connected to the specific database you want to monitor (not to postgres db) in order to see those stats.
Bizarre.. would have at least expected to see them all in pg_stat_all_indexes - it should have been named pg_stat_all_indexes_for_current_database_plus_pg_catalog - I know.. but pg_stat_all_indexes is not all indexes at all..
Related
I am using Grafana 9.3.1 for monitoring of our system. Among other things, I am trying to monitor the remaining FUP of a phone number for each unit we operate.
Basically, we intend to use two data sources.
Database mapping of the unit ID to its phone number (e.g. "unit_id=123, phone_number="00 123456789")
Prometheus time series remaining_fup{phone_number="00 123456789"}. However, remaining_fup is a 3rd party data and does not include unit_id.
In my unit-detail dashboard I have unit_id variable which indicates which unit FUP should be displayed (among other things depending on unit_id)
My original approach was this:
Create a mixed datasource dashboard
Add database datasource as data A. SELECT phone_number FROM units WHERE unit_id='$unit_id'
Add prometheus datasource remaining_fup and filter it based on A.phone_number: remaining_fup{phone_number="${A.phone_number}"}
Unfortunatelly such use of A isn't supported. I used to hope for applying some transformation like Merge or Join by field and then Filter but with no success. After a lot of googling and trying I feel hopeless.
Could you help please? Is such filter even possible? Thanks!
TL;DR: In grafana dashboard I want to query one datasource in order to obtain a value which I subsequently want to use in another datasource query.
1.) Create variable - name phone_number, type: Query and query your database datasource SELECT phone_number FROM units WHERE unit_id='$unit_id'. You can hide this variable if you don't want it to be visible for the dashboard users.
2.) Variable phone_number may have multiple values, so use advance variable formatting to create valid regex query syntax for your prometheus datasource, e.g.
remaining_fup{phone_number=~"${phone_number:pipe}"}
Of course this queries are just examples and they may need some (syntax) tweaking for the use case. Main idea: don't use 2 queries, but one variable and one query (where you use that variable).
I am unsure how to design security policies for a following system including counters in postgres/supabase. My database includes two tables:
Users:
uuid|name|follower_counter
------------------------------
xyz |tobi| 1
Following-Relationship
follower| following
---------------------------
uuid_1 | uuid_2
Once a user follows a different user, I would like to use a postgres function/transaction to
Insert a new following-follower relationship
Update the followed users' counter
BEGIN
create follower_relationship(follower_id, following_id);
update increment_counter_of_followed_person(following_id);
END;
The constraint should be that the users table (e.g. the name column) can only be altered by the user owning the row. However, the follower_counter should open to changes from users who start following that user.
What is the best security policy design here? Should I add column security or should exclude the counters to a different table?
Do I have to pass parameters to the "block transaction" to ensure that the update and insert functions are called with the needed rights? With which rights should I call the block function?
It might be better to take a different approach to solve this problem. Instead of having a column dedicated to counting the followers, I would recommend actually counting the number of followers when you query the users. Since you already have Following-Relationship table, we just need to count the rows within the table where following or follower is the querying user.
When you have a counter, it might be hard to keep the counter accurate. You have to make sure the number gets decremented when someone unfollows. What if someone blocks a user? What if a user was deleted? There could be a lot of situations that could throw off the counter.
If you count the number of followings/followers on the fly, you don't need to worry about those situations at all.
Now obvious concern with this approach that you might have is performance, but you should not worry too much about it. Postgres is a powerful database that has been battle tested for decades, and with a proper index in place, it can easily perform these query on the fly.
The easiest way of doing this in Supabase would be to create a view like this the following. Once you create a view, you can query it from your Supabase client just like a typical table!
create or replace view profiles as
select
id,
name,
(select count(*) from following_relationship where followed_user_id = id) as follower_count,
(select count(*) from following_relationship where following_user_id = id) as following_count
from users;
I have a multi-tenant database in postgres. So, I have one schema per customer and each schema has a fixed set of tables.
When I connect to the DB using Google Data Studio(GDS), I only see the table names without their associated schema.
How do I connect to tables belonging to one or more schemas?
Also, what do I do if my tables have more than 700k rows, as GDS has a limit on number of rows that can be queried right?
You'll have to use the "Custom Query" option instead of the basic table selection if you need anything more complex.
Regarding the row limit. I wasn't aware of the limit, but if that is true I'd suggest using the Custom Query to pre-group your rows in the query into whatever makes sense...days...months...etc to bring the row count down.
Data Studio will likely choke on anywhere near that many rows and make for a horrible user experience. Let Postgres do as much of the heavy lifting as you can.
Answering only: "what do I do if my tables have more than 700k rows, as GDS has a limit on number of rows that can be queried right?"
Not exactly. The limit is on the number of rows returned, not the amount of rows queried. And that matters since Data Studio will almost always push down queries to connectors.
Here's an example: Lets say you have a purchase table in a PostgreSQL db with 1M+ rows where each record is a purchase event. You add this table as a data source in your report and add a bar chart that shows average purchase by customer type. Let's say you have 12 customer types. Data Studio will then push down the GROUP BY clause to the PostgreSQL db. Thus, your result will have only 12 rows of data instead of 1M+. In most chart types, Data Studio will aggregate or page the results thus issuing a query statement that limits the number of rows returned.
You will only run into the limit if you end up creating a scenario where Data Studio cannot issue an aggregation or paging over the query results or if the aggregated results cross the row limit.
I have been searching for the condition, where, lets say when we enable time travel to a certain table in DB2 , but don't want to capture all the updates done, but only the updates that's done by some specific user.
Wanted to know if this is at all possible with the DB2 time travel and how we can achieve it .
It's not possible with DB2 temporal tables.
Alter the temporal table add a user column maintained by system.
db2 for Iseries column shown
EMP_CHANGE_USER VARCHAR(18) GENERATED ALWAYS AS (USER)
The new column will go automatically to the history table of the temporal table. You can report on the history table and have emp_change user.
Note: IRL Don't single out users. You can give management a report that lists out all users and management can filter it down to individuals. Programmers do not single out users for reporting and logging.
I would like to write a query that uses the IBM DB2 system tables (ex. SYSIBM) to pull a query that exports the following:
LIBRARY_NAME, LIBRARY_DESC, FILE_NAME, FILE_DESC, FIELD_NAME, FIELD_DESC
I can access the descriptions via the UI, but wanted to generate a dynamic query.
Thanks.
Along with SYSTABLES and SYSCOLUMNS, there is also a SYSSCHEMAS which appears to contain the data you need. Please note that accessing this information through QSYS2 will restrict rows returned to those objects with which you have some access - the SYSIBM schema appears to disregard this (check the reference - for V6R1 it's about page 1267).
You also shouldn't need to retrieve this with a dynamic query - static with host variables (if necessary) will work just fine.