I'm currently reviewing the usage of indexes in a complex PostgreSQL database. One of the queries that look useful is this
SELECT idstat.schemaname AS schema_name, idstat.relname AS table_name,
indexrelname AS index_name,
idstat.idx_scan AS times_used,
pg_size_pretty(pg_relation_size(idstat.relid)) AS table_size, pg_size_pretty(pg_relation_size(indexrelid)) AS index_size,
n_tup_upd + n_tup_ins + n_tup_del as num_writes,
indexdef AS definition
FROM pg_stat_user_indexes AS idstat JOIN pg_indexes ON (indexrelname = indexname AND idstat.schemaname = pg_indexes.schemaname)
JOIN pg_stat_user_tables AS tabstat ON idstat.relid = tabstat.relid
WHERE idstat.idx_scan < 200
AND indexdef !~* 'unique'
ORDER BY idstat.relname, indexrelname;
It tells me how often the index was used, how much space it uses etc.
However:
I get the database backup from the client site. When I restore the database, the query returns zeros for times_used. I suspect that all indexes are rebuild on restore.
Finally, question:
What is the easiest way to capture (backup) the data from pg_catalog so that actual client data on index use can be restored and analyzed?
Provide an SQL script that the client can run with psql -f report_on_live_db.sql live_db_name . Have it SELECT the desired data.
Alternately, you might be able to get a raw file-system level copy of the client database from the client and then fire it up on your local machine. This will only work if you can run PostgreSQL on the same operating system and architecture; you can't use a 64-bit PostgreSQL to access a 32-bit database, nor can you use a Linux PostgreSQL to access a Windows or BSD database. The same PostgreSQL major version must be used; you can't use 9.1 to access a 9.0 database or vice versa.
Related
I am new to DBT. I understand that DBT is used for transforming data once the raw data is available in your DWH. However I was trying to see if anyone has used DBT to do an initial select/import of data into staging tables from a remote database server using DB link in a select statement where both the staging database server and the remote database server are Postgres databases.
Also in my case the data volume isn’t much.
I was able to do some testing and looks like this is doable.
Below was the set-up:
The profiles.yml file was pointing to target DB server.
Below was the query used in the model targetTable.sql file.
{{ config(materialized='table') }}
SELECT i.col1,
i.col2
FROM dblink('dbname=sourceDBName port=portNo hostaddr=sourceHostName user=userID password=****',
'SELECT
a.col1,
a.col2
from
(SELECT
col1,col2
FROM
public.tableName) a
')
i (col1 integer, col2 varchar(20))
I am migrating Oracle database to Postgres Aurora. There is one Oracle PL/SQL block which checks if the database is in read write open mode. Below is the query like:
Select open_mode into v_open_mode from v$database;
if v_open_mode = 'READ WRITE' then
-- perform some steps.
I want to know if we have any equivalent query in Postgres. Or even if I can know the postgres node is WRITE mode.
I am also open to get anything which is native to Aurora which show if the node is reader or writer.
I am not sure what the Oracle thing does, but I assume the closest thing would be to check if Postgres is in recovery mode using pg_is_in_recovery()
So something like:
if not pg_is_in_recovery() then
-- do some steps
end if;
That is from "stock" Postgres, I don't know if Amazon Aurora does anything different or provides other functions.
When I run:
select * from sys.server_role_members ;
got below error:
SQLCODE=-204, SQLSTATE=42704, DRIVER=4.18.60
As you seem to be a beginner, you need to know that when asking for help with Db2, it is important to always write your Db2-version and the Db2-server operating system (Z/OS, i-Series(formerly AS/400), or Linux/Unix/Windows ). That's because the three platforms have different capabilities and sometimes the SQL is different also, and the catalog is different.
You get that error because your query is meant for Microsoft SQL-Server databases only.
Your query cannot work on Db2 because Db2 does not allow user-created schema names to begin with SYS. Some IBM-created schema-names can start with SYS for example SYSIBM and SYSCAT but these are reserved for the catalog and they get created by Db2 during database creation.
Db2 has tables like SYSIBM.SYSROLEAUTH and SYSIBM.SYSROLES and (on Linux/Unix/Windows) a set of views on these in the SYSCAT schema.
For example, if your Db2-server runs on Linux/Unix/Windows, then these views exist SYSCAT.ROLES and SYSCAT.ROLEAUTH .
In addition, Db2 for Linux/Unix/Windows provides table function AUTH_LIST_AUTHORITIES_FOR_AUTHID which is quite useful.
If your Db2-server runs on i-Series, then different tables/views exist. That's the reason you need to know which platform of Db2 you are using, and write that in your questions.
I have 2 PostgreSQL databases. The first one is version 9.1, and the second is version 9.3. They are configured the same way (including setting standard_conforming_strings=off). The following query returns one result on the version 9.1 database but returns nothing when run on the 9.3 database. Why?
select 'WORKS' where 'test.123' < 'test/';
Your databases may have different collation settings, which affects sort order. Check your collation with:
select datname, datcollate from pg_database;
If that's the case, you'll need to drop and recreate your 9.3 database with a collation matching your 9.1 copy.
I have a query which gives about 14M rows (I was not aware of this). When I use psql to run the query, my Fedora machine froze. Also after the query was done, I could not use Fedora anymore and had to restart my machine. When I redirected standard output to a file, Fedora also froze.
So how should I handle large resultsets with psql?
psql accumulates complete results in client memory by default. This behavior is usual for all libpq based Postgres applications or drivers. The solutions are cursors - then you are fetching only N rows from server. Cursors can be used by psql too. You can change it by setting FETCH_COUNT variable, then it will use cursors with batch retrieval size FETCH_COUNT.
postgres=# \set FETCH_COUNT 1000
postgres=# select * from generate_series(1,100000); -- big query