Unused indexes take lot of space and slow down data modifications. How can I check if my indexes are really useful and delete unused ones?
Make sure your applications have run long enough since the last stat reset. Think especially of periodic jobs that runs once every week and may require an index.
This is the code to make a human readable view with usefull indices information. The indices on partitionned table are aggregated. If you have templates tables for your partitions, make sure the filters for template tables names are accurate.
CREATE VIEW stats_index_aggregate as
-- Index for partitionned tables
select
'partitioned index' as indextype,
nsp.nspname as schemaname,
table_class.relname as tablename,
parent_class.relname as indexname,
index_columns.idx_columns as idx_columns,
seek_childs.nb_child_index,
seek_childs.nb_scans,
seek_childs.index_size
from pg_class parent_class
join pg_index parent_index on parent_index.indexrelid = parent_class.oid
join pg_namespace nsp on nsp.oid = parent_class.relnamespace -- to get schemaname
join pg_class table_class on table_class.oid = parent_index.indrelid
, lateral (
select count(stats_child.idx_scan) as nb_child_index,
sum(stats_child.idx_scan) as nb_scans,
sum(pg_relation_size(stats_child.indexrelid)) as index_size
from pg_catalog.pg_stat_user_indexes stats_child
join pg_inherits pi on pi.inhrelid = stats_child.indexrelid
where pi.inhparent = parent_class.oid
) seek_childs
, LATERAL (
SELECT string_agg(attname, ', ' order by attnum) AS idx_columns
FROM pg_attribute
WHERE attrelid = parent_class.oid
) index_columns
where parent_class.relkind = 'I'
AND 0 <>ALL (parent_index.indkey) -- no index column is an expression
AND NOT parent_index.indisunique -- is not a UNIQUE index
AND NOT EXISTS -- does not enforce a constraint
(SELECT 1 FROM pg_catalog.pg_constraint cc WHERE cc.conindid = parent_index.indexrelid)
and table_class.relname not like '%template' -- filter for template tables
union
-- Index for regular tables
select
'regular index' as indextype,
stats_child.schemaname,
stats_child.relname AS tablename,
c.relname as indexname,
index_columns.idx_columns as idx_columns,
null as nb_child_index,
stats_child.idx_scan as id_scan_count,
pg_relation_size(stats_child.indexrelid) as index_size
from pg_class c
join pg_index idx_parent on idx_parent.indexrelid = c.oid
join pg_catalog.pg_stat_user_indexes stats_child on c.oid = stats_child.indexrelid
, LATERAL (
SELECT string_agg(attname, ', ' order by attnum) AS idx_columns
FROM pg_attribute
WHERE attrelid = c.oid
) index_columns
where c.relkind = 'i'
AND 0 <>ALL (idx_parent.indkey) -- no index column is an expression
AND NOT idx_parent.indisunique -- is not a UNIQUE index
AND NOT EXISTS -- does not enforce a constraint
(SELECT 1 FROM pg_catalog.pg_constraint cc
WHERE cc.conindid = idx_parent.indexrelid)
AND NOT EXISTS -- is not a child index
(SELECT 1 FROM pg_inherits pi
where pi.inhrelid = c.oid)
and stats_child.relname not like '%template'; -- filter for template tables
Then you can check how much space you can save with this summary :
select count(*) || ' indices are not used, you can drop them and save ' || pg_size_pretty(sum(index_size)) as pretty_size from stats_index_aggregate where nb_scans = 0;
Execute this query output to clean your database :
select
case when nb_scans = 0 then 'DROP INDEX ' || indexname || ';' else '' end as drop_command, -- make sure we don't print drop command for used index
*, pg_size_pretty(index_size) as pretty_size
from stats_index_aggregate;
I'm running a project on a Postgres database and need to retrieve the comments on columns within the DB to be used as table headings and such. I have seen that there are a couple of built in functions (pg_description and col_description) but i haven't been able to find examples on how to use them and playing around with them has proved pretty futile.
So I was wondering if any has been able to do this before and if so, how?
select
c.table_schema,
c.table_name,
c.column_name,
pgd.description
from pg_catalog.pg_statio_all_tables as st
inner join pg_catalog.pg_description pgd on (
pgd.objoid = st.relid
)
inner join information_schema.columns c on (
pgd.objsubid = c.ordinal_position and
c.table_schema = st.schemaname and
c.table_name = st.relname
);
It all works by oid,
mat=> SELECT c.oid FROM pg_catalog.pg_class c WHERE c.relname = 'customers';
oid
-------
23208
(1 row)
Now, I have the oid for that table, so I can ask :
mat=> select pg_catalog.obj_description(23208);
obj_description
-------------------
Customers
(1 row)
Then, I can ask for the description of the fourth column :
mat=> select pg_catalog.col_description(23208,4);
col_description
-----------------------------------------
Customer codes, CHS, FACTPOST, POWER...
(1 row)
If you want to know which queries does psql run when you do \dt+ or \d+ customers, just run it with -E.
Just to be here if somebody will need it.
There are many answers here, but none of them was as simple as I would like it to be. So, based on previous answers and current postgres 9.4, I have created this query:
SELECT
obj_description(format('%s.%s',isc.table_schema,isc.table_name)::regclass::oid, 'pg_class') as table_description,
pg_catalog.col_description(format('%s.%s',isc.table_schema,isc.table_name)::regclass::oid,isc.ordinal_position) as column_description
FROM
information_schema.columns isc
It fetches table and column descriptions, without any confusing joins and ugly string concatenations.
Take care with schemas, this code considers them:
SELECT
cols.column_name, (
SELECT
pg_catalog.col_description(c.oid, cols.ordinal_position::int)
FROM
pg_catalog.pg_class c
WHERE
c.oid = (SELECT ('"' || cols.table_name || '"')::regclass::oid)
AND c.relname = cols.table_name
) AS column_comment
FROM
information_schema.columns cols
WHERE
cols.table_catalog = 'your_database'
AND cols.table_name = 'your_table'
AND cols.table_schema = 'your_schema';
References:
Postgresql Document Table and Column Description Comments on Table and Column
Determining the OID of a table in Postgres 9.1?
A slight change to one of the other answers which only gives you columns that have comments on them, this gives you all columns whether they have a comment or not.
select c.table_schema, st.relname as TableName, c.column_name,
pgd.description
from pg_catalog.pg_statio_all_tables as st
inner join information_schema.columns c
on c.table_schema = st.schemaname
and c.table_name = st.relname
left join pg_catalog.pg_description pgd
on pgd.objoid=st.relid
and pgd.objsubid=c.ordinal_position
where st.relname = 'YourTableName';
This works for me using the PostBooks 3.2.2 DB:
select cols.column_name,
(select pg_catalog.obj_description(oid) from pg_catalog.pg_class c where c.relname=cols.table_name) as table_comment
,(select pg_catalog.col_description(oid,cols.ordinal_position::int) from pg_catalog.pg_class c where c.relname=cols.table_name) as column_comment
from information_schema.columns cols
where cols.table_catalog='postbooks' and cols.table_name='apapply'
Regards,
Sylnsr
If you just need to show the comments for your columns among other data, you can also use:
\d+ my_table
Enhance for #Nick and #mat suggestions: use
SELECT obj_description('schemaName.tableName'::regclass, 'pg_class');
when you have string name (not oid).
To avoid to remember 'pg_class' parameter, and to avoid ugly concatenations at the function calls, as (tname||'.'||schema)::regclass, an useful overload for obj_description:
CREATE FUNCTION obj_description(
p_rname text, p_schema text DEFAULT NULL,
p_catalname text DEFAULT 'pg_class'
) RETURNS text AS $f$
SELECT obj_description((CASE
WHEN strpos($1, '.')>0 OR $2 IS NULL OR $2='' THEN $1
ELSE $2||'.'||$1
END)::regclass, $3);
$f$ LANGUAGE SQL IMMUTABLE;
-- USAGE: obj_description('mytable')
-- SELECT obj_description('s.t');
-- PS: obj_description('s.t', 'otherschema') is a syntax error,
-- but not generates exception: returns the same as ('s.t')
Now is easy to use, because the table name (rname parameter) is a varchar and can be expressed with a separated field for schema name, as in the main tables and queries.
See also "Getting list of table comments in PostgreSQL" or the new pg9.3 Guide
This answer is a little late, but it popped up on a google search I did to research this problem. We only needed Table descriptions, but the method would be the same for columns.
The column descriptions are in the pg_description table also, referenced by objoid.
Add this view:
CREATE OR REPLACE VIEW our_tables AS
SELECT c.oid, n.nspname AS schemaname, c.relname AS tablename, d.description,
pg_get_userbyid(c.relowner) AS tableowner, t.spcname AS "tablespace",
c.relhasindex AS hasindexes, c.relhasrules AS hasrules, c.reltriggers > 0 AS hastriggers
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
LEFT JOIN pg_tablespace t ON t.oid = c.reltablespace
LEFT JOIN pg_description d ON c.oid = d.objoid
WHERE c.relkind = 'r'::"char";
ALTER TABLE our_tables OWNER TO postgres;
GRANT SELECT, UPDATE, INSERT, DELETE, REFERENCES, TRIGGER ON TABLE our_tables TO postgres;
GRANT SELECT ON TABLE our_tables TO public;
Then run:
SELECT tablename, description FROM our_tables WHERE schemaname = 'public'
The view is a modified version of the pg_tables view which adds in the description column.
You could also monkey around with the view definition to make it a single query.
I accessed table comments like this:
select c.relname table_name, pg_catalog.obj_description(c.oid) as comment from pg_catalog.pg_class c where c.relname = 'table_name';
and column comments thusly:
SELECT c.column_name, pgd.description FROM pg_catalog.pg_statio_all_tables as st inner join pg_catalog.pg_description pgd on (pgd.objoid=st.relid) inner join information_schema.columns c on (pgd.objsubid=c.ordinal_position and c.table_schema=st.schemaname and c.table_name=st.relname and c.table_name = 'table_name' and c.table_schema = 'public');
I asked a similar question about Postgresql comments last month. If you dig through that, you'll come across some Perl code over on my blog that automates the process of extracting a comment.
To pull out the column names of a table, you can use something like the following:
select
a.attname as "colname"
,a.attrelid as "tableoid"
,a.attnum as "columnoid"
from
pg_catalog.pg_attribute a
inner join pg_catalog.pg_class c on a.attrelid = c.oid
where
c.relname = 'mytable' -- better to use a placeholder
and a.attnum > 0
and a.attisdropped is false
and pg_catalog.pg_table_is_visible(c.oid)
order by a.attnum
You can then use the tableoid,columnoid tuple to extract the comment of each column (see my question).
I just found this here. It will provide you with all kind of metadata on one specific table (type, default value, not null flag, length, comment, foreign key name, primary key name). It seems to work well.
SELECT pg_tables.tablename, pg_attribute.attname AS field,
format_type(pg_attribute.atttypid, NULL) AS "type",
pg_attribute.atttypmod AS len,
(SELECT col_description(pg_attribute.attrelid,
pg_attribute.attnum)) AS comment,
CASE pg_attribute.attnotnull
WHEN false THEN 1 ELSE 0
END AS "notnull",
pg_constraint.conname AS "key", pc2.conname AS ckey,
(SELECT pg_attrdef.adsrc FROM pg_attrdef
WHERE pg_attrdef.adrelid = pg_class.oid
AND pg_attrdef.adnum = pg_attribute.attnum) AS def
FROM pg_tables, pg_class
JOIN pg_attribute ON pg_class.oid = pg_attribute.attrelid
AND pg_attribute.attnum > 0
LEFT JOIN pg_constraint ON pg_constraint.contype = 'p'::"char"
AND pg_constraint.conrelid = pg_class.oid AND
(pg_attribute.attnum = ANY (pg_constraint.conkey))
LEFT JOIN pg_constraint AS pc2 ON pc2.contype = 'f'::"char"
AND pc2.conrelid = pg_class.oid
AND (pg_attribute.attnum = ANY (pc2.conkey))
WHERE pg_class.relname = pg_tables.tablename
-- AND pg_tables.tableowner = "current_user"()
AND pg_attribute.atttypid <> 0::oid
AND tablename='your_table'
ORDER BY field ASC
Source: http://golden13.blogspot.de/2012/08/how-to-get-some-information-about_7.html
Ok, so i worked it out to degree...
select col_description(table id, column number)...
ie: select col_description(36698,2);
That worked, but is there an easier way to do this maybe bringing all the comments on all the columns and using the table name instead of the oid???
To display comments from all columns of all table :
SELECT
cols.table_name,
cols.column_name, (
SELECT
pg_catalog.col_description(c.oid, cols.ordinal_position::int)
FROM
pg_catalog.pg_class c
WHERE
c.oid = (SELECT ('"' || cols.table_name || '"')::regclass::oid)
AND c.relname = cols.table_name
) AS column_comment
FROM
information_schema.columns cols
WHERE
cols.table_name IN (SELECT cols.table_name FROM information_schema.columns)
AND cols.table_catalog = 'your_database_name'
AND cols.table_schema = 'your_schema_name';
You need to execute this query outside any schema/catalog/db
This query is based on another answer in this question which display comments from one table only
To extend on the response provided by #amxy; I found that adding a schema filter can help in some environments. As I found #amxy's solution didn't work until I added by schema filters
SELECT
pg_tables.schemaname,
pg_tables.TABLENAME,
pg_attribute.attname AS field,
format_type(pg_attribute.atttypid, NULL) AS "type",
pg_attribute.atttypmod AS len,
(
SELECT col_description(pg_attribute.attrelid, pg_attribute.attnum)) AS COMMENT,
CASE pg_attribute.attnotnull
WHEN FALSE THEN 1
ELSE 0
END AS "notnull",
pg_constraint.conname AS "key", pc2.conname AS ckey,
(
SELECT pg_attrdef.adsrc
FROM pg_attrdef
WHERE pg_attrdef.adrelid = pg_class.oid
AND pg_attrdef.adnum = pg_attribute.attnum) AS def
FROM pg_tables, pg_class
JOIN pg_attribute
ON pg_class.oid = pg_attribute.attrelid
AND pg_attribute.attnum > 0
LEFT JOIN pg_constraint
ON pg_constraint.contype = 'p'::"char"
AND pg_constraint.conrelid = pg_class.oid
AND
(pg_attribute.attnum = ANY (pg_constraint.conkey))
LEFT JOIN pg_constraint AS pc2
ON pc2.contype = 'f'::"char"
AND pc2.conrelid = pg_class.oid
AND (pg_attribute.attnum = ANY (pc2.conkey))
WHERE pg_class.relname = pg_tables.TABLENAME
AND pg_tables.schemaname IN ('op', 'im', 'cs','usr','li')
-- AND pg_tables.tableowner = "current_user"()
AND pg_attribute.atttypid <> 0::oid
---AND TABLENAME='your_table'
ORDER BY pg_tables.schemaname,
pg_tables.TABLENAME ASC;
RESULTS:
SELECT
relname table_name,
obj_description(oid) table_description,
column_name,
pgd.description column_description
FROM pg_class
INNER JOIN
information_schema.columns
ON table_name = pg_class.relname
LEFT JOIN
pg_catalog.pg_description pgd
ON pgd.objsubid = ordinal_position
WHERE
relname = 'your_table_name'
SELECT sc.table_schema , sc.table_name, sc.column_name, col_description(pc."oid" , sc.ordinal_position) col_description FROM pg_class pc
INNER JOIN pg_namespace ns ON ns."oid" =pc.relnamespace
INNER JOIN information_schema.COLUMNS sc ON sc.table_name=pc.relname AND sc.table_schema=ns.nspname
WHERE 1=1
AND upper(ns.nspname) = 'TABLE_SCHEMA'
AND upper(pc.relname) = 'TABLE_NAME'
Retrieving Comments from a PostgreSQL DB
I'm using "ALTER SEQUENCE sequence OWNED BY table.column" to change sequence association in Postgres. Is there a way to view this new information using \ds or something similar? \ds still shows the table owner as the sequence owner.
SELECT c.relname,u.usename
FROM pg_class c, pg_user u
WHERE c.relowner = u.usesysid and c.relkind = 'S'
AND relnamespace IN (
SELECT oid
FROM pg_namespace
WHERE nspname NOT LIKE 'pg_%'
AND nspname != 'information_schema'
);
OWNED BY table.column changes the associated column; OWNER TO newowner changes the role which owns the sequence. The owner role is what is displayed in \ds. I don't know a psql command to see the linked column of a sequence. OWNED BY is primary used to cascade a delete of the associated sequence when the referent column is removed.
You can get visibility into the owned by column by constructing a query against the system catalog tables pg_class, pg_depend, and pg_attribute. I'm not sure of all the exact semantics of pg_depend, but this query should show you a sequence's column dependency:
select tab.relname as tabname, attr.attname as column
from pg_class as seq
join pg_depend as dep on (seq.relfilenode = dep.objid)
join pg_class as tab on (dep.refobjid = tab.relfilenode)
join pg_attribute as attr on (attr.attnum = dep.refobjsubid and attr.attrelid = dep.refobjid)
where seq.relname = 'sequence';
Yuri Levinsky's answer modified to include the namespace column:
SELECT n.nspname, c.relname, u.usename
FROM pg_class c
INNER JOIN pg_user u ON c.relowner = u.usesysid
INNER JOIN pg_namespace n ON c.relnamespace = n.oid
WHERE c.relkind = 'S'
AND n.nspname NOT LIKE 'pg_%'
AND n.nspname != 'information_schema'
ORDER BY 1, 2, 3;
I can run the following line:
ALTER SEQUENCE seqName OWNED BY table.id;
How can I get the 'owner' set by OWNED BY for a sequence (in this case: table.id)?
You may use following query:
select s.relname as seq, n.nspname as sch, t.relname as tab, a.attname as col
from pg_class s
join pg_depend d on d.objid=s.oid and d.classid='pg_class'::regclass and d.refclassid='pg_class'::regclass
join pg_class t on t.oid=d.refobjid
join pg_namespace n on n.oid=t.relnamespace
join pg_attribute a on a.attrelid=t.oid and a.attnum=d.refobjsubid
where s.relkind='S' and d.deptype='a'
It returns all sequences with owner information. Just filter them in WHERE clause and that's it.
Get the "owning" table and column
ALTER SEQUENCE seqName OWNED BY table.id;
Your ALTER SEQUENCE statement causes an entry in the system catalog pg_depend with the dependency type (deptype) 'a' and a refobjsubid greater than 0, pointing to the attribute number (attnum) in pg_attribute. With that knowledge you can devise a simple query:
SELECT d.refobjid::regclass, a.attname
FROM pg_depend d
JOIN pg_attribute a ON a.attrelid = d.refobjid
AND a.attnum = d.refobjsubid
WHERE d.objid = 'public."seqName"'::regclass -- your sequence here
AND d.refobjsubid > 0
AND d.classid = 'pg_class'::regclass;
Double quotes ("") are only needed for otherwise illegal names (mixed case, reserved words, ...).
No need to assert that refclassid is of type regclass since the join to pg_attribute does that automatically.
No need to assert that the sequence is a sequence since schema-qualified object names are unique across the database.
No need to join to pg_class or pg_namespace at all.
The schema name is only needed to disambiguate or if it's not in the search_path.
The same table name (or sequence name for that matter) can be used in multiple schemas. A cast to the object identifier type regclass observes the current search_path to pick the best match if you omit the schema qualification. If the table is not visible, you get an error message.
What's more, a regclass type is displayed as text to the user automatically. (If not, cast to text.) The schema-name is prepended automatically where necessary to be unambiguous in your session.
Get the actual "owner" (the role)
To get the role owning a specific sequence, as requested:
SELECT c.relname, u.usename
FROM pg_class c
JOIN pg_user u ON u.usesysid = c.relowner
WHERE c.oid = '"seqName"'::regclass; -- your sequence here
I was able to list the table and corresponding sequence for a particular column using the following SQL statement:
SELECT table_schema
, table_name
, column_name
, LTRIM(RTRIM(RTRIM(column_default, '::regclass)'),''''),'nextval(''') AS SEQUENCE_NAME
FROM information_schema.columns
WHERE column_default like '%nextval%';
SELECT c.relname,u.usename
FROM pg_class c, pg_user u
WHERE c.relowner = u.usesysid and c.relkind = 'S'
AND relnamespace IN (
SELECT oid
FROM pg_namespace
WHERE nspname NOT LIKE 'pg_%'
AND nspname != 'information_schema'
);
I use that query for get all queries.
Change CTE to filter the result
WITH table_with_sequence as (
SELECT
d.refobjid::regclass::text tablename,
c.relname::text sequencename,
np.nspname::text schemaname,
a.attname::text attname,
u.usename::text
FROM
pg_depend d
INNER JOIN pg_class c ON c.oid = d.objid
AND c.relkind = 'S'
INNER JOIN pg_namespace np ON np.oid = c.relnamespace
AND (np.nspname NOT LIKE 'pg_%'
AND np.nspname != 'information_schema')
INNER JOIN pg_user u ON u.usesysid = c.relowner
INNER JOIN pg_attribute a ON a.attrelid = d.refobjid
AND a.attnum = d.refobjsubid
)
SELECT
'ALTER SEQUENCE '|| QUOTE_LITERAL(QUOTE_IDENT(schemaname) || '.' || QUOTE_IDENT(sequencename)) ||' OWNED BY ' || tablename || '.' || QUOTE_IDENT(attname)
FROM table_with_sequence