This query is working as expected:
select nspname, relname, max(attnum) as num_cols
from pg_attribute a, pg_namespace n, pg_class c
where n.oid = c.relnamespace and a.attrelid = c.oid
and c.relname not like '%pkey'
and n.nspname not like 'pg%'
and n.nspname not like 'information%'
group by 1, 2
order by 1, 2;
nspname | relname | num_cols
--------+----------+----------
public | category | 4
public | date | 8
public | event | 6
public | listing | 8
public | sales | 10
public | users | 18
public | venue | 5
But how do I get the list of columns per table?
Expected output:
nspname | relname | num_cols
--------+----------+----------
public | category | col1, col2, col3, col4
public | date | col1, col2, col3, col4 ..., col8
Mysql has group_concat function that would apply here.
http://docs.aws.amazon.com/redshift/latest/dg/c_join_PG_examples.html
The following query mentioned on that page does not return any rows for me.
select distinct attrelid, rtrim(name), attname, typname
from pg_attribute a, pg_type t, stv_tbl_perm p
where t.oid=a.atttypid and a.attrelid=p.id
and a.attrelid between 100100 and 110000
and typname not in('oid','xid','tid','cid')
order by a.attrelid asc, typname, attname;
Make sure to include all the schemas that you want to look at in search path.
http://docs.aws.amazon.com/redshift/latest/dg/r_search_path.html
set search_path to '$user', public, enterprise;
Redshift without the schemas in your user id's search path cant show you the columns. For accessing table structure you may want to have usage rights on the schema too.
Related
I would like to show only the list of top level tables without child partitioned tables in PostgreSQL. (Currently using PostgreSQL 12.)
\dt in psql gives me all tables, including partitions of tables. I see this:
postgres=# \dt
List of relations
Schema | Name | Type | Owner
--------+------------------------------+-------------------+--------
public | tablea | table | me
public | partitionedtable1 | partitioned table | me
public | partitionedtable1_part1 | table | me
public | partitionedtable1_part2 | table | me
public | partitionedtable1_part3 | table | me
public | tableb | table | me
But I want a list like this, without child partitions of the parent partitioned table:
List of relations
Schema | Name | Type | Owner
--------+------------------------------+-------------------+--------
public | tablea | table | me
public | partitionedtable1 | partitioned table | me
public | tableb | table | me
Query to get all ordinary tables, including root partitioned tables, but excluding all non-root partitioned tables:
SELECT n.nspname AS "Schema"
, c.relname AS "Name"
, CASE c.relkind
WHEN 'p' THEN 'partitioned table'
WHEN 'r' THEN 'ordinary table'
-- more types?
ELSE 'unknown table type'
END AS "Type"
, pg_catalog.pg_get_userbyid(c.relowner) AS "Owner"
FROM pg_catalog.pg_class c
JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind = ANY ('{p,r,""}') -- add more types?
AND NOT c.relispartition -- exclude child partitions
AND n.nspname !~ ALL ('{^pg_,^information_schema$}') -- exclude system schemas
ORDER BY 1, 2;
The manual about relispartition:
... True if table or index is a partition
pg_get_userbyid() is instrumental to get the name of the owning role.
There are more types of "tables" in Postgres 12. The manual about relkind:
r = ordinary table, i = index, S = sequence, t = TOAST table,
v = view, m = materialized view, c = composite type, f =
foreign table, p = partitioned table, I = partitioned index
Postgres 12 also added the meta-command \dP to psql:
The manual:
\dP[itn+] [ pattern ]
Lists partitioned relations. If pattern is specified, only entries whose name matches the pattern are listed. The modifiers t (tables) and i (indexes) can be appended to the command, filtering the kind of relations to list. By default, partitioned tables and indexes are listed.
If the modifier n (“nested”) is used, or a pattern is specified, then non-root partitioned relations are included, and a column is shown displaying the parent of each partitioned relation.
So \dPt gets all root partitioned tables - but not ordinary tables.
Version 1
I can't test this right now, but this ought to give you only top-level tables that are partitioned
select relname
from pg_class
where oid in (select partrelid from pg_partitioned_table);
You should be able to refine/expand that to get more details.
Version 2
Here's a comically verbose solution:
with
partition_parents as (
select relnamespace::regnamespace::text as schema_name,
relname as table_name,
'partition_parent' as info,
*
from pg_class
where relkind = 'p'), -- The parent table is relkind 'p', the partitions are regular tables, relkind 'r'
unpartitioned_tables as (
select relnamespace::regnamespace::text as schema_name,
relname as table_name,
'unpartitioned_table' as info,
*
from pg_class
where relkind = 'r'
and not relispartition
) -- Regular table
select * from partition_parents where schema_name not in ('information_schema','pg_catalog','api','extensions') -- Whatever you've got for schemas
union
select * from unpartitioned_tables where schema_name not in ('information_schema','pg_catalog','api','extensions') -- Whatever you've got for schemas
order by 1,2,3
You should be able to cut the size of that way down to match what you really want. Someone here who really gets the system catalogs should likely be able to provide a more concise version. In the plus column, it's kind of nice to add in the "partition_parent" versus "unpartitioned_table" detail, as above.
If you don't have to use a \d* shortcut, this query should work (though you'll need to filter out schemas not in your search_path):
SELECT relname FROM pg_class WHERE relkind IN ('r','p') AND NOT relispartition;
--do some test in greenplum 6.14
--to find normal table ,partition table
SELECT
*
FROM
(
SELECT
t1.schemaname,
t1.tablename,
t1.tableowner,
CASE
WHEN t3.tablename IS NULL
AND t2.tablename IS NULL THEN
'r'
WHEN t3.tablename IS NOT NULL
AND t2.tablename IS NULL THEN
'p-root' ELSE'p-child'
END AS tabletype,
t4.actionname,
t4.statime
FROM
pg_tables t1
LEFT JOIN pg_partitions t2 ON t1.tablename = t2.partitiontablename
AND t1.schemaname = t2.partitionschemaname
LEFT JOIN ( SELECT DISTINCT schemaname, tablename FROM pg_partitions WHERE schemaname IN ( 'public', 'ods', 'tmp' ) ) t3 ON t1.tablename = t3.tablename
AND t1.schemaname = t3.schemaname
LEFT JOIN (
SELECT
*
FROM
(
SELECT
schemaname,
objname,
actionname,
statime,
ROW_NUMBER ( ) OVER ( PARTITION BY schemaname, objname ORDER BY statime DESC ) AS rn
FROM
pg_catalog.pg_stat_operations
WHERE
classname = 'pg_class'
AND schemaname IN ( 'public', 'ods', 'tmp' )
AND actionname IN ( 'CREATE', 'ALTER' )
) n
WHERE
n.rn = 1
) t4 ON t1.schemaname = t4.schemaname
AND t1.tablename = t4.objname
WHERE
t1.schemaname IN ( 'public', 'ods', 'tmp' )
) o
WHERE
tabletype IN ( 'r', 'p-root' )
ORDER BY
1,
2,
3,
4
The documentation about disk-usage have no "fast and simple" clue or information... So is faster to check here.
When I try SELECT pg_relation_filepath(oid), relpages FROM pg_class WHERE relname = 'big' it is working fine... but "big" is not a "public.big", it is a "othername.big", if there are public also, will be ambiguous.
When I try SELECT pg_relation_filepath(oid), relpages FROM pg_class WHERE relname = 'othername.big' not works.
I need to compare disk-usage of tables and to check (or sum) all schema disk-usage.
To get Schema size :
SELECT schemaname, pg_size_pretty(t.taille::bigint) AS taille_table, pg_size_pretty(t.taille_totale::bigint) AS taille_totale_table
FROM (SELECT schemaname,
sum(pg_relation_size(schemaname || '.' || tablename)) AS taille,
sum(pg_total_relation_size(schemaname || '.' || tablename)) AS taille_totale
FROM pg_tables
WHERE relname_exists(tablename,schemaname) -- see note
GROUP BY schemaname) as t ORDER BY taille_totale DESC;
And for Tables by Schema, you can do this :
SELECT schemaname, tablename, tablespace, pg_size_pretty(taille) AS taille_table, pg_size_pretty(taille_totale) AS taille_totale_table
FROM (SELECT *,
pg_relation_size(schemaname || '.' || tablename) AS taille,
pg_total_relation_size(schemaname || '.' || tablename) AS taille_totale
FROM pg_tables) AS tables
WHERE relname_exists(tablename,schemaname) -- see note
ORDER BY taille_totale DESC;
NOTE: to avoid "ERROR: relation 'x.y' does not exist", needs to guard the string before use it, so
use to_regclass(rel_name) in Postgres 9.4+... See https://stackoverflow.com/a/24089729
or use a generic (see "SwissKnife libraries") function for any pg version, as below
CREATE or replace FUNCTION relname_exists(text,text default NULL) RETURNS boolean AS $$
SELECT EXISTS (
SELECT 1
FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace,
regexp_split_to_array($1,'\.') t(x) -- not work with quoted names
WHERE CASE
WHEN COALESCE(x[2],'')>'' THEN n.nspname = x[1] AND c.relname = x[2]
WHEN $2 IS NULL THEN n.nspname = 'public' AND c.relname = $1
ELSE n.nspname = $2 AND c.relname = $1
END
)
$$ language SQL IMMUTABLE;
Using pg_class and simplifying layout
Same results as Hervé's queries (that used pg_tables)... Was adapted from https://wiki.postgresql.org/wiki/Disk_Usage
-- -- -- -- --
-- DISK-USAGE
CREATE VIEW pgvw_class_usage AS
SELECT *, pg_size_pretty(table_bytes) AS table_size
FROM (
SELECT nspname , relname, total_bytes
, total_bytes-index_bytes-COALESCE(toast_bytes,0) AS table_bytes
FROM (
SELECT nspname , relname
, pg_total_relation_size(c.oid) AS total_bytes
, pg_indexes_size(c.oid) AS index_bytes
, pg_total_relation_size(reltoastrelid) AS toast_bytes
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE relkind = 'r'
) a
) t
ORDER BY 1,2
; -- eg. SELECT * FROM pgvw_class_usage WHERE relname='foo' AND nspname='bar';
CREATE VIEW pgvw_nsclass_usage AS
SELECT *, pg_size_pretty(table_bytes) as table_size
FROM (
SELECT nspname, count(*) as n_tables,
sum(total_bytes) as total_bytes, sum(table_bytes) as table_bytes
FROM pgvw_class_usage
GROUP BY nspname
) t
; -- eg. SELECT * FROM pgvw_nsclass_usage WHERE nspname='bar';
EXAMPLE:
nspname | n_tables | total_bytes | table_bytes | table_size
--------------------+----------+-------------+-------------+------------
bench1 | 8 | 4718592 | 3825664 | 3736 kB
dataset | 4 | 8552448 | 6225920 | 6080 kB
information_schema | 7 | 352256 | 294912 | 288 kB
pg_catalog | 54 | 9003008 | 4734976 | 4624 kB
(4 rows)
I'm looking to print schema & name of relations which all the materialized views in a schema depend on:
select c.relname, d.classid, d.objid,
pg_describe_object(
d.classid, d.objid, d.objsubid)
from pg_class c
join pg_namespace n on c.relnamespace = n.oid
left join pg_depend d on c.oid = d.objid
where n.nspname = 'direct' and d.deptype = 'n'
This gives something like:
relname | relname | classid | objid | pg_describe_object
------------------------+---------+---------+-------+---------------------------------------------------
cases | | 2618 | 33736 | rule _RETURN on materialized view case_categories
benefit_investigations | | 2618 | 33928 | rule _RETURN on materialized view bi_intervals
The description returned gives some hint, but it doesn't contain the schema of the relation. How do I get the actual dependency schema and name? [NB I'm using postgres 9.6]
Here you go:
SELECT DISTINCT view_cs.nspname, view_c.relname, tab_cs.nspname, tab_c.relname
FROM pg_depend view_d
JOIN pg_class view_c ON view_c.oid = view_d.refobjid AND view_c.relkind = 'm'
JOIN pg_type view_ct ON view_ct.oid = view_c.reltype
JOIN pg_namespace view_cs ON view_cs.oid = view_ct.typnamespace
JOIN pg_depend tab_d ON tab_d.objid = view_d.objid
JOIN pg_class tab_c ON tab_c.oid = tab_d.refobjid AND tab_c.relkind = 'r'
JOIN pg_type tab_ct ON tab_ct.oid = tab_c.reltype
JOIN pg_namespace tab_cs ON tab_cs.oid = tab_ct.typnamespace
WHERE view_d.deptype = 'n'
The trick is that you have to JOIN pg_depend twice - once for view-rule relation and once for rule-table relation.
Add
AND view_c.relname != tab_c.relname
to WHERE statement of the accepted answer by #Boris to avoid showing dependency of the table on itself
I have a list of foreign keys. I'd like to find out the tables where these FK's point to and the actual key the point to.
I've got a list of FK's like so:
columnName0, columnName1, columnName2
Foreign key references
columnName0 references table0.idTable0
columnName1 references table1.idTable1
columnName2 references table2.idTable2
Some sample tables:
Table0:
idTable0, PK
name
Table1:
idTable1, PK
age
Table2:
idTable2, PK
createdOn
A sample result:
| column | referenced_column | referenced_table |
|-------------|-------------------|------------------|
| columnName0 | idTable0 | table0 |
| columnName1 | idTable1 | table1 |
| columnName2 | idTable2 | table2 |
I'm trying to translate something I do in MySQL like this:
SELECT DISTINCT
COLUMN_NAME AS column,
REFERENCED_COLUMN_NAME AS referenced_column,
REFERENCED_TABLE_NAME AS referenced_table
FROM
INFORMATION_SCHEMA.KEY_COLUMN_USAGE
WHERE
COLUMN_NAME IN (?);
I'm going to have to use straight-up queries (unfortunately, no stored procedures).
You can query pg_constraint. For column names you should lookup pg_attribute. A foreign key may be based on multiple columns, so conkey and confkey of pg_constraint are arrays. You have to unnest the arrays to get a list of column names. Example:
select
conrelid::regclass table_name,
a1.attname column_name,
confrelid::regclass referenced_table,
a2.attname referenced_column,
conname constraint_name
from (
select conname, conrelid::regclass, confrelid::regclass, col, fcol
from pg_constraint c,
lateral unnest(conkey) col,
lateral unnest(confkey) fcol
where contype = 'f' -- foreign keys constraints
) s
join pg_attribute a1 on a1.attrelid = conrelid and a1.attnum = col
join pg_attribute a2 on a2.attrelid = confrelid and a2.attnum = fcol;
table_name | column_name | referenced_table | referenced_column | constraint_name
------------+-------------+------------------+-------------------+------------------------
products | image_id | images | id | products_image_id_fkey
(1 row)
In Postgres 9.4 or later the function unnest() may have multiple arguments and the inner query may look like this:
...
select conname, conrelid::regclass, confrelid::regclass, col, fcol
from pg_constraint c,
lateral unnest(conkey, confkey) u(col, fcol)
where contype = 'f' -- foreign keys constraints
...
I am using the following query to discover (1) the primary key columns and (2) if the columns have a default value from the information_schema in Postgresql 9.1.
SELECT kcu.column_name, (c.column_default is not null) AS has_default
FROM information_schema.key_column_usage kcu
JOIN information_schema.table_constraints tc ON tc.constraint_name = kcu.constraint_name
JOIN information_schema.columns c on c.column_name = kcu.column_name and c.table_name = kcu.table_name
WHERE tc.constraint_type = 'PRIMARY KEY' AND kcu.table_name like :tablename
It works fine when run as the database owner, but when I run it as a "read-only" user (which I need to do in my application), it returns no data. Some research revealed that the problem is the information.table_constraints view; from the documentation:
The view table_constraints contains all constraints belonging to tables that the current user owns or has some non-SELECT privilege on.
So in order to retrieve table_constraints, my login role needs more than SELECT on the table? Is there no way to get the information from information_schema without giving write permissions to the login role?
Use pg_* views instead of information_schema views. pg_* views display all information regardles of granted privileges.
Try this query:
select
t.relname as table_name,
i.relname as index_name,
a.attname as column_name,
d.adsrc as default_value
from
pg_class t
join pg_attribute a on a.attrelid = t.oid
join pg_index ix on t.oid = ix.indrelid AND a.attnum = ANY(ix.indkey)
join pg_class i on i.oid = ix.indexrelid
left join pg_attrdef d on d.adrelid = t.oid and d.adnum = a.attnum
where
t.relkind = 'r'
and t.relname in ( 'aa', 'bb', 'cc' )
order by
t.relname,
i.relname,
a.attnum;
An example of the query results:
create table aa(
x int primary KEY
);
create table bb(
x int default 1,
constraint pk primary key ( x )
);
create table cc(
x int default 20,
y varchar(10) default 'something',
constraint cc_pk primary key ( x, y )
);
table_name | index_name | column_name | default_value
------------+------------+-------------+--------------------------------
aa | aa_pkey | x |
bb | pk | x | 1
cc | cc_pk | x | 20
cc | cc_pk | y | 'something'::character varying
This is correct, the official postgresql query is below
http://wiki.postgresql.org/wiki/Retrieve_primary_key_columns
if schema is needed the query is as follows
SELECT
pg_attribute.attname,
format_type(pg_attribute.atttypid, pg_attribute.atttypmod)
FROM pg_index, pg_class, pg_attribute, pg_namespace
WHERE
pg_class.oid = 'MY TABLE'::regclass AND
indrelid = pg_class.oid AND
nspname = 'MY CLASS' AND
pg_class.relnamespace = pg_namespace.oid AND
pg_attribute.attrelid = pg_class.oid AND
pg_attribute.attnum = any(pg_index.indkey)
AND indisprimary
The difference can be up to 6000~7000 times. The pg_ one runs often in 0.56ms where the schema based one can run up 6500ms. This is a huge difference especially if you have a high load on the server.
There is another way to provide access for data in information_schema.
A. Envelop SQL in a function with SECURITY DEFINER modifier
CREATE FUNCTION fn_inf(name)
RETURNS TABLE (column_name information_schema.sql_identifier, has_default bool)
LANGUAGE SQL
SECURITY DEFINER
AS $$
SELECT kcu.column_name, (c.column_default is not null) AS has_default
FROM information_schema.key_column_usage kcu
JOIN information_schema.table_constraints tc ON tc.constraint_name = kcu.constraint_name
JOIN information_schema.columns c on c.column_name = kcu.column_name and c.table_name = kcu.table_name
WHERE tc.constraint_type = 'PRIMARY KEY' AND kcu.table_name like $1;
$$;
B. GRANT EXECUTE to user read_only
GRANT EXECUTE ON FUNCTION fn_inf to read_only;
C. Use as user read_only
SELECT * FROM fn_inf('spatial_ref_sys');
column_name
has_default
srid
false