Show only list of tables without child partitions - postgresql

I would like to show only the list of top level tables without child partitioned tables in PostgreSQL. (Currently using PostgreSQL 12.)
\dt in psql gives me all tables, including partitions of tables. I see this:
postgres=# \dt
List of relations
Schema | Name | Type | Owner
--------+------------------------------+-------------------+--------
public | tablea | table | me
public | partitionedtable1 | partitioned table | me
public | partitionedtable1_part1 | table | me
public | partitionedtable1_part2 | table | me
public | partitionedtable1_part3 | table | me
public | tableb | table | me
But I want a list like this, without child partitions of the parent partitioned table:
List of relations
Schema | Name | Type | Owner
--------+------------------------------+-------------------+--------
public | tablea | table | me
public | partitionedtable1 | partitioned table | me
public | tableb | table | me

Query to get all ordinary tables, including root partitioned tables, but excluding all non-root partitioned tables:
SELECT n.nspname AS "Schema"
, c.relname AS "Name"
, CASE c.relkind
WHEN 'p' THEN 'partitioned table'
WHEN 'r' THEN 'ordinary table'
-- more types?
ELSE 'unknown table type'
END AS "Type"
, pg_catalog.pg_get_userbyid(c.relowner) AS "Owner"
FROM pg_catalog.pg_class c
JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind = ANY ('{p,r,""}') -- add more types?
AND NOT c.relispartition -- exclude child partitions
AND n.nspname !~ ALL ('{^pg_,^information_schema$}') -- exclude system schemas
ORDER BY 1, 2;
The manual about relispartition:
... True if table or index is a partition
pg_get_userbyid() is instrumental to get the name of the owning role.
There are more types of "tables" in Postgres 12. The manual about relkind:
r = ordinary table, i = index, S = sequence, t = TOAST table,
v = view, m = materialized view, c = composite type, f =
foreign table, p = partitioned table, I = partitioned index
Postgres 12 also added the meta-command \dP to psql:
The manual:
\dP[itn+] [ pattern ]
Lists partitioned relations. If pattern is specified, only entries whose name matches the pattern are listed. The modifiers t (tables) and i (indexes) can be appended to the command, filtering the kind of relations to list. By default, partitioned tables and indexes are listed.
If the modifier n (“nested”) is used, or a pattern is specified, then non-root partitioned relations are included, and a column is shown displaying the parent of each partitioned relation.
So \dPt gets all root partitioned tables - but not ordinary tables.

Version 1
I can't test this right now, but this ought to give you only top-level tables that are partitioned
select relname
from pg_class
where oid in (select partrelid from pg_partitioned_table);
You should be able to refine/expand that to get more details.
Version 2
Here's a comically verbose solution:
with
partition_parents as (
select relnamespace::regnamespace::text as schema_name,
relname as table_name,
'partition_parent' as info,
*
from pg_class
where relkind = 'p'), -- The parent table is relkind 'p', the partitions are regular tables, relkind 'r'
unpartitioned_tables as (
select relnamespace::regnamespace::text as schema_name,
relname as table_name,
'unpartitioned_table' as info,
*
from pg_class
where relkind = 'r'
and not relispartition
) -- Regular table
select * from partition_parents where schema_name not in ('information_schema','pg_catalog','api','extensions') -- Whatever you've got for schemas
union
select * from unpartitioned_tables where schema_name not in ('information_schema','pg_catalog','api','extensions') -- Whatever you've got for schemas
order by 1,2,3
You should be able to cut the size of that way down to match what you really want. Someone here who really gets the system catalogs should likely be able to provide a more concise version. In the plus column, it's kind of nice to add in the "partition_parent" versus "unpartitioned_table" detail, as above.

If you don't have to use a \d* shortcut, this query should work (though you'll need to filter out schemas not in your search_path):
SELECT relname FROM pg_class WHERE relkind IN ('r','p') AND NOT relispartition;

--do some test in greenplum 6.14
--to find normal table ,partition table
SELECT
*
FROM
(
SELECT
t1.schemaname,
t1.tablename,
t1.tableowner,
CASE
WHEN t3.tablename IS NULL
AND t2.tablename IS NULL THEN
'r'
WHEN t3.tablename IS NOT NULL
AND t2.tablename IS NULL THEN
'p-root' ELSE'p-child'
END AS tabletype,
t4.actionname,
t4.statime
FROM
pg_tables t1
LEFT JOIN pg_partitions t2 ON t1.tablename = t2.partitiontablename
AND t1.schemaname = t2.partitionschemaname
LEFT JOIN ( SELECT DISTINCT schemaname, tablename FROM pg_partitions WHERE schemaname IN ( 'public', 'ods', 'tmp' ) ) t3 ON t1.tablename = t3.tablename
AND t1.schemaname = t3.schemaname
LEFT JOIN (
SELECT
*
FROM
(
SELECT
schemaname,
objname,
actionname,
statime,
ROW_NUMBER ( ) OVER ( PARTITION BY schemaname, objname ORDER BY statime DESC ) AS rn
FROM
pg_catalog.pg_stat_operations
WHERE
classname = 'pg_class'
AND schemaname IN ( 'public', 'ods', 'tmp' )
AND actionname IN ( 'CREATE', 'ALTER' )
) n
WHERE
n.rn = 1
) t4 ON t1.schemaname = t4.schemaname
AND t1.tablename = t4.objname
WHERE
t1.schemaname IN ( 'public', 'ods', 'tmp' )
) o
WHERE
tabletype IN ( 'r', 'p-root' )
ORDER BY
1,
2,
3,
4

Related

Why is pg_table_def table returning tablenames containing the text "_$mig"

I am using pg_table_def to work out the primary keys associated to a given table, example below:
SELECT pg_table_def.tablename,
pg_table_def.column
FROM pg_table_def
WHERE schemaname = 'myschema'
AND tablename LIKE '%mytable%'
AND tablename LIKE '%_pkey%';
Example output from the above query:
tablename column
mytable_$mig_pkey id
This is correctly identifying the primary keys, but in some cases the format of the tablename returned has the text _$mig in it, for example: mytable_$mig_pkey.
I use the tablename in a later part of the code so currently have to remove this text. My questions are:
Why for some tables is the tablename formatted like this with _$mig?
Are there other strings that could appear between tablename and _pkey that should be taken into consideration?
Or is there a better way to identify the primary keys associated to a
table?
I've so far tried:
Checking the AWS documentation it recommends to check the search_path, so I've done this and made sure the schema is referenced in the where clause.
Checked that mytable_$mig doesn't exist / isn't available.
Other googling hasn't got me any further...
Update
Based on the comments below, running the below query:
select tablename, "column", type, encoding, distkey, sortkey, "notnull"
from pg_table_def
where schemaname = 'myschema' AND tablename LIKE '%mytable%';
Returns:
tablename column type encoding distkey sortkey notnull
mytable id integer none true 1 true
mytable desc character varying(50) lzo false 0 false
mytable_$mig_pkey id integer none true 1 false
Update:
So it looks like you have your actual table and one that was probably created accidentally. If you do select count(*) from mytable and select count(*) from mytable_$mig_pkey;, hopefully mytable has all your data and the other one is
empty. If so you can clean it up with drop table mytable_$mig_pkey.
I see you have the id column as the distkey which is Redshift's version of a primary key. You may also be trying to define constraints, which Redshift will allow you to do, but they are not enforced.
You can view the defined table constraints for a table like so:
select * from information_schema.table_constraints
where table_name='mytable';
On one of my tables gives a result like:
constraint_catalog | constraint_schema | constraint_name | table_catalog | table_schema | table_name | constraint_type | is_deferrable | initially_deferred
--------------------+-------------------+-----------------+---------------+--------------+------------+-----------------+---------------+--------------------
mydb | myschema | mytable_pkey | mydb | myschema | mytable | PRIMARY KEY | NO | NO
(1 row)
My current guess is that you attempted to define constraints on your table and accidentally created a second table at some point, which is why the other table has _pkey in its name.
I use a query like this myself:
select tablename, "column", type, encoding, distkey, sortkey, "notnull"
from pg_table_def
where schemaname = 'myschema' AND tablename LIKE '%mytable%';
Running something more general like that might help you figure out what's going on.
The \d I referenced in the comments is a way to list all the tables in the database that is a convenience function built into the psql command line app. The query it runs to do that is this:
SELECT n.nspname as "Schema",
c.relname as "Name",
CASE c.relkind WHEN 'r' THEN 'table' WHEN 'v' THEN 'view' WHEN 'm' THEN 'materialized view' WHEN 'i' THEN 'index' WHEN 'S' THEN 'sequence' WHEN 's' THEN 'special' WHEN 'f' THEN 'foreign table' END as "Type",
pg_catalog.pg_get_userbyid(c.relowner) as "Owner"
FROM pg_catalog.pg_class c
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r','v','m','S','f','')
AND n.nspname <> 'pg_catalog'
AND n.nspname <> 'information_schema'
AND n.nspname !~ '^pg_toast'
AND pg_catalog.pg_table_is_visible(c.oid)
ORDER BY 1,2;
Running that might help.

Getting referenced tables in Postgres

I have a list of foreign keys. I'd like to find out the tables where these FK's point to and the actual key the point to.
I've got a list of FK's like so:
columnName0, columnName1, columnName2
Foreign key references
columnName0 references table0.idTable0
columnName1 references table1.idTable1
columnName2 references table2.idTable2
Some sample tables:
Table0:
idTable0, PK
name
Table1:
idTable1, PK
age
Table2:
idTable2, PK
createdOn
A sample result:
| column | referenced_column | referenced_table |
|-------------|-------------------|------------------|
| columnName0 | idTable0 | table0 |
| columnName1 | idTable1 | table1 |
| columnName2 | idTable2 | table2 |
I'm trying to translate something I do in MySQL like this:
SELECT DISTINCT
COLUMN_NAME AS column,
REFERENCED_COLUMN_NAME AS referenced_column,
REFERENCED_TABLE_NAME AS referenced_table
FROM
INFORMATION_SCHEMA.KEY_COLUMN_USAGE
WHERE
COLUMN_NAME IN (?);
I'm going to have to use straight-up queries (unfortunately, no stored procedures).
You can query pg_constraint. For column names you should lookup pg_attribute. A foreign key may be based on multiple columns, so conkey and confkey of pg_constraint are arrays. You have to unnest the arrays to get a list of column names. Example:
select
conrelid::regclass table_name,
a1.attname column_name,
confrelid::regclass referenced_table,
a2.attname referenced_column,
conname constraint_name
from (
select conname, conrelid::regclass, confrelid::regclass, col, fcol
from pg_constraint c,
lateral unnest(conkey) col,
lateral unnest(confkey) fcol
where contype = 'f' -- foreign keys constraints
) s
join pg_attribute a1 on a1.attrelid = conrelid and a1.attnum = col
join pg_attribute a2 on a2.attrelid = confrelid and a2.attnum = fcol;
table_name | column_name | referenced_table | referenced_column | constraint_name
------------+-------------+------------------+-------------------+------------------------
products | image_id | images | id | products_image_id_fkey
(1 row)
In Postgres 9.4 or later the function unnest() may have multiple arguments and the inner query may look like this:
...
select conname, conrelid::regclass, confrelid::regclass, col, fcol
from pg_constraint c,
lateral unnest(conkey, confkey) u(col, fcol)
where contype = 'f' -- foreign keys constraints
...

Is there a psql command to show table definitions?

The sqlite3 CLI has a command .schema that will display the columns of all tables in the database.
The psql CLI for PostgreSQL has a meta-commands \d that shows columns for all "relations" (table, view, index, sequence, or foreign table) and a meta-command \dt that lists the relations that are tables but does not show the columns of those tables.
Is there a way to get psql to show output like sqlite3's .schema - show the output of \d on just relations that are tables? \d * shows columns for all relations, which in my database of 32 tables contains 63 tables and sequences. The pattern (* in this example) seems able to match on relation name but not relation type. Is there a pattern for "match all tables"?
If you want this like output:
sqlite> .schema t24
CREATE TABLE t24 (
i integer,
t text
);
You should use pg_dump -s, not psql:
bash>pg_dump -t t24 -s
CREATE TABLE t24 (
i integer,
t text
);
It created DDL just like .schema...
Now regarding the queries in comments, if you modify them a little:
t=# \d t24
Table "public.t24"
Column | Type | Modifiers
--------+---------+-----------
i | integer |
t | text |
t=# SELECT a.attname as "Column",pg_catalog.format_type(a.atttypid, a.atttypmod) as "Type",
(SELECT substring(pg_catalog.pg_get_expr(d.adbin, d.adrelid) for 128)
FROM pg_catalog.pg_attrdef d
WHERE d.adrelid = a.attrelid AND d.adnum = a.attnum AND a.atthasdef) as "Modifiers"
FROM pg_catalog.pg_attribute a
join pg_catalog.pg_class c on a.attrelid = c.oid
WHERE true
AND relname like 't24'
AND c.relkind = 'r'::"char"
AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY relname, a.attnum;
Column | Type | Modifiers
--------+---------+-----------
i | integer |
t | text |
(2 rows)
This query will show only tables in same manner as \d meta-command.
BTW I took the query from this metacommand. If you launch psql -E, and run '\d table_name` you will see that all meta commands are just wraps for select queries...

Find all partition tables "inheriting" from master table

Say I have a table, "foo", with partition tables "foo1", "foo2", and "foo3". But at the current moment all I know is there are parition tables which inherit from table "foo". How can I find that foo has 3 partitions, foo1, foo2, and foo3?
To list all your partitions (child tables) - tested with PG v9-v13:
SELECT c.relname FROM pg_inherits i JOIN pg_class p ON i.inhparent = p.oid
JOIN pg_class c ON i.inhrelid = c.oid WHERE p.relname='parentTableName';
Use pg_inherits. Example:
create table my_parent_table (id int);
create table my_child_table_no_1 (check (id < 10)) inherits (my_parent_table);
create table my_child_table_no_2 (check (id >= 10)) inherits (my_parent_table);
select relname
from pg_inherits i
join pg_class c on c.oid = inhrelid
where inhparent = 'my_parent_table'::regclass
relname
---------------------
my_child_table_no_1
my_child_table_no_2
(2 rows)
You can also select check constraints using pg_constraint:
select relname "child table", consrc "check"
from pg_inherits i
join pg_class c on c.oid = inhrelid
join pg_constraint on c.oid = conrelid
where contype = 'c'
and inhparent = 'my_parent_table'::regclass
child table | check
---------------------+------------
my_child_table_no_1 | (id < 10)
my_child_table_no_2 | (id >= 10)
(2 rows)
Starting with Postgres 12, there is a built-in function:
select *
from pg_partition_tree('the_table'::regclass)
where parentrelid is not null;
The way I dig into such catalog information is by using psql. Start psql with the -eE options:
psql -eE <your database>
And then show your table:
\d <your table>
This will list all the queries which psql generates to fetch the information from the catalog. This includes inherited tables.
Keep in mind that it is possible that the catalog changes from one major version to another - although for such basic functionality it is unlikely.

How to find inherited tables programatically in PostgreSQL?

I have a PostgreSQL 8.3 database where table inheritance is being used. I would like to get a list of all tables along with its schema name which is inherited from a base table using query. Is there any way we can get this using PGSQL?
Since you're on such an old version of PostgreSQL you'll probably have to use a PL/PgSQL function to handle inheritance depths of > 1. On modern PostgreSQL (or even 8.4) you'd use a recursive common table expression (WITH RECURSIVE).
The pg_catalog.pg_inherits table is the key. Given:
create table pp( ); -- The parent we'll search for
CREATE TABLE notpp(); -- Another root for multiple inheritance
create table cc( ) inherits (pp); -- a 1st level child of pp
create table dd( ) inherits (cc,notpp); -- a 2nd level child of pp that also inherits aa
create table notshown( ) inherits (notpp); -- Table that inherits only notpp
create table ccdd () inherits (cc,dd) -- Inheritance is a graph not a tree; join node
A correct result will find cc, dd, and ccdd, but not find notpp or notshown.
A single-depth query is:
SELECT pg_namespace.nspname, pg_class.relname
FROM pg_catalog.pg_inherits
INNER JOIN pg_catalog.pg_class ON (pg_inherits.inhrelid = pg_class.oid)
INNER JOIN pg_catalog.pg_namespace ON (pg_class.relnamespace = pg_namespace.oid)
WHERE inhparent = 'pp'::regclass;
... but this will only find cc.
For multi-depth inheritance (ie tableC inherits tableB inherits tableA) you have to extend that via a recursive CTE or a loop in PL/PgSQL, using the children of the last loop as parents in the next.
Update: Here's an 8.3 compatible version that should recursively find all tables that inherit directly or indirectly from a given parent. If multiple inheritance is used, it should find any table that has the target table as one of its parents at any point along the tree.
CREATE OR REPLACE FUNCTION find_children(oid) RETURNS SETOF oid as $$
SELECT i.inhrelid FROM pg_catalog.pg_inherits i WHERE i.inhparent = $1
UNION
SELECT find_children(i.inhrelid) FROM pg_catalog.pg_inherits i WHERE i.inhparent = $1;
$$ LANGUAGE 'sql' STABLE;
CREATE OR REPLACE FUNCTION find_children_of(parentoid IN regclass, schemaname OUT name, tablename OUT name) RETURNS SETOF record AS $$
SELECT pg_namespace.nspname, pg_class.relname
FROM find_children($1) inh(inhrelid)
INNER JOIN pg_catalog.pg_class ON (inh.inhrelid = pg_class.oid)
INNER JOIN pg_catalog.pg_namespace ON (pg_class.relnamespace = pg_namespace.oid);
$$ LANGUAGE 'sql' STABLE;
Usage:
regress=# SELECT * FROM find_children_of('pp'::regclass);
schemaname | tablename
------------+-----------
public | cc
public | dd
public | ccdd
(3 rows)
Here's the recursive CTE version, which will work if you update Pg, but won't work on your current version. It's much cleaner IMO.
WITH RECURSIVE inh AS (
SELECT i.inhrelid FROM pg_catalog.pg_inherits i WHERE inhparent = 'pp'::regclass
UNION
SELECT i.inhrelid FROM inh INNER JOIN pg_catalog.pg_inherits i ON (inh.inhrelid = i.inhparent)
)
SELECT pg_namespace.nspname, pg_class.relname
FROM inh
INNER JOIN pg_catalog.pg_class ON (inh.inhrelid = pg_class.oid)
INNER JOIN pg_catalog.pg_namespace ON (pg_class.relnamespace = pg_namespace.oid);
The following statement retrieves all child tables of the table public.base_table_name:
select bt.relname as table_name, bns.nspname as table_schema
from pg_class ct
join pg_namespace cns on ct.relnamespace = cns.oid and cns.nspname = 'public'
join pg_inherits i on i.inhparent = ct.oid and ct.relname = 'base_table_name'
join pg_class bt on i.inhrelid = bt.oid
join pg_namespace bns on bt.relnamespace = bns.oid
It should work with 8.3 although I'm not 100% sure.
For those who are running a version of PostgreSQL with RECURSIVE support here's a function that finds derived tables for the specified base table.
CREATE OR REPLACE FUNCTION tables_derived_from(base_namespace name, base_table name)
RETURNS TABLE (table_schema name, table_name name, oid oid)
AS $BODY$
WITH RECURSIVE inherited_id AS
(
SELECT i.inhrelid AS oid
FROM pg_inherits i
JOIN pg_class base_t ON i.inhparent = base_t.oid
JOIN pg_namespace base_ns ON base_t.relnamespace = base_ns.oid
WHERE base_ns.nspname = base_namespace AND base_t.relname = base_table
UNION
SELECT i.inhrelid AS oid
FROM pg_inherits i
JOIN inherited_id b ON i.inhparent = b.oid
)
SELECT child_ns.nspname as table_schema, child_t.relname as table_name, child_t.oid
FROM inherited_id i
JOIN pg_class child_t ON i.oid = child_t.oid
JOIN pg_namespace child_ns ON child_t.relnamespace = child_ns.oid
ORDER BY 1, 2, 3;
$BODY$ LANGUAGE sql STABLE;
It's important to note that one table can inherit multiple tables, and none of the solutions listed really expose that; they just walk down the tree of a single parent. Consider:
CREATE TABLE a();
CREATE TABLE b();
CREATE TABLE ab_() INHERITS (a,b);
CREATE TABLE ba_() INHERITS (b,a);
CREATE TABLE ab__() INHERITS (ab_);
CREATE TABLE ba__() INHERITS (ba_);
CREATE TABLE ab_ba_() INHERITS (ab_, ba_);
CREATE TABLE ba_ab_() INHERITS (ba_, ab_);
WITH RECURSIVE inh AS (
SELECT i.inhparent::regclass, i.inhrelid::regclass, i.inhseqno FROM pg_catalog.pg_inherits i WHERE inhparent = 'a'::regclass
UNION
SELECT i.inhparent::regclass, i.inhrelid::regclass, i.inhseqno FROM inh INNER JOIN pg_catalog.pg_inherits i ON (inh.inhrelid = i.inhparent)
) SELECT * FROM inh;
inhparent | inhrelid | inhseqno
-----------+----------+----------
a | ab_ | 1
a | ba_ | 2
ab_ | ab__ | 1
ba_ | ba__ | 1
ab_ | ab_ba_ | 1
ba_ | ab_ba_ | 2
ba_ | ba_ab_ | 1
ab_ | ba_ab_ | 2
(8 rows)
Notice that b doesn't show up at all which is incorrect, as both ab_ and ba_ inherit b.
I suspect the "best" way to handle this would be a column that's text[] and contains (array[inhparent::regclass])::text for each table. That would give you something like
inhrelid path
ab_ {"{a,b}"}
ba_ {"{b,a}"}
ab_ba_ {"{a,b}","{b,a}"}
While obviously not ideal, that would at least expose the complete inheritance path and allow you to access it with enough gymnastics. Unfortunately, constructing that is not at all easy.
A somewhat simpler alternative is not to include the full inheritance path at each level, only each tables direct parents. That would give you this:
inhrelid parents
ab_ {a,b}
ba_ {b,a}
ab_ba_ {ab_,ba_}