In the same vein as pg_dump without comments on objects?, is anyone aware of a command to quickly get rid of the comments (created with COMMENT ON) on all objects at once ?
For now, I resorted to bash generating a SQL script that would void one by one the comments on each table/view/column, but it is quite slow, especially with >4000 columns.
Example:
COMMENT ON COLUMN table1.column1 IS NULL;
COMMENT ON COLUMN table1.column2 IS NULL;
COMMENT ON COLUMN table1.column3 IS NULL;
...
I have faced a very similar problem some time ago and came up with a very simple solution: delete from the system catalog table pg_description directly. Comments are just "attached" to objects and don't interfere otherwise.
DELETE FROM pg_description WHERE description = 'something special';
Disclaimer:
Manipulating catalog tables directly is dangerous and a measure of last resort. You have to know what you are doing and you are doing it at your own risk! If you screw up, you may screw up your database (cluster).
I asked about the idea on pgsql-admin list and got an encouraging answer from Tom Lane:
> DELETE FROM pg_description WHERE description = 'My very special
> totally useless comment.';
> AFAICS, there are no side-effects. Are there any?
It's safe enough, as long as you don't delete the wrong comments.
There's no hidden infrastructure for a comment.
regards, tom lane
You should make sure that there aren't any comments you'd rather keep.
Inspect what your are going to delete first. Be aware that many built-in Postgres objects have comments, too.
For instance, to only delete all comments on table columns, you could use:
SELECT *
-- DELETE
FROM pg_description
WHERE objsubid > 0;
The manual informs about the column objsubid:
For a comment on a table column, this is the column number (the objoid and
classoid refer to the table itself). For all other object types, this column is zero.
Ok, thanks to your help, I found the following commands pretty useful:
To delete a comment from a given column position of a specific object (here, mytable), you could go:
DELETE FROM pg_description WHERE (SELECT relname FROM pg_class WHERE oid=objoid)='mytable' AND objsubid=2;
...but note that it's not more efficient than using COMMENT ON mytable.myfield IS NULL;
Now, to delete all comments from my user-defined views and underlying columns, here's what works very well:
DELETE FROM pg_description WHERE (SELECT relkind FROM pg_class WHERE oid=objoid)='v' AND (SELECT relname FROM pg_class WHERE oid=objoid) ~ 'v_';
where:
(SELECT relkind FROM pg_class WHERE oid=objoid)='v': all views
(SELECT relname FROM pg_class WHERE oid=objoid) ~ 'v_' : additional security, my views' names all start with 'v_'
If you want to do this without hacking a system table, then this will generate the statements for you:
SELECT 'COMMENT ON COLUMN ' || quote_ident(pg_namespace.nspname) || '.' || quote_ident(pg_class.relname) || '.' || quote_ident(columns.column_name) || ' IS NULL;'
FROM pg_description
JOIN pg_class ON pg_class.oid = pg_description.objoid
JOIN pg_namespace ON pg_namespace.oid = pg_class.relnamespace
JOIN information_schema.columns
ON columns.table_schema = pg_namespace.nspname
AND columns.table_name = pg_class.relname
AND columns.ordinal_position = pg_description.objsubid
You should probably add a WHERE clause that constrains it to your schemas.
Related
I have a database with plenty of tables.
I want to tidy up relations that do not fit namewise anymore, due to name changes of the tables.
I was able to fix the constraints, but I am not able to put the lines together to list the sequences and the related columns. As pgAdmin shows under dependencies the column a sequence is connected to, it should be possible to create a SELECT to show sequences and their related column.
Try this:
SELECT a.attrelid::regclass AS table_name,
a.attname AS column_name,
pg_get_serial_sequence(a.attrelid::regclass::text, a.attname) AS sequence_name
FROM pg_attribute AS a
JOIN pg_class AS t ON a.attrelid = t.oid
WHERE t.relkind IN ('r', 'P')
AND NOT a.attisdropped
AND pg_get_serial_sequence(a.attrelid::regclass::text, a.attname) IS NOT NULL;
What is the "classoid" column in pg_description table? I see it changing for different kinds of objects like table, function etc.,
The PostgreSQL Official documentations says classoid is "The OID of the system catalog this object appears in", but I don't understand it.
https://www.postgresql.org/docs/8.2/static/catalog-pg-description.html
If I'm trying to insert some comments into the pg_description table, is there a specific value based on the object? Like for table --> 1259 for Function --> 1259 etc.,
If that's true, may I know where can I find that list of classoids'?
They're oid values from pg_class, which is Postgres' internal list of tables (among other things). The simplest way to work with them is via the regclass type, e.g.:
/* Show catalog table name for all entries */
SELECT classoid::regclass, * FROM pg_description
/* Show all entries referencing pg_proc (i.e. functions) */
SELECT objoid::regprocedure, * FROM pg_description WHERE classoid = 'pg_proc'::regclass
However, you really, really shouldn't insert into the catalog tables directly. There may be associated entries to add, locks to acquire, validation to do, etc., and unless you know exactly what's going on under the hood, you could easily corrupt your database.
If you want to add an entry to pg_description, use a COMMENT statement.
The official docs and the accepted answer here did not add clarity for me, so I'm going to try to restate the answer in a way that might be helpful to others.
pg_description has four columns, objoid, classoid, objsubid, and description. description has the comment, but the other three values are needed to know exactly what is being commented on.
objoid tells whatyou specific thing in the DB this comment applies to, i.e. what actual table, column, procedure. But, there is way to know what this is refering to without consulting classoid.
classoid is the type of thing being described. This is a foreign key into pg_class. The relname column in pg_class tells you the type of thing.
pg_class is somewhat confusing. If you do select * from pg_class where relname = 'my_table' you will get a result, but this is not what pg_description.classoid refers to!
pg_description.classoid should refer to the entry in pg_class with the relname of…the string 'pg_class'. That is because the type of the thing being described is a class. And, of course, in this case, you then use objoid to look up the table in pg_class.
To clarify this, suppose pg_description's classoid referred to the entry in pg_class with the value for relname of 'pg_proc' That means that objoid is a reference to pg_proc (not to pg_class).
OK, so how do you figure out what's what?
SELECT
pg_class_objoid.relname AS "Table/View Name",
pg_description.description
FROM
pg_description
LEFT JOIN
pg_class AS pg_class_objoid ON pg_description.objoid = pg_class_objoid.oid
LEFT JOIN
pg_class AS pg_class_classoid ON pg_description.classoid = pg_class_classoid.oid
WHERE
pg_class_classoid.relname = 'pg_class'
This says to join pg_description.objoid to pg_class.oid and also to make a separate join pg_description.classoid to pg_class.oid.
We then restrict the results to only those where classoid refers to the pg_class with the relname 'pg_class'.
Now, if you do this, you'll see a lot of results. You'll see comments on tables and columns. This is where objsubid comes in.
For the table comments, the objsubid is 0, thus:
SELECT
pg_class_objoid.relname AS "Table/View Name",
pg_description.description
FROM
pg_description
LEFT JOIN
pg_class AS pg_class_objoid ON pg_description.objoid = pg_class_objoid.oid
LEFT JOIN
pg_class AS pg_class_classoid ON pg_description.classoid = pg_class_classoid.oid
WHERE
pg_class_classoid.relname = 'pg_class' AND
pg_description.objsubid = 0
Will show you what you want.
For completeness, the values of objsubid that are not 0 but where classoid indicates the type of thing is a table/view, the value for objsubid should correspond to the value pg_attribute.attnum where pg_attribute.attrelid equals pg_description.objoid. Thus, to see the table and column comments with their respective names:
SELECT
pg_class_objoid.relname AS "Table/View Name",
pg_attribute.attname AS "Column Name",
pg_description.description
FROM
pg_description
LEFT JOIN
pg_class AS pg_class_objoid ON pg_description.objoid = pg_class_objoid.oid
LEFT JOIN
pg_class AS pg_class_classoid ON pg_description.classoid = pg_class_classoid.oid
LEFT JOIN
pg_attribute ON pg_attribute.attnum = pg_description.objsubid AND
pg_attribute.attrelid = pg_description.objoid
WHERE
pg_class_classoid.relname = 'pg_class'
I have a utility that introspects columns of tables using:
select column_name, data_type from information_schema.columns
where table_name=%s
How can I extend this to introspect columns of materialized views?
Your query carries a few shortcomings / room for improvement:
A table name is not unique inside a database, you would have to narrow down to a specific schema, or could get surprising / misleading / totally incorrect results.
It's much more effective / convenient to cast the (optionally) schema-qualified table name to regclass ... see below.
A cast to regtype gives you generic type names instead of internal ones. But that's still only the base type.
Use the system catalog information functions format_type() instead to get an exact type name including modifiers.
With the above improvements you don't need to join to additional tables. Just pg_attribute.
Dropped columns reside in the catalog until the table is vacuumed (fully). You need to exclude those.
SELECT attname, atttypid::regtype AS base_type
, format_type(atttypid, atttypmod) AS full_type
FROM pg_attribute
WHERE attrelid = 'myschema.mytable'::regclass
AND attnum > 0
AND NOT attisdropped; -- no dead columns
As an aside: the views in the information schema are only good for standard compliance and portability (rarely works anyway). If you don't plan to switch your RDBMS, stick with the catalog tables, which are much faster - and more complete, apparently.
It would seem that postgres 9.3 has left materialized views out of the information_schema. (See http://postgresql.1045698.n5.nabble.com/Re-Materialized-views-WIP-patch-td5740513i40.html for a discussion.)
The following will work for introspection:
select attname, typname
from pg_attribute a
join pg_class c on a.attrelid = c.oid
join pg_type t on a.atttypid = t.oid
where relname = %s and attnum >= 1;
The clause attnum >= 1 suppresses system columns. The type names are pg_specific this way, I guess, but good enough for my purposes.
I have an application that reads the structure of an existing PostgreSQL 9.1 database, compares it against a "should be" state and updates the database accordingly. That works fine, most of the time. However, I had several instances now when reading the current database structure deadlocked. The query responsible reads the existing foreign keys:
SELECT tc.table_schema, tc.table_name, tc.constraint_name, kcu.column_name,
ccu.table_schema, ccu.table_name, ccu.column_name
FROM information_schema.table_constraints AS tc
JOIN information_schema.key_column_usage AS kcu
ON tc.constraint_name = kcu.constraint_name
JOIN information_schema.constraint_column_usage AS ccu
ON ccu.constraint_name = tc.constraint_name
WHERE constraint_type = 'FOREIGN KEY'
Viewing the server status in pgAdmin shows this to be the only active query/transaction that's running on the server. Still, the query doesn't return.
The error is reproducible in a way: When I find a database that produces the error, it will produce the error every time. But not all databases produce the error. This is one mysterious bug, and I'm running out of options and ideas on what else to try or how to work around this. So any input or ideas are highly appreciated!
PS: A colleague of mine just reported he produced the same error using PostgreSQL 8.4.
I tested and found your query very slow, too. The root of this problem is that "tables" in information_schema are in fact complicated views to provide catalogs according to the SQL standard. In this particular case, matters are further complicated as foreign keys can be built on multiple columns. Your query yields duplicate rows for those cases which, I suspect, may be an undesired.
Correlated subqueries with unnest, fed to ARRAY constructors avoid the problem in my query.
This query yields the same information, just without duplicate rows and 100x faster. Also, I would venture to guarantee, without deadlocks.
Only works for PostgreSQL, not portable to other RDBMSes.
SELECT c.conrelid::regclass AS table_name
, c.conname AS fk_name
, ARRAY(SELECT a.attname
FROM unnest(c.conkey) x
JOIN pg_attribute a
ON a.attrelid = c.conrelid AND a.attnum = x) AS fk_columns
, c.confrelid::regclass AS ref_table
, ARRAY(SELECT a.attname
FROM unnest(c.confkey) x
JOIN pg_attribute a
ON a.attrelid = c.confrelid AND a.attnum = x) AS ref_columns
FROM pg_catalog.pg_constraint c
WHERE c.contype = 'f';
-- ORDER BY c.conrelid::regclass::text,2
The cast to ::regclass yields table names as seen with your current search_path. May or may not be what you want. For this query to include the absolute path (schema) for every table name you can set the search_path like this:
SET search_path = pg_catalog;
SELECT ...
To continue your session with your default search_path:
RESET search_path;
Related:
Get column names and data types of a query, table or view
What is the recommended way to figure out if a user got a certain right (e.g. select or execute) on a certain class (e.g. table or function) in PostgreSQL?
At the moment I got something like
aclcontains(
someColumnWithAclitemArray,
makeaclitem(userOid,grantorOid,someRight,false))
but it's terrible since I have to check for every grantorOid that is possible and for every userOid the user can belong to.
On a related note: what are the possible rights you can test for?
I haven't found any documentation but reading the source code I guess:
INSERT
SELECT
UPDATE
DELETE
TRUNCATE
REFERENCES
TRIGGER
EXECUTE
USAGE
CREATE
CONNECT
There also seems to be a CREATE TEMP right, but I can't figure out the correct text to use in the makeaclitem-function.
I've found that a better approach (and I seem to remember this was taken from some queries built into psql, or maybe the information_schema views) is to use the has_*_privilege functions, and simply apply them to a set of all possible combinations of user and object. This will take account of having access to an object via some group role as well.
For example, this will show which users have which access to non-catalogue tables and views:
select usename, nspname || '.' || relname as relation,
case relkind when 'r' then 'TABLE' when 'v' then 'VIEW' end as relation_type,
priv
from pg_class join pg_namespace on pg_namespace.oid = pg_class.relnamespace,
pg_user,
(values('SELECT', 1),('INSERT', 2),('UPDATE', 3),('DELETE', 4)) privs(priv, privorder)
where relkind in ('r', 'v')
and has_table_privilege(pg_user.usesysid, pg_class.oid, priv)
and not (nspname ~ '^pg_' or nspname = 'information_schema')
order by 2, 1, 3, privorder;
The possible privileges are detailed in the description of the has_*_privilege functions at http://www.postgresql.org/docs/current/static/functions-info.html#FUNCTIONS-INFO-ACCESS-TABLE.
'CREATE TEMP' is a database-level privilege: it permits a user to use a pg_temp_* schema. It can be tested with has_database_privilege(useroid, datoid, 'TEMP').
Take a look at the "Access Privilege Inquiry Functions" and also the "GRANT" reference page.
Because Redshift is supporting values() only in INSERT INTO queries, the below query can be used with the obviously not-so-nice union all select.
select usename, nspname || '.' || relname as relation,
case relkind when 'r' then 'table' when 'v' then 'view' end as relation_type,
priv
from pg_class join pg_namespace on pg_namespace.oid = pg_class.relnamespace,
pg_user,
(select 'select' as priv,1 as privorder union all select 'insert',2 union all select 'update',3 union all select 'delete',4)
where relkind in ('r', 'v')
and has_table_privilege(pg_user.usesysid, pg_class.oid, priv)
and not (nspname ~ '^pg_' or nspname = 'information_schema')
order by 2, 1, 3, privorder;
Edit:
Also, I realized, that in our db Dataiku creates tables that can have CAPITAL letter in them, so, if table not exist error happens, you should use the lower() function