with a moderate PostgreSQL installation we accumulated quite a few stored procedures/functions and types.
Now in the lowest level composite type (i.e. 3 types are built with it and a myriad functions reference any of those types) one element of the type is of wrong type (i.e. smallint instead of bigint), thus handling it is identical, only the range is different.
How do I know all types depending on
a type (pg_catalog.pg_type seems
insufficient)?
How can I know all
functions depending on a type (as
arguments and locally scoped vars)?
Can I refactore a composite type
(maybe change smallint to bigint)
without dropping/rebuilding every
single function depending on it?
Is there any kind of
automation/tool/best practice for
such a refactoring?
I know its 4 questions in one, but atm this is kind of frustrating and any help would be appreciated!
Many Thanks!
The system catalogue "pg_depend" contains some useful dependency information. You can find objects depending on particular types a bit like this:
select * from pg_depend where refclassid = 'pg_type'::regclass
and refobjid = 'information_schema.sql_identifier'::regtype;
This finds objects dependent on the "information_schema.sql_identifier" type. In the result, classid is the OID of a catalogue- for instance, for a column depending on a user type, classid is 'pg_class'::regclass, objid is the OID of the pg_class row, and objsubid is the attnum value from pg_attribute, so for this case you can format the results like this:
select objid::regclass, attname from pg_depend
join pg_attribute on pg_attribute.attrelid = pg_depend.objid and pg_attribute.attnum = pg_depend.objsubid
where refclassid = 'pg_type'::regclass and refobjid = 'information_schema.sql_identifier'::regtype
and classid = 'pg_class'::regclass
limit 10
So in pg_depend, (classid,objid,objsubid) describe some object that depends on the object described by (refclassid,refobjid).
Related
What is the "classoid" column in pg_description table? I see it changing for different kinds of objects like table, function etc.,
The PostgreSQL Official documentations says classoid is "The OID of the system catalog this object appears in", but I don't understand it.
https://www.postgresql.org/docs/8.2/static/catalog-pg-description.html
If I'm trying to insert some comments into the pg_description table, is there a specific value based on the object? Like for table --> 1259 for Function --> 1259 etc.,
If that's true, may I know where can I find that list of classoids'?
They're oid values from pg_class, which is Postgres' internal list of tables (among other things). The simplest way to work with them is via the regclass type, e.g.:
/* Show catalog table name for all entries */
SELECT classoid::regclass, * FROM pg_description
/* Show all entries referencing pg_proc (i.e. functions) */
SELECT objoid::regprocedure, * FROM pg_description WHERE classoid = 'pg_proc'::regclass
However, you really, really shouldn't insert into the catalog tables directly. There may be associated entries to add, locks to acquire, validation to do, etc., and unless you know exactly what's going on under the hood, you could easily corrupt your database.
If you want to add an entry to pg_description, use a COMMENT statement.
The official docs and the accepted answer here did not add clarity for me, so I'm going to try to restate the answer in a way that might be helpful to others.
pg_description has four columns, objoid, classoid, objsubid, and description. description has the comment, but the other three values are needed to know exactly what is being commented on.
objoid tells whatyou specific thing in the DB this comment applies to, i.e. what actual table, column, procedure. But, there is way to know what this is refering to without consulting classoid.
classoid is the type of thing being described. This is a foreign key into pg_class. The relname column in pg_class tells you the type of thing.
pg_class is somewhat confusing. If you do select * from pg_class where relname = 'my_table' you will get a result, but this is not what pg_description.classoid refers to!
pg_description.classoid should refer to the entry in pg_class with the relname of…the string 'pg_class'. That is because the type of the thing being described is a class. And, of course, in this case, you then use objoid to look up the table in pg_class.
To clarify this, suppose pg_description's classoid referred to the entry in pg_class with the value for relname of 'pg_proc' That means that objoid is a reference to pg_proc (not to pg_class).
OK, so how do you figure out what's what?
SELECT
pg_class_objoid.relname AS "Table/View Name",
pg_description.description
FROM
pg_description
LEFT JOIN
pg_class AS pg_class_objoid ON pg_description.objoid = pg_class_objoid.oid
LEFT JOIN
pg_class AS pg_class_classoid ON pg_description.classoid = pg_class_classoid.oid
WHERE
pg_class_classoid.relname = 'pg_class'
This says to join pg_description.objoid to pg_class.oid and also to make a separate join pg_description.classoid to pg_class.oid.
We then restrict the results to only those where classoid refers to the pg_class with the relname 'pg_class'.
Now, if you do this, you'll see a lot of results. You'll see comments on tables and columns. This is where objsubid comes in.
For the table comments, the objsubid is 0, thus:
SELECT
pg_class_objoid.relname AS "Table/View Name",
pg_description.description
FROM
pg_description
LEFT JOIN
pg_class AS pg_class_objoid ON pg_description.objoid = pg_class_objoid.oid
LEFT JOIN
pg_class AS pg_class_classoid ON pg_description.classoid = pg_class_classoid.oid
WHERE
pg_class_classoid.relname = 'pg_class' AND
pg_description.objsubid = 0
Will show you what you want.
For completeness, the values of objsubid that are not 0 but where classoid indicates the type of thing is a table/view, the value for objsubid should correspond to the value pg_attribute.attnum where pg_attribute.attrelid equals pg_description.objoid. Thus, to see the table and column comments with their respective names:
SELECT
pg_class_objoid.relname AS "Table/View Name",
pg_attribute.attname AS "Column Name",
pg_description.description
FROM
pg_description
LEFT JOIN
pg_class AS pg_class_objoid ON pg_description.objoid = pg_class_objoid.oid
LEFT JOIN
pg_class AS pg_class_classoid ON pg_description.classoid = pg_class_classoid.oid
LEFT JOIN
pg_attribute ON pg_attribute.attnum = pg_description.objsubid AND
pg_attribute.attrelid = pg_description.objoid
WHERE
pg_class_classoid.relname = 'pg_class'
So if table A is:
no | username
1 | admin
2 | chicken
And table B is:
id | no
a | 1
b | 3
c | 4
Then, I do a NATURAL FULL OUTER JOIN as so:
SELECT no
FROM A NATURAL FULL OUTER JOIN
B;
Then, what is the result? And is the result the same for all PostgreSQL implementations?
Because does the 'no' come from table A, or table B, it is ambiguous. But, NATURAL joins combine the 'no'. But what if one of the 'no' is ambiguous, i.e. A.no IS NOT NULL, but B.no IS NULL, which of the 'no' does it pick? And what if A.no and B.no are both NULL?
TL;DR: So the question is, WHAT is the value of the no in SELECT no: Is it the A.no or B.no, or is it the COLAESCE of them?
SELECT no
FROM A NATURAL FULL OUTER JOIN
B;
First, don't use natural for joins. It is a bug waiting to happen. As you note in your question, natural chooses the join keys based on the names of columns. It doesn't take types into account. It doesn't even take explicitly declared foreign key relationships in to account.
The particularly insidious problem, though, is that someone reading the query does not see the join keys. That makes is much harder to debug queries or to modify/enhance them.
So, my advice is to use using instead.
SELECT no
FROM A FULL OUTER JOIN
B
USING (no);
What does a full join return? It returns all rows from both tables, regardless of whether the join matches or not. Because a NULL comparison always fails, NULL will not match in the join conditions.
For example, the following query returns 4 rows not 2 containing a NULL value:
with x as (
select NULL::int as id union all select NULL as id
)
select id
from x full join
x y
using (id);
You would get the same result with a natural join, but I simply don't use that construct.
I'm not 100% sure, but I'm pretty sure that all versions of Postgres that support full join would work the same way. This behavior is derived specifically from the ANSI definitions of joins and join conditions.
I have a utility that introspects columns of tables using:
select column_name, data_type from information_schema.columns
where table_name=%s
How can I extend this to introspect columns of materialized views?
Your query carries a few shortcomings / room for improvement:
A table name is not unique inside a database, you would have to narrow down to a specific schema, or could get surprising / misleading / totally incorrect results.
It's much more effective / convenient to cast the (optionally) schema-qualified table name to regclass ... see below.
A cast to regtype gives you generic type names instead of internal ones. But that's still only the base type.
Use the system catalog information functions format_type() instead to get an exact type name including modifiers.
With the above improvements you don't need to join to additional tables. Just pg_attribute.
Dropped columns reside in the catalog until the table is vacuumed (fully). You need to exclude those.
SELECT attname, atttypid::regtype AS base_type
, format_type(atttypid, atttypmod) AS full_type
FROM pg_attribute
WHERE attrelid = 'myschema.mytable'::regclass
AND attnum > 0
AND NOT attisdropped; -- no dead columns
As an aside: the views in the information schema are only good for standard compliance and portability (rarely works anyway). If you don't plan to switch your RDBMS, stick with the catalog tables, which are much faster - and more complete, apparently.
It would seem that postgres 9.3 has left materialized views out of the information_schema. (See http://postgresql.1045698.n5.nabble.com/Re-Materialized-views-WIP-patch-td5740513i40.html for a discussion.)
The following will work for introspection:
select attname, typname
from pg_attribute a
join pg_class c on a.attrelid = c.oid
join pg_type t on a.atttypid = t.oid
where relname = %s and attnum >= 1;
The clause attnum >= 1 suppresses system columns. The type names are pg_specific this way, I guess, but good enough for my purposes.
I'm running PostgreSQL 9.2 on Windows.
I have an existing table with some non nullable columns :
CREATE TABLE testtable
(
bkid serial NOT NULL,
bklabel character varying(128),
lacid integer NOT NULL
}
The I create a view on this table :
CREATE OR REPLACE VIEW test AS
SELECT testtable.bkid, testtable.lacid
from public.testtable;
I'm surprised that information_schema.columns for the view reports is_nullable to be YES for the selected columns ?
select * from information_schema.columns where table_name = 'test'
Reports :
"MyDatabase";"public";"test";"bkid";1;"";"YES";"integer";;;32;2;0;;"";;"";"";"";"";"";"";"";"";"";"MyDatabase";"pg_catalog";"int4";"";"";"";;"1";"NO";"NO";"";"";"";"";"";"";"NEVER";"";"NO"
"MyDatabase";"public";"test";"lacid";2;"";"YES";"integer";;;32;2;0;;"";;"";"";"";"";"";"";"";"";"";"MyDatabase";"pg_catalog";"int4";"";"";"";;"2";"NO";"NO";"";"";"";"";"";"";"NEVER";"";"NO"
Is it an expected behavior ?
My problem is that I'm trying to import such views in an Entity Framework Data Model and it fails because all columns are marked as nullable.
EDIT 1 :
The following query :
select attrelid, attname, attnotnull, pg_class.relname
from pg_attribute
inner join pg_class on attrelid = oid
where relname = 'test'
returns :
attrelid;attname;attnotnull;relname
271543;"bkid";f;"test"
271543;"lacid";f;"test"
As expected, attnotnull is 'false'.
As #Mike-Sherrill-Catcall suggested, I could manually set them to true :
update pg_attribute
set attnotnull = 't'
where attrelid = 271543
And the change is reflected in the information_schema.columns :
select * from information_schema.columns where table_name = 'test'
Output is :
"MyDatabase";"public";"test";"bkid";1;"";"NO";"integer";;;32;2;0;;"";;"";"";"";"";"";"";"";"";"";"MyDatabase";"pg_catalog";"int4";"";"";"";;"1";"NO";"NO";"";"";"";"";"";"";"NEVER";"";"NO"
"MyDatabase";"public";"test";"lacid";2;"";"NO";"integer";;;32;2;0;;"";;"";"";"";"";"";"";"";"";"";"MyDatabase";"pg_catalog";"int4";"";"";"";;"2";"NO";"NO";"";"";"";"";"";"";"NEVER";"";"NO"
I'll try to import the views in the Entity Framework data model.
EDIT 2 :
As guessed, it works, the view is now correctly imported in the Entity Framework Data Model.
Of course, I won't set all columns to be non nullable, as demonstrated above, only those non nullable in the underlying table.
I believe this is expected behavior, but I don't pretend to fully understand it. The columns in the base table seem to have the right attributes.
The column in the system tables underlying the information_schema here seems to be "attrnotnull". I see only one thread referring to "attnotnull" on the pgsql-hackers listserv: cataloguing NOT NULL constraints. (But that column might have had a different name in an earlier version. It's probably worth researching.)
You can see the behavior with this query. You'll need to work with the WHERE clause to get exactly what you need to see.
select attrelid, attname, attnotnull, pg_class.relname
from pg_attribute
inner join pg_class on attrelid = oid
where attname like 'something%'
On my system, columns that have a primary key constraint and columns that have a NOT NULL constraint have "attnotnull" set to 't'. The same columns in a view have "attnotnull" set to 'f'.
If you tilt your head and squint just right, that kind of makes sense. The column in the view isn't declared NOT NULL. Just the column in the base table.
The column pg_attribute.attnotnull is updatable. You can set it to TRUE, and that change seems to be reflected in the information_schema views. Although you can set it to TRUE directly, I think I'd be more comfortable setting it to match the value in the base table. (And by more comfortable, I don't mean to imply I'm comfortable at all with mucking about in the system tables.)
Why:
A view could be computed but references the column from a table. That computation could result in a NULL value on what is otherwise a non-null column. So basically, they put it into the too hard basket.
There is a way to see the underlying nullability for yourself with the following query:
select vcu.column_name, c.is_nullable, c.data_type
from information_schema.view_column_usage vcu
join information_schema."columns" c
on c.column_name = vcu.column_name
and c.table_name = vcu.table_name
and c.table_schema = vcu.table_schema
and c.table_catalog = vcu.table_catalog
where view_name = 'your_view_here'
If you know that you are only projecting the columns raw without functions, then it will work. Ideally, the Postgres provider for EF would use this view and also read the view definition to confirm nullability.
The nullability tracking in PostgreSQL is not developed very much at all. In most places, it will default to claiming everything is potentially nullable, which is in many cases allowed by the relevant standards. This is the case here as well: Nullability is not tracked through views. I wouldn't rely on it for an application.
How to see all the built-in general-purpose data types, which support postgresql? For example from phppgadmin is possible to browse all types, but how to get types list via query, something like this:
SELECT data_types from ....
Something like this:
select ns.nspname as schema_name, t.typname as type_name
from pg_type t
join pg_namespace ns on ns.oid = t.typnamespace
where t.typtype in ('b')
and t.typelem = 0;
pg_type contains an entry for each and every type in the database, that includes the composite type that is created for a table and so on. The above query tries to filter out those that might not be interesting for you. You will have to play around with it to make it fit your needs.
pg_type is documented in the manual: http://www.postgresql.org/docs/current/static/catalog-pg-type.html