When deleting table in Redshift I get an error message :
Amazon Invalid operation: cannot drop table
rawdata.bss_edw_customer_account because other objects depend on it;
I don't want to use CASCADE because I don't know which other tables/views it will kill. How do I find out which tables or views are dependent on the table I want to drop? I want to make sure I don't step on somebody else's work.
Thanks
Redshift Admin Views for object [1] and constraint [2] dependencies can help in identifying dependent objects. You can also create the find_depend view as described in the DROP table documentation [3] to view any dependencies.
I also found that these views may not always list materialized views as dependent objects. If you have created any MV in the past, then you might want to check for dependencies using the view DDLs. The following query could help:
select schemaname, viewname from pg_views where schemaname not like 'pg_catalog' and schemaname not like 'information_schema' and definition like '%<tablename>%';
[1] https://github.com/awslabs/amazon-redshift-utils/blob/master/src/AdminViews/v_object_dependency.sql
[2] https://github.com/awslabs/amazon-redshift-utils/blob/master/src/AdminViews/v_constraint_dependency.sql
[3] https://docs.aws.amazon.com/redshift/latest/dg/r_DROP_TABLE.html
If you need a more general overview of the (materialized) view dependencies in a schema you can use the following snippit. Just replace schema_name with the name of the schema. If you need to display dependencies across schemas this should be easily doable by adapting the regex or by doing a union.
create view admin.v_view_dependencies_fixed as (
with h1 as (
select generate_series as i
from generate_series(1, 100) -- we can use this since the query only touches the leader node
),
h2 as (
select schemaname as dependent_schema,
viewname as dependent_view,
schemaname || '.' || viewname as dependent_full,
regexp_substr(definition, 'schema_name\\.\\w+', 1, i) as dependency
from pg_views
cross join h1
where schemaname = 'schema_name'
and dependency is not null
and dependency != ''
)
select distinct
dependent_full,
dependent_schema,
dependent_view,
dependency as source_full,
split_part(dependency, '.', 1) as source_schema,
split_part(dependency, '.', 2) as source_object
from h2
where dependent_full != source_full
);
Related
Is it possible to easily see what tables exist in what schemas, at a glance?
So far I have had to connect to a database, view the schemas, then change the search path to one of the schemas and then list the tables. I had to do this for multiple schemas until I found the table I was looking for.
What if there is a scenario where you inherit a poorly documented database and you want to find a specific table in hundreds of schemas?
Ideally I imagine some output like so;
SCHEMA TABLE
--------------------
schema1 table1
schema2 table2
schema2 table1
--------------------
Or even the more standard <SCHEMA_NAME>.<TABLE_NAME>;
schema1.table1
schema2.table2
schema2.table1
The latter output would be even better since you could simply check the table using copy-paste;
my-database=# \d schema2.table1
Ideally I'm hoping I missed a built-in command to find this. I don't really want to create and memorize a lengthy SQL command to get this (somewhat basic) information.
You can make use of pg_tables
SELECT schemaname, tablename,
quote_ident(schemaname) || '.' || quote_ident(tablename)
FROM pg_tables
WHERE tablename = 'test';
I created a unique index for a materialized view as :
create unique index if not exists matview_key on
matview (some_group_id, some_description);
I can't tell if it has been created
How do I see the index?
Thank you!
Two ways to verify index creation:
--In psql
\d matview
--Using SQL
select
*
from
pg_indexes
where
indexname = 'matview_key'
and
tablename = 'matview';
More information on pg_indexes.
Like has been commented, if the command finishes successfully and you don't get an error message, the index was created. Possible caveat: while the transaction is not committed, nobody else can see it (except the unique name is reserved now), and it still might get rolled back. Check in a separate transaction to be sure.
To be absolutely sure:
SELECT pg_get_indexdef(oid)
FROM pg_catalog.pg_class
WHERE relname = 'matview_key'
AND relkind = 'i'
-- AND relnamespace = 'public'::regnamespace -- optional, to make sure of the schema, too
This way you see whether an index of the given name exists, and also its exact definition to rule out a different index with the same name. Pure SQL, works from any client. (There is nothing special about an index on materialized views.)
Also filter for the schema to be absolutely sure. Would be the "default" schema (a.k.a. "current" schema) in your case, since you did not specify in the creation. See:
How does the search_path influence identifier resolution and the "current schema"
Related:
Create index if it does not exist
How to check if a table exists in a given schema
In psql:
\di public.matview_key
To only find indexes. Again, the schema is optional to narrow down.
Progress Reporting
If creating an index takes a long time, you can look up progress in pg_stat_progress_create_index since Postgres 12:
SELECT * FROM pg_stat_progress_create_index
-- WHERE relid = 'public.matview'::regclass -- optionally narrow down
Un alternative to looking into pg_indexes is pg_matviews (for a materialized view only)
select *
from pg_matviews
where matviewname = 'my_matview_name';
I've been reading a lot about PostgreSQL's TOAST, and there's one thing I seem to be missing. They mention in the documentation that, "there are four different strategies for storing TOAST-able columns on disk," those being: PLAIN, EXTENDED, EXTERNAL, and MAIN. They also have a very clear way to define which strategy to use for your column, which can be found here. Essentially, it would be something like this:
ALTER TABLE table_name ALTER COLUMN column_name SET STORAGE EXTERNAL
The one thing I don't see is how to easily retrieve that setting. My question is, is there a simple way (either through commands or pgAdmin) to retrieve the storage strategy being used by a column?
This is stored pg_attribute.attstorage, e.g.:
select att.attname,
case att.attstorage
when 'p' then 'plain'
when 'm' then 'main'
when 'e' then 'external'
when 'x' then 'extended'
end as attstorage
from pg_attribute att
join pg_class tbl on tbl.oid = att.attrelid
join pg_namespace ns on tbl.relnamespace = ns.oid
where tbl.relname = 'table_name'
and ns.nspname = 'public'
and not att.attisdropped;
Note that attstorage is only valid if attlen is > -1
While I like #a_horse_with_no_name's method, after I posted this question, I expanded my search to just general table information and found that if you use psql, you can use the command described here, and the result will be a table listing all of the columns, their types, modifiers, storage types, stats targets, and descriptions.
So, using psql this info can be found with:
\d+ table_name
I just figured I'd post this in case anyone wanted another solution.
I know that to drop a table in a database is as follows, but if I have a database that has over a dozen tables and I only need to keep 3 of them, am I able to delete all but the 3 with only one command?
DROP TABLE IF EXISTS c_note RESTRICT;
Yes, but you need to enumerate over all the tables you want to drop. There's no command to drop all but 3. So, if you had the following tables:
foo
bar
baz
narf
poit
troz
And you wanted to drop the first three and keep the last three, you would use the command:
DROP TABLE foo, bar, baz;
If you know the tables all exist, there's no need for IF EXISTS, although it won't hurt. RESTRICT is also not needed -- that's the default (the opposite is CASCADE, where you also drop dependant objects).
SQL Doc
The following query will simmply delete all the tables but keep your schema
select 'drop table if exists ' || tablename || ' cascade;'
from pg_tables
where schemaname = 'public'
Write down a query where you can skip three tables by name
select 'drop table if exists ' || tablename || ' cascade;'
from pg_tables
where schemaname = 'public'
and tablename not in (:tbaleNamesToKeep);
I had 192 tables, so manually writing all tables out was infeasible. My strategy was to select all table names, remove the tables I wanted to keep and then run a DROP on the rest.
1. Select all tables
SELECT tablename FROM pg_tables
When you select, copy and paste the results, the text should look something like this:
"table1"
"table2"
"table3"
...
2. Remove the tables you want to keep
So let's say you want to keep table2, you just remove the line.
"table1"
"table3"
...
3. Add commas with text replacement
Replace "\n with ",\n.
To do this, I use regexr.com, but any text editor will do.
"table1",
"table3",
...
4. Create and run the final PSQL statement
DROP TABLE
"table1",
"table3",
...
How do I drop all tables whose name start with, say, doors_? Can I do some sort of regex using the drop table command?
I prefer not writing a custom script but all solutions are welcomed. Thanks!
This script will generate the DDL commands to drop them all:
SELECT 'DROP TABLE ' || t.oid::regclass || ';'
FROM pg_class t
-- JOIN pg_namespace n ON n.oid = t.relnamespace -- to select by schema
WHERE t.relkind = 'r'
AND t.relname ~~ E'doors\_%' -- enter search term for table here
-- AND n.nspname ~~ '%myschema%' -- optionally select by schema(s), too
ORDER BY 1;
The cast t.oid::regclass makes the syntax work for mixed case identifiers, reserved words or special characters in table names, too. It also prevents SQL injection and prepends the schema name where necessary. More about object identifier types in the manual.
About the schema search path.
You could automate the dropping, too, but it's unwise not to check what you actually delete before you do.
You could append CASCADE to every statement to DROP depending objects (views and referencing foreign keys). But, again, that's unwise unless you know very well what you are doing. Foreign key constraints are no big loss, but this will also drop all dependent views entirely. Without CASCADE you get error messages informing you which objects prevent you from dropping the table. And you can then deal with it.
I normally use one query to generate the DDL commands for me based on some of the metadata tables and then run those commands manually. For example:
SELECT 'DROP TABLE ' || tablename || ';' FROM pg_tables
WHERE tablename LIKE 'prefix%' AND schemaname = 'public';
This will return a bunch of DROP TABLE xxx; queries, which I simply copy&paste to the console. While you could add some code to execute them automatically, I prefer to run them on my own.