I inherit a database with a specific schema that have 288 tables. These tables have all exactly the same column name. I would like to merge all these 288 tables in 1.
My first try was to CREATE TABLE AS :
CREATE TABLE bigTable AS SELECT X.* FROM (SELECT tablename FROM pg_tables WHERE tablename LIKE '%xxx') AS X
But this obviously doesn't work.
So I tried with a plpgsql script:
DO $$
DECLARE
r RECORD;
BEGIN
FOR r in (SELECT tablename FROM pg_tables WHERE tablename LIKE '%iti') LOOP
INSERT INTO xxx(gid, shape_len, geom)
SELECT * FROM r;
END LOOP;
END;
$$;
But it tells me that it doesn't know what r is.
I think I'm missing something about how pg handle this kind of things.
When you are creating syntax dynamically, use Execute command that performs SQL supplied with string data.
I've just made 3 tables with example values and made 4th table that has been filled using this statement
DO $$
DECLARE
tbl_name text;
BEGIN
FOR tbl_name IN (SELECT tablename FROM pg_tables WHERE tablename LIKE 'table%') LOOP
execute 'INSERT INTO table4 select * FROM '||tbl_name;
END LOOP;
END;
$$
Use this shell script to fetch all the tables and insert into the new table
#!/bin/bash
psql **DATABASE_NAME** -c "select tablename from pg_tables where
schemaname='public'" | sed 1,2d | head -n -2 > hello.out
while read line;
do
psql **DATABASE_NAME** -c "insert into **TABLE_NAME** (select * from $line )"
done < hello.out
Related
I have a set of tables with suffix _ABC
The table names are like:
TEST_ABC
MYTB_ABC
AB_1_ER_ABC
I know I can do:
TRUNCATE TABLE TEST_ABC;
But I don't want to run this 100s of times.
I have 100s of those. Is there a way using plain SQL(only 1 query) to truncate tables with suffix _ABC?
You need dynamic SQL for this. This can be done with an anonymous PL/pgSQL block:
do
$$
declare
l_sql text;
begin
select 'truncate '||string_agg(tablename, ',')||' cascade'
into l_sql
from pg_tables
where tablename like '%\_abc' escape '\'
and schemaname = 'public';
execute l_sql;
end;
$$
I am trying to create the query as a string and execute that in PostgreSQL 10.
As far as I know, I can use the EXECUTE command to execute my query from a defined string.
Unfortunately, I have got an error: SQL Error [42601]: ERROR: syntax error at or near "execute"
Below is my code:
drop table if exists delinquent;
create table delinquent
(
report_date date
,account_id text
)
;
INSERT INTO delinquent VALUES('2019-07-23', 'a1234');
INSERT INTO delinquent VALUES('2019-07-23', 'b5679');
--------------
drop table if exists output1;
create temp table output1
(
report_date date
,account_id text
)
;
--------------
do $$
declare table_name text := 'delinquent';
begin
truncate table output1;
insert into output1
execute concat('select * from ',table_name);
end; $$;
select * from output1;
Anybody has an idea on what is wrong and what to do about it?
Many thanks,
You need to run the complete INSERT statement as dynamic SQL. And to build dynamic SQL, using format() is highly recommended to properly deal with identifiers and literals:
do $$
declare
table_name text := 'delinquent';
some_value text := 'a1234';
begin
truncate table output1;
execute format('insert into output1 select * from %I where some_column = %L',
table_name, some_value);
end; $$;
select *
from output1;
I need to remove selected tables from the Postgres SQL. It would better to use like or where clause.
Like I have
TABLE_A
TABLE_B
TABLE_C
-
-
-
TABLE_N
I need to delete
TABLE_A to TABLE_X
Can be done with a single command, which is faster - in case this is a recurring task.
Add IF EXISTS if the existence of any tables is uncertain. This way we save an extra trip to the system catalogs (information_schema.tables or pg_catalog.pg_tables).
And you may want to add CASCADE:
DO
$do$
BEGIN
-- child safety device: quote RAISE instead of EXECUTE to prime the bomb
-- EXECUTE (
RAISE NOTICE '%', (
SELECT 'DROP TABLE IF EXISTS'
|| string_agg('table_' || chr(ascii('a') + g) , ', ')
|| ' CASCADE;'
FROM generate_series(0,13) g
);
END
$do$;
Generates a command of the form:
DROP TABLE IF EXISTS table_a, table_b, ... , table_n CASCADE;
Using generate_series() to generate requested table names. More here:
How to drop many (but not all) tables in one fell swoop?
How to check if a table exists in a given schema
DO
$$
DECLARE
r RECORD;
BEGIN
FOR r IN SELECT oid::REGCLASS table_name
FROM pg_class
WHERE relname <= 'table_x'
AND relkind = 'r'
LOOP
EXECUTE 'DROP TABLE' || r.table_name;
END LOOP;
END;
$$ LANGUAGE plpgsql;
For some fancy database maintenance for my developer database I'd like to be able to use queries to generate commands to alter the database. The thing is: I'm a complete greenhorn to PostgreSQL. I've made my attempt but have failed colorfully.
So in the end, I would like to have a table with a single column and each row would be a command (or group of commands, depending on the case) that I would think would look something like this...
DO $$
DECLARE
command_entry RECORD;
BEGIN
FOR command_entry IN SELECT * FROM list_of_commands
LOOP
EXECUTE command_entry;
END LOOP;
END;
$$;
Where the table list_of_commands could be populated with something like the following (which in this example would remove all tables from the public schema)...
CREATE TEMP TABLE list_of_commands AS
SELECT 'drop table if exists "' || tablename || '" cascade;'
FROM pg_tables
WHERE schemaname = 'public';
However, with this I get the following error...
ERROR: syntax error at or near ""drop table if exists ""dummy_table"" cascade;""
LINE 1: ("drop table if exists ""dummy_table"" cascade;")
I assume this is a matter of escaping characters, but I'm not entirely sure how to fit that into either A) the population of the table or B) the execution of each row. Does anyone know what I could do to achieve the desired result?
The command_entry variable is of type record while the EXECUTE command expects a string. What is apparently happening is that PostgreSQL turns the record into a double-quoted string, but that messes up your command. Also, your temp table does not use a column name, making things a bit awkward to work with (the column name becomes ?column?), so change both as follows:
CREATE TEMP TABLE list_of_commands AS
SELECT 'drop table if exists public.' || quote_ident(tablename) || ' cascade' AS cmd
FROM pg_tables
WHERE schemaname = 'public';
DO $$
DECLARE
command_entry varchar;
BEGIN
FOR command_entry IN SELECT cmd FROM list_of_commands
LOOP
EXECUTE command_entry;
END LOOP;
END;
$$;
But seeing that you do all of this at session level (temp table, anonymous code block), why not write a stored procedure that performs all of this housekeeping when you are ready to do spring cleaning?
CREATE FUNCTION cleanup() RETURNS void AS $$
BEGIN
FOR tbl IN SELECT tablename FROM pg_tables WHERE schemaname = 'public'
LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(tbl) || ' CASCADE';
END LOOP;
-- More housekeeping jobs
END;
$$ LANGUAGE plpgsql;
This saves a lot of typing: SELECT cleanup();. Any other housekeeping jobs you have you simply add to the stored procedure.
I had trouble with Patrick's answers, so here is an updated version for postgreSQL 10.
CREATE FUNCTION droptables(sn varchar) RETURNS void AS $$
DECLARE
tbl varchar;
BEGIN
FOR tbl IN SELECT tablename FROM pg_tables WHERE schemaname = sn
LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(tbl) || ' CASCADE';
END LOOP;
END;
$$ LANGUAGE plpgsql;
And then "SELECT droptables('public');".
I want to loop through all my tables to count rows in each of them. The following query gets me an error:
DO $$
DECLARE
tables CURSOR FOR
SELECT tablename FROM pg_tables
WHERE tablename NOT LIKE 'pg_%'
ORDER BY tablename;
tablename varchar(100);
nbRow int;
BEGIN
FOR tablename IN tables LOOP
EXECUTE 'SELECT count(*) FROM ' || tablename INTO nbRow;
-- Do something with nbRow
END LOOP;
END$$;
Errors:
ERROR: syntax error at or near ")"
LINE 1: SELECT count(*) FROM (sql_features)
^
QUERY: SELECT count(*) FROM (sql_features)
CONTEXT: PL/pgSQL function inline_code_block line 8 at EXECUTE statement
sql_features is a table's name in my DB. I already tried to use quote_ident() but to no avail.
I can't remember the last time I actually needed to use an explicit cursor for looping in PL/pgSQL.
Use the implicit cursor of a FOR loop, that's much cleaner:
DO
$$
DECLARE
rec record;
nbrow bigint;
BEGIN
FOR rec IN
SELECT *
FROM pg_tables
WHERE tablename NOT LIKE 'pg\_%'
ORDER BY tablename
LOOP
EXECUTE 'SELECT count(*) FROM '
|| quote_ident(rec.schemaname) || '.'
|| quote_ident(rec.tablename)
INTO nbrow;
-- Do something with nbrow
END LOOP;
END
$$;
You need to include the schema name to make this work for all schemas (including those not in your search_path).
Also, you actually need to use quote_ident() or format() with %I or a regclass variable to safeguard against SQL injection. A table name can be almost anything inside double quotes. See:
Table name as a PostgreSQL function parameter
Minor detail: escape the underscore (_) in the LIKE pattern to make it a literal underscore: tablename NOT LIKE 'pg\_%'
How I might do it:
DO
$$
DECLARE
tbl regclass;
nbrow bigint;
BEGIN
FOR tbl IN
SELECT c.oid
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind = 'r'
AND n.nspname NOT LIKE 'pg\_%' -- system schema(s)
AND n.nspname <> 'information_schema' -- information schema
ORDER BY n.nspname, c.relname
LOOP
EXECUTE 'SELECT count(*) FROM ' || tbl INTO nbrow;
-- raise notice '%: % rows', tbl, nbrow;
END LOOP;
END
$$;
Query pg_catalog.pg_class instead of tablename, it provides the OID of the table.
The object identifier type regclass is handy to simplify. n particular, table names are double-quoted and schema-qualified where necessary automatically (also prevents SQL injection).
This query also excludes temporary tables (temp schema is named pg_temp% internally).
To only include tables from a given schema:
AND n.nspname = 'public' -- schema name here, case-sensitive
The cursor returns a record, not a scalar value, so "tablename" is not a string variable.
The concatenation turns the record into a string that looks like this (sql_features). If you had selected e.g. the schemaname with the tablename, the text representation of the record would have been (public,sql_features).
So you need to access the column inside the record to create your SQL statement:
DO $$
DECLARE
tables CURSOR FOR
SELECT tablename
FROM pg_tables
WHERE tablename NOT LIKE 'pg_%'
ORDER BY tablename;
nbRow int;
BEGIN
FOR table_record IN tables LOOP
EXECUTE 'SELECT count(*) FROM ' || table_record.tablename INTO nbRow;
-- Do something with nbRow
END LOOP;
END$$;
You might want to use WHERE schemaname = 'public' instead of not like 'pg_%' to exclude the Postgres system tables.