I have created the following function to truncate bunch of tables starting with "irm_gtresult". There are no syntax errors in my function, but the function doesn't truncate the tables when I run it. What could be wrong here?
My Postgres db version is 8.4.
create or replace function br()
RETURNS void
LANGUAGE plpgsql
AS
$$
DECLARE
row text;
BEGIN
FOR row IN
select table_name from information_schema.tables where table_name ILIKE 'irm_gtresult%'
LOOP
EXECUTE 'TRUNCATE TABLE ' || row;
END LOOP;
END;
$$;
Call:
select br();
Your code is valid. I tested and it works for me in Postgres 9.4.
Using the outdated and unsupported version 8.4 (like you added) may be the problem. The version is just too old, consider upgrading to a current version.
However, I have a couple of suggestions:
Don't use key word row as variable name.
You don't need to loop, you can TRUNCATE all tables in a single command. Faster, shorter.
You may need to add CASCADE if there are dependencies. Be aware of the effect.
CREATE OR REPLACE FUNCTION br()
RETURNS void AS
$func$
BEGIN
EXECUTE (
SELECT 'TRUNCATE TABLE '
|| string_agg(format('%I.%I', schemaname, tablename), ',')
|| ' CASCADE'
FROM pg_tables t
WHERE tablename ILIKE 'irm_gtresult%'
AND schemaname = 'public'
-- AND tableowner = 'postgres' -- optionaly restrict to one user
);
END
$func$ LANGUAGE plpgsql;
Call:
SELECT br();
I am using the view pg_tables from the system catalog. You can as well use information_schema.tables like you did. Note the subtle differences:
How to check if a table exists in a given schema
Related answers with more explanation:
Can I truncate tables dynamically?
Truncating all tables in a Postgres database
To truncate in postgres you just have to use the TRUNC() function.
Example:
SELECT TRUNC(price, 0) AS truncated_price
FROM product;
Related
I want to truncate the whole database while maintaining the sequence identity. I came up with something like this:
WITH tables_to_be_truncated AS (
SELECT table_name
FROM information_schema.tables
WHERE table_type='BASE TABLE'
AND table_schema='public'
AND table_name NOT IN ('admins', 'admin_roles')
)
TRUNCATE TABLE (SELECT table_name FROM tables_to_be_truncated) CONTINUE IDENTITY RESTRICT;
I get this error:
ERROR: syntax error at or near "TRUNCATE"
LINE 9: TRUNCATE TABLE (SELECT table_name FROM tables_to_be_truncated...
I do have the permissions to truncate the tables and when I run for a single table like TRUNCATE TABLE access_tokens it works fine.
I also tried with this
TRUNCATE TABLE (SELECT string_agg(table_name, ', ') FROM tables_to_be_truncated) CONTINUE IDENTITY RESTRICT
which didn't work as well.
From what I see in other posts, people are doing it with functions. I didn't want to go down this path honestly but if this is the only way...
You don't need a function for that. An anonymous code block will do:
DO $$
DECLARE row RECORD;
BEGIN
FOR row IN SELECT table_name
FROM information_schema.tables
WHERE table_type='BASE TABLE'
AND table_schema='public'
AND table_name NOT IN ('admins', 'admin_roles')
LOOP
EXECUTE format('TRUNCATE TABLE %I CONTINUE IDENTITY RESTRICT;',row.table_name);
END LOOP;
END;
$$;
Other than that I don't think you'll be able to run dynamic queries with pure SQL.
Demo: db<>fiddle
I'm trying to drop tables returned from a query using EXECUTE. Here's an example:
CREATE TABLE test_a (id BIGINT);
CREATE TABLE test_b (id BIGINT);
DO
$f$
DECLARE command TEXT;
BEGIN
SELECT INTO command 'SELECT ARRAY_TO_STRING(ARRAY_AGG($$DROP TABLE $$ || table_name), $$;$$) FROM information_schema.tables WHERE table_name ILIKE $$test%$$';
EXECUTE command;
END;
$f$;
The SELECT statement returns "DROP TABLE test_a; DROP TABLE test_b", which I'm passing into the declared variable and trying to run using EXECUTE, but to no effect. What am I doing wrong?
PostgreSQL 9.5.18, compiled by Visual C++ build 1800, 64-bit
You are storing the string SELECT ARRAY_TO_STRING ... in that variable, not the result of the SELECT statement.
You can also simplify ARRAY_TO_STRING(ARRAY_AGG(..)) to string_agg() and it's highly recommended to use format() to generate dynamic SQL, to properly deal with identifiers that need quoting.
Use the following:
DO
$f$
DECLARE
command TEXT;
BEGIN
SELECT string_agg(format('drop table %I', table_name), ';')
INTO command
FROM information_schema.tables
WHERE table_name ILIKE 'test%';
execute command;
END;
$f$;
I am trying to create tables from all the tables that have been imported into my foreign tables via import foreign schema.
The reason I want to do this is simply to copy data from one PostgreSQL server database to another. I know that I won't get sequences and other table-related indexes, but that's alright.
I don't wanna do a pg_dump, so I've been trying to do it this way, but so far I haven't had any luck with the below code.
When I run the function, the query is running, but the tables don't seem to start being created in my database and I am having a hard time figuring out if my function is actually working or not.
DROP FUNCTION gv.create_tables_from_foreign_schema(character varying,character varying);
CREATE OR REPLACE FUNCTION gv.create_tables_from_foreign_schema(_foreign_schema character varying, _local_schema character varying)
RETURNS void AS
$BODY$
declare
selectrow record;
begin
for selectrow in
select 'CREATE TABLE ' || quote_ident(_local_schema) || '.' ||quote_ident(t.table_name) || ' AS SELECT * FROM ' || quote_ident(_foreign_schema) || '.' ||quote_ident(t.table_name) || '' AS qry
from (
SELECT table_name
FROM information_schema.tables
WHERE table_schema = _foreign_schema
)t
loop
execute selectrow.qry;
end loop;
end;
$BODY$
LANGUAGE plpgsql
SELECT gv.create_tables_from_foreign_schema('gv_import_fra_pgv_2018a', 'gv')
Does the function make sense to you?
The function looks ok.
I suspect that everything is fine; you don't see any new tables because the transaction isn't finished yet. If you look into the data directory, you should see new files being created and written.
Below is a great function to check the real count of all tables in PostgreSQL database. I found it here.
From my local test, it seems that the function returns the all result only after it finished all counting for 100 tables.
I am trying to make it more practical. If we could save the result of each table counting as soon as it finished with the table, then we can check the progress of all counting jobs instead of waiting for the end.
I think if I could UPDATE the result in this function immediately after finishing the first table, it will be great for my requirement.
Can you let me know how I can update the result into the table after this function finishes the counting of the first table?
CREATE FUNCTION rowcount_all(schema_name text default 'public')
RETURNS table(table_name text, cnt bigint) as
$$
declare
table_name text;
begin
for table_name in SELECT c.relname FROM pg_class c
JOIN pg_namespace s ON (c.relnamespace=s.oid)
WHERE c.relkind = 'r' AND s.nspname=schema_name
ORDER BY c.relname
LOOP
RETURN QUERY EXECUTE format('select count(*) from %I.%I',
table_name, schema_name, table_name);
END LOOP;
end
$$ language plpgsql;
-- Query
WITH rc(schema_name,tbl) AS (
select s.n,rowcount_all(s.n) from (values ('schema1'),('schema2')) as s(n)
)
SELECT schema_name,(tbl).* FROM rc;
Updated
I have decided to use a shell script to run the function below as a background process. The function would generate a processing log file so that I can check the current process.
I think your idea is good, but I also don't think it will work "out of the box" on PostgreSQL. I'm by no means the expert on this, but the way MVCC works on PostgreSQL, it's basically doing all of the DML in what can best be understood as temporary space, and then if and when everything works as expected it moves it all in at the end.
This has a lot of advantages, most notably that when someone is updating tables it doesn't prevent others from querying from those same tables.
If this were Oracle, I think you could accomplish this within the stored proc by using commit, but this isn't Oracle. And to be fair, Oracle doesn't allow truncates to be rolled back within a stored proc the way PostgreSQL does, so there are gives and takes.
Again, I'm not the expert, so if I've messed up a detail or two, feel free to correct me.
So, back to the solution. One way you COULD accomplish this is to set up your server as a remote server. Something like this would work:
CREATE SERVER pgprod
FOREIGN DATA WRAPPER dblink_fdw
OPTIONS (dbname 'postgres', host 'localhost', port '5432');
Assuming you have a table that stores the tables and counts:
create table table_counts (
table_name text not null,
record_count bigint,
constraint table_counts_pk primary key (table_name)
);
Were it not for your desire to see these results as they occur, something like this would work, for a single schema. It's easy enough to make this all schemas, so this is for illustration:
CREATE or replace FUNCTION rowcount_all(schema_name text)
returns void as
$$
declare
rowcount integer;
tablename text;
begin
for tablename in SELECT c.relname FROM pg_class c
JOIN pg_namespace s ON (c.relnamespace=s.oid)
WHERE c.relkind = 'r' AND s.nspname=schema_name
ORDER BY c.relname
LOOP
EXECUTE 'select count(*) from ' || schema_name || '.' || tablename into rowcount;
insert into table_counts values (schema_name || '.' || tablename, rowcount)
on conflict (table_name) do
update set record_count = rowcount;
END LOOP;
end
$$ language plpgsql;
(this presupposes 9.5 or greater -- if not, hand-roll your own upsert).
However, since you want real-time updates to the table, you could then put that same upsert into a dblink expression:
perform dblink_exec('pgprod', '
<< your upsert statement here >>
');
Of course the formatting of the SQL within the DBlink is now a little extra tricky, but the upside is once you nail it, you can run the function in the background and query the table while it's running to see the dynamic results.
I'd weigh that against the need to really have the information real-time.
I have a query that I have to update every month and generate a new table. There are several references to this table, and I always seem to miss one. I was wondering if there is a way that I can set a local variable and reuse it through out the query. As an example:
DECLARE 'table'||to_char(curent_timestamp, 'MON') ||
to_char(current_timestanp,"YY") AS table_ref;
CREATE TABLE table_ref AS select * FROM base_table;
SELECT * FROM table_ref;
Thanks.
You can use FORMAT() and EXECUTE to execute your dynamic SQL, like so:
DO $$
DECLARE table_name TEXT;
BEGIN
SELECT FORMAT('table%I',TO_CHAR(CURRENT_TIMESTAMP,'MONYY')) INTO table_name; -- ex. tableJUN16
EXECUTE FORMAT('CREATE TABLE %I AS SELECT * FROM base_table;',table_name);
END; $$ LANGUAGE PLPGSQL;
This will create your new table from the base_table with your dynamic name.
https://www.postgresql.org/docs/current/static/functions-string.html