Hi I have some variables in a plpgsql script. It's all case sensitive.
schema := 'myschema';
table_name := 'MyTable'
column_name := 'MyColumn'
I need to put those variables in a simple statement:
select max(column_name) from schema.table_name.
I feel like I am battling the case sensitivity with PostgreSQL. I basically need to recreate this:
select max("MyColumn") from myschema."MyTable";
For the life of me I can't get this work and am clearly too dumb for PostgreSQL. Trying with EXECUTE(), EXECUTE() w/ FORMAT(), quote_indent() etc etc. Any thoughts?
Got it with this gem
execute format('select max(%I) from %s.%I', column_name, schema_name, table_name);
Depressing that that took hours of my life...
Related
I want to give all privileges to each table in the database。
sql as follows：
CREATE OR REPLACE FUNCTION test()
RETURNS void
AS $$
DECLARE
tb RECORD;
sql1 TEXT;
BEGIN
FOR tb IN (select tablename from pg_tables where schemaname='public')
LOOP
sql1 := 'GRANT ALL ON table ' || tb.tablename || ' TO tmp_admin';
EXECUTE sql1;
--RAISE NOTICE '%s', sql1;
END LOOP;
END
$$ LANGUAGE PLPGSQL;
select * FROM test();
when i RAISE NOTICE, it can normal output the sql i wanted。but this sql always effect。
but this sql always effect
I don't know what this means. Presumably you mean it doesn't do the right thing, otherwise you wouldn't have a question to ask.
As the horse mentioned, you don't to complicate this with a function at all. But regardless, it does work for me, as long as your table names don't need to be double-quoted. You should use FORMAT() rather than || to construct your string, to avoid that problem.
If you get an error, what is the error you get? If you think it silently has no effect, what did you do to infer the lack of effect?
Below is a great function to check the real count of all tables in PostgreSQL database. I found it here.
From my local test, it seems that the function returns the all result only after it finished all counting for 100 tables.
I am trying to make it more practical. If we could save the result of each table counting as soon as it finished with the table, then we can check the progress of all counting jobs instead of waiting for the end.
I think if I could UPDATE the result in this function immediately after finishing the first table, it will be great for my requirement.
Can you let me know how I can update the result into the table after this function finishes the counting of the first table?
CREATE FUNCTION rowcount_all(schema_name text default 'public')
RETURNS table(table_name text, cnt bigint) as
$$
declare
table_name text;
begin
for table_name in SELECT c.relname FROM pg_class c
JOIN pg_namespace s ON (c.relnamespace=s.oid)
WHERE c.relkind = 'r' AND s.nspname=schema_name
ORDER BY c.relname
LOOP
RETURN QUERY EXECUTE format('select count(*) from %I.%I',
table_name, schema_name, table_name);
END LOOP;
end
$$ language plpgsql;
-- Query
WITH rc(schema_name,tbl) AS (
select s.n,rowcount_all(s.n) from (values ('schema1'),('schema2')) as s(n)
)
SELECT schema_name,(tbl).* FROM rc;
Updated
I have decided to use a shell script to run the function below as a background process. The function would generate a processing log file so that I can check the current process.
I think your idea is good, but I also don't think it will work "out of the box" on PostgreSQL. I'm by no means the expert on this, but the way MVCC works on PostgreSQL, it's basically doing all of the DML in what can best be understood as temporary space, and then if and when everything works as expected it moves it all in at the end.
This has a lot of advantages, most notably that when someone is updating tables it doesn't prevent others from querying from those same tables.
If this were Oracle, I think you could accomplish this within the stored proc by using commit, but this isn't Oracle. And to be fair, Oracle doesn't allow truncates to be rolled back within a stored proc the way PostgreSQL does, so there are gives and takes.
Again, I'm not the expert, so if I've messed up a detail or two, feel free to correct me.
So, back to the solution. One way you COULD accomplish this is to set up your server as a remote server. Something like this would work:
CREATE SERVER pgprod
FOREIGN DATA WRAPPER dblink_fdw
OPTIONS (dbname 'postgres', host 'localhost', port '5432');
Assuming you have a table that stores the tables and counts:
create table table_counts (
table_name text not null,
record_count bigint,
constraint table_counts_pk primary key (table_name)
);
Were it not for your desire to see these results as they occur, something like this would work, for a single schema. It's easy enough to make this all schemas, so this is for illustration:
CREATE or replace FUNCTION rowcount_all(schema_name text)
returns void as
$$
declare
rowcount integer;
tablename text;
begin
for tablename in SELECT c.relname FROM pg_class c
JOIN pg_namespace s ON (c.relnamespace=s.oid)
WHERE c.relkind = 'r' AND s.nspname=schema_name
ORDER BY c.relname
LOOP
EXECUTE 'select count(*) from ' || schema_name || '.' || tablename into rowcount;
insert into table_counts values (schema_name || '.' || tablename, rowcount)
on conflict (table_name) do
update set record_count = rowcount;
END LOOP;
end
$$ language plpgsql;
(this presupposes 9.5 or greater -- if not, hand-roll your own upsert).
However, since you want real-time updates to the table, you could then put that same upsert into a dblink expression:
perform dblink_exec('pgprod', '
<< your upsert statement here >>
');
Of course the formatting of the SQL within the DBlink is now a little extra tricky, but the upside is once you nail it, you can run the function in the background and query the table while it's running to see the dynamic results.
I'd weigh that against the need to really have the information real-time.
I have created the following function to truncate bunch of tables starting with "irm_gtresult". There are no syntax errors in my function, but the function doesn't truncate the tables when I run it. What could be wrong here?
My Postgres db version is 8.4.
create or replace function br()
RETURNS void
LANGUAGE plpgsql
AS
$$
DECLARE
row text;
BEGIN
FOR row IN
select table_name from information_schema.tables where table_name ILIKE 'irm_gtresult%'
LOOP
EXECUTE 'TRUNCATE TABLE ' || row;
END LOOP;
END;
$$;
Call:
select br();
Your code is valid. I tested and it works for me in Postgres 9.4.
Using the outdated and unsupported version 8.4 (like you added) may be the problem. The version is just too old, consider upgrading to a current version.
However, I have a couple of suggestions:
Don't use key word row as variable name.
You don't need to loop, you can TRUNCATE all tables in a single command. Faster, shorter.
You may need to add CASCADE if there are dependencies. Be aware of the effect.
CREATE OR REPLACE FUNCTION br()
RETURNS void AS
$func$
BEGIN
EXECUTE (
SELECT 'TRUNCATE TABLE '
|| string_agg(format('%I.%I', schemaname, tablename), ',')
|| ' CASCADE'
FROM pg_tables t
WHERE tablename ILIKE 'irm_gtresult%'
AND schemaname = 'public'
-- AND tableowner = 'postgres' -- optionaly restrict to one user
);
END
$func$ LANGUAGE plpgsql;
Call:
SELECT br();
I am using the view pg_tables from the system catalog. You can as well use information_schema.tables like you did. Note the subtle differences:
How to check if a table exists in a given schema
Related answers with more explanation:
Can I truncate tables dynamically?
Truncating all tables in a Postgres database
To truncate in postgres you just have to use the TRUNC() function.
Example:
SELECT TRUNC(price, 0) AS truncated_price
FROM product;
This question already has answers here:
PostgreSQL "DESCRIBE TABLE"
(24 answers)
Closed 6 years ago.
I had employed MySQL for a couple of former projects. But now have
decided to switch to PostgreSQL. Not that version 8 also works on that,
ahem other OS which I'm stuck with at work.
But alas, two of the most useful commands appear to be missing:
SHOW TABLES
DESCRIBE table
Inasmuch as my prototyping DB is on my NetBSD server at home while my
data waiting to be 'based is at work, such that I have to connect via
Perl/DBI and XML-RPC (not psql, alas). The IT dept here just says, "Use
MS-Access", so no help there.
While I'm in the initial stage I need an informative way to blunder
around and see what's what as I try different ways to build this thing.
For that I had always relied on the two above from MySQL.
I can't believe there is no way for PostgreSQL to tell me what the
current DB's table structure is via simple SQL queries executed remotely.
Surely there must be. But I can't seem to find out from the couple of
books I have. All I dug up was some ultra-lame hack to get column names
for an already known table name by doing a "WHERE 1 != 1" or some such
so that no actual rows could be returned. Not very informative, that.
Surely I've missed the point, somewhere.
So enlighten me, please. What, pray tell, are the PostgreSQL-ish SQL
queries one uses so as to explore a given DB's table structure? What is
the PostgreSQL translation for "SHOW TABLES" and "DESCRIBE table"?
SHOW TABLES and DESCRIBE TABLE are MySQL-specific admin commands, and nothing to do with standard SQL.
You want the:
\d
and
\d+ tablename
commands from psql.
These are implemented client-side. I find this odd myself, and would love to move them server-side as built-in SQL commands one day.
Other clients provide other ways to browse the structure - for example, PgAdmin-III.
If you want a portable way to get table structure in code, you should use the information_schema views, which are SQL-standard. See information_schema. They're available in MySQL, PostgreSQL, Ms-SQL, and most other DBs. The downside is that they're fiddlier to use, so they aren't convenient for quick access when you're just browsing a DB structure.
As per the Documentation
SELECT
table_schema || '.' || table_name as show_tables
FROM
information_schema.tables
WHERE
table_type = 'BASE TABLE'
AND
table_schema NOT IN ('pg_catalog', 'information_schema');
for more convenience make it as a function
create or replace function show_tables() returns SETOF text as $$
SELECT
table_schema || '.' || table_name as show_tables
FROM
information_schema.tables
WHERE
table_type = 'BASE TABLE'
AND
table_schema NOT IN ('pg_catalog', 'information_schema');
$$
language sql;
So we can get the tables using
select show_tables()
For the table description
select column_name, data_type, character_maximum_length
from INFORMATION_SCHEMA.COLUMNS where table_name ='table_name';
as a Function
create or replace function describe_table(tbl_name text) returns table(column_name
varchar, data_type varchar,character_maximum_length int) as $$
select column_name, data_type, character_maximum_length
from INFORMATION_SCHEMA.COLUMNS where table_name = $1;
$$
language 'sql';
select * from describe_table('a_table_name');
I see from the Postgres 8.1 docs that EXPLAIN generated something like tabular data:
Prior to PostgreSQL 7.3, the plan was emitted in the form of a NOTICE
message. Now it appears as a query result (formatted like a table with
a single text column).
I'm working with 9.0, for which, the docs say, the output can be a variety of types, including TEXT and XML. What I'd really like to do is treat the output as a standard query result so that I could generate a simple report for a query or a set of queries, e.g.,
SELECT maxcost FROM (
EXPLAIN VERBOSE
SELECT COUNT(*)
FROM Mytable
WHERE value>17);
The above doesn't work in any form that I've tried, and I made up the attribute maxcost to demonstrate how neat it would be to pull out specific bits of data (in this case, the maximum estimated cost of the query). Is there anything I can do that would get me part of the way there? I'd prefer to be able to work within a simple SQL console.
No other answers so far, so here's my own stab at it.
It's possible to read the results of explain into a variable within plpgsql, and since the output can be in XML one can wrap EXPLAIN in a stored function to yield the top-level costs using xpath:
CREATE OR REPLACE FUNCTION estimate_cost(IN query text,
OUT startup numeric,
OUT totalcost numeric,
OUT planrows numeric,
OUT planwidth numeric)
AS
$BODY$
DECLARE
query_explain text;
explanation xml;
nsarray text[][];
BEGIN
nsarray := ARRAY[ARRAY['x', 'http://www.postgresql.org/2009/explain']];
query_explain :=e'EXPLAIN(FORMAT XML) ' || query;
EXECUTE query_explain INTO explanation;
startup := (xpath('/x:explain/x:Query/x:Plan/x:Startup-Cost/text()', explanation, nsarray))[1];
totalcost := (xpath('/x:explain/x:Query/x:Plan/x:Total-Cost/text()', explanation, nsarray))[1];
planrows := (xpath('/x:explain/x:Query/x:Plan/x:Plan-Rows/text()', explanation, nsarray))[1];
planwidth := (xpath('/x:explain/x:Query/x:Plan/x:Plan-Width/text()', explanation, nsarray))[1];
RETURN;
END;
$BODY$
LANGUAGE plpgsql;
Hence the example from the question becomes:
SELECT totalcost
FROM estimate_cost('SELECT COUNT(*)
FROM Mytable
WHERE value>17');
This message suggests using "FOR rec IN EXECUTE EXPLAIN..." which seems to do the trick e.g.:
drop table if exists temp_a;
create temp table temp_a
(
"QUERY PLAN" text
);
DO
$$
DECLARE
rec record;
BEGIN
FOR rec IN EXECUTE 'EXPLAIN VERBOSE select version()'
LOOP
-- RAISE NOTICE 'rec=%', row_to_json(rec);
insert into temp_a
select rec."QUERY PLAN";
END LOOP;
END
$$ language plpgsql;
select *
from temp_a;