I am trying to follow this guide from AWS docs: http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.PostGIS
On "Step 2: Load the PostGIS extensions", in the docs it shows (4 rows) but there are three rows there. Running the same commands myself up to that point, I see four rows, and the row that was missing from the docs is tiger_data. In Step 3, should ownership of tiger_data be given to rds_superuser as well?
In "Step 4: Transfer ownership of the objects to the rds_superuser role", I'm getting a syntax error from using the query provided in the docs and I don't know what to do about this:
postgres=> CREATE FUNCTION exec(text) returns text language plpgsql volatile AS $f$ BEGIN EXECUTE $1; RETURN $1; END; $f$;
CREATE FUNCTION
postgres=> SELECT exec('ALTER TABLE ' || quote_ident(s.nspname) || '.' || quote_ident(s.relname) || ' OWNER TO rds_superuser,')
postgres-> FROM (
postgres(> SELECT nspname, relname
postgres(> FROM pg_class c JOIN pg_namespace n ON (c.relnamespace = n.oid)
postgres(> WHERE nspname in ('tiger','topology') AND
postgres(> relkind IN ('r','S','v') ORDER BY relkind = 'S')
postgres-> s;
ERROR: syntax error at end of input
LINE 1: ALTER TABLE tiger.loader_variables OWNER TO rds_superuser,
^
QUERY: ALTER TABLE tiger.loader_variables OWNER TO rds_superuser,
CONTEXT: PL/pgSQL function exec(text) line 1 at EXECUTE statement
That looks to me like a typo in the docs - there is a , where there should be a ;. The query being constructed is:
ALTER TABLE tiger.loader_variables OWNER TO rds_superuser,
But should be:
ALTER TABLE tiger.loader_variables OWNER TO rds_superuser;
So change the SELECT line to:
SELECT exec('ALTER TABLE ' || quote_ident(s.nspname) || '.' || quote_ident(s.relname) || ' OWNER TO rds_superuser;')
FROM ...
Related
I have a crosstab function that I've used successfully many times in the past, but now it's dumping all the data at the end instead of pivoting it into the output table. It can't seem to find Crosstab. I've researched it doing the following;
create extension if not exists tablefunc; --- answer is: extension "tablefunc" already exists
create extension tablefunc with schema animals; answer is: as above
select count(*) from information_schema.routines where routine_name like 'crosstab%'; ---- answer is 6.
The following is a section of the function code:
BEGIN
str := '" " text,'; -- blanks in A1 cell
FOR rec IN SELECT DISTINCT col_name
FROM an_in_tbl
ORDER BY col_name
LOOP
str := str || '"' || rec.col_name || '" text' ||',';
END LOOP;
str:= substring(str, 0, length(str));
EXECUTE 'CREATE EXTENSION IF NOT EXISTS tablefunc;
DROP TABLE IF EXISTS an_out_tbl;
CREATE TABLE an_out_tbl AS
SELECT *
FROM crosstab(''select row_name, col_name, row_value from an_in_tbl order by 1'',
''SELECT DISTINCT col_name FROM an_in_tbl ORDER BY 1'')
AS final_result ('|| str ||')';
select animal_pivot_fn()
NOTICE: extension "tablefunc" already exists, skipping
NOTICE: table "an_out_tbl" does not exist, skipping
ERROR: function crosstab(unknown, unknown) does not exist
LINE 5: FROM crosstab('select row_name, col_name, row_value from...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
QUERY: CREATE EXTENSION IF NOT EXISTS tablefunc;
DROP TABLE IF EXISTS an_out_tbl;
CREATE TABLE an_out_tbl AS
SELECT *
FROM crosstab('select row_name, col_name, row_value from an_in_tbl order by 1',
'SELECT DISTINCT col_name FROM an_in_tbl ORDER BY 1')
AS final_result (" " text,"CAT" text,"DOG" text,"SNAKE" text,"HORSE" text,"ELEPHANT" text,"MOUSE" text,"MONKEY"... and many more... HERE IS WHERE THE DATA GETS DUMPED AND NO PIVOTED TABLE GETS CREATED.
Need to run below query
Run \dx command .
if result like below need to run following query
CREATE EXTENSION tablefunc;
Run \dx command again,result should be like below.
Now you can run crosstab query it should be solved.
I need to remove selected tables from the Postgres SQL. It would better to use like or where clause.
Like I have
TABLE_A
TABLE_B
TABLE_C
-
-
-
TABLE_N
I need to delete
TABLE_A to TABLE_X
Can be done with a single command, which is faster - in case this is a recurring task.
Add IF EXISTS if the existence of any tables is uncertain. This way we save an extra trip to the system catalogs (information_schema.tables or pg_catalog.pg_tables).
And you may want to add CASCADE:
DO
$do$
BEGIN
-- child safety device: quote RAISE instead of EXECUTE to prime the bomb
-- EXECUTE (
RAISE NOTICE '%', (
SELECT 'DROP TABLE IF EXISTS'
|| string_agg('table_' || chr(ascii('a') + g) , ', ')
|| ' CASCADE;'
FROM generate_series(0,13) g
);
END
$do$;
Generates a command of the form:
DROP TABLE IF EXISTS table_a, table_b, ... , table_n CASCADE;
Using generate_series() to generate requested table names. More here:
How to drop many (but not all) tables in one fell swoop?
How to check if a table exists in a given schema
DO
$$
DECLARE
r RECORD;
BEGIN
FOR r IN SELECT oid::REGCLASS table_name
FROM pg_class
WHERE relname <= 'table_x'
AND relkind = 'r'
LOOP
EXECUTE 'DROP TABLE' || r.table_name;
END LOOP;
END;
$$ LANGUAGE plpgsql;
I want to loop through all my tables to count rows in each of them. The following query gets me an error:
DO $$
DECLARE
tables CURSOR FOR
SELECT tablename FROM pg_tables
WHERE tablename NOT LIKE 'pg_%'
ORDER BY tablename;
tablename varchar(100);
nbRow int;
BEGIN
FOR tablename IN tables LOOP
EXECUTE 'SELECT count(*) FROM ' || tablename INTO nbRow;
-- Do something with nbRow
END LOOP;
END$$;
Errors:
ERROR: syntax error at or near ")"
LINE 1: SELECT count(*) FROM (sql_features)
^
QUERY: SELECT count(*) FROM (sql_features)
CONTEXT: PL/pgSQL function inline_code_block line 8 at EXECUTE statement
sql_features is a table's name in my DB. I already tried to use quote_ident() but to no avail.
I can't remember the last time I actually needed to use an explicit cursor for looping in PL/pgSQL.
Use the implicit cursor of a FOR loop, that's much cleaner:
DO
$$
DECLARE
rec record;
nbrow bigint;
BEGIN
FOR rec IN
SELECT *
FROM pg_tables
WHERE tablename NOT LIKE 'pg\_%'
ORDER BY tablename
LOOP
EXECUTE 'SELECT count(*) FROM '
|| quote_ident(rec.schemaname) || '.'
|| quote_ident(rec.tablename)
INTO nbrow;
-- Do something with nbrow
END LOOP;
END
$$;
You need to include the schema name to make this work for all schemas (including those not in your search_path).
Also, you actually need to use quote_ident() or format() with %I or a regclass variable to safeguard against SQL injection. A table name can be almost anything inside double quotes. See:
Table name as a PostgreSQL function parameter
Minor detail: escape the underscore (_) in the LIKE pattern to make it a literal underscore: tablename NOT LIKE 'pg\_%'
How I might do it:
DO
$$
DECLARE
tbl regclass;
nbrow bigint;
BEGIN
FOR tbl IN
SELECT c.oid
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind = 'r'
AND n.nspname NOT LIKE 'pg\_%' -- system schema(s)
AND n.nspname <> 'information_schema' -- information schema
ORDER BY n.nspname, c.relname
LOOP
EXECUTE 'SELECT count(*) FROM ' || tbl INTO nbrow;
-- raise notice '%: % rows', tbl, nbrow;
END LOOP;
END
$$;
Query pg_catalog.pg_class instead of tablename, it provides the OID of the table.
The object identifier type regclass is handy to simplify. n particular, table names are double-quoted and schema-qualified where necessary automatically (also prevents SQL injection).
This query also excludes temporary tables (temp schema is named pg_temp% internally).
To only include tables from a given schema:
AND n.nspname = 'public' -- schema name here, case-sensitive
The cursor returns a record, not a scalar value, so "tablename" is not a string variable.
The concatenation turns the record into a string that looks like this (sql_features). If you had selected e.g. the schemaname with the tablename, the text representation of the record would have been (public,sql_features).
So you need to access the column inside the record to create your SQL statement:
DO $$
DECLARE
tables CURSOR FOR
SELECT tablename
FROM pg_tables
WHERE tablename NOT LIKE 'pg_%'
ORDER BY tablename;
nbRow int;
BEGIN
FOR table_record IN tables LOOP
EXECUTE 'SELECT count(*) FROM ' || table_record.tablename INTO nbRow;
-- Do something with nbRow
END LOOP;
END$$;
You might want to use WHERE schemaname = 'public' instead of not like 'pg_%' to exclude the Postgres system tables.
I have seen that it is possible to convert all tables to case insensitive names using the following commands in psql:
\o /tmp/go_to_lower
select 'ALTER TABLE '||'"'||tablename||'"'||' RENAME TO ' ||
lower(tablename)||';' from pg_tables where schemaname = 'public';
psql -U username database < /tmp/go_to_lower
I have been unable to unearth a command to convert all columns to case insensitive in the same way. How can this be achieved?
EDIT: Apparently the above code only converts table names to lower case. I am aware that this code ALTER TABLE "YourTableName" RENAME TO YourTableName; will convert to case insensitive for a table name. Is there a way to do a similar function on mass for column names?
Along the same lines as the original, then, you should be able to do the following. This renames all columns that are not already in lower case, by extracting them from the information_schema, generating SQL for the changes, storing it to a file then executing the SQL again.
\t on
select 'ALTER TABLE '||'"'||table_name||'"'||' RENAME COLUMN '||'"'||column_name||'"'||' TO ' || lower(column_name)||';'
from information_schema.columns
where table_schema = 'public' and lower(column_name) != column_name
\g /tmp/go_to_lower
\i /tmp/go_to_lower
By default, all you identifiers are case insensitive, and internally PostgreSQL stores them in lowercase. In case you need to have:
case sensitive
non-ASCII characters
special characters
within your identifiers, you should use double quotes (") around your identifiers.
Please, check this bit of the PostgreSQL documentation.
EDIT: After your clarification, you can use:
SELECT 'ALTER TABLE '||quote_ident(t.relname)||' RENAME TO '||t.relname||';'
FROM pg_class t, pg_namespace s
WHERE s.oid = t.relnamespace AND s.nspname = 'public'
AND t.relkind='r' AND t.relname != lower(t.relname)
ORDER BY 1;
and for columns:
SELECT 'ALTER TABLE '||quote_ident(t.relname)||
' RENAME COLUMN '||quote_ident(a.attname)||
' TO '||a.attname||';'
FROM pg_class t, pg_namespace s, pg_attribute a
WHERE s.oid = t.relnamespace AND s.nspname = 'public'
AND t.relkind='r'
AND a.attrelid = t.oid AND NOT a.attisdropped AND a.attnum > 0
AND a.attname != lower(a.attname)
ORDER BY 1;
Then copy-paste the output into your client.
If you're using psql, you can use \t to enable rows-only mode, \o <full_file_path> to save output into the temporary file and, finally, \i <full_file_path> to execute actual statements.
I created a SQL query on Database Administrators that does just this.
Converts all identifiers to lower case
Converts spaces ' ' to '_'
Does this for all schema, table, and column names
For more information see,
How do I adopt the PostgreSQL naming convention in legacy database?
do language plpgsql $$
declare
r record;
begin
for r in
select relname, attname
from pg_attribute a
inner join pg_class c on a.attrelid = c.oid
inner join pg_namespace n on c.relnamespace = n.oid
where
n.nspname = 'public'
and
attname != lower(attname)
and
not attisdropped
loop
execute format('
alter table %1$I rename column %2$I to %3$s
', r.relname, r.attname, lower(r.attname));
end loop;
end;
$$;
Issue a begin; before trying this. Check if it is correct. Only then issue a commit;. If you are using a namespace then substitute it in the where clause.
Let me add a step by step guide using PgAdmin for beginners like myself and those who are not used to command line tools like psql:
Execute the following query in PgAdmin:
SELECT 'ALTER TABLE ' || quote_ident(c.table_schema) || '.'
|| quote_ident(c.table_name) || ' RENAME "' || c.column_name || '" TO ' || quote_ident(lower(c.column_name)) || ';' As ddlsql
FROM information_schema.columns As c
WHERE c.table_schema NOT IN('information_schema', 'pg_catalog')
AND c.column_name <> lower(c.column_name)
ORDER BY c.table_schema, c.table_name, c.column_name;
*Source: https://www.postgresonline.com/article_pfriendly/141.html. Note that you do not need to change anything here.
Now export the result to a textfile.
Open the .csv file with Excel, Notepad or any texteditor of your choice. Copy all lines except the first one ("ddlsql") and past them into a new query in PgAdmin. Make sure to remove doubled text quotations. Run it and done.
Suppose I create a table in Postgresql with a comment on a column:
create table t1 (
c1 varchar(10)
);
comment on column t1.c1 is 'foo';
Some time later, I decide to add another column:
alter table t1 add column c2 varchar(20);
I want to look up the comment contents of the first column, and associate with the new column:
select comment_text from (what?) where table_name = 't1' and column_name = 'c1'
The (what?) is going to be a system table, but after having looked around in pgAdmin and searching on the web I haven't learnt its name.
Ideally I'd like to be able to:
comment on column t1.c1 is (select ...);
but I have a feeling that's stretching things a bit far. Thanks for any ideas.
Update: based on the suggestions I received here, I wound up writing a program to automate the task of transferring comments, as part of a larger process of changing the datatype of a Postgresql column. You can read about that on my blog.
The next thing to know is how to obtain the table oid. I think that using this as part of comment on will not work, as you suspect.
postgres=# create table comtest1 (id int, val varchar);
CREATE TABLE
postgres=# insert into comtest1 values (1,'a');
INSERT 0 1
postgres=# select distinct tableoid from comtest1;
tableoid
----------
32792
(1 row)
postgres=# comment on column comtest1.id is 'Identifier Number One';
COMMENT
postgres=# select col_description(32792,1);
col_description
-----------------------
Identifier Number One
(1 row)
Anyhow, I whipped up a quick plpgsql function to copy comments from one table/column pair to another. You have to createlang plpgsql on the database and use it like this:
Copy the comment on the first column of table comtest1 to the id
column of the table comtest2. Yes, it should be improved but
that's left as work for the reader.
postgres=# select copy_comment('comtest1',1,'comtest2','id');
copy_comment
--------------
1
(1 row)
CREATE OR REPLACE FUNCTION copy_comment(varchar,int,varchar,varchar) RETURNS int AS $PROC$
DECLARE
src_tbl ALIAS FOR $1;
src_col ALIAS FOR $2;
dst_tbl ALIAS FOR $3;
dst_col ALIAS FOR $4;
row RECORD;
oid INT;
comment VARCHAR;
BEGIN
FOR row IN EXECUTE 'SELECT DISTINCT tableoid FROM ' || quote_ident(src_tbl) LOOP
oid := row.tableoid;
END LOOP;
FOR row IN EXECUTE 'SELECT col_description(' || quote_literal(oid) || ',' || quote_literal(src_col) || ')' LOOP
comment := row.col_description;
END LOOP;
EXECUTE 'COMMENT ON COLUMN ' || quote_ident(dst_tbl) || '.' || quote_ident(dst_col) || ' IS ' || quote_literal(comment);
RETURN 1;
END;
$PROC$ LANGUAGE plpgsql;
You can retrieve comments on columns using the system function col_description(table_oid, column_number). See this page for further details.