Why cannot create partitioning table - postgresql

I'm trying to create simple table with partitions.
this is my command:
CREATE TABLE measurement (
city_id int not null,
logdate date not null,
peaktemp int,
unitsales int
) PARTITION BY RANGE (logdate);
this is the error I got:
SQL Error [42601]: ERROR: syntax error at or near "PARTITION"
Unable to understand with is the problem..
I am using PostgreSQL 9.6.3

"Declarative table partitioning", that is partitioning as a first-class feature of the DBMS with its own syntax, was added in PostgreSQL 10.
In earlier versions, you can achieve the same effect with a bit more effort using "table inheritance". There is a page in the manual describing how to do this manually, summarised as:
Create the "master" table, from which all of the partitions will inherit.
Create several "child" tables that each inherit from the master table.
Add table constraints to the partition tables to define the allowed key values in each partition.
For each partition, create an index on the key column(s), as well as any other indexes you might want.
Optionally, define a trigger or rule to redirect data inserted into the master table to the appropriate partition.
Ensure that the constraint_exclusion configuration parameter is not disabled in postgresql.conf. If it is, queries will not be optimized as desired.
To make this easier, if you can't upgrade to version 10, you can use an extension such as pg_partman which gives you additional functions for setting up and managing partition sets.

Here is the example of automatically creating monthly partition for 9.6 version, may be compatible with other versions:
`----------Function to create partitions by system------------------
CREATE OR REPLACE FUNCTION schema.insert_function()
RETURNS TRIGGER
AS $$
DECLARE
partition_date TEXT;
partition_schema TEXT;
partition_name TEXT;
start_of_month TEXT;
end_of_month TEXT;
BEGIN
partition_date := to_char(NEW.created_dtm,'YYYY_MM');
partition_schema := 'temp';
partition_name := 'patition_name_' || partition_date;
start_of_month := to_char((NEW.created_dtm),'YYYY-MM'|| '-01');
end_of_month := to_char((NEW.created_dtm + interval '1 month'),'YYYY-MM'|| '-01');
IF NOT EXISTS
(SELECT 1
FROM information_schema.tables
WHERE table_name = partition_name
AND table_schema = partition_schema)
THEN
EXECUTE 'CREATE TABLE '|| partition_schema ||' . '|| partition_name ||' (check (created_dtm >= ''' || start_of_month || ''' and created_dtm < ''' || end_of_month || ''' ), ' || 'LIKE master_schema.master_table INCLUDING ALL) INHERITS (master_schema.master_table)';
EXECUTE format('ALTER TABLE %I.%I OWNER TO role1', partition_schema, partition_name);
EXECUTE format('GRANT SELECT ON TABLE %I.%I TO read_only_role', partition_schema, partition_name);
EXECUTE format('GRANT INSERT, SELECT, UPDATE, DELETE ON TABLE %I.%I TO read_write_role', partition_schema, partition_name);
EXECUTE format('CREATE INDEX ON %I.%I(column1, column2, column3)', partition_schema, partition_name);
EXECUTE format('CREATE UNIQUE INDEX ON %I.%I(column4)', partition_schema, partition_name);
........
RAISE NOTICE 'A partition has been created %.%', partition_schema, partition_name;
RAISE NOTICE 'All necessary indices are created on %.%', partition_schema, partition_name;
END IF;
EXECUTE format('INSERT INTO %I.%I VALUES($1.*)', partition_schema, partition_name) using NEW;
RETURN NULL;
END
$$
LANGUAGE plpgsql;
ALTER FUNCTION schema.insert_function()
OWNER TO super_user;
-----------------------------------------Trigger on master table--------------
CREATE TRIGGER insert_trigger
BEFORE INSERT
ON master_schema.master_table
FOR EACH ROW
EXECUTE PROCEDURE schema.insert_function();`

Related

Create partition table using execute

I would like to create N partition tables for the last N days. I have created a table like the following
create table metrics.my_table (
id bigserial NOT NULL primary key,
...
logdate date NOT NULL
) PARTITION BY LIST (logdate);
Then I have the following function to create those tables:
CREATE OR REPLACE function metrics.create_my_partitions(init_date numeric default 30, current_date_parameter timestamp default current_date)
returns void as $$
DECLARE
partition_date TEXT;
partition_name TEXT;
begin
for cnt in 0..init_date loop
partition_date := to_char((current_date_parameter - (cnt * interval '1 day')),'YYYY-MM-DD');
raise notice 'cnt: %', cnt;
raise notice 'partition_date: %', partition_date;
partition_name := 'my_table_' || partition_date;
raise notice 'partition_name: %', partition_name;
EXECUTE format('CREATE table if not exists metrics.%I PARTITION OF metrics.my_table for VALUES IN ($1)', partition_name) using partition_date;
end loop;
END
$$
LANGUAGE plpgsql;
select metrics.create_my_partitions(30, current_date);
But it throws the following error in the EXECUTE format line:
SQL Error [42P02]: ERROR: there is no parameter $1
Any idea on how to create those tables?
The EXECUTE ... USING ... option only works for data values in DML commands (SELECT,INSERT, etc.). Since CREATE TABLE is a DDL command, use a parameter in format():
execute format(
'create table if not exists metrics.%I partition of metrics.my_table for values in (%L)',
partition_name, partition_date);

Delete selected tables from Postgres sql

I need to remove selected tables from the Postgres SQL. It would better to use like or where clause.
Like I have
TABLE_A
TABLE_B
TABLE_C
-
-
-
TABLE_N
I need to delete
TABLE_A to TABLE_X
Can be done with a single command, which is faster - in case this is a recurring task.
Add IF EXISTS if the existence of any tables is uncertain. This way we save an extra trip to the system catalogs (information_schema.tables or pg_catalog.pg_tables).
And you may want to add CASCADE:
DO
$do$
BEGIN
-- child safety device: quote RAISE instead of EXECUTE to prime the bomb
-- EXECUTE (
RAISE NOTICE '%', (
SELECT 'DROP TABLE IF EXISTS'
|| string_agg('table_' || chr(ascii('a') + g) , ', ')
|| ' CASCADE;'
FROM generate_series(0,13) g
);
END
$do$;
Generates a command of the form:
DROP TABLE IF EXISTS table_a, table_b, ... , table_n CASCADE;
Using generate_series() to generate requested table names. More here:
How to drop many (but not all) tables in one fell swoop?
How to check if a table exists in a given schema
DO
$$
DECLARE
r RECORD;
BEGIN
FOR r IN SELECT oid::REGCLASS table_name
FROM pg_class
WHERE relname <= 'table_x'
AND relkind = 'r'
LOOP
EXECUTE 'DROP TABLE' || r.table_name;
END LOOP;
END;
$$ LANGUAGE plpgsql;

Postgres create universal function on all table

I have a few db tables.
I want write universtal postgres function on copy rows to history tables
I have tables:
table1
table1_h
table2
table2_h
I wrote function (with help stackoverflow)
CREATE OR REPLACE FUNCTION copy_history_f() RETURNS TRIGGER AS
$BODY$
DECLARE
tablename_h text:= TG_TABLE_NAME || '_h';
BEGIN
EXECUTE 'INSERT INTO ' || quote_ident(TG_TABLE_SCHEMA) || '.' || quote_ident(tablename_h) || ' VALUES (' || OLD.* ||')';
RETURN NULL;
END;
$BODY$
LANGUAGE plpgsql VOLATILE;
And functions was create, but after update is error.
ERROR: syntax error at or near ","
ROW 1: ...RT INTO table1_h VALUES ((12,,,0,,"Anto...
I know where is error in this insert but I don't know how I repair that.
Structure tables table1 and table1_h are identical but table1_h has one more column (id_h)
Can you help me, how I have create psql function?
Thnak you.
drop table if exists t;
drop table if exists t_h;
drop function if exists ftg();
create table t(i serial, x numeric);
insert into t(x) values(1.1),(2.2);
create table t_h(i int, x numeric);
create function ftg() returns trigger language plpgsql as $ftg$
declare
tablename_h text:= TG_TABLE_NAME || '_h';
begin
execute format($q$ insert into %I.%I select $1.*; $q$, TG_TABLE_SCHEMA, tablename_h) using old;
return null;
end $ftg$;
create trigger tg_t after delete on t for each row execute procedure ftg();
delete from t where i = 1;
select * from t_h;
dbfiddle
Update It solves your problem, but I think that you want to have a bit more info in your history tables. It will be more complex a bit:
drop table if exists t;
drop table if exists t_h;
drop function if exists ftg();
create table t(i serial, x numeric);
insert into t(x) values(1.1),(2.2);
create table t_h(
hi serial, -- just ID
hd timestamp, -- timestamp
hu text, -- user who made changes
ha text, -- action
i int, x numeric
);
create function ftg() returns trigger language plpgsql as $ftg$
declare
tablename_h text:= TG_TABLE_NAME || '_h';
begin
execute format(
$q$
insert into %I.%I
select
nextval(%L || '_hi_seq'),
clock_timestamp(),
current_user,
%L,
$1.*
$q$, TG_TABLE_SCHEMA, tablename_h, tablename_h, TG_OP) using old;
return null;
end $ftg$;
create trigger tg_t after delete or update on t for each row execute procedure ftg();
update t set x = x * 2;
update t set x = x * 2 where i = 2;
delete from t where i = 1;
select * from t_h;
dbfiddle
I assume you are inserting the 'old' values from table1 into table1_h.
The additional column is your problem. When you using an insert without naming columns you must use a matching number and type for the insert.
You must use column referencing.
eg.
Insert into table1_h(column1, column2, column3)
values (a,b,c)
Consider a default value for the additional column in table table1_h.

SELECTing commands into a temp table to EXECUTE later in PostgreSQL

For some fancy database maintenance for my developer database I'd like to be able to use queries to generate commands to alter the database. The thing is: I'm a complete greenhorn to PostgreSQL. I've made my attempt but have failed colorfully.
So in the end, I would like to have a table with a single column and each row would be a command (or group of commands, depending on the case) that I would think would look something like this...
DO $$
DECLARE
command_entry RECORD;
BEGIN
FOR command_entry IN SELECT * FROM list_of_commands
LOOP
EXECUTE command_entry;
END LOOP;
END;
$$;
Where the table list_of_commands could be populated with something like the following (which in this example would remove all tables from the public schema)...
CREATE TEMP TABLE list_of_commands AS
SELECT 'drop table if exists "' || tablename || '" cascade;'
FROM pg_tables
WHERE schemaname = 'public';
However, with this I get the following error...
ERROR: syntax error at or near ""drop table if exists ""dummy_table"" cascade;""
LINE 1: ("drop table if exists ""dummy_table"" cascade;")
I assume this is a matter of escaping characters, but I'm not entirely sure how to fit that into either A) the population of the table or B) the execution of each row. Does anyone know what I could do to achieve the desired result?
The command_entry variable is of type record while the EXECUTE command expects a string. What is apparently happening is that PostgreSQL turns the record into a double-quoted string, but that messes up your command. Also, your temp table does not use a column name, making things a bit awkward to work with (the column name becomes ?column?), so change both as follows:
CREATE TEMP TABLE list_of_commands AS
SELECT 'drop table if exists public.' || quote_ident(tablename) || ' cascade' AS cmd
FROM pg_tables
WHERE schemaname = 'public';
DO $$
DECLARE
command_entry varchar;
BEGIN
FOR command_entry IN SELECT cmd FROM list_of_commands
LOOP
EXECUTE command_entry;
END LOOP;
END;
$$;
But seeing that you do all of this at session level (temp table, anonymous code block), why not write a stored procedure that performs all of this housekeeping when you are ready to do spring cleaning?
CREATE FUNCTION cleanup() RETURNS void AS $$
BEGIN
FOR tbl IN SELECT tablename FROM pg_tables WHERE schemaname = 'public'
LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(tbl) || ' CASCADE';
END LOOP;
-- More housekeeping jobs
END;
$$ LANGUAGE plpgsql;
This saves a lot of typing: SELECT cleanup();. Any other housekeeping jobs you have you simply add to the stored procedure.
I had trouble with Patrick's answers, so here is an updated version for postgreSQL 10.
CREATE FUNCTION droptables(sn varchar) RETURNS void AS $$
DECLARE
tbl varchar;
BEGIN
FOR tbl IN SELECT tablename FROM pg_tables WHERE schemaname = sn
LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(tbl) || ' CASCADE';
END LOOP;
END;
$$ LANGUAGE plpgsql;
And then "SELECT droptables('public');".

How to add column if not exists on PostgreSQL?

Question is simple. How to add column x to table y, but only when x column doesn't exist ? I found only solution here how to check if column exists.
SELECT column_name
FROM information_schema.columns
WHERE table_name='x' and column_name='y';
With Postgres 9.6 this can be done using the option if not exists
ALTER TABLE table_name ADD COLUMN IF NOT EXISTS column_name INTEGER;
Here's a short-and-sweet version using the "DO" statement:
DO $$
BEGIN
BEGIN
ALTER TABLE <table_name> ADD COLUMN <column_name> <column_type>;
EXCEPTION
WHEN duplicate_column THEN RAISE NOTICE 'column <column_name> already exists in <table_name>.';
END;
END;
$$
You can't pass these as parameters, you'll need to do variable substitution in the string on the client side, but this is a self contained query that only emits a message if the column already exists, adds if it doesn't and will continue to fail on other errors (like an invalid data type).
I don't recommend doing ANY of these methods if these are random strings coming from external sources. No matter what method you use (client-side or server-side dynamic strings executed as queries), it would be a recipe for disaster as it opens you to SQL injection attacks.
Postgres 9.6 added ALTER TABLE tbl ADD COLUMN IF NOT EXISTS column_name.
So this is mostly outdated now. You might use it in older versions, or a variation to check for more than just the column name.
CREATE OR REPLACE function f_add_col(_tbl regclass, _col text, _type regtype)
RETURNS bool
LANGUAGE plpgsql AS
$func$
BEGIN
IF EXISTS (SELECT FROM pg_attribute
WHERE attrelid = _tbl
AND attname = _col
AND NOT attisdropped) THEN
RETURN false;
ELSE
EXECUTE format('ALTER TABLE %s ADD COLUMN %I %s', _tbl, _col, _type);
RETURN true;
END IF;
END
$func$;
Call:
SELECT f_add_col('public.kat', 'pfad1', 'int');
Returns true on success, else false (column already exists).
Raises an exception for invalid table or type name.
Why another version?
This could be done with a DO statement, but DO statements cannot return anything. And if it's for repeated use, I would create a function.
I use the object identifier types regclass and regtype for _tbl and _type which a) prevents SQL injection and b) checks validity of both immediately (cheapest possible way). The column name _col has still to be sanitized for EXECUTE with quote_ident(). See:
Table name as a PostgreSQL function parameter
format() requires Postgres 9.1+. For older versions concatenate manually:
EXECUTE 'ALTER TABLE ' || _tbl || ' ADD COLUMN ' || quote_ident(_col) || ' ' || _type;
You can schema-qualify your table name, but you don't have to.
You can double-quote the identifiers in the function call to preserve camel-case and reserved words (but you shouldn't use any of this anyway).
I query pg_catalog instead of the information_schema. Detailed explanation:
How to check if a table exists in a given schema
Blocks containing an EXCEPTION clause are substantially slower.
This is simpler and faster. The manual:
Tip
A block containing an EXCEPTION clause is significantly more
expensive to enter and exit than a block without one.
Therefore, don't use EXCEPTION without need.
Following select query will return true/false, using EXISTS() function.
EXISTS(): The argument of EXISTS is an arbitrary SELECT statement, or
subquery. The subquery is evaluated to determine whether it returns
any rows. If it returns at least one row, the result of EXISTS is
"true"; if the subquery returns no rows, the result of EXISTS is
"false"
SELECT EXISTS(SELECT column_name
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'x'
AND column_name = 'y');
and use the following dynamic SQL statement to alter your table
DO
$$
BEGIN
IF NOT EXISTS (SELECT column_name
FROM information_schema.columns
WHERE table_schema = 'public'
AND table_name = 'x'
AND column_name = 'y') THEN
ALTER TABLE x ADD COLUMN y int DEFAULT NULL;
ELSE
RAISE NOTICE 'Already exists';
END IF;
END
$$
For those who use Postgre 9.5+(I believe most of you do), there is a quite simple and clean solution
ALTER TABLE if exists <tablename> add if not exists <columnname> <columntype>
the below function will check the column if exist return appropriate message else it will add the column to the table.
create or replace function addcol(schemaname varchar, tablename varchar, colname varchar, coltype varchar)
returns varchar
language 'plpgsql'
as
$$
declare
col_name varchar ;
begin
execute 'select column_name from information_schema.columns where table_schema = ' ||
quote_literal(schemaname)||' and table_name='|| quote_literal(tablename) || ' and column_name= '|| quote_literal(colname)
into col_name ;
raise info ' the val : % ', col_name;
if(col_name is null ) then
col_name := colname;
execute 'alter table ' ||schemaname|| '.'|| tablename || ' add column '|| colname || ' ' || coltype;
else
col_name := colname ||' Already exist';
end if;
return col_name;
end;
$$
This is basically the solution from sola, but just cleaned up a bit. It's different enough that I didn't just want to "improve" his solution (plus, I sort of think that's rude).
Main difference is that it uses the EXECUTE format. Which I think is a bit cleaner, but I believe means that you must be on PostgresSQL 9.1 or newer.
This has been tested on 9.1 and works. Note: It will raise an error if the schema/table_name/or data_type are invalid. That could "fixed", but might be the correct behavior in many cases.
CREATE OR REPLACE FUNCTION add_column(schema_name TEXT, table_name TEXT,
column_name TEXT, data_type TEXT)
RETURNS BOOLEAN
AS
$BODY$
DECLARE
_tmp text;
BEGIN
EXECUTE format('SELECT COLUMN_NAME FROM information_schema.columns WHERE
table_schema=%L
AND table_name=%L
AND column_name=%L', schema_name, table_name, column_name)
INTO _tmp;
IF _tmp IS NOT NULL THEN
RAISE NOTICE 'Column % already exists in %.%', column_name, schema_name, table_name;
RETURN FALSE;
END IF;
EXECUTE format('ALTER TABLE %I.%I ADD COLUMN %I %s;', schema_name, table_name, column_name, data_type);
RAISE NOTICE 'Column % added to %.%', column_name, schema_name, table_name;
RETURN TRUE;
END;
$BODY$
LANGUAGE 'plpgsql';
usage:
select add_column('public', 'foo', 'bar', 'varchar(30)');
Can be added to migration scripts invoke function and drop when done.
create or replace function patch_column() returns void as
$$
begin
if exists (
select * from information_schema.columns
where table_name='my_table'
and column_name='missing_col'
)
then
raise notice 'missing_col already exists';
else
alter table my_table
add column missing_col varchar;
end if;
end;
$$ language plpgsql;
select patch_column();
drop function if exists patch_column();
In my case, for how it was created reason it is a bit difficult for our migration scripts to cut across different schemas.
To work around this we used an exception that just caught and ignored the error. This also had the nice side effect of being a lot easier to look at.
However, be wary that the other solutions have their own advantages that probably outweigh this solution:
DO $$
BEGIN
BEGIN
ALTER TABLE IF EXISTS bobby_tables RENAME COLUMN "dckx" TO "xkcd";
EXCEPTION
WHEN undefined_column THEN RAISE NOTICE 'Column was already renamed';
END;
END $$;
You can do it by following way.
ALTER TABLE tableName drop column if exists columnName;
ALTER TABLE tableName ADD COLUMN columnName character varying(8);
So it will drop the column if it is already exists. And then add the column to particular table.
Simply check if the query returned a column_name.
If not, execute something like this:
ALTER TABLE x ADD COLUMN y int;
Where you put something useful for 'x' and 'y' and of course a suitable datatype where I used int.