I would like to do a two phase commit transaction with prepare transaction for PostgreSQL.
Could you help with the error?
I can not understand how to connect to the remote database via dblick with prepare transaction?
create or replace function insert_into_table_a() returns void as $$
declare
trnxprepare text;
trnxcommit text;
trnxrollback text;
trnxid varchar;
begin
select uuid_generate_v4() into trnxid;
select 'prepare transaction ' || ' ''' || trnxid || ' ''' into trnxprepare;
select 'commit prepared ' || ' ''' || trnxid || ' ''' into trnxcommit;
select 'rollback prepared ' || ' ''' || trnxid || ' ''' into trnxrollback;
insert into table_a values ('test');
perform dblink_connect('cn','dbname=test2 user=test2user password=123456');
perform dblink_exec('cn','insert into table_b values (''test 2'');');
perform dblink_disconnect('cn');
execute trnxprepare;
execute trnxcommit;
exception
when others then
execute trnxrollback;
perform dblink_disconnect('cn');
raise notice '% %', sqlerrm, sqlstate;
end;
$$ language plpgsql;
select insert_into_table_a();
ERROR: ERROR: transactions can not be started in PL / pgSQL
HINT: Use the BEGIN block with the EXCEPTION clause instead.
CONTEXT: insert_into_table_a () PL / pgSQL function, line 24, in EXECUTE
SQL state: 0A000
So, in Postgres, you can't control transactions from inside functions for the most part. You can raise errors to abort them indirectly, if they aren't caught, but you can't begin or end them directly like this.
To manage transactions, you'd either need a worker process as a loadable module, or to control the transaction from a client through a connection.
Related
I am a newbie wrt functions and I am struggling with using the name of a table in the function body. I get an error "SQL Error [42703]: ERROR: column "tname" does not exist" when I call the function using
select "JsonToView"('data_import.import_360xero_report');
My code is below
create or replace
function data_import."JsonToView"(tname text) returns numeric
language plpgsql
as $function$
begin
do
$$
declare
l_keys text;
begin
drop view if exists v_json_view cascade;
select
string_agg(distinct format('import_data ->> %L as %I', jkey, jkey), ', ')
into
l_keys
from
import_360xero_report,
json_object_keys(import_data) as t(jkey);
execute 'create view v_json_view as select ' || l_keys || ' from ' || tname;
end;
$$;
return 0;
end $function$ ;
I have modified the code and the second create view query works with the table name but the first one does not.
Below if my modified code
create or replace
function data_import."JsonToView"(tname text) returns numeric
language plpgsql
as $function$
declare
l_keys text;
begin
drop view if exists v_json_view cascade;
execute $a$select
string_agg(distinct format('import_data ->> %L as %I', jkey, jkey), ', ')
into
l_keys
from $a$ ||
tname || $b$,
json_object_keys(import_data) as t(jkey)$b$;
execute 'create view v_json_view as select ' || l_keys || ' from ' || tname;
return 0;
end $function$ ;
The error I am getting is
SQL Error [0A000]: ERROR: EXECUTE of SELECT ... INTO is not implemented
Hint: You might want to use EXECUTE ... INTO or EXECUTE CREATE TABLE ... AS instead.
Where: PL/pgSQL function "JsonToView"(text) line 10 at EXECUTE
The problem is the superfluous nested DO statement.
The variable tname exists only in the scope of the function, not in the nested DO statement. DO is an SQL statement, not a PL/pgSQL statement, and there are no variables in SQL. Also, DO does not allow parameters.
Get rid of the DO and you will be fine.
When initializing my process, it runs the PL/pgSQL statement below creating two functions. However, every time I create multiple processes simultaneously as part of an end-to-end test, parallel execution of this statement leads to a tuple concurrently updated error that I can't seem to get around. Any help would be much appreciated.
CREATE OR REPLACE FUNCTION
count_rows(schema text, tablename text) returns integer
AS
$body$
DECLARE
result integer;
query varchar;
BEGIN
query := 'SELECT count(1) FROM "' || schema || '"."' || tablename || '"';
execute query into result;
return result;
END;
$body$
LANGUAGE plpgsql;
CREATE OR REPLACE FUNCTION
delete_if_empty(schema text, tablename text) RETURNS INTEGER
AS
$$
DECLARE
result integer;
query varchar;
BEGIN
query := 'SELECT COUNT(*) FROM "' || schema || '"."' || tablename || '"';
execute query into result;
IF result = 0 THEN
EXECUTE 'DROP TABLE "' || schema || '"."' || tablename || '" CASCADE;';
EXECUTE 'NOTIFY "' || schema || '", ''DESTROY_TABLE:' || tablename || ''';';
RETURN 1;
END IF;
RETURN 0;
END;
$$
LANGUAGE plpgsql;
SELECT version()
As described here, postgres doesn't currently allow you to use CREATE FUNCTION concurrently:
It'd be necessary to add
some kind of locking scheme if you want to avoid "tuple concurrently
updated" errors. This is not really any different from the situation
where two transactions both want to update the same row in a user table:
unless the application takes extra steps to serialize the updates, you're
going to get "tuple concurrently updated" errors.
We do have such locking for DDL on tables/indexes, but the theory in the
past has been that it's not worth the trouble for objects represented by
single catalog rows, such as functions or roles.
A solution to this is to ensure that no two transaction try to do the CREATE FUNCTION at the same time.
You can use posgres advisory locks for that.
A good introduction to advisory locks can be found here: https://vladmihalcea.com/how-do-postgresql-advisory-locks-work/
For example, you can use:
BEGIN; -- start of transaction
SELECT pg_advisory_xact_lock(2142616474639426746); -- random 64-bit signed ('bigint') lock number
CREATE OR REPLACE FUNCTION myfunction ...
COMMIT;
This takes a transaction-level exclusive advisory lock, so that no two concurrent transaction can run create the function at the same time. At the end of the transaction, the lock is automatically released.
I am writing a stored procedure to update multiple table's data in Postgres 9.4. I will execute this procedure from python.
CREATE OR REPLACE FUNCTION update_data(
list_table_name TEXT[],
list_parameters TEXT[]
)
RETURNS BOOLEAN LANGUAGE plpgsql SECURITY DEFINER AS $$
BEGIN
FOR i IN list_table_name LOOP
FOR j IN dict_parameters LOOP
EXECUTE '
UPDATE ' || i || '
SET default_value = ' || j.param_val || '
WHERE ' || i.id = j.id ||''
END LOOP;
END LOOP;
END;
$$
LANGUAGE 'plpgsql';
-- dict_parameters would be python list of dictionary [{"id":val,"param_val":[val1,val2,val3]},{"id":val,"param_val":[val1,val2,val3]}]
-- list_table_name would be python list containing table names.
First of all this gives me error as -
ERROR: syntax error at or near "BEGIN"
LINE 8: BEGIN
^
optimation=# END LOOP;
ERROR: syntax error at or near "LOOP"
LINE 1: END LOOP;
^
And even if it runs I am not sure if I am extracting the values the right way. If someone could guide me.
For some fancy database maintenance for my developer database I'd like to be able to use queries to generate commands to alter the database. The thing is: I'm a complete greenhorn to PostgreSQL. I've made my attempt but have failed colorfully.
So in the end, I would like to have a table with a single column and each row would be a command (or group of commands, depending on the case) that I would think would look something like this...
DO $$
DECLARE
command_entry RECORD;
BEGIN
FOR command_entry IN SELECT * FROM list_of_commands
LOOP
EXECUTE command_entry;
END LOOP;
END;
$$;
Where the table list_of_commands could be populated with something like the following (which in this example would remove all tables from the public schema)...
CREATE TEMP TABLE list_of_commands AS
SELECT 'drop table if exists "' || tablename || '" cascade;'
FROM pg_tables
WHERE schemaname = 'public';
However, with this I get the following error...
ERROR: syntax error at or near ""drop table if exists ""dummy_table"" cascade;""
LINE 1: ("drop table if exists ""dummy_table"" cascade;")
I assume this is a matter of escaping characters, but I'm not entirely sure how to fit that into either A) the population of the table or B) the execution of each row. Does anyone know what I could do to achieve the desired result?
The command_entry variable is of type record while the EXECUTE command expects a string. What is apparently happening is that PostgreSQL turns the record into a double-quoted string, but that messes up your command. Also, your temp table does not use a column name, making things a bit awkward to work with (the column name becomes ?column?), so change both as follows:
CREATE TEMP TABLE list_of_commands AS
SELECT 'drop table if exists public.' || quote_ident(tablename) || ' cascade' AS cmd
FROM pg_tables
WHERE schemaname = 'public';
DO $$
DECLARE
command_entry varchar;
BEGIN
FOR command_entry IN SELECT cmd FROM list_of_commands
LOOP
EXECUTE command_entry;
END LOOP;
END;
$$;
But seeing that you do all of this at session level (temp table, anonymous code block), why not write a stored procedure that performs all of this housekeeping when you are ready to do spring cleaning?
CREATE FUNCTION cleanup() RETURNS void AS $$
BEGIN
FOR tbl IN SELECT tablename FROM pg_tables WHERE schemaname = 'public'
LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(tbl) || ' CASCADE';
END LOOP;
-- More housekeeping jobs
END;
$$ LANGUAGE plpgsql;
This saves a lot of typing: SELECT cleanup();. Any other housekeeping jobs you have you simply add to the stored procedure.
I had trouble with Patrick's answers, so here is an updated version for postgreSQL 10.
CREATE FUNCTION droptables(sn varchar) RETURNS void AS $$
DECLARE
tbl varchar;
BEGIN
FOR tbl IN SELECT tablename FROM pg_tables WHERE schemaname = sn
LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(tbl) || ' CASCADE';
END LOOP;
END;
$$ LANGUAGE plpgsql;
And then "SELECT droptables('public');".
CREATE OR REPLACE FUNCTION mover(src text, dst text, cpquery text, conname text, ifbin boolean) returns void as
$$
DECLARE
cnt integer;
dlcnt integer;
del_count integer;
ret text;
BEGIN
SELECT pg_catalog.dblink_copy_open(conname, dst, ifbin) INTO ret ;
RAISE LOG 'dblink_open %',ret;
execute 'SELECT 1 as check FROM ' || src ||' limit 1' into cnt;
IF cnt=0 THEN
PERFORM pg_sleep(2);
END IF;
IF ifbin=true THEN
RAISE DEBUG 'Start to Copy data with binary';
execute 'COPY (' || cpquery || ' ) to function pg_catalog.dblink_copy_write with binary';
RAISE DEBUG 'Finish Copy data';
ELSE
RAISE DEBUG 'Start to Copy data without binary';
execute 'COPY (' || cpquery || ' ) to function pg_catalog.dblink_copy_write';
RAISE DEBUG 'Finish Copy data';
END IF;
execute 'DELETE FROM ' || src;
GET DIAGNOSTICS del_count=ROW_COUNT;
RAISE INFO 'DELETE % rows',del_count;
SELECT pg_catalog.dblink_copy_end() INTO ret;
RAISE LOG 'dblink_end %',ret;
END;
$$
language plpgsql;
As code, I want to put some message into log by using RAISE, but where is the location
of my log file ? and where RAISE DEBUG output?
They can either be output to the Postgres log, reported back to the client, or both. These are controlled by server-side settings, log_min_messages and client_min_messages.
See the following doc for more details:
http://www.postgresql.org/docs/current/static/plpgsql-errors-and-messages.html
http://www.postgresql.org/docs/current/static/runtime-config-logging.html
As #a_horse_with_no_name suggested: These parameters can also be set via the SET command from the client.
It can be set via the SQL: set client_min_messages to 'debug';