Create non conflicting temporary tables in a Pl/pgSQL function - postgresql

I want to create a TEMPORARY TABLE in a Pl/pgSQL function because I want to index it before doing some process. The fact that any concurrent call to the function will try to reuse the same table seems to be a problem.
e.g. A first call to the function creates and uses a temporary table named "test" with data depending on the function parameters. A second concurrent call tries also to create and use the temporary table with the same name but with different data...
The doc says
"Temporary tables are automatically dropped at the end of a session,
or optionally at the end of the current transaction"
I guess the problem would not exist if temporary tables created with the "ON COMMIT DROP" option would only be visible to the current transaction. Is this the case?
If not, how to automatically create independent tables from two different function calls?
I could probably try to create a temporary name and check if a table with this name already exists but that seems like a lot of management to me...

Temporary tables of distinct sessions cannot conflict because each session has a dedicated temporary schema, only visible to the current session.
In current Postgres only one transaction runs inside the same session at a time. So only two successive calls in the same session can see the same temporary objects. ON COMMIT DROP, like you found, limits the lifespan of temp tables to the current transaction, avoiding conflicts with other transactions.
If you (can) have temp tables that don't die with the transaction (like if you want to keep using some of those tables after the end of the current transaction), then an alternative approach would be to truncate instead of create if the temp table already exists - which is a bit cheaper, too.
Wrapped into a function:
CREATE OR REPLACE FUNCTION f_create_or_trunc_temp_table(_tbl text, OUT _result "char") AS
$func$
BEGIN
SELECT INTO _result relkind
FROM pg_catalog.pg_class
WHERE relnamespace = pg_my_temp_schema() -- only temp objects!
AND relname = _tbl;
IF NOT FOUND THEN -- not found
EXECUTE format('CREATE TEMP TABLE %I(id int)', _tbl);
ELSIF _result = 'r' THEN -- table exists
EXECUTE format('TRUNCATE TABLE %I', _tbl); -- assuming identical table definition
ELSE -- other temp object occupies name
RAISE EXCEPTION 'Other temp object of type >>%<< occupies name >>%<<', _result, _tbl;
-- or do nothing, return more info or raise a warning / notice instead of an exception
END IF;
END
$func$ LANGUAGE plpgsql;
Call:
SELECT f_create_or_trunc_temp_table('my_tbl');
This assumes identical table definition if the table exists. You might do more and also return more informative messages, etc. This is just the basic concept.
Related:
How can I determine if a table exists in the current search_path with PLPGSQL?
How to check if a table exists in a given schema

Temporary tables are visible only in the current session. Concurrent processes do not see each other's temporary tables even when they share the same names. Per the documentation:
PostgreSQL requires each session to issue its own CREATE TEMPORARY TABLE command for each temporary table to be used. This allows different sessions to use the same temporary table name for different purposes (...)

Related

How do I create a temporary trigger in Postgres? [duplicate]

This question already has an answer here:
Drop trigger/function at end of session in PostgreSQL?
(1 answer)
Closed last month.
I'm trying to create a system in Postgres where each client can create its own subscriptions via listen + notify + triggers. I.e. the client will listen to a channel, and then create a trigger which runs notify on that channel when its conditions have been met. The issue is that I want Postgres to clean up properly in case of improper client termination (e.g. the remote client process dies). To be more specific, I want that trigger which is calling the notify to be removed as there is no longer a listener anyways. How can I accomplish this?
I've thought about having a table to map triggers to client ids and then using that to remove triggers where the client is gone, but it seems like a not so great solution.
I found an answer to this in another question: Drop trigger/function at end of session in PostgreSQL?
In reasonably recent Postgres versions you can create a function in
pg_temp schema:
create function pg_temp.get_true() returns boolean language sql as $$ select true; $$;
select pg_temp.get_true();
This is the schema in which temporary tables are created. All its
contents, including your function, will be deleted on end of session.
You can also create triggers using temporary functions on tables. I've
just tested this and it works as expected:
create function pg_temp.ignore_writes() returns trigger language plpgsql as $$
begin
return NULL;
end;
$$;
create table test (id int);
create trigger test_ignore_writes
before insert, update, delete on test
for each row execute procedure pg_temp.ignore_writes();
Because this trigger function always returns NULL and is before [event] it should make any writes to this table to be ignored. And
indeed:
insert into test values(1);
select count(*) from test;
count
-------
0
But after logout and login this function and the trigger would not be
present anymore, so writes would work:
insert into test values(1);
select count(*) from test;
count
-------
1
But you should be aware that this is somewhat hackish — not often used
and might not be very thoroughly tested.
That's not how it works. CREATE TRIGGER requires that you either own the table or have the TRIGGER privilege on it (which nobody in their right mind will give you, because it enables you to run arbitrary code in their name). Moreover, CREATE TRIGGER requires an SHARE ROW EXCLUSIVE lock on the table and DROP TRIGGER requires an ACCESS EXCLUSIVE lock, which can be disruptive.
Create a single trigger and keep that around.

Cannot drop priorly modified new table in execute block

I'm not well acquainted with FB database and its subtleties.
On script executing, the problem occurres:
EXECUTE ibeblock
AS
BEGIN
-- 1. Create temporary table
execute statement 'recreate GLOBAL TEMPORARY table TMPTBL (ID bigint) /*on commit delete rows*/;';
commit;
-- 2. dummy fill of temporary table
insert into tmptbl (ID)
values (0xFE);
commit; -- not necessary
-- 3. perform some actions...
-- 4. Delete temporary table
execute statement 'drop table TMPTBL;';
commit; -- FAILURE!
END
The idea of script is primitive: 1) create temporary table; 2) fill it with records; 3) perform actions on other DB objects using populated records; 4) drop temp table.
For simulation, step-3 is useless (skipped). Step-4 leads to an error on commit: "This operation is not defined for system tables. unsuccessful metadata update. object TABLE "TMPTBL" is in use.".
Neither triggers nor constraints are applied for the table. Obviously, there should be nothing locking temp table.
Help, please, with resolution. Hopefully I missed something.
P.S.: FB 2.5, IBExpert 2017.12.13.1 used as DB managing tool
There are a number of problems with your code:
A global temporary table is intended as a permanent object, it is just the content that is temporary (either for transaction or connection duration). So normally you would create a global temporary table once, and not drop it, but instead reuse its definition.
Although you technically can execute DDL using execute statement, you are not supposed to, and it is not guaranteed to work. Your code is specifically an example of one of the things that will not work.
The problem here, is that you are trying to drop the table in the same transaction that used it (though to be honest, I'm surprised the insert even worked, because normally you can't insert into a table that was created in the same transaction).
The insert you executed on TMPTBL will mark the table in use, and given the transaction isn't committed yet, you can't drop the table: it is in use.
You shouldn't call commit in PSQL code (to be honest, I thought this wasn't even possible).
In short, you need to rethink how you use global temporary tables: define it once, and do not use execute statement to create it, but create it separately.
If you do want to create and drop it and not retain the definition of the global temporary table, then create it before the execute block, commit, then the execute block (with only the inserts and the 'perform some actions'), commit, and then drop it (and commit).
Alternatively, you might get away with executing the create using execute statement ... with autonomous transaction, the inserts and the 'perform some actions' in another execute statement ... with autonomous transaction, and finally the drop in yet another execute statement ... with autonomous transaction. However that makes your code very brittle, and this is not a recommend approach.
I have been forced again by devops guys to find robust solution to provide DB structure upgrades. Requirements: safely combine DDL and DML statements; ability to create temporary tables (for heavy selections); leave no garbage. Of course, upgrade is handled within single connection.
Referencing to the clues given by Mark a deeper insight and lots of experiments were made.
Here is template filescript that really worked out (isql native utility used):
SET TERM #;
-- 1. Create temporary table
EXECUTE BLOCK
AS
BEGIN
execute statement 'recreate GLOBAL TEMPORARY table TMPTBL (ID bigint) /*on commit preserve rows*/;';
END#
commit#
-- Data manipulations
EXECUTE BLOCK
AS
declare xid bigint;
BEGIN
-- 2. dummy fill of temporary table
begin
insert into TMPTBL (ID) values (0xFE);
end
-- 3. perform some actions...
for
select tt.ID
from TMPTBL tt
into :xid
do
begin
-- use :xid var
end
END#
commit#
-- 4. Delete temporary table
EXECUTE BLOCK
AS
BEGIN
execute statement 'drop table TMPTBL;';
END#
commit#
SET TERM ;#
Might be usefull for someone.
Damn, Firebird do drives crazy!

PostgreSQL TEMP table alternating between exist and not exist

I'm using PostgreSQL 9.6.2, with Toad client on Mac. Auto-commit is set to ON.
I first created a simple temp table like this:
CREATE TEMP TABLE demo_pairs
AS
WITH t (name, value) AS (VALUES ('a', 'b'), ('c', 'd'))
SELECT * FROM t;
Then something weird happens when I ran:
SELECT * FROM demo_pairs;
Every time I run the select (without re-running the create), it alternates between successfully selecting the values and error with table does not exist!
Can anyone help me understand what's going on?
https://www.postgresql.org/docs/current/static/sql-createtable.html
TEMPORARY or TEMP
If specified, the table is created as a temporary table. Temporary
tables are automatically dropped at the end of a session, or
optionally at the end of the current transaction (see ON COMMIT
below). Existing permanent tables with the same name are not visible
to the current session while the temporary table exists, unless they
are referenced with schema-qualified names. Any indexes created on a
temporary table are automatically temporary as well.
If you use session pooler that can close session for your or just close it yourself (eg network problem), the temp table will be dropped.
Also you can create it the way it is dropped on transaction end as well:
ON COMMIT
The behavior of temporary tables at the end of a transaction block can
be controlled using ON COMMIT. The three options are:
PRESERVE ROWS
No special action is taken at the ends of transactions. This is the
default behavior.
DELETE ROWS
All rows in the temporary table will be deleted at the end of each
transaction block. Essentially, an automatic TRUNCATE is done at each
commit.
DROP
The temporary table will be dropped at the end of the current transaction block.

Prevent query assigning ID when column has a default

I have a table that has a column defined as id serial. Therefore, it has a default on the column to call a sequencer. Which is what I want. However, my boss (and owner of the company) insists on having full access to the database and likes to write his own queries (including inserts) on this table. And even more unfortunately, he seems to always forgets that these IDs should be auto-generated for him, and he creates the record with max(id) + 1. Then the sequencer is out of sync with the table and the next time someone calls the API we get a 500.
Question: Given that I don't have the ability to lock him out of the database, what is the best way to prevent him from doing this?
My first idea was create a before insert trigger and raise an error if the ID was assigned. Here is what I did:
CREATE OR REPLACE FUNCTION md.tf_no_manual_keys()
RETURNS trigger
LANGUAGE plpgsql
AS $function$
DECLARE
BEGIN
IF NEW.id IS NOT NULL THEN
RAISE EXCEPTION 'ID must be assigned from sequencer';
END IF;
END;
$function$
trigger_no_manual_keys BEFORE INSERT ON vendor FOR EACH ROW EXECUTE PROCEDURE tf_no_manual_keys()
But, this doesn't work because the sequencer has already fired (I didn't think that was the case for a "BEFORE" trigger).
I guess I could always call the trigger on BEFORE INSERT, but that seems like it would use 2 IDs whenever it's called.
I'm using PostgreSQL 9.4
If you have grant privilege on the table you can revoke a role's insert privilege on a certain column:
revoke insert (id) on table t from role_name;
But that will not work if that role is a member of another role that has that privilege.

Selecting column name from other database table through function in PostgreSQL

Here i need to select a column name by using function(stored procedure) which is present in other database table using PostgreSQL.
I have sql server query as shown below.
Example:
create procedure sp_testing
as
if not exists ( select ssn from testdb..testtable) /*ssn is the column-name of testtable which exists in testdb database */
...
Q: Can i do the same in PostgreSQL?
Your question is not very clear, but if you want to know if a column by a certain name exists in a table by a certain name in a remote PostgreSQL database, then you should first set up a foreign data wrapper, which is a multi-stage process. Then to test the existence of a certain column in a table you need to formulate a query that conforms to the standards of the particular DBMS that you are connecting to. Use the remote information_schema.tables table for optimal compatibility (which is here specified as remote_tables which you must have defined with a prior CREATE FOREIGN TABLE command):
CREATE FUNCTION sp_testing () AS $$
BEGIN
PERFORM *
FROM remote_tables
WHERE table_name = 'testtable'
AND column_name = 'ssn';
IF NOT FOUND THEN
...
END IF;
END;
$$ LANGUAGE plpgsql;
If you want to connect to another type of DBMS, you need to write some custom function in f.i. C or perl and then call that from within a PostgreSQL function on your local machine. The test on the column is then best done inside the function which should therefore take connection parameters, table name and column name as parameters, and return a boolean to inform the result.
Before you start testing this, make sure that you read all the documentation on connecting to remote servers and learning PL/pgSQL first would also be a nice gesture to demonstrate your own efforts before you ask for help.