I have a very long cleaning procedure with 1000s of "replaces".
I saved this in a separate file.
All other processes are defined in another query.
I would like to execute the cleaning file from the main query.
Is this possible and how to do that?
Example:
Main query file
create table a as
select * from b;
--a lot of other stuff--
execute cleaning query here!! -- I want to execute the cleaning query within my main query
-- cleaning query looks as follows (don't want to paste this into main query):
create table a_ as
select *, replace(replace(replace(...(a1
, 'WORD1', '')
, 'WORD2', '')
, 'WORD3', '')
... from a ;
-- end of cleaning query
--again a lot of stuff --
I suggest you using a trigger. PostgreSqlDocumentation: trigger
A trigger allows you to call functions (query); you can also declare it for each row so you can write a query which refers to only one generic row, than the trigger will apply it to every row of the table (you can also ask for a condition to be fulfilled)
Related
I have a job which runs every night to load changes into a temporary table and apply those changes to the main table.
CREATE TEMP TABLE IF NOT EXIST tmp AS SELECT * FROM mytable LIMIT 0;
COPY tmp FROM PROGRAM '';
11 SQL queries to update 'mytable' based on data from 'tmp'
I have a large number of queries to delete duplicates from tmp, update values in tmp, update values in the main table and insert new rows into the main table. Is it possible to loop over both tables using plpgsql instead?
UPDATE mytable m
SET "Field" = t."Field" +1
FROM tmp t
WHERE (t."ID" = m."ID");
In this example, it is simple change of a column value. Instead, I want to do more complex operations on both the main table as well as the temp table.
EDIT: so here is some is some PSEUDO code of what I imagine.
LOOP tmp t, mytable m
BEGIN
-- operation in plpgsql including UPDATE, INSERT, DELETE
END
WHERE t.ID = m.ID;
You can use plpgsql FOR to loop over query results.
DECLARE
myrow RECORD;
BEGIN
FOR myrow IN SELECT * FROM table1 JOIN table2 USING (id)
LOOP
... do something with the row ...
END LOOP;
END
If you want to update a table while looping over it, you can create a FOR UPDATE cursor, but that won't work if the query is a join, because then you're not opening an update cursor on a table.
Note writing to/updating temp tables is much faster than writing to normal tables because temp tables don't have WAL and crash recovery overhead, and they're owned by one single connection, so you don't have to worry about locks.
If you put a query inside the loop, it will be executed many times though, which could get pretty slow. It's usually faster to use bulk queries, even if they're complicated.
If you want to UPDATE many rows in the temp table with values that depend on other tables and joins, it could be faster to run several updates on the temp table with different join and WHERE conditions.
In a function, I need an array of values which is a result of a simple query like:
SELECT array_agg( some_col ) FROM some_table;
I could declare it in function like:
my_array text[] := SELECT array_agg( some_col ) FROM some_table;
But:
this dataset changes maybe once in some years
this dataset is really small
this function would be called a lot
this dataset needs to be up to date
Is there a way to avoid executing the same query over and over? It is not particularly expensive to call, but due to its static nature, I'd like to avoid it.
I could set trigger on some_table to generate the cached version of my_array on any mutation on the table, but is there a way to hold such a variable all the time for every connection?
I'd like to write this function in SQL or PLPGSQL.
In Postgres you can create materialized views (see the docs). It allows you to store the result of a query, and refresh it whenever you want.
It acts like a virtual table, so it is very cheap to query against.
CREATE MATERIALIZED VIEW mymatview AS SELECT array_agg( some_col ) FROM some_table;
And when you want to refresh it:
REFRESH MATERIALIZED VIEW mymatview;
I need to insert either set A or set B of records into a #temptable, depending on certain condition
My pseudo-code:
IF OBJECT_ID('tempdb..#t1') IS NOT NULL DROP TABLE #t1;
IF {some-condition}
SELECT {columns}
INTO #t1
FROM {some-big-table}
WHERE {some-filter}
ELSE
SELECT {columns}
INTO #t1
FROM {some-other-big-table}
WHERE {some-other-filter}
The two SELECTs above are exclusive (guaranteed by the ELSE operator). However, SQL compiler tries to outsmart me and throws the following message:
There is already an object named '#t1' in the database.
My idea of "fixing" this is to create #t1 upfront and then executing a simple INSERT INTO (instead of SELECT... INTO). But I like minimalism and am wondering whether this can be achieved in an easier way i.e. without explicit CREATE TABLE #t1 upfront.
Btw why is it NOT giving me an error on a conditional DROP TABLE in the first line? Just wondering.
You can't have 2 temp tables with the same name in a single SQL batch. One of the MSDN article says "If more than one temporary table is created inside a single stored procedure or batch, they must have different names". You can have this logic with 2 different temp tables or table variable/temp table declared outside the IF-Else block.
Using a Dyamic sql we can handle this situation. As a developoer its not a good practice. Best to use table variable or temp table.
IF 1=2
BEGIN
EXEC ('SELECT 1 ID INTO #TEMP1
SELECT * FROM #TEMP1
')
END
ELSE
EXEC ('SELECT 2 ID INTO #TEMP1
SELECT * FROM #TEMP1
')
I'm using Firebird as DB and I need to do this:
INSERT INTO TG (ID, UID, GID)
SELECT (INSERT INTO TBO VALUES (GEN_ID('o',1)) RETURNING ID), UID, 10
FROM TBL l
WHERE l.is=1
the part with select is OK when I use:
SELECT (GEN_ID('o',1)), UID, 10
FROM TBL l
WHERE l.is=1
but I need the ID in other table for dependency first.
I know about something called procedures but I have no idea how to use them. Is there an option to do this using SQL?
Take a look at EXECUTE BLOCK statement. It allow you to execute multiple statements in one "batch" or write complex logic if you cannot embed it in one SQL query.
Inside EXECUTE BLOCK you can write mutiple commands using PSQL.
EB allow input params, output params (yes you can use it as a table), local variables, if statement, while, for select etc... very powerfull tool.
Just prepare your block and execute it like simple SQL query.
A simpler aproach would be to use triggers.
In this example it would be a before insert trigger.
Something like this:
CREATE TRIGGER TG_BI0 FOR TABLE TG ACTIVE BEFORE INSERT 0
AS
BEGIN
/* Before insert create a new record in TBO */
INSERT INTO TBO (ID) VALUES (NEW.ID);
END
After having this trigger you shloud only insert records in TG.
INSERT INTO TG (ID, UID, GID)
VALUES (GEN_ID('o',1), 'SOME_UUID', 10)
Due to a legacy report generation system, I need to use a cursor to traverse the result set from a stored procedure. The system generates report output by PRINTing data from each row in the result set. Refactoring the report system is way beyond scope for this problem.
As far as I can tell, the DECLARE CURSOR syntax requires that its source be a SELECT clause. However, the query I need to use lives in a 1000+ line stored procedure that generates and executes dynamic sql.
Does anyone know of a way to get the result set from a stored procedure into a cursor?
I tried the obvious:
Declare Cursor c_Data For my_stored_proc #p1='foo', #p2='bar'
As a last resort, I can modify the stored procedure to return the dynamic sql it generates instead of executing it and I can then embed this returned sql into another string and, finally, execute that. Something like:
Exec my_stored_proc #p1='foo', #p2='bar', #query='' OUTPUT
Set #sql = '
Declare Cursor c_Data For ' + #query + '
Open c_Data
-- etc. - cursor processing loop etc. goes here '
Exec #sql
Any thoughts? Does anyone know of any other way to traverse the result set from a stored proc via a cursor?
Thanks.
You could drop the results from the stored proc into a temp table and select from that for your cursor.
CREATE TABLE #myResults
(
Col1 INT,
Col2 INT
)
INSERT INTO #myResults(Col1,Col2)
EXEC my_Sp
DECLARE sample_cursor CURSOR
FOR
SELECT
Col1,
Col2
FROM
#myResults
Another option may be to convert your stored procedure into a table valued function.
DECLARE sample_cursor CURSOR
FOR
SELECT
Col1,
Col2
FROM
dbo.NewFunction('foo', 'bar')
You use INSERT ... EXEC to push the result of the procedure into a table (can be a temp #table or a #table variable), the you open the cursor over this table. The article in the link discusses the problems that may occur with this technique: it cannot be nested and it forces a transaction around the procedure.
You could execute your SP into a temporary table and then iterate over the temporary table with the cursor
create table #temp (columns)
insert into #temp exec my_stored_proc ....
perform cursor work
drop table #temp