I have a rather large while loop which looks like this:
WHILE x
SELECT * INTO #temp FROM y
INSERT INTO x FROM #temp
The INSERT won't always work because of schema changes.
How would I go about just skipping this INSERT (if there is an error) and moving on to the next instance of the loop?
Related
I have a job which runs every night to load changes into a temporary table and apply those changes to the main table.
CREATE TEMP TABLE IF NOT EXIST tmp AS SELECT * FROM mytable LIMIT 0;
COPY tmp FROM PROGRAM '';
11 SQL queries to update 'mytable' based on data from 'tmp'
I have a large number of queries to delete duplicates from tmp, update values in tmp, update values in the main table and insert new rows into the main table. Is it possible to loop over both tables using plpgsql instead?
UPDATE mytable m
SET "Field" = t."Field" +1
FROM tmp t
WHERE (t."ID" = m."ID");
In this example, it is simple change of a column value. Instead, I want to do more complex operations on both the main table as well as the temp table.
EDIT: so here is some is some PSEUDO code of what I imagine.
LOOP tmp t, mytable m
BEGIN
-- operation in plpgsql including UPDATE, INSERT, DELETE
END
WHERE t.ID = m.ID;
You can use plpgsql FOR to loop over query results.
DECLARE
myrow RECORD;
BEGIN
FOR myrow IN SELECT * FROM table1 JOIN table2 USING (id)
LOOP
... do something with the row ...
END LOOP;
END
If you want to update a table while looping over it, you can create a FOR UPDATE cursor, but that won't work if the query is a join, because then you're not opening an update cursor on a table.
Note writing to/updating temp tables is much faster than writing to normal tables because temp tables don't have WAL and crash recovery overhead, and they're owned by one single connection, so you don't have to worry about locks.
If you put a query inside the loop, it will be executed many times though, which could get pretty slow. It's usually faster to use bulk queries, even if they're complicated.
If you want to UPDATE many rows in the temp table with values that depend on other tables and joins, it could be faster to run several updates on the temp table with different join and WHERE conditions.
I am writing a Sybase stored procedure with a cursor fetching records from table and inserting records back. Surprisingly I found that fetches see amd return records inserted in the same loop. Under some conditions this results in endless loop, sometimes the loop ends inseting a number of extra records. Looking through Sybase documentation I could not find a cure. Transaction isolation does not help since we are acting inside a single transaction. Of course I could solve the problem by inserting into a temporary table in the loop and then inserting back into main table after the loop ends.
But the question remain: how can I isolate cursor fetches from inserts into the same table?
Pseudocode follows:
create table t (v int)
go
insert into t(v) values (1)
create procedure p as
begin
declare #v int
declare c cursor for select v from t
begin transaction
open c
while 1=1 begin
fetch c into #v
if (##sqlstatus != 0) break
insert into t (v) values (#v+1)
end
close c
commit
end
go
exec p
go
I was surprised by this behavior, too. Cursors only iterate over rows, they are not immune to changes during the loop. The manual states this:
A searched or positioned update on an allpages-locked table can change
the location of the row; for example, if it updates key columns of a
clustered index. The cursor does not track the row; it remains
positioned just before the next row at the original location.
Positioned updates are not allowed until a subsequent fetch returns
the next row. The updated row may be visible to the cursor a second
time, if the row moves to a later position in the search order
My solution was to insert into a temporary table and copy the results back at the end, this also sped up the process by a factor of about 10.
Pseudo-code:
select * into #results from OriginalTable where 1<>1 --< create temp table with same columns
WHILE fetch...
BEGIN
insert into #results
select -your-results-here
END
insert into OriginalTable select * from #results
This is related to this question but slightly different, I have while loop that inserts records and I want it to continue even if some inserts fail. So, the insertrecords procedure inserts records, by doing a where on the temp table for top 50 rows at a time.
The problem is that it won't continue if any of the inserts inside the insertrecords fail? How can I modify the sql to continue with the next 50 rows, even if it fails for current 50 records. I guess is there something like try/catch exception handling in sybase?
SELECT id INTO #temp FROM myTable
-- Loop through the rows of the temp table
WHILE EXISTS(SELECT 1 FROM #temp)
BEGIN
BEGIN TRANSACTION
exec insertrecords
IF ##error = 0
begin
print 'commited'
commit
end
else
begin
print 'rolled back'
rollback
end
DELETE TOP 50 FROM #temp order by id
END
-- Drop the temp table.
DROP TABLE #temp
Try putting the content inside your while block inside try cactch.
NOTE: The below sample is in SQL, try similar code in sybase.
`WHILE(SOME CONDITION)
BEGIN --start of while block
BEGIN TRY-start of try block
--your code here
END TRY
BEGIN CATCH
PRINT ERR_MESSAGE();
END CATCH
END --end of while loop.
`
I have a TSQL sproc that does three loops in order to find relevant data. If the first loop renders no results, then the second one normally does. I append another table that has multiple values that I can use later on.
So at most I should only have two tables returned in the dataset from the sproc.
The issue is that if the first loop is blank, I then end up with three data tables in my data set.
In my C# code, I can remove this empty table, but would rather not have it returned at all from the sproc.
Is there a way to remove the empty table from within the sproc, given the following:
EXEC (#sqlTop + #sqlBody + #sqlBottom)
SET #NumberOfResultsReturned = ##ROWCOUNT;
.
.
.
IF #NumberOfResultsReturned = 0
BEGIN
SET #searchLoopCount = #searchLoopCount + 1
END
ELSE
BEGIN
-- we have data, so no need to run again
BREAK
END
The process goes as follows: On the first loop there could be no results. Thus the rowcount will be zero because the EXEC executes a dynamically created SQL query. That's one table.
In the next iteration, results are returned, making that two data tables in the dataset output, plus my third one added on the end.
I didn't want to do a COUNT(*) then if > 0 then perform the query as I want to minimize the queries.
Thanks.
You can put the result for your SP in a table variable and then check if the table variable has any data in it.
Something like this with a SP named GetData that returns one integer column.
declare #T table(ID int)
declare #SQL varchar(25)
-- Create dynamic SQL
set #SQL = 'select 1'
-- Insert result from #SQL to #T
insert into #T
exec (#SQL)
-- Check for data
if not exists(select * from #T)
begin
-- No data continue loop
set #searchLoopCount = #searchLoopCount + 1
end
else
begin
-- Have data so wee need to query the data
select *
from #T
-- Terminate loop
break
end
I have a table with some persistent data in it. Now when I query it, I also have a pretty complex CTE which computes the values required for the result and I need to insert missing rows into the persistent table. In the end I want to select the result consisting of all the rows identified by the CTE but with the data from the table if they were already in the table, and I need the information whether a row has been just inserted or not.
Simplified this works like this (the following code runs as a normal query if you like to try it):
-- Set-up of test data, this would be the persisted table
DECLARE #target TABLE (id int NOT NULL PRIMARY KEY) ;
INSERT INTO #target (id) SELECT v.id FROM (VALUES (1), (2)) v(id);
-- START OF THE CODE IN QUESTION
-- The result table variable (will be several columns in the end)
DECLARE #result TABLE (id int NOT NULL, new bit NOT NULL) ;
WITH Source AS (
-- Imagine a fairly expensive, recursive CTE here
SELECT * FROM (VALUES (1), (3)) AS Source (id)
)
MERGE INTO #target AS Target
USING Source
ON Target.id = Source.id
-- Perform a no-op on the match to get the output record
WHEN MATCHED THEN
UPDATE SET Target.id=Target.id
WHEN NOT MATCHED BY TARGET THEN
INSERT (id) VALUES (SOURCE.id)
-- select the data to be returned - will be more columns
OUTPUT source.id, CASE WHEN $action='INSERT' THEN CONVERT(bit, 1) ELSE CONVERT(bit, 0) END
INTO #result ;
-- Select the result
SELECT * FROM #result;
I don't like the WHEN MATCHED THEN UPDATE part, I'd rather leave the redundant update away but then I don't get the result row in the OUTPUT clause.
Is this the most efficient way to do this kind of completing and returning data?
Or would there be a more efficient solution without MERGE, for instance by pre-computing the result with a SELECT and then perform an INSERT of the rows which are new=0? I have difficulties interpreting the query plan since it basically boils down to a "Clustered Index Merge" which is pretty vague to me performance-wise compared to the separate SELECT followed by INSERT variant. And I wonder if SQL Server (2008 R2 with CU1) is actually smart enough to see that the UPDATE is a no-op (e.g. no write required).
You could declare a dummy variable and set its value in the WHEN MATCHED clause.
DECLARE #dummy int;
...
MERGE
...
WHEN MATCHED THEN
UPDATE SET #dummy = 0
...
I believe it should be less expensive than the actual table update.