ADF Copy activity Sink Pre-copy script timeout - azure-data-factory

I have a pre-copy script DELETE FROM mytable where ID=123
This timed out (after 4 hours)
Then I gave TRUNCATE TABLE mytable and I got the error 'Table does not exist or no permission' .
I am able to insert from ADF copy. But on pre-copy or on lookup query . I get the errors state above. What could be wrong?

In Lookup active, your query DELETE FROM mytable where ID=123 and TRUNCATE TABLE mytable doesn't return any result.
Please ref the Look up active note:
When you use query or stored procedure to lookup data, make sure to
return one and exact one result set. Otherwise, Lookup activity
fails.
Just according the error message, please make sure you're using the user/account which have enough permission to delete the data in Sink linked server dataset.

Related

Snowflake How to get Records failed in Copy command

is it possible to get records which failed during Copy command in Snowflake from internal stage to snowflake table?
I am trying to load error recrods in a error table during Copy command execution . Copy Command used:
Copy into table ( col1, col2,col3,col4) from ( select $1,$2,$3,56 from #%table) ON_ERROR=CONTINUE
To get all the bad records, you can run the copy with VALIDATION_MODE = 'RETURN ERRORS'. Then use the RESULT_SCAN from the validation in an insert statement.
If one of your columns is unique (i.e. col1), maybe you can compare rows in the table with the rows in the stage:
select $1 from #%table
MINUS
select col1 from table;
Please check below select statement after copy command
select rejected_record from table(validate(test_copy , job_id => '_last')) ;

Redshift COPY throws error but 'stl_load_errors' system table does not provide details

When I attempt to copy a CSV from S3 into a new table in Redshift (which normally works for other tables) I get this error
ERROR: Load into table 'table_name' failed. Check 'stl_load_errors'
system table for details.
But, when I run the standard query to investigate stl_load_errors
SELECT errors.tbl, info.table_id::integer, info.table_id, *
FROM stl_load_errors errors
INNER JOIN svv_table_info info
ON errors.tbl = info.table_id
I don't see any results related to this COPY. I see errors from previous failed COPY commands, but none related to the most recent one that I am interested in.
Please make sure that you are querying stl_load_errors table with same user you are performing COPY command. You can also try to avoid using ssv_table_info table in query or change INNER to LEFT join.

"No current record for fetch operation" for select insert

Can anyone see why I'm getting the "No current record for fetch operation" below?
I'm successfully skipping duplicate records by catching and not re-throwing the unique key violation exception below. There is another error however.
FOR SELECT
...
FROM
P_SELECT_CLAIM_FILE cf
ORDER BY
cf.DATESBM, cf.TIMESBM, cf.TRANSCDE
INTO
...variables
DO BEGIN
INSERT INTO
CLAIM_TABLE (...)
VALUES (...variables);
WHEN GDSCODE unique_key_violation DO
TX_ID=0;
WHEN GDSCODE no_cur_rec DO
/* Why does this happen?
-508 335544348 no_cur_rec No current record for fetch operation
*/
EXCEPTION E 'no_cur_rec ' || TX_ID;
END
The procedure P_SELECT_CLAIM_FILE contains another FOR SELECT INTO with lots of trimming and finally a SUSPEND command. This reads from a fixed width text file.
I'm tempted to change this into a single INSERT SELECT where not exists statement. I prefer to make a minimal fix instead however; the holidays already here.

How to Retrieve autoincremnt value after inserting 1 record in single query (sql server)

I am have two fields in my table:
One is Primary key auto increment value and second is text value.
lets say: xyzId & xyz
So I can easily insert like this
insert into abcTable(xyz) Values('34')
After performing above query it must insert these information
xyzId=1 & xyz=34
and for retrieving I can retrieve like this
select xyzId from abcTable
But for this I have to write down two operation. Cant I retrieve in single/sub query ?
Thanks
If you are on SQL Server 2005 or later you can use the output clause to return the auto created id.
Try this:
insert into abcTable(xyz)
output inserted.xyzId
values('34')
I think you can't do an insert and a select in a single query.
You can use a Store Procedures to execute the two instructions as an atomic operation or you can build a query in code with the 2 instructions using ';' (semicolon) as a separator betwen instructions.
Anyway, for select identity values in SQL Server you must check ##IDENTITY, SCOPE_IDENTITY and IDENT_CURRENT. It's faster and cleaner than a select in the table.

Mirth Question: what does a Run On-Update Statement do?

I am trying to document a Mirth channel with Connector Type: Database Reader. It has a SQL statement that it uses to read input to the Mirth channel. But then it has another box called On-Update SQL with more SQL code. Does that SQL run after the SQL input statement? What does the On-Update Statement do?
On-Update SQL should be used to update a record once it has been read so that it is not read again. For example, if your SQL statement is:
SELECT id, firstName, lastName FROM person WHERE status = 0;
Then should have the On-Update SQL as:
UPDATE person SET status = 1 WHERE id = ${id};
Notice that the ${id} variable is used. This would be replaced with the ID value selected in the original SQL statement. This allows you to update the same record that was selected.
You can use any of the columns in your UPDATE that you retrieve in your SELECT (ex. ${firstName}).
Source