is it possible to get records which failed during Copy command in Snowflake from internal stage to snowflake table?
I am trying to load error recrods in a error table during Copy command execution . Copy Command used:
Copy into table ( col1, col2,col3,col4) from ( select $1,$2,$3,56 from #%table) ON_ERROR=CONTINUE
To get all the bad records, you can run the copy with VALIDATION_MODE = 'RETURN ERRORS'. Then use the RESULT_SCAN from the validation in an insert statement.
If one of your columns is unique (i.e. col1), maybe you can compare rows in the table with the rows in the stage:
select $1 from #%table
MINUS
select col1 from table;
Please check below select statement after copy command
select rejected_record from table(validate(test_copy , job_id => '_last')) ;
Related
I have a pre-copy script DELETE FROM mytable where ID=123
This timed out (after 4 hours)
Then I gave TRUNCATE TABLE mytable and I got the error 'Table does not exist or no permission' .
I am able to insert from ADF copy. But on pre-copy or on lookup query . I get the errors state above. What could be wrong?
In Lookup active, your query DELETE FROM mytable where ID=123 and TRUNCATE TABLE mytable doesn't return any result.
Please ref the Look up active note:
When you use query or stored procedure to lookup data, make sure to
return one and exact one result set. Otherwise, Lookup activity
fails.
Just according the error message, please make sure you're using the user/account which have enough permission to delete the data in Sink linked server dataset.
I've created a temporary table DETAILS and follow the same syntax of creating and inserting in it. But I have not received any result set However, the CREATE and INSERT statements ran successfully and the Row was also affected in the INSERT statement . But the result set was empty when I ran the last SELECT statement to view the record .
DROP TABLE DETAILS ;
CREATE GLOBAL TEMPORARY TABLE DETAILS AS (
SELECT ins_id , firstname , pages FROM
INSTRUCTOR)DEFINITION ONLY;
INSERT INTO DETAILS
SELECT ins_id , firstname , pages
FROM INSTRUCTOR WHERE ins_id = '1';
SELECT * FROM DETAILS ;
If you want to preserve rows in CGTT after commit, you have to specify ON COMMIT PRESERVE ROWS option of the CREATE GLOBAL TEMPORARY TABLE statement.
ON COMMIT DELETE ROWS option is in effect otherwise, and such a table is cleared on commit.
I have to insert current date (timestamp) in a table via hive query. The query is failing for some reason. Can someone please help me out.
CREATE EXTERNAL TABLE IF NOT EXISTS dataFlagTest(
date string
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
LOCATION 's3://bckt1/hive_test/dateFlag/';
Now To insert into it, I run following query :
INSERT OVERWRITE TABLE dataFlagTest
SELECT from_unixtime(unix_timestamp()) ;
It failed with the following error :
FAILED: NullPointerException null
Can someone please help me out
Solution is you have to do select from a table. You cannot run select without from clause.
So, create a sample table with 1 row or use an existing table like below :
Insert OVERWRITE TABLE dataflagtest SELECT from_unixtime(unix_timestamp()) as date FROM EXISTING_TABLE TABLESAMPLE(1 ROWS);
I am having problems with my query.
Basically, what I am trying to do is empty out a table and copy the records from the same table in another database.
I did use the SET IDENTITY_INSERT code to make sure that the identity column is turned off before I perform my insert. But somehow, it still throws me the error message:
Msg 8101, Level 16, State 1, Line 3
An explicit value for the identity column in table 'dbo.UI_PAGE' can only be specified when a column list is used and IDENTITY_INSERT is ON.
Below is my query:
DELETE FROM [DB1].[dbo].[MY_TABLE]
SET IDENTITY_INSERT [DB1].[dbo].[MY_TABLE] ON
INSERT INTO [DB1].[dbo].[MY_TABLE]
SELECT *
FROM [DB2].[dbo].[MY_TABLE]
SET IDENTITY_INSERT [DB1].[dbo].[MY_TABLE] OFF
Can someone point me as to which step I am doing wrong?
Thanks a lot!
You have to specify all the column names when inserting with IDENTITY INSERT ON when using INSERT INTO
INSERT INTO [DB1].[dbo].[MY_TABLE](TabelID,Field1,Field2,Field3...)
SELECT * FROM [DB2].[dbo].[MY_TABLE]
In case you did not know there is a nifty little trick in ssms. If select a table and expand its' nodes you ctrl-c copy on the Columns node and that will place a comma-delimited list of the field names on your clipboards text buffer.
Addition to the first answer given by Ross Bush,
If your table has many columns then to get those columns name by using this command.
SELECT column_name + ','
FROM information_schema.columns
WHERE table_name = 'TableName'
for xml path('')
(after removing the last comma(',')) Just copy past columns name.
Generally, I am bulding dynamic SQL statement that is executing using sp_executsql like this:
EXEC sp_executesql #TempSQLStatement
I need to insert the return result row set in something (table variable or temporary table), but I am getting the following error:
Msg 208, Level 16, State 0, Line 1746
Invalid object name '#TempTable'.
after executing this:
INSERT INTO #TempTable
EXEC sp_executesql #TempSQLStatement
From what I have read, I believe the issue is caused because I am not specifying the columns of the temporary table, but I am not able to do this because the return columns count varies.
I have read that I can use global temporary tables, but I have done this before and wonder is there an other way to do that.
You can't. There is simply no way to create a #temptable from an EXEC output schema.
INSERT ... EXEC requires the table to exists (thus must know the schema before execution).
SELECT ... INTO does not support EXEC as a source.
If you use INSERT INTO statement you have to create a table first.
Another way if you want to store SQL statement result into the temp table you can use SELECT ... INTO but in this case you should change #TempSQLStatement and add INTO #TempTable before FROM to get it.
For example if your #TempSQLStatement contains only one FROM keyword:
SET #TempSQLStatement=REPLACE(#TempSQLStatement,' FROM ',' INTO ##TempTable FROM ');
EXEC sp_executesql #TempSQLStatement;
SELECT * from ##TempTable;