I am trying to run a create Shema/table query in the ADF lookup activity, with a dummy select in the end.
CREATE SCHEMA [schemax] AUTHORIZATION [auth1];
SELECT 0 AS dummyValue
but I got the below error
A database operation failed with the following error: 'Parse error at line: 2, column: 1: Incorrect syntax near 'SELECT'.',Source=,''Type=System.Data.SqlClient.SqlException,Message=Parse error at line: 2, column: 1: Incorrect syntax near 'SELECT'.,Source=.Net SqlClient Data Provider,SqlErrorNumber=103010,Class=16,ErrorCode=-2146232060,State=1
Data factory pipeline
I was able to run a similar query without SELECT query in the end but got another error mandating lookup must return a value.
You can only write select statements in lookup activity query settings.
To create schema or table, use copy data activity pre-copy script in sink settings. You can select a dummy table for the source and sink dataset and write your create script in pre-copy activity as below.
Source settings: (using dummy table which pulls 0 records)
Sink settings:
Pre-copy script: CREATE SCHEMA test1 AUTHORIZATION [user]
Related
I have source query in Copy data activity of Azure Data Factory. Currently it is static query, but I would like to get entire statement from database and use that in Query in Activity.
I try to pass SQL statement as variable there is issue.
Lookup SQL Query for fetching Select statement:
SELECT [source_query] FROM [SalesLT].[configuration] WHERE [Id] = '1'
Query returns and set variable:
SELECT Count(CustomerId) Source_Rows FROM [SalesLT].[Customer]
Error with Copy data:
Type=System.Data.SqlClient.SqlException,Message=Incorrect syntax near
')'.,Source=.Net SqlClient Data
You can't directly use the pipeline parameter in Query instead need to use #concat function to concatenate your SQL Query with parameters. Please refer the example below:
I suggest you to please go through this tutorial from #TechBrothersIT to get more clarification on the related issue.
I have a pre-copy script DELETE FROM mytable where ID=123
This timed out (after 4 hours)
Then I gave TRUNCATE TABLE mytable and I got the error 'Table does not exist or no permission' .
I am able to insert from ADF copy. But on pre-copy or on lookup query . I get the errors state above. What could be wrong?
In Lookup active, your query DELETE FROM mytable where ID=123 and TRUNCATE TABLE mytable doesn't return any result.
Please ref the Look up active note:
When you use query or stored procedure to lookup data, make sure to
return one and exact one result set. Otherwise, Lookup activity
fails.
Just according the error message, please make sure you're using the user/account which have enough permission to delete the data in Sink linked server dataset.
Executing an insert..select query over Tez on a Hortonworks HDP 3 cluster with hive3, I get the following error:
java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. MetaException(message:
Put request failed : INSERT INTO "PARTITION_PARAMS" ("PARAM_VALUE","PART_ID","PARAM_KEY") VALUES (?,?,?) )
The destination table has 200 fields and it is partitioned by two fields. Performing some testing, the error dissapears when the destination table has 143 fields. If I change the names of destination table fields with shorter ones, I can get the query working without error with more fields, but I can't get it working with the 200 fields I need.
Hive Metastore is configured to use a PostgreSQL database
We where hitting HIVE-20221
We can get the query executing correctly, setting hive.stats.autogather=false
I have a CLOB Json in a column of a table with the following structure:
{"sources": [1,2,4]}
I'm trying to write a trigger that read the array [1,2,4] and performs some checks:
I'm trying using:
DECLARE
TYPE source_type IS TABLE OF NUMBER;
SOURCES source_type;
[...]
json_query(:NEW.COL, '$.sources') BULK COLLECT INTO SOURCES FROM dual;
but I got the error:
Row 1: ORA-01722: invalid number
Any ideas?
I am trying to document a Mirth channel with Connector Type: Database Reader. It has a SQL statement that it uses to read input to the Mirth channel. But then it has another box called On-Update SQL with more SQL code. Does that SQL run after the SQL input statement? What does the On-Update Statement do?
On-Update SQL should be used to update a record once it has been read so that it is not read again. For example, if your SQL statement is:
SELECT id, firstName, lastName FROM person WHERE status = 0;
Then should have the On-Update SQL as:
UPDATE person SET status = 1 WHERE id = ${id};
Notice that the ${id} variable is used. This would be replaced with the ID value selected in the original SQL statement. This allows you to update the same record that was selected.
You can use any of the columns in your UPDATE that you retrieve in your SELECT (ex. ${firstName}).
Source