I have source query in Copy data activity of Azure Data Factory. Currently it is static query, but I would like to get entire statement from database and use that in Query in Activity.
I try to pass SQL statement as variable there is issue.
Lookup SQL Query for fetching Select statement:
SELECT [source_query] FROM [SalesLT].[configuration] WHERE [Id] = '1'
Query returns and set variable:
SELECT Count(CustomerId) Source_Rows FROM [SalesLT].[Customer]
Error with Copy data:
Type=System.Data.SqlClient.SqlException,Message=Incorrect syntax near
')'.,Source=.Net SqlClient Data
You can't directly use the pipeline parameter in Query instead need to use #concat function to concatenate your SQL Query with parameters. Please refer the example below:
I suggest you to please go through this tutorial from #TechBrothersIT to get more clarification on the related issue.
Related
I am executing this query:
select * from schema.country where country_code='abc' and country='INDIA'
inside ADF dataflow. I am getting this query from the database so I don't want to hardcode it inside the data flow. All the data flow inputs are adding a single quotes while executing. The dataflow is failing because of parsing error.
How can I escape single quotes ?
You can just pass your query in double quotes ("") in the dataflow to escape single quotes.
Please go through the below 3 approaches based on your usage of query in the dataflow.
This is my sample data from database:
If you are using Query in the source of dataflow, you can give the below expression in the Expression builder.
"select * from [dbo].[country] where country_code='abc' and country_name='INDIA'"
Output Data preview:
If you want to get your query from your database, then create a stored procedure for the query and access it inside the dataflow.
My stored procedure in the database:
create or alter procedure dbo.proc2
as
begin
select * from [dbo].[country] where country_code='abc' and country_name='INDIA'
end;
Use this query inside stored procedure in the dataflow like below. You will get the same output as above.
If you are using pipeline paramter to pass query to the dataflow, then create a parameter in the dataflow with default value. Give your query in double quotes in the pipeline parameters expression. Check the Expression check box after this.
Then give the $parameter1 in the expression builder of query and execute. Please check this SO thread to learn more it.
My csv in sink after pipeline execution in this process:
I am trying to run a create Shema/table query in the ADF lookup activity, with a dummy select in the end.
CREATE SCHEMA [schemax] AUTHORIZATION [auth1];
SELECT 0 AS dummyValue
but I got the below error
A database operation failed with the following error: 'Parse error at line: 2, column: 1: Incorrect syntax near 'SELECT'.',Source=,''Type=System.Data.SqlClient.SqlException,Message=Parse error at line: 2, column: 1: Incorrect syntax near 'SELECT'.,Source=.Net SqlClient Data Provider,SqlErrorNumber=103010,Class=16,ErrorCode=-2146232060,State=1
Data factory pipeline
I was able to run a similar query without SELECT query in the end but got another error mandating lookup must return a value.
You can only write select statements in lookup activity query settings.
To create schema or table, use copy data activity pre-copy script in sink settings. You can select a dummy table for the source and sink dataset and write your create script in pre-copy activity as below.
Source settings: (using dummy table which pulls 0 records)
Sink settings:
Pre-copy script: CREATE SCHEMA test1 AUTHORIZATION [user]
I have a PostgreSQL function which has a ref cursor as output parameter;
I know i can get output using below set of commands
begin;
select uspfillactions();
fetch All in "<unnamed portal 1>";
rollback;
But just want to know is there any way to get function output using single query?
Any dynamic query solution or anything else?
I don't want to mention Begin and rollback/commit also.
I am have two fields in my table:
One is Primary key auto increment value and second is text value.
lets say: xyzId & xyz
So I can easily insert like this
insert into abcTable(xyz) Values('34')
After performing above query it must insert these information
xyzId=1 & xyz=34
and for retrieving I can retrieve like this
select xyzId from abcTable
But for this I have to write down two operation. Cant I retrieve in single/sub query ?
Thanks
If you are on SQL Server 2005 or later you can use the output clause to return the auto created id.
Try this:
insert into abcTable(xyz)
output inserted.xyzId
values('34')
I think you can't do an insert and a select in a single query.
You can use a Store Procedures to execute the two instructions as an atomic operation or you can build a query in code with the 2 instructions using ';' (semicolon) as a separator betwen instructions.
Anyway, for select identity values in SQL Server you must check ##IDENTITY, SCOPE_IDENTITY and IDENT_CURRENT. It's faster and cleaner than a select in the table.
Currently my development environment is using SQL server express 2008 r2 and VS2010 for my project development.
My question is like this by providing a scenario:
Development goal:
I develop window services something like data mining or data warehousing using .net C#.
That meant I have a two or more database involved.
my senario is like this:
I have a database with a table call SQL_Stored inside provided with a coloum name QueryToExec.
I first idea that get on my mind is written a stored procedure and i tried to came out a stored procedure name Extract_Sources with two parameter passed in thats ID and TableName.
My first step is to select out the sql need to be execute from table SQL_Stored. I tried to get the SQL by using a simple select statement such as:
Select Download_Sql As Query From SQL_Stored
Where ID=#ID AND TableName=#TableName
Is that possible to get the result or is there another way to do so?
My Second step is to excecute the Sql that i get from SQL_Stored Table.Is possible to
to execute the query that select on the following process of this particular stored proc?
Need to create a variable to store the sql ?
Thank you,Appreciate for you all help.Please don't hesitate to voice out my error or mistake because I can learn from it. Thank you.
PS_1:I am sorry for my poor English.
PS_2:I am new to stored procedure.
LiangCk
Try this:
DECLARE #download_sql VARCHAR(MAX)
Select
#download_sql = Download_Sql
From
SQL_Stored
Where
AreaID = #AreaID
AND TableName = #TableName
EXEC (#download_sql)