I'm calling a procedure using a lookup activity in Azure Data Factory.
NOTE: The reason to use Lookup here is, I wanted to store the OUTPUT parameter value from procedure into a variable in ADF for future use.
Below works,
DECLARE #ADFOutputMsg [VARCHAR](500);
EXEC Test.spAsRunTVA #ReportDate = '2022-06-01', #OutputMsg = #ADFOutputMsg OUTPUT;
SELECT #ADFOutputMsg As OutputMsg;
But when I want to pass dynamic parameters, it doesn't like,
DECLARE #ADFOutputMsg [VARCHAR](500);
EXEC #{pipeline().parameters.SchemaName}.spAsRunTVA #ReportDate = #{substring(pipeline().parameters.FileName,8,10)}, #OutputMsg = ADFOutputMsg OUTPUT;
SELECT #ADFOutputMsg As OutputMsg;
I also tried to keep the date As-Is and just updated SchemaName to be dynamic but I still get the error.
Please Provide single quote ' ' at your dynamic content
'#{substring(pipeline().parameters.FileName,8,10)}'
I tried to reproduce similar kind of approach in my environment and I got below results:
Use the below dynamic content in the query with lookup activity. and also Added dynamic content with single quotes ' '
select * from for_date where date1='#{string(pipeline().parameters.Date)}'
Added Date Parameter
Got this Output:
Related
I am trying to encapsulate the functionality from this sample code here, inside a Table-Function.
I can run the sample alone without any problem.
But when I create a table function, just with a single call to OPEN_CURSOR , I receive SQL0577N
CREATE FUNCTION ROW_CHECKSUM
( IN sSchema VARCHAR(128) ,
IN sTable VARCHAR(128) ,
IN sColumnList VARCHAR(1024) ,
IN sWhere VARCHAR(1023),
IN iRows INTEGER
)
RETURNS TABLE (ROW_PK_VALUES VARCHAR(3000), CHECKSUM INTEGER )
LANGUAGE SQL
SPECIFIC ROW_CHECKSUM
--NO EXTERNAL ACTION
--MODIFIES SQL DATA
--NOT DETERMINISTIC
BEGIN
DECLARE iCheckSum INTEGER ;
DECLARE sKyes VARCHAR(1024) ;
DECLARE iCursor INTEGER;
DECLARE sQuery VARCHAR(32000) ;
SET sQuery = 'SELECT ' || sColumnList || ' FROM "' || sSchema || '"."' || sTable || '" WHERE ' || sWhere || ' FETCH FIRST ' || TO_CHAR(iRows) || ' ONLY' ;
CALL DBMS_SQL.OPEN_CURSOR(iCursor);
--CALL DBMS_SQL.PARSE(iCursor, sQuery, DBMS_SQL.native) ;
--PIPE (sKeys, iCheckSum) ;
--PIPE ('abcd', 1234) ;
RETURN ;
END
----
SQL0577N User defined routine "DB2ADMIN.ROW_CHECKSUM" (specific name "")
attempted to modify data but was not defined as MODIFIES SQL DATA. LINE
NUMBER=33. SQLSTATE=38002
it seems, OPEN_CURSOR demands to have the MODIFY SQL DATA specified.. ok.. let's go!
But, when I specify it, then I get the following error, instead:
SQL0628N Multiple or conflicting keywords involving the "MODIFIES SQL DATA"
clause are present. LINE NUMBER=33. SQLSTATE=42613
The error details for -628 error is too generic and does not help me to determine what's really going on here.
I need to perform dynamic SQL queries using DBMS_SQL module, and return the result set using PIPE , like this other sample here.
I have been reading spread documentations the entire day.. and so far was not able to determine exactly what rule I am violating.
Also, found some inconsistencies on documentation, which I don't understand:
This page, says:
SQL table functions cannot contain compiled compound statements.
While, the Rules from RETURN statement says the opposite, and matches with PIPE sample code:
In an SQL table function using a compound SQL (compiled) statement, an expression, NULL, or fullselectcannot be specified. Rows are returned from the function using the PIPE statement and the RETURN statement is required as the last statement to execute when the function exits (SQLSTATE 2F005).
Appreciate any help!
Look at the note about the MODIFIES SQL DATA in the CREATE FUNCTION statement description:
4 Valid only for compiled scalar function definition and an inlined
table function definition.
But you can't use PIPE in an inlined function.
So, you want to use different functionalities, which can't be used together.
The inconsistency you found in the documentation is not related to you problem.
I am considering using RCP to run a generic datastage job, but the initial SQL changes each time it's called. Is there a process in which I can use a User Activity Variable to inject SQL from a text file or something so I can use the same datastage?
I know this Routine can read a file to look up parameters:
Routine = ‘ReadFile’
vFileName = Arg1
vArray = ”
vCounter = 0
OPENSEQ vFileName to vFileHandle
Else Call DSLogFatal(“Error opening file list: “:vFileName,Routine)
Loop
While READSEQ vLine FROM vFileHandle
vCounter = vCounter + 1
vArray = Fields(vLine,’,’,1)
vArray = Fields(vLine,’,’,2)
vArray = Fields(vLine,’,’,3)
Repeat
CLOSESEQ vFileHandle
Ans = vArray
Return Ans
But does that mean I just store the SQL in one Single line, even if it's long?
Thanks.
Why not just have the SQL within the routine itself and propagate parameters?
I have multiple queries within a single routine that does just that (one for source and one for AfterSQL statement)
This is an example and apologies I'm answering this on my mobile!
InputCol=Trim(pTableName)
If InputCol='Table1' then column='Day'
If InputCol='Table2' then column='Quarter, Day'
SQLCode = ' Select Year, Month, '
SQLCode := column:", Time, "
SQLCode := " to_date(current_timestamp, 'YYYY-MM-DD HH24:MI:SS'), "
SQLCode := \ "This is example text as output" \
SQLCode := "From DATE_TABLE"
crt SQLCode
I've used the multiple encapsulations in the example above, when passing out to a parameter make sure you check the ', " have either been escaped or are displaying correctly
Again, apologies for the quality but I hope it gives you some ideas!
You can give this a try
As you mentioned ,maintain the SQL in a file ( again , if the SQL keeps changing , you need to build a logic to automate populating the new SQL)
In the Datastage Sequencer , use a Execute Command Activity to open the SQL file
eg : cat /home/bk/query.sql
In the job activity which calls your generic job . you should map the command output of your EC activity to a job parameter
so if EC activity name is exec_query , then the job parameter will be
exec_query.$CommandOuput
When you run the sequence , your query will flow from
SQL file --> EC activity-->Parameter in Job activity-->DB stage( query parameterised)
Has you thinked to invoke a shellscript who connect to database and execute the SQL script from the sequential job? You could use sqlplus to connect in the shellscript and read the file with the SQL and use it. To execute the shellscript from the sequential job use a ExecCommand Stage (sh, ./, ...), it depends from the interpreter.
Other way to solve this, depends of the modification degree of your SQL; you could invoke a routine base who handle the parameters and invokes your parallel job.
The principal problem that I think you could have, is the limit of the long of the variable where you could store the parameter.
Tell me what option you choose and I could help you more.
this should be a simple thing but I've spent hours to no avail. Basically, I need to look up a salesrep # in a SQL database using the user's Window's user id. The format of the user id is
"Norstar\kjones" and I need the "kjones" portion of it.
using the split function, I am able to pull just the 'kjones' part out:
split(User!UserID,"\").GetValue(1)
I've created a parameter called SlsmnNum and created a dataset to be used to look up the salesrep # using the user id (the slsm_num field is a varchar, not an integer):
select slsm_num from Salesman_Msid where slsm_msid = ''' + split(User!UserID,"\").GetValue(1) + '''
However, I get no results. How can I get the select to work?
alternatively, I tried the following:
in parameter SlsmnNum, I set the default to an expression using:
split(User!UserID,"\").GetValue(1) and this returns 'kjones', as expected.
I created a SECOND parameter (which is positioned BELOW the SlsmnNum parameter), SlsmnNum2, that has a default (and an available) value using a query, which is a dataset containing the following select statement:
select slsm_num from Salesman_Msid where slsm_msid = (#SlsmnNum)
When I run the query on the Data tab, when I type in 'kjones' into the parameter box, it returns '1366', the salesrep # I'm expecting.
But, when I Preview the report, all I get in SlsmnNum2 box is Select a Value and nothing is there (it should return '1366').
Any help would be greatly appreciated!
Try your first approach with Query Text as
="select slsm_num from Salesman_Msid where slsm_msid = '" & split(User!UserID,"\").GetValue(1) & "'"
I know this has been asked earlier.
Most of the answers were not relevant.
Google, shows that the solution is to configure the expression in the "data flow task" and set the query.
However in the ADO .NET source, when I try to preview the output I keep getting "Must declare the variable '#'"
It does not show the full variable in this error - "#[User::GLOBAL_PARAMETER]"
I think that's because "[USER::" isn't the correct syntax inside a SQL; but then how does one set it ?!
From your description it seems like you are having an error due to using the variable name inside the query string as opposed to the processed variable value. In other words:
"SELECT * FROM #[User::TABLE]" in the expression builder would be WRONG
"SELECT * FROM " + #[User::TABLE] would be CORRECT
It would help if you shared the expression you are using as a query
Is there any way to reference the collection of parameters passed to a stored proc, without referencing each one by name?
The context here is error logging. I'd like to develop a generic CATCH clause, or sub-clause, that grabs all parameter values as well as other error and execution info and logs it to a table (one or more).
The basic version looks something like this:
CREATE PROC dbo.Proc1 #Param1 INT, #Param2 INT
AS
BEGIN TRY
SELECT 1/0
END TRY
BEGIN CATCH
DECLARE #params XML
SET #params = (
SELECT
#Param1 AS [Param1]
, #Param2 AS [Param2]
FOR XML PATH('params'), TYPE
)
EXEC dbo.LogError #procid = ##PROCID, #params = #params
EXEC dbo.RethrowError
END CATCH
Now, this template does work, except that for each individual procedure, I would have to edit the SET #params section manually.
I could script it out easily enough, but then I would still need to copy it in.
I could leave some sort of placeholder, and then dynamically update each definition w/ a correct SET statement. I could even write a database trigger that listens for CREATE and ALTER PROC statements and have it do this for me automagically.
But what I really want is just to grab the collection of parameters directly and be done with it. Is there any way?
EDIT: getting parameter metadata from the system tables is not enough. For example:
DECLARE #sql NVARCHAR(MAX)
SET #sql =
'SET #params = (SELECT '
+ STUFF((
SELECT ', '+name+' AS '+STUFF(name,1,1,'')
FROM sys.parameters WHERE object_id = ##PROCID
FOR XML PATH(''), TYPE).value('.','NVARCHAR(MAX)')
,1,2,'')
+ ' FOR XML PATH(''params''), TYPE)'
-- inline execute here....alas, there's isn't a mode for that
EXEC sp_executesql #sql, N'#params XML OUTPUT', #params OUTPUT
This, of course, generates an error, since I haven't declared parameters for the nested scope of sp_executesql. While I could script out the second param of sp_executesql, I can't assign them without explicitly assigning them one at a time by name.
Original question, rephrased: is there another flavor of sp_executesql that inherits variable "context" from the outer scope, that executes in the same frame, that inlines the command, rather than executing in a nested scope?
Why not use sys.dm_exec_sql_text(#sql_handle) to retrieve the SQL command, using sys.sysprocesses.sql_handle?