Expression for Lookup activity result when stored procedure returns scalar value - azure-data-factory

I'm using a Lookup activity that returns an scalar result and I want to use that result to build a dinamic query using a concat expression. However, I'm receiving an error complaining that my SQL query is not well formated. This is how I'm building the query:
#concat('SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId
WHERE v.CscadaEventId IN(', activity('LookupUnfinishedAlarms').output.firstRow, ') AND e.EndDate IS NOT NULL;')
I expect that to return a query like this:
SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId WHERE v.CscadaEventId IN(2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,2378594,2379357) AND e.EndDate IS NOT NULL;
I had see some examples where the lookup return multiple columns, and the right expression is activity('LookupUnfinishedAlarms').output.firstRow.myColumnName but what about when the lookup activity return an scalar value, as in my case?
This is the full error so far:
You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use
near
'\"output\":\"2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,237859'
at line
6,Source=Microsoft.DataTransfer.Runtime.GenericOdbcConnectors,''Type=System.Data.Odbc.OdbcException,Message=ERROR
[42000] [Microsoft][MariaDB] You have an error in your SQL syntax;
check the manual that corresponds to your MariaDB server version for
the right syntax to use near
'\"output\":\"2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,237859'
at line 6,Source=MariaDBODBC_sb64.dll

Ok, just for the records, I found the solution. The expression must be:
#concat('SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId
WHERE v.CscadaEventId IN(', activity('LookupUnfinishedAlarms').output.firstRow.output, ') AND e.EndDate IS NOT NULL;')
So, the default column becomes output

Related

Oracle sql missing expression

I have an Oracle SQL query and running the query, it gives ORA-00936: missing expression. When I hover over the red in Oracle Sql Developer, it says "Syntax Error. Partially Recognized Rules, railroad diagrams. I think there's something wrong with my Group By. I think Group by needs to have all query columns in it, but I know the last 3 are min/max/avg, so I don't think it makes sense to add those to the group by separately. What is the proper way to add them to the group by?
select
do.dcode,
ds.SERIALNO,
ds.BASECOMPONENTCODE,
TO_CHAR (strt.DLOCALECRTDT,'MON') as MON,--this looks like 13-OCT-15 05.19.03.000000000 PM
Max (do.METRICVALUE) as MaxCount,
min (do.METRICVALUE) as MinCount,
avg (do.METRICVALUE) as AvgCount
FROM
TECH_DWH.D_DIM_OUTPUTCOUNT_TBL do
join (
Select d1.dcode,d1.organizationid
from K_D_VW d1
where
d1.isactive='Y'
and d1.organizationid = 7500 -- company id
) d on d.dcode=do.dcode
left join
TECH_DWH.D_COMPSTAT_SERIAL_NO_MAP_TBL csm on csm.DCOMPONENTSTATEID = do.DCOMPONENTSTATEID
join TECH_D.D_DIM_SERIAL_NO_TBL ds on ds.serialnoid = csm.serialnoid
left join TECH_DWH.d_dim_medianumber_tbl dm on dm.DCOMPONENTSTATEID = csm.DCOMPONENTSTATEID
left join TECH_DWH.D_DEVICE_COMPSTATE_STRT_TBL strt on strt.DCOMPONENTSTATEID = csm.DCOMPONENTSTATEID
WHERE
instr(upper(ds.basecomponentcode),'PRINT')>0 AND --- return only device components
LENGTH(TRIM(TRANSLATE((do.METRICVALUE), ' +-.0123456789',' '))) is null -- test for only rows with numberic metrivalue's
AND do.dcode like '0046'
AND strt.COMPONENTSTATECODE like '%EP_DEVICE%'
and strt.DLOCALECRTDT >= to_date ( '30-12-2021', 'DD-MM-YYYY' )
and
Group by --red squiggly at "by", but error line number is following line
do.dcode,
ds.SERIALNO,
ds.BASECOMPONENTCODE,
TO_CHAR(strt.DLOCALECRTDT,'MON'),
do.METRICVALUE;
What's obvious, is
and
Group by --red squiggly at "by", but error line number is following line
do.dcode,
What's that AND doing alone? Remove it.

Errors with declared array/table variable values in SAP HanaDB SQL

I am trying to add a declared variable to replace a hardcoded list of values in a "where in" clause.
Researching how Hana handles array variables it seems like I can do this by declaring an array and then either using a select directly on it or by unnesting it first into a table but I keep getting errors I can't resolve.
When I try it this way:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM :CODES_ARRAY); -- line 9 where error occurs
END;
I get this error message sql syntax error: incorrect syntax near ")": line 9 col 54 (at pos 239)
I can't figure out what the syntax error resolution is.
So then I tried inserting a declared table variable like this:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE = UNNEST(:CODES_ARRAY) AS ("code"); -- line 5 where error occurs
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM CODES_TABLE); -- I know : is missing here but when adding, the same error from previous block shows up
END;
and I get this error message: identifier must be declared: 1: line 5 col 38 (at pos 123)
As far as I can tell the array variable is declared as it should be so I don't know how to resolve the error.
I've read the SAP Hana SQL Reference documentation (for array/table variables, unnest function, etc.) over and over and it seems like I've got everything setup correctly but can't figure out these errors. I would like to be able to use both of these approaches at different times if possible (the "array variable to table variable" and the "array variable only" approaches)
I don't know exactly what is going on here, but one thing I notice that the two different error messages referenced in my post (see difference from errors in the first two code blocks) is that each error is occurring either immediately before the use of the variable with the : (in the case of the UNNEST) or immediately following the variable with the : (in the case of using in the SELECT * FROM in the query).
Because of that, I wondered if the issue is "upstream" at the Hana ADO.NET application query preparation and execution call level, but I ran a test and when I double checked the query string just before it is executed, it appears unchanged and the variables with : still look as they should, so at least as far as just before execution through Hana ADO.NET HanaCommand it looks correct - but once executing the query using HanaDataReader or HanaDataAdapter it returns the error messages referred to above. It may be a red herring to chase the problem from the Hana ADO.NET level but don't know what else to do.
Update
To further troubleshoot, I tried executing this code block below using hdbsql.exe -n XXX.XXX.XXX.XXX:30015 -u XXX -p XXX -m -I "c:\temp\test.sql" -c "#" and it works! So, the errors I see only show up when executing the same query through the Hana ADO.NET interface.
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(10) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE ("code" NVARCHAR(10)) = UNNEST(:CODES_ARRAY) AS ("code");
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM :CODES_TABLE); -- line 10 where error occurs when using Hana ADO.NET
END;
#
The above fails when through Hana ADO.NET with the error message: sql syntax error: incorrect syntax near ")": line 10 col 54 (at pos 325) but works when executed through hdbsql.
Update
The C# code that executes the query is fairly straight forward, but for completeness of troubleshooting effort I am including the interesting parts of our HanaHelper class. This code works successfully to execute 100s of queries a day without errors or problems. This is the first time where a variable of any type has been attempted to be declared or used in a query through this code and when the errors started showing up. As far as I can tell, the issue is tied to the use of the : when using the variable in the query.
public class HanaHelper
{
public HanaConnection objConn = null;
public HanaHelper(string ConnectionString)
{
try
{
objConn = new HanaConnection(ConnectionString);
}
catch (Exception e)
{
Console.WriteLine(#"Exception thrown by HanaConnection: {0}\n{1}", e.Message, e.InnerException);
}
}
public DataSet GetData(string strSQL)
{
using (HanaCommand objCmd = new HanaCommand(strSQL, objConn))
{
using (HanaDataAdapter objDA = new HanaDataAdapter(objCmd))
{
DataSet objDS = new DataSet();
try
{
objDA.Fill(objDS);
}
catch (Exception)
{
throw;
}
finally
{
// do something interesting regardless of success or failure
}
objConn.Close();
return objDS;
}
}
}
}
Any clue here why the same query works through hdbsql but fails when executing through Hana ADO.NET?
Update
I figured out how to use HanaSQLTrace in the C# code so that I can inspect the prepared query text and viola, the source of error messages becomes apparent, all occurrences of ":VARNAME" are replaced with "? " (a ? replaces the : and a space for each character in the variable name). I suppose it is trying to pre-substitute occurrences of : with a ? as if there were parameters to be substituted.
How can this behavior be disabled, or worked with, or worked around so that I can use variables in a query in Hana ADO.NET effectively?
Updated based on the OP feedback.
To refer to a variable (in order to access its value(s)) in SQLScript it's
necessary to put a colon : in front of the variable name.
The main issue, however, turns out to be the declaration of the CODES_TABLE table variable.
With HANA 2 SPS 4 the error message is
`SAP DBTech JDBC: [264]: invalid datatype: unknown type SYSTEM.TABLE: line 5 col 23`
This points to the declaration of the TABLE typed variable CODES_TABLE which lacks the definition of what columns should be in the table.
Adding this fixes the issue.
With these changes, your code should work:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE ("code" NVARCHAR(100)) = UNNEST(:CODES_ARRAY) AS ("code");
-- ^^^^^^^^^^^^^^^^^^^^^^
-- |
---------------------------------------+
SELECT
T0."ItemCode"
FROM
OITM T0
INNER JOIN OITW T1
ON T0."ItemCode" = T1."ItemCode"
WHERE
"WhsCode" IN (SELECT "code" FROM :CODES_TABLE);
-- ^
-- |
---------------------------------------+
END;
An alternative option to declare and assign the table variable is to not use the DECLARE command.
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
CODES_TABLE = UNNEST(:CODES_ARRAY) AS ("code");
SELECT
T0."ItemCode"
FROM
OITM T0
INNER JOIN OITW T1
ON T0."ItemCode" = T1."ItemCode"
WHERE
"WhsCode" IN (SELECT "code" FROM :CODES_TABLE);
END;

PostgreSQL: Why does this join query throw a syntax error

I don't understand, why the following query produced an error. Whereas in other dbms (eg mysql) this sort of query is absolutely legal.
ERROR: Syntax error at the end of input
LINE 1: SELECT * FROM "addr_country" AS c JOIN "addr_state" AS s
^
SQL state: 42601
Character: 59
Adding an on-clause to the query makes it working:
SELECT * FROM "addr_country" AS c JOIN "addr_state" AS s
ON c."id" = s."country" AND s.id = 10
Due to this answer, MySQL makes a cross join out of a join w/o conditions

Orientdb sql , select from multiple tables

dont run "select from multiple tables" commands in orientdb 3.0 (centos)
i tested like this following commands
SELECT *
FROM Employee A, City B
WHERE A.city = B.id
Error Codes ; "Error parsing query: ^ Encountered " "SELECT "" at line 1, column 1. Was expecting one of: ... ... ";" ... DB name="
The most important difference between OrientDB and a Relational Database is that relationships are represented by LINKS instead of JOINs.
For this reason, the classic JOIN syntax is not supported. OrientDB uses the "dot (.) notation" to navigate LINKS. Example 1 : In SQL you might create a join such as:
SELECT *
FROM Employee A, City B
WHERE A.city = B.id
AND B.name = 'Rome'
In OrientDB an equivalent operation would be:
SELECT * FROM Employee WHERE city.name = 'Rome'
For more information: https://orientdb.com/docs/2.2.x/SQL.html#joins
Hope it helps
Regards

Ecto fragment without parentheses

My Postgres table structure:
id | stuff
--------+------------------------------------------------------------
123 | {"type1": {"ref": "ref_1", "...": "..."}, "type2": {"ref": "ref_1", "...": "..."}}
I'd like to query by ref in each type of stuff, I have a working SQL query for this:
SELECT * FROM "stuff" AS c0 CROSS JOIN jsonb_each(c0."stuff") AS f1 WHERE value->>'ref' = 'ref_1';
But using this Ecto query:
(from c in Stuff,
join: fragment("jsonb_each(?)", c.stuff),
where: fragment("value->>'ref' = ?", ^ref)
)
|> Repo.all
I get a Postgres syntax error in the CROSS JOIN statement:
** (Postgrex.Error) ERROR 42601 (syntax_error): syntax error at or near ")"
Inspecting the generated query:
[debug] QUERY ERROR source="stuff" db=0.3ms
SELECT ... FROM "stuff" AS c0 CROSS JOIN (jsonb_each(c0."stuff")) AS f1 WHERE (value->>'ref' = $1) ["ref_1"]
The above works when I remove the outer parentheses around (jsonb_each(c0."stuff")).
Is there a way to have the fragment generate the query without these parentheses or do I have to redesign the query?
Thanks
It seems that Ecto always wraps the join clause in parentheses, which is usually fine. The times when its not unfortunately includes certain calls like the jsonb_each above. There is a wiki here for such cases: The parentheses rules of PostgreSQL, is there a summarized guide?
The linked raw sql example had a much less upvoted answer that seems to work well with both making this query and getting back the expected struct.
sql = "SELECT * FROM "stuff" AS c0 CROSS JOIN jsonb_each(c0."stuff") AS f1 WHERE value->>'ref' = 'ref_1';"
result = JsonbTest.Repo.query!(sql)
Enum.map(result.rows, &JsonbTest.Repo.load(StuffStruct, {result.columns, &1}))
This is a bug in ecto, has been fixed here https://github.com/elixir-ecto/ecto/issues/2537