Errors with declared array/table variable values in SAP HanaDB SQL - ado.net

I am trying to add a declared variable to replace a hardcoded list of values in a "where in" clause.
Researching how Hana handles array variables it seems like I can do this by declaring an array and then either using a select directly on it or by unnesting it first into a table but I keep getting errors I can't resolve.
When I try it this way:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM :CODES_ARRAY); -- line 9 where error occurs
END;
I get this error message sql syntax error: incorrect syntax near ")": line 9 col 54 (at pos 239)
I can't figure out what the syntax error resolution is.
So then I tried inserting a declared table variable like this:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE = UNNEST(:CODES_ARRAY) AS ("code"); -- line 5 where error occurs
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM CODES_TABLE); -- I know : is missing here but when adding, the same error from previous block shows up
END;
and I get this error message: identifier must be declared: 1: line 5 col 38 (at pos 123)
As far as I can tell the array variable is declared as it should be so I don't know how to resolve the error.
I've read the SAP Hana SQL Reference documentation (for array/table variables, unnest function, etc.) over and over and it seems like I've got everything setup correctly but can't figure out these errors. I would like to be able to use both of these approaches at different times if possible (the "array variable to table variable" and the "array variable only" approaches)
I don't know exactly what is going on here, but one thing I notice that the two different error messages referenced in my post (see difference from errors in the first two code blocks) is that each error is occurring either immediately before the use of the variable with the : (in the case of the UNNEST) or immediately following the variable with the : (in the case of using in the SELECT * FROM in the query).
Because of that, I wondered if the issue is "upstream" at the Hana ADO.NET application query preparation and execution call level, but I ran a test and when I double checked the query string just before it is executed, it appears unchanged and the variables with : still look as they should, so at least as far as just before execution through Hana ADO.NET HanaCommand it looks correct - but once executing the query using HanaDataReader or HanaDataAdapter it returns the error messages referred to above. It may be a red herring to chase the problem from the Hana ADO.NET level but don't know what else to do.
Update
To further troubleshoot, I tried executing this code block below using hdbsql.exe -n XXX.XXX.XXX.XXX:30015 -u XXX -p XXX -m -I "c:\temp\test.sql" -c "#" and it works! So, the errors I see only show up when executing the same query through the Hana ADO.NET interface.
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(10) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE ("code" NVARCHAR(10)) = UNNEST(:CODES_ARRAY) AS ("code");
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM :CODES_TABLE); -- line 10 where error occurs when using Hana ADO.NET
END;
#
The above fails when through Hana ADO.NET with the error message: sql syntax error: incorrect syntax near ")": line 10 col 54 (at pos 325) but works when executed through hdbsql.
Update
The C# code that executes the query is fairly straight forward, but for completeness of troubleshooting effort I am including the interesting parts of our HanaHelper class. This code works successfully to execute 100s of queries a day without errors or problems. This is the first time where a variable of any type has been attempted to be declared or used in a query through this code and when the errors started showing up. As far as I can tell, the issue is tied to the use of the : when using the variable in the query.
public class HanaHelper
{
public HanaConnection objConn = null;
public HanaHelper(string ConnectionString)
{
try
{
objConn = new HanaConnection(ConnectionString);
}
catch (Exception e)
{
Console.WriteLine(#"Exception thrown by HanaConnection: {0}\n{1}", e.Message, e.InnerException);
}
}
public DataSet GetData(string strSQL)
{
using (HanaCommand objCmd = new HanaCommand(strSQL, objConn))
{
using (HanaDataAdapter objDA = new HanaDataAdapter(objCmd))
{
DataSet objDS = new DataSet();
try
{
objDA.Fill(objDS);
}
catch (Exception)
{
throw;
}
finally
{
// do something interesting regardless of success or failure
}
objConn.Close();
return objDS;
}
}
}
}
Any clue here why the same query works through hdbsql but fails when executing through Hana ADO.NET?
Update
I figured out how to use HanaSQLTrace in the C# code so that I can inspect the prepared query text and viola, the source of error messages becomes apparent, all occurrences of ":VARNAME" are replaced with "? " (a ? replaces the : and a space for each character in the variable name). I suppose it is trying to pre-substitute occurrences of : with a ? as if there were parameters to be substituted.
How can this behavior be disabled, or worked with, or worked around so that I can use variables in a query in Hana ADO.NET effectively?

Updated based on the OP feedback.
To refer to a variable (in order to access its value(s)) in SQLScript it's
necessary to put a colon : in front of the variable name.
The main issue, however, turns out to be the declaration of the CODES_TABLE table variable.
With HANA 2 SPS 4 the error message is
`SAP DBTech JDBC: [264]: invalid datatype: unknown type SYSTEM.TABLE: line 5 col 23`
This points to the declaration of the TABLE typed variable CODES_TABLE which lacks the definition of what columns should be in the table.
Adding this fixes the issue.
With these changes, your code should work:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE ("code" NVARCHAR(100)) = UNNEST(:CODES_ARRAY) AS ("code");
-- ^^^^^^^^^^^^^^^^^^^^^^
-- |
---------------------------------------+
SELECT
T0."ItemCode"
FROM
OITM T0
INNER JOIN OITW T1
ON T0."ItemCode" = T1."ItemCode"
WHERE
"WhsCode" IN (SELECT "code" FROM :CODES_TABLE);
-- ^
-- |
---------------------------------------+
END;
An alternative option to declare and assign the table variable is to not use the DECLARE command.
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
CODES_TABLE = UNNEST(:CODES_ARRAY) AS ("code");
SELECT
T0."ItemCode"
FROM
OITM T0
INNER JOIN OITW T1
ON T0."ItemCode" = T1."ItemCode"
WHERE
"WhsCode" IN (SELECT "code" FROM :CODES_TABLE);
END;

Related

Creating anonymous block on Postgres to update rows

I ran this anonymous block on Postgres and got this error:
DO
$$BEGIN
FOR diag_file IN
SELECT d_file.diagnostic_file_id AS diagnostic_file_id,
controller.controller_id AS controller_id
FROM diagnostic_file d_file
JOIN decoder_mapping d_mapping ON d_mapping.decoder_mapping_id = d_file.decoder_mapping_id
JOIN device_model d_model ON d_model.device_model_id = d_mapping.device_model_id
JOIN controller ON controller.device_model_id = d_model.device_model_id
WHERE d_file.controller_id IS NULL
LOOP
UPDATE diagnostic_file SET controller_id = diag_file.controller_id WHERE diagnostic_file.diagnostic_file_id = diag_file.diagnostic_file_id;
COMMIT;
END LOOP;
END;$$;
An It returns me this error:
WARNING: there is no transaction in progress
Query 1 OK: COMMIT
An It returns me this error:
WARNING: there is no transaction in progress
Not an error, a warning. You tried to commit, but there's no transaction.
Often this happens when you've turned on autocommit and then commit. It's warning you that the commit does nothing.
Your loop should not work at all in PostgreSQL 9.5, you can't begin nor end transactions inside PL/pgSQL in that version. THat is not introduced until PostgreSQL 11.
ERROR: cannot begin/end transactions in PL/pgSQL
HINT: Use a BEGIN block with an EXCEPTION clause instead.
CONTEXT: PL/pgSQL function inline_code_block line 11 at SQL statement
If you're writing a loop in SQL, you can probably do it easier and faster without the loop. In this case with an update from a sub-select.
update diagnostic_file set controller_id = diag_file.controller_id
from (
SELECT d_file.diagnostic_file_id AS diagnostic_file_id,
controller.controller_id AS controller_id
FROM diagnostic_file d_file
JOIN decoder_mapping d_mapping ON d_mapping.decoder_mapping_id = d_file.decoder_mapping_id
JOIN device_model d_model ON d_model.device_model_id = d_mapping.device_model_id
JOIN controller ON controller.device_model_id = d_model.device_model_id
WHERE d_file.controller_id IS NULL
) diag_file
WHERE diagnostic_file.diagnostic_file_id = diag_file.diagnostic_file_id
The update docs have many examples.

Don't understand 'CREATE VIEW' must be the first statement in a query batch

I am trying to create a View for temporary data I only need during the SQL run.
I get the error:
"'CREATE VIEW' must be the first statement in a query batch."
I searched on that and see it was asked about on Stack Overflow already here: 'CREATE VIEW' must be the first statement in a query batch
However, my SQL code looks like it should but I still get the error. First, I create a table called #StatusTable but that table wasn't working like I wanted. So I tried creating a View.
DECLARE #DOC INT;
DECLARE #StatusTable TABLE
(
status VARCHAR(8000)
)
INSERT INTO #StatusTable
Select Status = ( select CONCAT(ls.ItemText, '/', st.ItemText ) as Status from Inmate_Legal_Status ils
Left join jt_list_items ls on ls.ItemID = ils.LegalStatusID
Left join jt_list_items st on st.ItemID = ils.StatusTypeId
where ArrestNo = #doc and ils.RecordDeleted = 0
order by ils.CreatedTime desc FOR XML PATH('')
)
-- table
USE #StatusTable;
GO
create view MyView as
Select Status = ( select CONCAT(ls.ItemText, '/', st.ItemText ) as Status from Inmate_Legal_Status ils
Left join jt_list_items ls on ls.ItemID = ils.LegalStatusID
Left join jt_list_items st on st.ItemID = ils.StatusTypeId
where ArrestNo = #doc and ils.RecordDeleted = 0
GO
I get three errors:
Incorrect syntax near GO.
'CREATE VIEW' must be the first statement in a query batch
Incorrect syntax near #StatusTable

Expression for Lookup activity result when stored procedure returns scalar value

I'm using a Lookup activity that returns an scalar result and I want to use that result to build a dinamic query using a concat expression. However, I'm receiving an error complaining that my SQL query is not well formated. This is how I'm building the query:
#concat('SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId
WHERE v.CscadaEventId IN(', activity('LookupUnfinishedAlarms').output.firstRow, ') AND e.EndDate IS NOT NULL;')
I expect that to return a query like this:
SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId WHERE v.CscadaEventId IN(2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,2378594,2379357) AND e.EndDate IS NOT NULL;
I had see some examples where the lookup return multiple columns, and the right expression is activity('LookupUnfinishedAlarms').output.firstRow.myColumnName but what about when the lookup activity return an scalar value, as in my case?
This is the full error so far:
You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use
near
'\"output\":\"2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,237859'
at line
6,Source=Microsoft.DataTransfer.Runtime.GenericOdbcConnectors,''Type=System.Data.Odbc.OdbcException,Message=ERROR
[42000] [Microsoft][MariaDB] You have an error in your SQL syntax;
check the manual that corresponds to your MariaDB server version for
the right syntax to use near
'\"output\":\"2329390,2340616,2342078,2345857,2361240,2362088,2362574,2377062,237859'
at line 6,Source=MariaDBODBC_sb64.dll
Ok, just for the records, I found the solution. The expression must be:
#concat('SELECT v.CscadaEventId as EventId,
v.EndDate as EndDateUtc
FROM v_cscadaevents v
INNER JOIN cscadaevents e
ON e.cscadaeventId = v.CscadaEventId
WHERE v.CscadaEventId IN(', activity('LookupUnfinishedAlarms').output.firstRow.output, ') AND e.EndDate IS NOT NULL;')
So, the default column becomes output

How do I avoid syntax error in FOR loops in postgresql?

I'm using PostgreSQL 10.5 and I have the following SQL:
FOR temprow IN
SELECT o.objectid, o.nametag, cor.userseqno, cor.commseqno
FROM "commuserobjectrights" as cor
LEFT JOIN "object" as o ON cor.objectid = o.objectid
WHERE o.nametag LIKE 'commission.video_questions'
LOOP
INSERT INTO u commuserobjectrights (objectid, commseqno, userseqno, access)
VALUES (temprow.objectid, temprow.commseqno, temprow.userseqno, TRUE);
END LOOP;
which throws the following error:
ERROR: syntax error at or near "FOR" Position: 3
I have never used loops before but according the documentation, postgresql should have support for these types of loops. And yes, I have checked and double checked that all tables and column names are spelled correctly.
You can't use FOR loops outside of procedural code. But, in general Postgres (and SQL) is optimized to already do set based operations. So, you may phrase this as an INSERT INTO ... SELECT:
INSERT INTO commuserobjectrights (objectid, commseqno, userseqno, access)
SELECT o.objectid, o.nametag, cor.userseqno, TRUE
FROM "commuserobjectrights" as cor
LEFT JOIN "object" as o ON cor.objectid = o.objectid
WHERE o.nametag LIKE 'commission.video_questions';
FOR is procedural code, so you need to use DO or use it in stored code.
DO $$
DECLARE
temprow record ;
BEGIN
FOR temprow IN
SELECT o.objectid, o.nametag, cor.userseqno, cor.commseqno
FROM "commuserobjectrights" as cor
LEFT JOIN "object" as o ON cor.objectid = o.objectid
WHERE o.nametag LIKE 'commission.video_questions'
LOOP
INSERT INTO commuserobjectrights (objectid, commseqno, userseqno, access)
VALUES (temprow.objectid, temprow.commseqno, temprow.userseqno, TRUE);
END LOOP;
END;
$$;
This is not the most efficient way to do this task but for other tasks where you can't easily write SQL DO may be useful.

PostgreSQL ERROR: invalid input syntax for integer: "1e+06"

The full error message is:
ERROR: invalid input syntax for integer: "1e+06"
SQL state: 22P02
Context: In PL/R function sample
The query I'm using is:
WITH a as
(
SELECT a.tract_id_alias,
array_agg(a.pgid ORDER BY a.pgid) as pgids,
array_agg(a.sample_weight_geo ORDER BY a.pgid) as block_weights
FROM results_20161109.block_microdata_res_joined a
WHERE a.tract_id_alias in (66772, 66773, 66785, 66802, 66805, 66806, 66813)
AND a.bldg_count_res > 0
GROUP BY a.tract_id_alias
)
SELECT NULL::INTEGER agent_id,
a.tract_id_alias,
b.year,
unnest(shared.sample(a.pgids,
b.n_agents,
1 * b.year,
True,
a.block_weights)
) as pgid
FROM a
LEFT JOIN results_20161109.initial_agent_count_by_tract_res_11 b
ON a.tract_id_alias = b.tract_id_alias
ORDER BY b.year, a.tract_id_alias, pgid;
And the shared.sample function I'm using is:
CREATE OR REPLACE FUNCTION shared.sample(ids bigint[], size integer, seed integer DEFAULT 1, with_replacement boolean DEFAULT false, probabilities numeric[] DEFAULT NULL::numeric[])
RETURNS integer[] AS
$BODY$
set.seed(seed)
if (length(ids) == 1) {
s = rep(ids,size)
} else {
s = sample(ids,size, with_replacement,probabilities)
}
return(s)
$BODY$
LANGUAGE plr VOLATILE
COST 100;
ALTER FUNCTION shared.sample(bigint[], integer, integer, boolean, numeric[])
OWNER TO "server-superusers";
I'm pretty new to this stuff, so any help would be appreciated.
Not a problem of the function. Like the error messages says: The string '1e+06' cannot be cast to integer.
Obviously, the columns n_agents in your table results_20161109.initial_agent_count_by_tract_res_11 is not an integer column. Probably type text or varchar? (That info would help in your question.)
Either way, the assignment cast does not work for the target type integer. But it does for numeric:
Does not work:
SELECT '1e+06'::text::int; -- error as in question
Works:
SELECT '1e+06'::text::numeric::int;
If my assumptions hold, you can use this as stepping stone.
Replace b.n_agents in your query with b.n_agents::numeric::int.
It's your responsibility that numbers stay in integer range, or you get the next exception.
If that did not nail it, you need to look into function overloading:
Is there a way to disable function overloading in Postgres
And function type resolution:
PostgreSQL function call
The schema search path is relevant in many related cases, but you did schema-qualify all objects, so we can rule that out.
How does the search_path influence identifier resolution and the "current schema"
Your query generally looks good. I had a look and only found minor improvements:
SELECT NULL::int AS agent_id -- never omit the AS keyword for column alias
, a.tract_id_alias
, b.year
, s.pgid
FROM (
SELECT tract_id_alias
, array_agg(pgid) AS pgids
, array_agg(sample_weight_geo) AS block_weights
FROM ( -- use a subquery, cheaper than CTE
SELECT tract_id_alias
, pgid
, sample_weight_geo
FROM results_20161109.block_microdata_res_joined
WHERE tract_id_alias IN (66772, 66773, 66785, 66802, 66805, 66806, 66813)
AND bldg_count_res > 0
ORDER BY pgid -- sort once in a subquery. cheaper.
) sub
GROUP BY 1
) a
LEFT JOIN results_20161109.initial_agent_count_by_tract_res_11 b USING (tract_id_alias)
LEFT JOIN LATERAL
unnest(shared.sample(a.pgids
, b.n_agents
, b.year -- why "1 * b.year"?
, true
, a.block_weights)) s(pgid) ON true
ORDER BY b.year, a.tract_id_alias, s.pgid;