Not sure if this is a Supabase issue or a psycopg2 issue honestly and would love some help debugging.
I have the following code:
args = [('HaurGlass','60000','2022-10-20T21:15:39.751Z','10130506261','ac76e8db-ace0-40df-b6fa-f470641805e9','ad43639e-f66e-49d5-8fe8-d1ce5cd26193','{}')]
statement = ('''
INSERT INTO %s (%s) VALUES %s ON CONFLICT (company_id, crm_id)
DO UPDATE SET (%s)=(%s) RETURNING crm_id, id''')
statement = cur.mogrify(statement,
(AsIs(db_table), AsIs(','.join(keys)),
AsIs("%s"), AsIs(','.join(update_keys)),
AsIs(','.join(excluded_keys))))
output = execute_values(cur, statement, args, fetch=True)
The weird thing is that if args is <=100 rows in length, this query works without any problems. As soon as I increase the length of args to 101 rows or more, my Postgres logs show:
INSERT INTO licenses (name,value,subscription_end,crm_id,company_id,csm_id,custom_data) VALUES ('HaurGlass','60000','2022-10-20T21:15:39.751Z','10130506261','ac76e8db-ace0-40df-b6fa-f470641805e9','ad43639e-f66e-49d5-8fe8-d1ce5cd26193','{}')...
which would be good, except that it's immediately followed by:
INSERT INTO licenses (name,value,subscription_end,crm_id,company_id,csm_id,custom_data) VALUES ('HaurGlass','60000','2022-10-20T21:15:39.751Z','10130506261','ac76e8db-ace0-40df-b6fa-f470641805e9',NULL,'{}'),...
I've also confirmed that the number of records in the second "NULLifying" query is exactly equal to len(args)-100.
Any idea what is going on?
OK so it turns out I was missing the page_size parameter. All I had to do was:
output = execute_values(cur, statement, args, fetch=True, page_size=len(args))
Related
I have a table fepu00 and trigger on it.
The part of code causing tha problem looks like follows:
if (old_record.potime = NEW.putime --new AP (read in statement above)
or old_record.pupisb::NUMERIC != NEW.pupisb::NUMERIC) then
insert into dl356_table
values (old_record.poid, old_record.poidma, old_record.ponmaf,
old_record.adstr, old_record.adpsc, old_record.adcit, old_record.adidze,
NEW.pupisb::numeric,
case when old_record.potime = NEW.putime then '1' else '2' end,
NEW.putime);
end if;
The query I'm executing is really simple:
update fepu00 set pudas = ? where puid = ?
It works, but only for some quantity of times. Then it throws:
ERROR: type of parameter 25 (numeric) does not match that when preparing the plan (text)
Where: PL/pgSQL function dl356_trigger() line 75 at IF
Number of updated records varies from a few to a few hundreds.
When I run the same query (with the same parameters) again, it works properly until the next fail.
Thanks for any suggestions.
I am trying to pipe a webscraped pandas dataframe into a MySql table with mysql.connector but I can't seem to link df values to the %s variable. The connection is good (I can add individual rows) but it just returns errors when I replace the value witht he %s.
cnx = mysql.connector.connect(host = 'ip', user = 'user', passwd = 'pass', database = 'db')
cursor = cnx.cursor()
insert_df = ("""INSERT INTO table"
"(page_1, date_1, record_1, task_1)"
"VALUES ('%s','%s','%s','%s')""")
cursor.executemany(insert_df, df)
cnx.commit()
cnx.close()
This returns "ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()."
If I add any additional oiperations it returns "ProgrammingError: Parameters for query must be an Iterable."
I am very new to this so any help is appreciated
Work around for me was to redo my whole process. I ran sqlalchemy, all the documentation makes this very easy. message if you want the code I used.
I am trying to add a declared variable to replace a hardcoded list of values in a "where in" clause.
Researching how Hana handles array variables it seems like I can do this by declaring an array and then either using a select directly on it or by unnesting it first into a table but I keep getting errors I can't resolve.
When I try it this way:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM :CODES_ARRAY); -- line 9 where error occurs
END;
I get this error message sql syntax error: incorrect syntax near ")": line 9 col 54 (at pos 239)
I can't figure out what the syntax error resolution is.
So then I tried inserting a declared table variable like this:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE = UNNEST(:CODES_ARRAY) AS ("code"); -- line 5 where error occurs
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM CODES_TABLE); -- I know : is missing here but when adding, the same error from previous block shows up
END;
and I get this error message: identifier must be declared: 1: line 5 col 38 (at pos 123)
As far as I can tell the array variable is declared as it should be so I don't know how to resolve the error.
I've read the SAP Hana SQL Reference documentation (for array/table variables, unnest function, etc.) over and over and it seems like I've got everything setup correctly but can't figure out these errors. I would like to be able to use both of these approaches at different times if possible (the "array variable to table variable" and the "array variable only" approaches)
I don't know exactly what is going on here, but one thing I notice that the two different error messages referenced in my post (see difference from errors in the first two code blocks) is that each error is occurring either immediately before the use of the variable with the : (in the case of the UNNEST) or immediately following the variable with the : (in the case of using in the SELECT * FROM in the query).
Because of that, I wondered if the issue is "upstream" at the Hana ADO.NET application query preparation and execution call level, but I ran a test and when I double checked the query string just before it is executed, it appears unchanged and the variables with : still look as they should, so at least as far as just before execution through Hana ADO.NET HanaCommand it looks correct - but once executing the query using HanaDataReader or HanaDataAdapter it returns the error messages referred to above. It may be a red herring to chase the problem from the Hana ADO.NET level but don't know what else to do.
Update
To further troubleshoot, I tried executing this code block below using hdbsql.exe -n XXX.XXX.XXX.XXX:30015 -u XXX -p XXX -m -I "c:\temp\test.sql" -c "#" and it works! So, the errors I see only show up when executing the same query through the Hana ADO.NET interface.
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(10) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE ("code" NVARCHAR(10)) = UNNEST(:CODES_ARRAY) AS ("code");
SELECT T0."ItemCode"
FROM OITM T0
INNER JOIN OITW T1 ON T0."ItemCode" = T1."ItemCode"
WHERE "WhsCode" IN (SELECT "code" FROM :CODES_TABLE); -- line 10 where error occurs when using Hana ADO.NET
END;
#
The above fails when through Hana ADO.NET with the error message: sql syntax error: incorrect syntax near ")": line 10 col 54 (at pos 325) but works when executed through hdbsql.
Update
The C# code that executes the query is fairly straight forward, but for completeness of troubleshooting effort I am including the interesting parts of our HanaHelper class. This code works successfully to execute 100s of queries a day without errors or problems. This is the first time where a variable of any type has been attempted to be declared or used in a query through this code and when the errors started showing up. As far as I can tell, the issue is tied to the use of the : when using the variable in the query.
public class HanaHelper
{
public HanaConnection objConn = null;
public HanaHelper(string ConnectionString)
{
try
{
objConn = new HanaConnection(ConnectionString);
}
catch (Exception e)
{
Console.WriteLine(#"Exception thrown by HanaConnection: {0}\n{1}", e.Message, e.InnerException);
}
}
public DataSet GetData(string strSQL)
{
using (HanaCommand objCmd = new HanaCommand(strSQL, objConn))
{
using (HanaDataAdapter objDA = new HanaDataAdapter(objCmd))
{
DataSet objDS = new DataSet();
try
{
objDA.Fill(objDS);
}
catch (Exception)
{
throw;
}
finally
{
// do something interesting regardless of success or failure
}
objConn.Close();
return objDS;
}
}
}
}
Any clue here why the same query works through hdbsql but fails when executing through Hana ADO.NET?
Update
I figured out how to use HanaSQLTrace in the C# code so that I can inspect the prepared query text and viola, the source of error messages becomes apparent, all occurrences of ":VARNAME" are replaced with "? " (a ? replaces the : and a space for each character in the variable name). I suppose it is trying to pre-substitute occurrences of : with a ? as if there were parameters to be substituted.
How can this behavior be disabled, or worked with, or worked around so that I can use variables in a query in Hana ADO.NET effectively?
Updated based on the OP feedback.
To refer to a variable (in order to access its value(s)) in SQLScript it's
necessary to put a colon : in front of the variable name.
The main issue, however, turns out to be the declaration of the CODES_TABLE table variable.
With HANA 2 SPS 4 the error message is
`SAP DBTech JDBC: [264]: invalid datatype: unknown type SYSTEM.TABLE: line 5 col 23`
This points to the declaration of the TABLE typed variable CODES_TABLE which lacks the definition of what columns should be in the table.
Adding this fixes the issue.
With these changes, your code should work:
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
DECLARE CODES_TABLE TABLE ("code" NVARCHAR(100)) = UNNEST(:CODES_ARRAY) AS ("code");
-- ^^^^^^^^^^^^^^^^^^^^^^
-- |
---------------------------------------+
SELECT
T0."ItemCode"
FROM
OITM T0
INNER JOIN OITW T1
ON T0."ItemCode" = T1."ItemCode"
WHERE
"WhsCode" IN (SELECT "code" FROM :CODES_TABLE);
-- ^
-- |
---------------------------------------+
END;
An alternative option to declare and assign the table variable is to not use the DECLARE command.
DO
BEGIN
DECLARE CODES_ARRAY NVARCHAR(100) ARRAY = ARRAY('01','02','03','04');
CODES_TABLE = UNNEST(:CODES_ARRAY) AS ("code");
SELECT
T0."ItemCode"
FROM
OITM T0
INNER JOIN OITW T1
ON T0."ItemCode" = T1."ItemCode"
WHERE
"WhsCode" IN (SELECT "code" FROM :CODES_TABLE);
END;
I have a SQL query which takes user inputs hence security flaw is present.
The existing query is:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID, 'I' ||
LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
, ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, Other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (" +TriggerType+ ");
I changed it to accept TriggerType from setString like:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID,
'I' || LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (?);
Setting trigger type as below:
if (StringUtils.isNotBlank(request.getTriggerType())) {
preparedStatement.setString(1, triggerType != null ? triggerType.toString() : "");
}
Getting error as
Caused by: com.ibm.db2.jcc.am.SqlDataException: DB2 SQL Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null, DRIVER=4.19.26
The -302 SQLCODE indicates a conversion error of some sort.
SQLSTATE 22001 narrows that down a bit by telling us that you are trying to force a big string into a small variable. Given the limited information in your question, I am guessing it is the XMLCAST that is the culprit.
DB2 won't jam 30 pounds of crap into a 4 pound bag so to speak, it gives you an error. Maybe giving XML some extra room in the cast might be a help. If you need to make sure it ends up being only 4 characters long, you could explicitly do a LEFT(XMLCAST( ... AS VARCHAR(64)), 4). That way the XMLCAST has the space it needs, but you cut it back to fit your variable on the fetch.
The other thing could be that the variable being passed to the parameter marker is too long. DB2 will guess the type and length based on the length of ALERT_TYPE. Note that you can only pass a single value through a parameter marker. If you pass a comma separated list, it will not behave as expected (unless you expect ALERT_TYPE to also contain a comma separated list). If you are getting the comma separated list from a table, you can use a sub-select instead.
Wrong IN predicate use with a parameter.
Do not expect that IN ('AAAA, M250, ABCD') (as you try to do passing a comma-separated string as a single parameter) works as IN ('AAAA', 'M250', 'ABCD') (as you need). These predicates are not equivalent.
You need some "string tokenizer", if you want to pass such a comma-separated string like below.
select t.*
from
(
select XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE
from table(values xmlparse(document '<alertTypeConfig><biqCode>M250, really big code</biqCode></alertTypeConfig>')) IAC(INTL_ALERT_TYPE_CONFIG)
) t
--WHERE ALERT_TYPE IN ('AAAA, M250, ABCD')
join xmltable('for $id in tokenize($s, ",\s?") return <i>{string($id)}</i>'
passing cast('AAA, M250 , ABCD' as varchar(200)) as "s"
columns token varchar(200) path '.') x on x.token=t.ALERT_TYPE
;
Run the statement as is. Then you may uncomment the string with WHERE clause and comment out the rest to see what you try to do.
P.S.:
The error you get is probably because you don't specify the data type of the parameter (you don't use something like IN (cast(? as varchar(xxx))), and db2 compiler assumes that its length is equal to the length of the ALERT_TYPE expression (4 bytes).
I'm working in a bank so I had to adjust the column names and information in the query to fit the external web, so if there're any weird mistakes know it is somewhat fine.
I'm trying to use the CASE clause to display data from a different table, I know this is a workaround but due to certain circumstances I'm obligated to use it, plus it is becoming interesting to figure out if there's an actual solution.
The error I'm receiving for the following query is:
"ERROR [21000] [IBM][CLI Driver][DB2] SQL0811N The result of a scalar
fullselect, SELECT INTO statement, or VALUES INTO statement is more
than one row."
select bank_num, branch_num, account_num, client_id,
CASE
WHEN exists(
select *
from bank.services BS
where ACCS.client_id= BS.sifrur_lakoach
)
THEN (select username from bank.services BS where BS.client_id = ACCS.client_id)
ELSE 'NONE'
END username_new
from bank.accounts accs
where bank_num = 431 and branch_num = 170
EDIT:
AFAIK we're using DB2 v9.7:
DSN11015 - DB21085I Instance "DB2" uses "64" bits and DB2 code release "SQL09075" with
level identifier "08060107".
Informational tokens are "DB2 v9.7.500.702", "s111017", "IP23287", and Fix Pack "5".
Use listagg function to include all results.
select bank_num, branch_num, account_num, client_id,
CASE
WHEN exists(
select *
from bank.services BS
where ACCS.client_id= BS.sifrur_lakoach
)
THEN (select LISTAGG(username, ', ') from bank.services BS
where BS.client_id = ACCS.client_id)
ELSE 'NONE'
END username_new
from bank.accounts accs
where bank_num = 431 and branch_num = 170