I want to execute following raw SQL with Korma:
k/exec-raw
["INSERT INTO events ?, VALUES ? ON CONFLICT (id) DO UPDATE SET title = EXCLUDED.title;" [keys values]]
with params equal to:
keys (str "(" (keys->str res) ")")
values (str "(" (serialize (merge res) ", ") ")" )
The both evaluate to correct strings and work in repl.
But in runtime i have following error in psql console:
ERROR: syntax error at or near "$1" at character 20
STATEMENT: INSERT INTO events $1, VALUES $2 ON CONFLICT (id) DO UPDATE SET title = EXCLUDED.title
Can't figure out what's the problem. Anybody have ever done insert with Korma?
PostgreSQL 9.5 + Korma 0.4.2
Only values can be used as parameters in a prepared statement. So to make it work:
INSERT INTO events (column) VALUES (?)
If you want to work it like you do you have to prepare the sql string yourself and make sure you don't have an sql injection.
Please see the manual
Related
I'm getting SQL exception while executing the query which contains
split_part() method as split_part(value::TEXT,':', 1).
String queryStr = " select split_part(value::TEXT,':', 1) from table";
Query query = entityManager.createNativeQuery(queryStr);
List results = query.getResultList();
ERROR 2020-02-10 14:54:37,926 [http-nio-7070-exec-1] 142 - ERROR: syntax error at or near ":"
Position: 855
Your obfuscation layer probably chokes on the :: operator. Use the cast() operator instead:
String queryStr = " select split_part(cast(value as text),':', 1) from table";
But why do you think you need the cast to begin with? If you are storing : characters that column, it is most probably a text (or varchar) column anyway and you don't need a cast at all.
I have a SQL query which takes user inputs hence security flaw is present.
The existing query is:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID, 'I' ||
LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
, ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, Other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (" +TriggerType+ ");
I changed it to accept TriggerType from setString like:
SELECT BUS_NM, STR_ADDR_1, CITY_NM, STATE_CD, POSTAL_CD, COUNTRY_CD,
BUS_PHONE_NB,PEG_ACCOUNT_ID, GDN_ALERT_ID, GBIN, GDN_MON_REF_NB,
ALERT_DT, ALERT_TYPE, ALERT_DESC,ALERT_PRIORITY
FROM ( SELECT A.BUS_NM, AE.STR_ADDR_1, A.CITY_NM, A.STATE_CD, A.POSTAL_CD,
CC.COUNTRY_CD, A.BUS_PHONE_NB, A.PEG_ACCOUNT_ID,
'I' || LPAD(INTL_ALERT_DTL_ID, 9,'0') GDN_ALERT_ID,
LPAD(IA.GBIN, 9,'0') GBIN, IA.GDN_MON_REF_NB,
DATE(IAD.ALERT_TS) ALERT_DT,
XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing
IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE,
ROW_NUMBER() OVER () AS "RN"
FROM ACCOUNT A, other tables
WHERE IA.GDN_MON_REF_NB = '100'
AND A.PEG_ACCOUNT_ID = IAAR.PEG_ACCOUNT_ID
AND CC.COUNTRY_CD = A.COUNTRY_ISO3_CD
ORDER BY IA.INTL_ALERT_ID ASC )
WHERE ALERT_TYPE IN (?);
Setting trigger type as below:
if (StringUtils.isNotBlank(request.getTriggerType())) {
preparedStatement.setString(1, triggerType != null ? triggerType.toString() : "");
}
Getting error as
Caused by: com.ibm.db2.jcc.am.SqlDataException: DB2 SQL Error: SQLCODE=-302, SQLSTATE=22001, SQLERRMC=null, DRIVER=4.19.26
The -302 SQLCODE indicates a conversion error of some sort.
SQLSTATE 22001 narrows that down a bit by telling us that you are trying to force a big string into a small variable. Given the limited information in your question, I am guessing it is the XMLCAST that is the culprit.
DB2 won't jam 30 pounds of crap into a 4 pound bag so to speak, it gives you an error. Maybe giving XML some extra room in the cast might be a help. If you need to make sure it ends up being only 4 characters long, you could explicitly do a LEFT(XMLCAST( ... AS VARCHAR(64)), 4). That way the XMLCAST has the space it needs, but you cut it back to fit your variable on the fetch.
The other thing could be that the variable being passed to the parameter marker is too long. DB2 will guess the type and length based on the length of ALERT_TYPE. Note that you can only pass a single value through a parameter marker. If you pass a comma separated list, it will not behave as expected (unless you expect ALERT_TYPE to also contain a comma separated list). If you are getting the comma separated list from a table, you can use a sub-select instead.
Wrong IN predicate use with a parameter.
Do not expect that IN ('AAAA, M250, ABCD') (as you try to do passing a comma-separated string as a single parameter) works as IN ('AAAA', 'M250', 'ABCD') (as you need). These predicates are not equivalent.
You need some "string tokenizer", if you want to pass such a comma-separated string like below.
select t.*
from
(
select XMLCAST(XMLQUERY('$A/alertTypeConfig/biqCode/text()' passing IAC.INTL_ALERT_TYPE_CONFIG as "A") AS CHAR(4)) ALERT_TYPE
from table(values xmlparse(document '<alertTypeConfig><biqCode>M250, really big code</biqCode></alertTypeConfig>')) IAC(INTL_ALERT_TYPE_CONFIG)
) t
--WHERE ALERT_TYPE IN ('AAAA, M250, ABCD')
join xmltable('for $id in tokenize($s, ",\s?") return <i>{string($id)}</i>'
passing cast('AAA, M250 , ABCD' as varchar(200)) as "s"
columns token varchar(200) path '.') x on x.token=t.ALERT_TYPE
;
Run the statement as is. Then you may uncomment the string with WHERE clause and comment out the rest to see what you try to do.
P.S.:
The error you get is probably because you don't specify the data type of the parameter (you don't use something like IN (cast(? as varchar(xxx))), and db2 compiler assumes that its length is equal to the length of the ALERT_TYPE expression (4 bytes).
I'm changing queries from an Oracle Database to PostgreSQL, and in this query I am getting this error:
ERROR: syntax error at or near "SET"
the query is:
{call UPDATE alarm_instance SET last_update_time=default, wait_expire_time=null, core_number=nextval(SEQ_ALRM_NUMBR)
where wait_time <= current_date RETURNING alarm_instance_id bulk collect INTO ?}
I am using JDBC to connect to the database and here is the call code
try (CallableStatement cs = super.prepareCall_(query)) {
cs.registerOutParameter(1, Types.ARRAY);
cs.execute();
...
I have taken a long look at Postgres documentation and cannot find what is wrong and didn't find any answer to this specific situation
An UPDATE statement can't be executed with a CallableStatement. A CallableStatement is essentially only intended to call stored procedures. In case of Oracle that includes anonymous PL/SQL blocks.
And bulk collect is invalid in Postgres to begin with.
It seems you want something like this:
String sql =
"UPDATE alarm_instance " +
" SET last_update_time=default, " +
" wait_expire_time=null, "
" core_number=nextval('SEQ_ALRM_NUMBR') " +
" where wait_time <= current_date RETURNING alarm_instance_id";
Statement stmt = connection.createStatement();
stmt.execute(sql);
int rowsUpdated = stmt.getUpdateCount();
ResultSet rs = stmt.getResultSet();
while (rs.next() {
// do something with the returned IDs
}
In python 3+, I want to insert values from a dictionary (or pandas dataframe) into a database. I have opted for psycopg2 with a postgres database.
The problems is that I cannot figure out the proper way to do this. I can easily concatenate a SQL string to execute, but the psycopg2 documentation explicitly warns against this. Ideally I wanted to do something like this:
cur.execute("INSERT INTO table VALUES (%s);", dict_data)
and hoped that the execute could figure out that the keys of the dict matches the columns in the table. This did not work. From the examples of the psycopg2 documentation I got to this approach
cur.execute("INSERT INTO table (" + ", ".join(dict_data.keys()) + ") VALUES (" + ", ".join(["%s" for pair in dict_data]) + ");", dict_data)
from which I get a
TypeError: 'dict' object does not support indexing
What is the most phytonic way of inserting a dictionary into a table with matching column names?
Two solutions:
d = {'k1': 'v1', 'k2': 'v2'}
insert = 'insert into table (%s) values %s'
l = [(c, v) for c, v in d.items()]
columns = ','.join([t[0] for t in l])
values = tuple([t[1] for t in l])
cursor = conn.cursor()
print cursor.mogrify(insert, ([AsIs(columns)] + [values]))
keys = d.keys()
columns = ','.join(keys)
values = ','.join(['%({})s'.format(k) for k in keys])
insert = 'insert into table ({0}) values ({1})'.format(columns, values)
print cursor.mogrify(insert, d)
Output:
insert into table (k2,k1) values ('v2', 'v1')
insert into table (k2,k1) values ('v2','v1')
I sometimes run into this issue, especially with respect to JSON data, which I naturally want to deal with as a dict. Very similar. . .But maybe a little more readable?
def do_insert(rec: dict):
cols = rec.keys()
cols_str = ','.join(cols)
vals = [ rec[k] for k in cols ]
vals_str = ','.join( ['%s' for i in range(len(vals))] )
sql_str = """INSERT INTO some_table ({}) VALUES ({})""".format(cols_str, vals_str)
cur.execute(sql_str, vals)
I typically call this type of thing from inside an iterator, and usually wrapped in a try/except. Either the cursor (cur) is already defined in an outer scope or one can amend the function signature and pass a cursor instance in. I rarely insert just a single row. . .And like the other solutions, this allows for missing cols/values provided the underlying schema allows for it too. As long as the dict underlying the keys view is not modified as the insert is taking place, there's no need to specify keys by name as the values will be ordered as they are in the keys view.
[Suggested answer/workaround - better answers are appreciated!]
After some trial/error I got the following to work:
sql = "INSERT INTO table (" + ", ".join(dict_data.keys()) + ") VALUES (" + ", ".join(["%("+k+")s" for k in dict_data]) + ");"
This gives the sql string
"INSERT INTO table (k1, k2, ... , kn) VALUES (%(k1)s, %(k2)s, ... , %(kn)s);"
which may be executed by
with psycopg2.connect(database='deepenergy') as con:
with con.cursor() as cur:
cur.execute(sql, dict_data)
Post/cons?
using %(name)s placeholders may solve the problem:
dict_data = {'key1':val1, 'key2':val2}
cur.execute("""INSERT INTO table (field1, field2)
VALUES (%(key1)s, %(key2)s);""",
dict_data)
you can find the usage in psycopg2 doc Passing parameters to SQL queries
Here is another solution inserting a dictionary directly
Product Model (has the following database columns)
name
description
price
image
digital - (defaults to False)
quantity
created_at - (defaults to current date)
Solution:
data = {
"name": "product_name",
"description": "product_description",
"price": 1,
"image": "https",
"quantity": 2,
}
cur = conn.cursor()
cur.execute(
"INSERT INTO products (name,description,price,image,quantity) "
"VALUES(%(name)s, %(description)s, %(price)s, %(image)s, %(quantity)s)", data
)
conn.commit()
conn.close()
Note: The columns to be inserted is specified on the execute statement .. INTO products (column names to be filled) VALUES ..., data <- the dictionary (should be the same **ORDER** of keys)
I'm doing a little application for the work and it includes a form. When the person using the form doesn't put a value in a data type "numeric" (lets say a PIN number) or even a date in one of my "date" Data Type fields (lets say his anniverssary), it returns me with the errors as follow:
Error Type – Type d'erreur
_Microsoft OLE DB Provider for ODBC Drivers (0x80004005)_
ERROR: invalid input syntax for type numeric: ""; Error while executing the query
Error Type – Type d'erreur
_Microsoft OLE DB Provider for ODBC Drivers (0x80004005)_
ERROR: syntax error at or near ")"; Error while executing the query
So it seems that when the person using the form enters nothing, it returns the string "empty" that is "".
Why can't the numeric type and data type read that as a NULL entry? These fields are not mandatory and so I need to have them be sometimes blank.
How can I make it possible? Is their a way to keep using "numeric" and "date" type so that when the user enters nothing in those fields, the table fills with a blank case insted of giving me this error?
Here is my sql statement :
trsConn.EXECUTE "INSERT INTO ""TRS"".employeetbl ( "& _
"""firstName"", " & _
"""lastName"" , " & _
"""detContract"", " & _
"sle, " & _
"""posNumber"" "& _
") VALUES (" & _
"'" & Request.Form("empFirst") & "', " & _**
"'" & Replace(Request.Form("empLast"), "'", "`") & "', " & _
"'" & Request.Form("dateContract") & "', " & _
"'" & Request.Form("sle") & "', " & _
"'" & Request.Form("posNum") & "');"
(The posNum and dateContract are both respectivly of type "numeric" and "date"
Thanks so much for the help. Looking forward for hearing what you geniuses have to say.
The concept of NULL in SQL is pretty muddled and inconsistent ... but it's very clear that '' is distinct from NULL.
'' isn't NULL, it's '', the empty string. You can't convert it to a date, number, etc:
regress=# SELECT CAST('' AS DATE);
ERROR: invalid input syntax for type date: ""
LINE 1: SELECT CAST('' AS DATE);
^
regress=# SELECT CAST('' AS NUMERIC);
ERROR: invalid input syntax for type numeric: ""
LINE 1: SELECT CAST('' AS NUMERIC);
^
Some products - notably Microsoft Access and old versions of MySQL - are confused about that matter. NULL is NULL, '' is the empty string; they aren't the same thing. You can't convert one to the other.
So it seems that when the person using the form enters nothing, it
returns the string "empty" that is "". Why can't the numeric type and
data type read that as a NULL entry? These fields are not mandatory
and so I need to have them be sometimes blank.
That's your application's job. When your app it sees the empty string come in on a form field for a numeric, date, or similar, it should send NULL to the database, not ''. That's normally a routine part of converting data from user input before it's supplied to the database. It is vital that you do such conversion; you should never just send values from the user straight to the database.
A quick search suggests that asp classic uses null or undefined as its null values; you should be able to pass them into your prepared statements when something is null.
The fact that you get a syntax error after the error about '' suggests that you're building your SQL statements as strings, not using prepared statements with placeholders. (Thanks JayC for the SO question ref). This is begging for SQL injection; in other words your application is critically insecure. Imagine what happens if the user enters the "date":
2012-01-01'); DROP SCHEMA public;--
and your app happily turns that into
INSERT INTO sometable (blah, blah, blah) VALUES (1, 2, DATE '2012-01-01'); DROP SCHEMA public;--');
The DROP SCHEMA then merrily executes and whoops, splat, there goes your database. That's just the dumbest, simplest kind of SQL injection attack too.