having syntax error trying to use on duplicate - executemany

sql_insert_query = "insert into TABLE1 (building, course, description, course_type, course_type_desc, dual_credit)
VALUES (?, ?, ?, ?, ?, ?) ON DUPLICATE KEY UPDATE building = VALUES(building), course = VALUES(course), description = VALUES(description), course_type = VALUES(course_type), course_type_desc = VALUES(course_type_desc), dual_credit = VALUES(dual_credit);"
cursor.executemany(sql_insert_query, listCourse)
pyodbc.ProgrammingError: ('42000', u"[42000] [Microsoft][SQL Server
Native Client 11.0][SQL Server]Incorrect syntax near the keyword 'ON'.
(156) (SQLExecDirectW); [42000] [Microsoft][SQL Server Native Client
11.0][SQL Server]Statement(s) could not be prepared. (8180)")
This one below only works, but adds duplicate when running again.
"insert into TABLE1 (building, course, description, course_type, course_type_desc, dual_credit) VALUES (?, ?, ?, ?, ?, ?)"

I tied '%s' this did not work so I am using '?'

I have resolved it by tile with tuple
listCourse = numpy.tile(courses, 2)
listCourse = map(tuple,numpy.tile(courses, 2))

Related

Issue in importing dashboard in superset

I facing the issue while trying to export dashboard from server to another server of same version 1.5, but showing error: Integrity error. How to import dashboard?
ERROR: (while importing Dashboard)
sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.IntegrityError) UNIQUE constraint failed: tables.table_name
[SQL: INSERT INTO tables (uuid, created_on, changed_on, description, default_endpoint, is_featured, filter_select_enabled, "offset", cache_timeout, params, perm, schema_perm, is_managed_externally, external_url, table_name, main_dttm_col, database_id, fetch_values_predicate, schema, sql, is_sqllab_view, template_params, extra, created_by_fk, changed_by_fk) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]
[parameters: (<memory at 0x7f3b7c882640>, '2022-11-18 11:22:47.890965', '2022-11-18 11:22:47.890971', None, None, 0, 0, 0, None, '{"remote_id": 19, "database_name": "PostgreSQL", "import_time": 1668750767}', None, None, 0, None, 'storefrequency', None, '7', None, None, None, 0, None, None, 1, 1)]
(Background on this error at: http://sqlalche.me/e/13/gkpj) (Background on this error at: http://sqlalche.me/e/13/7s2a)
2022-11-18 11:22:47,893:INFO:werkzeug:127.0.0.1 - - [18/Nov/2022 11:22:47] "POST /superset/import_
I also changed the VERSIONED_EXPORT=True and DASHBOARD_RBAC=True but still it showing a error.

Collation over DDEV TYPO3 instance

I got a TYPO3 8.7 instance running over a server whose database and tables collation is utf8mb4_unicode_ci. The character set is utf8mb4.
However, MySQL over the server shows the collation for the connection is latin1_swedish_ci and collation for the server is utf8_general_ci (I guess these are parameters that come pre-configured with the hosting):
+----------------------+--------------------+
| Variable_name | Value |
+----------------------+--------------------+
| collation_connection | latin1_swedish_ci |
| collation_database | utf8mb4_unicode_ci |
| collation_server | utf8_general_ci |
+----------------------+--------------------+
3 rows in set (0.00 sec)
I would like to reproduce all these settings over my DDEV instance. I got the last two variable names properly configured with these settings over my .ddev/mysql/no_utf8mb4.cnf file:
collation-server = utf8_general_ci
character-set-server = utf8
But I cannot get the collation_connection as latin1_swedish_ci. How could I achieve that?
I exported my database from the hosting and imported it into ddev, but with the current configuration, every time I login into the backend, I receive an error like this:
An exception occurred while executing 'INSERT INTO sys_log (userid, type, action, error, details_nr, details, log_data, tablename, recuid, IP, tstamp, event_pid, NEWid, workspace) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [1, 255, 1, 0, 1, "User %s logged in from %s (%s)", "a:3:{i:0;s:30:\"xxx#xxx.com\";i:1;s:10:\"172.18.0.6\";i:2;s:0:\"\";}", "", 0, "172.18.0.6", 1564691070, -1, "", -99]: Field 'request_id' doesn't have a default value
Has anyone experienced this issue?
My MariaDB version over my ddev instance is 10.1.37-MariaDB.

When Using Postgresql JDBC, INSERT statement that will result in 0 rows being inserted

When Using Postgresql JDBC, Insert row with executeUpdate statement but result was zero. I found that data was inserted. API document tells that executeUpdate method return affected row count.
mSqlInsertEventHist = "INSERT INTO " + DatabaseTables.EVENT_HIST +
" ( event_id, mntr_id, mntr_zone, mntr_name, mntr_class_name, mntr_class_sub_type, host_name, ip_addr, " +
" metric_id, metric_name, perf_name, object_name, event_type, event_master_id, " +
" event_name, event_level, event_msg, threshold, event_recv_day, event_reco_dt, event_recv_dt," +
" event_saved_dt, blackout_yn, msg_filter_yn ) " +
" VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, " + " now() " + ", ?, ? ) ";
mPstmtInstEvtHist.setLong(1, eventid);
mPstmtInstEvtHist.setInt(2, mntrid);
mPstmtInstEvtHist.setString(3, mntrZone);
mPstmtInstEvtHist.setString(4, mntrname);
mPstmtInstEvtHist.setString(5, mntrClsName);
mPstmtInstEvtHist.setString(6, mntrClsSubType);
mPstmtInstEvtHist.setString(7, hostname);
mPstmtInstEvtHist.setString(8, ipAddr);
mPstmtInstEvtHist.setInt(9, Integer.parseInt((String)dataobj.get(MQConstants.METRIC_ID)));
mPstmtInstEvtHist.setString(10, metricname);
mPstmtInstEvtHist.setString(11, perfname);
mPstmtInstEvtHist.setString(12, objectname);
mPstmtInstEvtHist.setString(13, dataType);
mPstmtInstEvtHist.setInt(14,
Integer.parseInt((String)dataobj.get(MQConstants.EVENT_MASTER_ID)));
mPstmtInstEvtHist.setString(15, eventname);
mPstmtInstEvtHist.setString(16, eventlvl);
mPstmtInstEvtHist.setString(17, evtMsg);
mPstmtInstEvtHist.setObject(18, jsonObj);
mPstmtInstEvtHist.setInt(19,
Integer.parseInt((String)dataobj.get(MQConstants.EVENT_RECV_DAY)));
mPstmtInstEvtHist.setTimestamp(20, colTs);
mPstmtInstEvtHist.setTimestamp(21, rcvTs);
mPstmtInstEvtHist.setString(22,
(String)dataobj.get(MQConstants.BLACKOUT_YN));
mPstmtInstEvtHist.setString(23,
(String)dataobj.get(MQConstants.MSG_FILTER_YN));
rtn = mPstmtInstEvtHist.executeUpdate();
if ( rtn == 1 ) {//NOPMD
Debug.getLogger().trace("Inserted Event into {} => {}, {}, {} "
,DatabaseTables.EVENT_HIST , mntrid, eventname, eventlvl);
} else {
Debug.getLogger().warn("Failed to insert Event into {} => {}",
DatabaseTables.EVENT_HIST, Debug.getString(jobj));
}
It might be that you have a trigger-based partitioning in your database, as described here in PostgreSQL manual.
It's happening completely on the DB side, except that it always returns "0 rows inserted".
Check your DB schema for partitioning triggers in DatabaseTables.EVENT_HIST table.

Calling prepare with mysqli - SQL syntax error

$q2 = "UPDATE `tasks` SET `title` = ?, task = ?, rules = ?, media = ?, type = ?, xp = ?, available = ?, timestamp = ? WHERE id = ?";
if ($stmt = $mysqli->prepare($q2)) {
$stmt->bind_param("sssssissi", $_POST["tasktitle"], $_POST["editor"], $_POST["rules"], serialize($_POST["media"]), $_POST["type"], $_POST["xp"], $a = 0, strtotime("now"), $_GET['id']);
$stmt->execute();
$stmt->close();
}
$r = $mysqli->query($q2) or die($mysqli->error);
I got this error msg:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '?, task = ?, rules = ?, media = ?, type = ?, xp = ?, available = ?, timestamp = ' at line 1
What is problem, and how can i solve it?
I'm pretty certain it's coming from the call to $mysqli->query() which needs a properly escaped query (ie, none of that nice safe parameter stuff). That explains why it's complaining at the first ?.
Quick way to check is to actually comment out the entire if statement and se if the error still appears. If so, you know it's the query rather than the prepared statement execution.
My question to you is: why are you executing the prepared statement and then trying to run it again as a query?
I think you'll find the execute does your update quite well enough. Get rid of the call to query and you should be fine.

perl DBI and placeholders

I have this query select * from table where ID in (1,2,3,5...)
How is it possible to build this query with the DBI using placeholders ?
for example :
my #list = (1, 2, 3, 4, 5);
my $sql = "select * from table where ID in (?)";
$sth->prepare($sql);
$sth->execute();
What argument should I send to execute? Is it a list or a string separated by , or something else?
This should build your query dynamically according to the number of items in your array
my #list =(1,2,3,4,5);
my $sql ="select * from table where ID in (#{[join',', ('?') x #list]})";
It's not possible in that way. You need to specify a placeholder for each item in your array:
my #list = (1,2,3,4,5);
my $sql = "select * from table where ID in (?,?,?,?,?)";
$sth->prepare($sql);
$sth->execute(#list);
If your #list is not a fixed size, you need to build the $sql with the proper number of placeholders.
Quoting DBI documentation:
Also, placeholders can only represent single scalar values. For example, the following statement won't work as expected
for more than one value:
SELECT name, age FROM people WHERE name IN (?) # wrong
SELECT name, age FROM people WHERE name IN (?,?) # two names
Rewrite to:
my $sql = 'select * from table where ID in ( ?, ?, ?, ?, ? )';
$sth->prepare($sql);
$sth->execute(#list);
If you are using DBI to access a PostgreSQL database with the DBD::Pg driver, you can use:
my #list = (1, 2, 3, 4, 5);
my $sql = "select * from table where ID = ANY(?::INT[]);";
$sth->prepare ($sql);
$sth->execute (\#list);
Unless you know the exact number of elements you cannot use placeholders. Try this:
my #list = (1, 2, 3, 4, 5); # any number of elements
my $in = join(',', map { $dbh->quote($_) } #list);
my $sql = "select * from table where someid IN ($in)";
If you switch to DBIx::Simple you can just say:
$db->query('INSERT INTO foo VALUES (??)', $foo, $bar, $baz);
?? Means "as many as needed"
Edit:
Actually, I was a little too optimistic: "If the string (??) is present in the query, it is replaced with a list of as many question marks as #values."
So this does not seem to work:
$db->query( "SELECT * FROM foo WHERE id IN (??) AND stuff=?", #ids, $stuff )
Still useful though..
For the curious, the code in the module is:
# Replace (??) with (?, ?, ?, ...)
sub _replace_omniholder {
my ($self, $query, $binds) = #_;
return if $$query !~ /\(\?\?\)/;
my $omniholders = 0;
my $q = $self->{dbd} =~ /mysql/ ? $quoted_mysql : $quoted;
$$query =~ s[($q|\(\?\?\))] {
$1 eq '(??)'
? do {
Carp::croak('There can be only one omniholder')
if $omniholders++;
'(' . join(', ', ('?') x #$binds) . ')'
}
: $1
}eg;
}
I found a sure way for this to work summarizing all of the above advice. My Production query (I posted a much simpler version here) uses IN <>, where neither the codes nor their quantity is unknown. It could be a single Code (e.g. FIN), or a series of them (FITLC FITLD FITLU FITSC FITSD FITSU MDYLC MDYLD MDYLU). Some function returns that as a list.
The code that makes this happen is
#codes = get_muni_evcode( $category );
my $in = join( ', ', ('?') x #codes );
print "\n\nProcessing Category: $category --> Codes: #codes .. in: $in\n";
my $sql = "select distinct cusip9
from material_event
where event_date between (trunc(sysdate) - 1) + 2/3 and trunc(sysdate) + 2/3
and event_code in ($in)";
my $sth2 = $dbh->prepare($sql);
$sth2->execute( #codes );
while (my $s2 = $sth2->fetchrow_hashref('NAME_lc'))
{
my $cusip9 = $s2->{cusip9};
print "$cusip9\t";
.................. further processing ..............
}
The result sample:
Processing Category: RatingChange --> Codes: FITLC FITLD FITLU FITSC FITSD FITSU MDYLC MDYLD MDYLU MDYSC MDYSD MDYSU SPLD SPLPR SPLU SPSD SPSPR SPSU .. in: ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?
359496HQ2 359496GB6 359496GH3 359496GL4 359496HU3 359496HS8 359496HA7 359496HF6 359496GM2 359496HM1 359496HR0 359496HT6 359496GY6 359496GJ9 359496HL3 359496GU4 359496HK5 359496HN9 359496HP4 359496GW0 359496GZ3 359496HC3 359496GC4 359496GK6 359496GP5 359496GV2 359496GX8 359496GN0
I'm extremely grateful to everybody who posted their ideas here that finally made me find the right way to do this. It must be a pretty common problem I think.