I have a spring batch with a step that:
read from table all message in "SCHEDULED" status
process message
update message status to "SENT" or other status according to process result
Using RepositoryItemReader i had the problem that updating items state messed up pagination, so i ended up using JpaCursorItemReader.
Everything goes fine, but now all the items are read again from DB before the update phase (this not happens when using RepositoryItemReader).
Here is my relevant code:
#Autowired
private MessageSenderProcessor messageSenderProcessor;
#Autowired
private GovioMessagesRepository govioMessagesRepository;
public Step getProfileStep(){
return steps.get("getProfileStep")
.<GovioMessageEntity, GovioMessageEntity>chunk(10)
.reader(expiredScheduledDateMessageReader())
.processor(this.messageSenderProcessor)
.writer(messageWriter())
.build();
}
private RepositoryItemWriter<GovioMessageEntity> messageWriter() {
final RepositoryItemWriter<GovioMessageEntity> repositoryItemWriter = new RepositoryItemWriter<>();
repositoryItemWriter.setRepository(govioMessagesRepository);
repositoryItemWriter.setMethodName("save");
return repositoryItemWriter;
}
private ItemReader<GovioMessageEntity> expiredScheduledDateMessageCursor() {
JpaCursorItemReader<GovioMessageEntity> itemReader = new JpaCursorItemReader<>();
itemReader.setQueryString("SELECT msg FROM GovioMessageEntity msg WHERE msg.status = :status AND msg.scheduledExpeditionDate < :now");
itemReader.setEntityManagerFactory(entityManager.getEntityManagerFactory());
itemReader.setSaveState(true);
Map<String, Object> parameters = new HashMap<String, Object>();
parameters.put("status", Status.SCHEDULED);
parameters.put("now", LocalDateTime.now());
itemReader.setParameterValues(parameters);
return itemReader;
}
Here an extract of execution log:
Hibernate: select goviomessa0_.id as id1_0_0_, govioservi1_.id as id1_1_1_, goviomessa0_.amount as amount2_0_0_, goviomessa0_.appio_message_id as appio_me3_0_0_, goviomessa0_.creation_date as creation4_0_0_, goviomessa0_.expedition_date as expediti5_0_0_, goviomessa0_.id_govio_service_instance as id_govi15_0_0_, goviomessa0_.invalid_after_due_date as invalid_6_0_0_, goviomessa0_.last_update_status as last_upd7_0_0_, goviomessa0_.markdown as markdown8_0_0_, goviomessa0_.notice_number as notice_n9_0_0_, goviomessa0_.payee as payee10_0_0_, goviomessa0_.scheduled_expedition_date as schedul11_0_0_, goviomessa0_.status as status12_0_0_, goviomessa0_.subject as subject13_0_0_, goviomessa0_.taxcode as taxcode14_0_0_ from govio_messages goviomessa0_ where goviomessa0_.status=? and goviomessa0_.scheduled_expedition_date<?
2022-11-07 10:03:06,990 INFO [spring_batch_msgsender1] it.govio.msgsender.step.GetProfileProcessor: Sending msg 1
2022-11-07 10:03:06,990 INFO [spring_batch_msgsender2] it.govio.msgsender.step.GetProfileProcessor: Sending msg 2
2022-11-07 10:03:06,990 INFO [spring_batch_msgsender3] it.govio.msgsender.step.GetProfileProcessor: Sending msg 3
....
2022-11-07 10:03:07,171 INFO [spring_batch_msgsender1] it.govio.msgsender.step.GetProfileProcessor: Message sent.
2022-11-07 10:03:07,171 INFO [spring_batch_msgsender2] it.govio.msgsender.step.GetProfileProcessor: Message sent.
2022-11-07 10:03:07,220 INFO [spring_batch_msgsender3]
....
Hibernate: select goviomessa0_.id as id1_0_0_, goviomessa0_.amount as amount2_0_0_, goviomessa0_.appio_message_id as appio_me3_0_0_, goviomessa0_.creation_date as creation4_0_0_, goviomessa0_.expedition_date as expediti5_0_0_, goviomessa0_.id_govio_service_instance as id_govi15_0_0_, goviomessa0_.invalid_after_due_date as invalid_6_0_0_, goviomessa0_.last_update_status as last_upd7_0_0_, goviomessa0_.markdown as markdown8_0_0_, goviomessa0_.notice_number as notice_n9_0_0_, goviomessa0_.payee as payee10_0_0_, goviomessa0_.scheduled_expedition_date as schedul11_0_0_, goviomessa0_.status as status12_0_0_, goviomessa0_.subject as subject13_0_0_, goviomessa0_.taxcode as taxcode14_0_0_ from govio_messages goviomessa0_ where goviomessa0_.id=?
Hibernate: select goviomessa0_.id as id1_0_0_, goviomessa0_.amount as amount2_0_0_, goviomessa0_.appio_message_id as appio_me3_0_0_, goviomessa0_.creation_date as creation4_0_0_, goviomessa0_.expedition_date as expediti5_0_0_, goviomessa0_.id_govio_service_instance as id_govi15_0_0_, goviomessa0_.invalid_after_due_date as invalid_6_0_0_, goviomessa0_.last_update_status as last_upd7_0_0_, goviomessa0_.markdown as markdown8_0_0_, goviomessa0_.notice_number as notice_n9_0_0_, goviomessa0_.payee as payee10_0_0_, goviomessa0_.scheduled_expedition_date as schedul11_0_0_, goviomessa0_.status as status12_0_0_, goviomessa0_.subject as subject13_0_0_, goviomessa0_.taxcode as taxcode14_0_0_ from govio_messages goviomessa0_ where goviomessa0_.id=?
....
Hibernate: update govio_messages set amount=?, appio_message_id=?, creation_date=?, expedition_date=?, id_govio_service_instance=?, invalid_after_due_date=?, last_update_status=?, markdown=?, notice_number=?, payee=?, scheduled_expedition_date=?, status=?, subject=?, taxcode=? where id=?
Hibernate: update govio_messages set amount=?, appio_message_id=?, creation_date=?, expedition_date=?, id_govio_service_instance=?, invalid_after_due_date=?, last_update_status=?, markdown=?, notice_number=?, payee=?, scheduled_expedition_date=?, status=?, subject=?, taxcode=? where id=?
...
So i have those questions:
Why the item is read again after process phase when using cursor?
Can i avoid this?
CursorItemReader as item reader is the right choice for my use case?
When trying to call procedure with timestamp parameters by the code:
SimpleJdbcCall simpleJdbcCall = new SimpleJdbcCall(dataSource)
.withSchemaName("time_series")
.withProcedureName("refresh_continuous_aggregate")
.declareParameters(new SqlParameter("continuous_aggregate", Types.VARCHAR),
new SqlParameter("window_start", Types.TIMESTAMP),
new SqlParameter("window_end", Types.TIMESTAMP));
MapSqlParameterSource inParams = new MapSqlParameterSource()
.addValue("continuous_aggregate", "time_series.current_data_hourly", Types.VARCHAR)
.addValue("window_start", "2020-11-12 01:02:03.123456789", Types.TIMESTAMP)
.addValue("window_end", "2021-02-12 01:02:03.123456789", Types.TIMESTAMP);
Map<String, Object> execute = simpleJdbcCall.execute(inParams);
I'm getting error:
ERROR o.s.i.handler.LoggingHandler ->
org.springframework.jdbc.BadSqlGrammarException:
CallableStatementCallback; bad SQL grammar [{call
time_series.refresh_continuous_aggregate(?, ?, ?)}]; nested exception
is org.postgresql.util.PSQLException: ERROR: could not determine data
type of parameter $2
The similar error I'm getting when I try to use native query with dynamic parameters:
String queryStr = "CALL time_series.refresh_continuous_aggregate(?1, ?2, ?3)";
Query query = entityManager.createNativeQuery(queryStr)
.setParameter(1, continuousAggregate)
.setParameter(2, windowStart, TemporalType.TIMESTAMP)
.setParameter(3, windowEnd, TemporalType.TIMESTAMP);
Object singleResult = query.getSingleResult();
ERROR o.h.e.jdbc.spi.SqlExceptionHelper -> ERROR: syntax error at or
near "$2"
And by using StoredProcedureQuery:
StoredProcedureQuery query = entityManager.createStoredProcedureQuery("time_series.refresh_continuous_aggregate")
.registerStoredProcedureParameter("continuous_aggregate", String.class, ParameterMode.IN)
.registerStoredProcedureParameter("window_start", Date.class, ParameterMode.IN)
.registerStoredProcedureParameter("window_end", Date.class, ParameterMode.IN)
.setParameter("continuous_aggregate", "time_series.current_data_hourly")
.setParameter("window_start", new Date(), TemporalType.TIMESTAMP)
.setParameter("window_end", new Date(), TemporalType.TIMESTAMP);
query.execute();
WARN o.h.e.jdbc.spi.SqlExceptionHelper -> SQL Error: 0, SQLState:
42P18 ERROR o.h.e.jdbc.spi.SqlExceptionHelper -> ERROR: could not
determine data type of parameter $2
It looks like JDBC drivers issue.
While executing insert and update command in oracle 11g I'm getting below error.
val stmt = con.createStatement()
//Insert
val query1 = "insert into audit values('D','abc','T','01-NOV-18','Inprogress')"
stmt.executeUpdate(query1)
//Update
val query2 = "Update audit Set status='test' where where product= = 'D'"
stmt.executeUpdate(query2)
I'm getting below error
//Error while updating record
Exception in thread "main" java.sql.SQLSyntaxErrorException: ORA-00936: missing expression
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:195)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1036)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1336)
at oracle.jdbc.driver.OracleStatement.executeUpdateInternal(OracleStatement.java:1845)
at oracle.jdbc.driver.OracleStatement.executeUpdate(OracleStatement.java:1810)
at oracle.jdbc.driver.OracleStatementWrapper.executeUpdate(OracleStatementWrapper.java:294)
at com.oracle.OracleConnection$.main(OracleConnection.scala:21)
at com.oracle.OracleConnection.main(OracleConnection.scala)
Process finished with exit code 1
Any help will be appreciated. Thanks in advance
You have two where and two = in your update query.
val query2 = "Update audit Set status='test' where product='D'"
Using the Java api on an Orient-DB 2.0 rc1. Is it possible to use a string as a RID?
Documentation : https://github.com/orientechnologies/orientdb/wiki/SQL-Update#example-3-add-a-value-into-a-collection
more specifiaclly the section SQL Commands
JSON received:
{
userRID:"#100:100",
group:"someCode"
}
JAVA CODE (xtend script):
def addUserInGroup(Message<JsonObject> message){
val params = new JsonObject() => [
putString("userId", message.body().getString("userRID"));
putString("code", message.body().getString("group"));
]
var queryCommand= "update Table add users=:userId where code=:code";
Map<String, Object> paramsFormatted = params.toMap();
database.command( new OCommandSQL(queryCommand) ).execute(paramsFormatted);
}
Table structure :
users : linklist (user object)
code : string
When I run the following commands in the studio they all work :
update Table add users=(Select #rid from User where #rid=#100:100) where code='someCode'
update Table add users=#100:100 where code='someCode'
But none of these command work in the API. I'm guessing there is something wrong in way I'm using the RID.
SEVERE: Exception in Java verticle
com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #29: Found unexpected keyword '=:USERID' while it was expected '[=]'. Use UPDATE <class>|cluster:<cluster>> [SET|ADD|PUT|REMOVE|INCREMENT|CONTENT {<JSON>}|MERGE {<JSON>}] [[,] <field-name> = <expression>|<sub-command>]* [LOCK <NONE|RECORD>] [UPSERT] [RETURN <COUNT|BEFORE|AFTER>] [WHERE <conditions>]
Command: update SecureAccess add users=:userId where code=:code
-------------------------------------^
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLAbstract.throwSyntaxErrorException(OCommandExecutorSQLAbstract.java:89)
at com.orientechnologies.common.parser.OBaseParser.parserRequiredKeyword(OBaseParser.java:317)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLUpdate.parseAddFields(OCommandExecutorSQLUpdate.java:605)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLUpdate.parse(OCommandExecutorSQLUpdate.java:129)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLUpdate.parse(OCommandExecutorSQLUpdate.java:59)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.parse(OCommandExecutorSQLDelegate.java:56)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.parse(OCommandExecutorSQLDelegate.java:37)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:1150)
at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:63)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1179)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:385)
at com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:216)
at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:65)
janv. 07, 2015 8:45:17 AM org.vertx.java.core.logging.impl.JULLogDelegate error
SEVERE: Exception in Java verticle
com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #29: Found unexpected keyword '=:USERID' while it was expected '[=]'. Use UPDATE <class>|cluster:<cluster>> [SET|ADD|PUT|REMOVE|INCREMENT|CONTENT {<JSON>}|MERGE {<JSON>}] [[,] <field-name> = <expression>|<sub-command>]* [LOCK <NONE|RECORD>] [UPSERT] [RETURN <COUNT|BEFORE|AFTER>] [WHERE <conditions>]
Command: update SecureAccess add users=:userId where code=:code
-------------------------------------^
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLAbstract.throwSyntaxErrorException(OCommandExecutorSQLAbstract.java:89)
at com.orientechnologies.common.parser.OBaseParser.parserRequiredKeyword(OBaseParser.java:317)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLUpdate.parseAddFields(OCommandExecutorSQLUpdate.java:605)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLUpdate.parse(OCommandExecutorSQLUpdate.java:129)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLUpdate.parse(OCommandExecutorSQLUpdate.java:59)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.parse(OCommandExecutorSQLDelegate.java:56)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.parse(OCommandExecutorSQLDelegate.java:37)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:1150)
at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:63)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1179)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:385)
at com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:216)
at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:65)
I think You should try something like :
putString("userId", new ORecordId(message.body().getString("userRID")));
Have anybody tried to remove data(triples) from the DB2 RDF simple store(Windows)? (INSERT works fine)
SPARQL DELETE statament:
DELETE { ?document ?property ?value} WHERE{ ?document <http://example.com#begin>
?begin .
FILTER(?begin > 200) ?document ?property ?value }
I've got the following exception:
com.ibm.rdf.store.exception.RdfStoreException: DB255006E ERRORCODE=-4499, SQLSTATE=08001. SQLSTATE: 08001.
at
com.ibm.rdf.store.internal.jena.impl.update.SingleTripleOperations.removeQuad(Unknown Source)
at com.ibm.rdf.store.internal.jena.impl.update.AbstractTripleOperation.removeQuad(Unknown Source)
.....
db2diag.log:
RETCODE : ZRC=0x87120007=-2028863481=SQLR_SEVERE_PGM_ERROR
"Severe programming error"
DIA8516C A severe internal processing error has occurred.
My Java code:
String queryString = "delete ...";
Dataset ds = RdfStoreFactory.connectDataset(storeP, conn);
GraphStore graphStore = GraphStoreFactory.create(ds) ;
UpdateAction.parseExecute(queryString, graphStore); // exception
//UpdateAction.parseExecute("DROP ALL", graphStore); // works fine
Thanks!
The problem solved by adding
prop.setProperty("enableExtendedIndicators", "2");
to the DB2 connection.