I am getting below exceptions in the logs, related to orientdb (2.2.19)
SYS_ERR: java.lang.IllegalStateException: Pool is closed
>
> at
> com.orientechnologies.orient.core.db.OPartitionedDatabasePool.checkForClose(OPartitionedDatabasePool.java:370)
>
> at
> com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire(OPartitionedDatabasePool.java:176)
>
> at
> com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.<init>(OrientBaseGraph.java:143)
>
> at
> com.tinkerpop.blueprints.impls.orient.OrientTransactionalGraph.<init>(OrientTransactionalGraph.java:77)
> at
> com.tinkerpop.blueprints.impls.orient.OrientGraph.<init>(OrientGraph.java:135)
> at
> com.tinkerpop.blueprints.impls.orient.OrientGraphFactory$1.getGraph(OrientGraphFactory.java:84)
>
> at
> com.tinkerpop.blueprints.impls.orient.OrientGraphFactory.getTx(OrientGraphFactory.java:221)
The code mainly perform adding the data in orientdb, and while getting the OrientGraphFactory.getTx() call, I am seeing all these exception.
I am calling commit() call, and then doing shutdown()
private void commitGraph(OrientBaseGraph graph){
try{
graph.commit();
}catch(Exception e){
e.printStackTrace();
}finally{
graph.shutdown();
}
}
Also these exception cased after NPE, while performing commit
java.lang.NullPointerException
at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.makeActive(OrientBaseGraph.java:362)
at com.tinkerpop.blueprints.impls.orient.OrientTransactionalGraph.commit(OrientTransactionalGraph.java:177)
The OrientGraphFactory is initialized as below
factory = new OrientGraphFactory(url, userName, password).setupPool(1,50);
Any pointers, when and why this error is thrown would be helpful.
Edit:
The configuration in my orientdb-server-config.xml is as below :
<parameters>
<parameter value="true" name="enabled"/>
<parameter value="50" name="graph.pool.max"/>
</parameters>
I could see that there are many exceptions for not reaching the remote server having orientdb installation. Is it something related to it. I could see the below exception
com.orientechnologies.orient.core.exception.OStorageException: Cannot create a connection to remote server address(es)
Related
i'm working in spring batch
public void beforeStep(StepExecution stepExecution) {
ExecutionContext context = new ExecutionContext();
context.putString("resourceFileName", file.getFilePath());
System.out.println("display resource path : ");
System.out.println(context.get("resourceFileName"));
}
this line works successfully i got the right path
System.out.println(context.get("resourceFileName"));
in the xml item reader i tried to get this resource like this
<property name="resource" value="file:#{stepExecutionContext['resourceFileName']}" />
but i got this error :
Caused by: java.lang.IllegalArgumentException: The Resource must not be null.
znd sometimes this error
Input resource must exist (reader is in 'strict' mode)
I am designing a Spring Batch, which reads multiple csv files. I have used partitioning to read each file in chunk and process it to decrypt a certain column in the csv. Before decrypting if i encounter any validation error , i throw custom exception.
Now what i want is if the processing finds any validation error in the first line, the other lines should not be processed, and the job should end.
How can i achieve this? I tried to implement ProcessorListener too but it has no StepExecution object so that i can call SetTerminateOnly() or ExitStatus=Failed
Also note that i have multiple thread accessing the file in different lines.I want to kill all threads in the event of the first encountered error.
Thanks in advance
So, I identified that running multiple asynchronous concurrent threads (Spring Batch partitioning) was the real issue. Though one of the thread threw an Exception, the other threads were parallely running, and finished executing till the end.
Ath the end, the Job FAILED overall and there was no output processed, but it consumed time to process rest of the data.
Well,the solution to it is as simple as it gets. We just need stop the Job while encountering an error during processing.
The Custom Processor
public class MultiThreadedFlatFileItemProcessor implements ItemProcessor<BinFileVO, BinFileVO>,JobExecutionListener{
private JobExecution jobExecution;
private RSADecrypter decrypter;
public RSADecrypter getDecrypter() {
return decrypter;
}
public void setDecrypter(RSADecrypter decrypter) {
this.decrypter = decrypter;
}
#Override
/**
This method is used process the encrypted data
#param item
* */
public BinFileVO process(BinFileVO item) throws JobException {
if(null!=item.getEncryptedText() && !item.getEncryptedText().isEmpty()){
String decrypted = decrypter.getDecryptedText(item.getEncryptedText());
if(null!=decrypted && !decrypted.isEmpty()){
if(decrypted.matches("[0-9]+")){
if(decrypted.length() >= 12 && decrypted.length() <= 19){
item.setEncryptedText(decrypted);
}else{
this.jobExecution.stop();
throw new JobException(PropertyLoader.getValue(ApplicationConstants.DECRYPTED_CARD_NO_LENGTH_INVALID),item.getLineNumber());
}
}
}else{
this.jobExecution.stop();
throw new JobException(PropertyLoader.getValue(ApplicationConstants.EMPTY_ENCRYPTED_DATA),item.getLineNumber());
}
return item;
}
#Override
public void beforeJob(JobExecution jobExecution) {
this.jobExecution=jobExecution;
}
#Override
public void afterJob(JobExecution jobExecution) {
}
}
The Job xml config
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
.....>
<!-- JobRepository and JobLauncher are configuration/setup classes -->
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<!-- Job Details -->
<job id="simpleMultiThreadsReaderJob" xmlns="http://www.springframework.org/schema/batch">
<step id="step" >
<partition step="step1" partitioner="partitioner">
<handler grid-size="5" task-executor="taskExecutor"/>
</partition>
</step>
<listeners>
<listener ref="decryptingItemProcessor"/>
</listeners>
</job>
<step id="step1" xmlns="http://www.springframework.org/schema/batch">
<tasklet>
<chunk reader="itemReader" writer="itemWriter" processor="decryptingItemProcessor" commit-interval="500"/>
<listeners>
<listener ref="customItemProcessorListener" />
</listeners>
</tasklet>
</step>
<!-- Processor Details -->
<bean id="decryptingItemProcessor" class="com.test.batch.io.MultiThreadedFlatFileItemProcessor">
<property name="decrypter" ref="rsaDecrypter" />
</bean>
<!-- RSA Decrypter class -->
<bean id="rsaDecrypter" class="test.batch.secure.rsa.client.RSADecrypter"/>
<!-- Partitioner Details -->
<bean class="org.springframework.batch.core.scope.StepScope" />
<bean id="partitioner" class="com.test.batch.partition.FlatFilePartitioner" scope="step">
<property name="resource" ref="inputFile"/>
</bean>
<bean id="taskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="10"/>
</bean>
<!-- Step will need a transaction manager -->
<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
........
.................
</beans>
Here are the logs
2016-09-01 06:32:40 INFO SimpleJobRepository:273 - Parent JobExecution is stopped, so passing message on to StepExecution
2016-09-01 06:32:43 INFO ThreadStepInterruptionPolicy:60 - Step interrupted through StepExecution
2016-09-01 06:32:43 INFO AbstractStep:216 - Encountered interruption executing step: Job interrupted status detected.
; org.springframework.batch.core.JobInterruptedException
2016-09-01 06:32:45 ERROR CustomJobListener:163 - exception :At line No. 1 : The decrypted card number is less than 12 or greater than 19 in length
2016-09-01 06:32:45 ERROR CustomJobListener:163 - exception :Job interrupted status detected.
2016-09-01 06:32:45 INFO SimpleJobLauncher:135 - Job: [FlowJob: [name=simpleMultiThreadsReaderJob]] completed with the following parameters: [{outputFile=/usr/local/pos/bulktokenization/csv/outputs/cc_output_EDWError_08162016.csv, partitionFile=/usr/local/pos/bulktokenization/csv/partitions/, inputFile=C:\usr\local\pos\bulktokenization\csv\inputs\cc_input_EDWError_08162016.csv, fileName=cc_input_EDWError_08162016}] and the following status: [FAILED]
2016-09-01 06:32:45 INFO BatchLauncher:122 - Exit Status : FAILED
2016-09-01 06:32:45 INFO BatchLauncher:123 - Time Taken : 8969
If we throw Custom Exception in Processor, Spring Batch will terminate and mark the job failed unless you setup 'skipable' exception. You have not mentioned where you perform validate step, are you doing in Processor or Reader? Let me know because it is where Spring Batch decides.
In my project, if I want to stop the job and throw Custom Exception, we put validation logic in a Tasklet or Processor and throw exception as below
private AccountInfoEntity getAccountInfo(Long partnerId) {
if(partnerId != null){
.....
return ....;
} else {
throw new ReportsException("XXXXX");
}
}
While trying to convert my app from using MyBatis [it was iBatis before], I've been getting the following error, in spite of trying to make so many different kinds of changes to the code: What am I doing wrong? Any help would be really really really great!!!
Error message:
`Caused by: org.springframework.jdbc.UncategorizedSQLException: ### Error querying database. Cause: com.ibm.db2.jcc.am.SqlException: [jcc][10271][10296][3.58.82] Unrecognized JDBC type: -10. ERRORCODE=-4228, SQLSTATE=null
`
`### The error may exist in Path-to-XML-File.XML`
`### The error may involve namespace.resultMap-name`
`### The error occurred while executing a query`
`### SQL: {call name-of-stored-proc(?)}`
`### Cause: com.ibm.db2.jcc.am.SqlException: [jcc][10271][10296][3.58.82] Unrecognized JDBC type: -10. ERRORCODE=-4228, SQLSTATE=null`
`; uncategorized SQLException for SQL []; SQL state [null]; error code [-4228]; [jcc][10271][10296][3.58.82] Unrecognized JDBC type: -10. ERRORCODE=-4228, SQLSTATE=null; nested exception is com.ibm.db2.jcc.am.SqlException: [jcc][10271][10296][3.58.82] Unrecognized JDBC type: -10. ERRORCODE=-4228, SQLSTATE=null`
XML File: `
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN"
"http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="Namespace-name">
<resultMap id="retrieveReports-results" type="Folder-structure-to-Java-file" autoMapping="true">
<result property="name" column="REPORT_NAME"/>
<result property="location" column="REPORT_DIRECTORY"/>
<result property="key" column="REPORT_KEY"/>
<result property="executable" column="EXECUTABLE"/>
<result property="commandLine" column="COMMAND_LINE"/>
<result property="commandLineFormat" column="COMMAND_LINE_FORMAT"/>
</resultMap>
<select id="retrieveReports" resultType="java.util.Map" statementType="CALLABLE">
{call prc_sel_reports(#{reports,jdbcType=CURSOR,javaType=java.sql.ResultSet,mode=OUT,resultMap=retrieveReports-results})}
</select>
</mapper>`
Java code:
try {
List < Report > reports = super.getSqlSession().selectList("retrieveReports");
if(log.isDebugEnabled()){
log.debug("Retrieved " + reports.size() + " reports, in method: retrieveReports()");
}
return reports;
// Attempt to catch different kinds of exceptions.
} catch (Exception e) {
throw new RuntimeException("Exception caught while trying to retrieve reports", e);
}
You have to specify a parameterType in the select statement, either map or custom class. An instance has to be passed on statement invocation.
resultType is not used, then irrelevant.
Procedure actually writes in a field of the "input" parameter passed to mybatis statement, and the statement does not return anything.
Mapper interface would be: void retrieveReports(Map<String, Object> params);
And your call:
Map<String, Object> params = new HashMap<String, Object>();
session.selectList("retrieveReports", params);
List<Report> reports = (List<Report>)params.get("reports");
Param may also be a custom type with property private List<Report> reports;.
I am trying to skip all the exceptions during the batch run using the following config:
<chunk reader="aaaFileReader" writer="aaaDBWriter"
commit-interval="100" skip-limit="100000">
<skippable-exception-classes>
<include class="java.lang.Exception" />
<exclude
class="org.springframework.jdbc.CannotGetJdbcConnectionException" />
</skippable-exception-classes>
</chunk>
<listeners>
<listener ref="aaabatchFailureListener" />
</listeners>
And I handle the exception in my listener. But when Spring Batch actually encounters an exception its not being skipped and the batch run ends with a failed state. The actual exception is a FlatFileParse Exception. How do I skip the FlatFileParseException?
Here is the log :
:18:21.257 [main] DEBUG o.s.b.repeat.support.RepeatTemplate - Handling fatal exception explicitly (rethrowing first of 1): org.springframework.batch.core.step.skip.NonSkippableReadException: Non-skippable exception during read
15:18:21.257 [main] ERROR o.s.batch.core.step.AbstractStep - Encountered an error executing the step
org.springframework.batch.core.step.skip.NonSkippableReadException: Non-skippable exception during read
at org.springframework.batch.core.step.item.FaultTolerantChunkProvider.read(FaultTolerantChunkProvider.java:81) ~[spring-batch-core.jar:na]
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:106) ~[spring-batch-core.jar:na]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:367) ~[spring-batch-infrastructure.jar:na]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infr
Caused by: org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 5, input=[0254285458908060150983101150983 AK00055002035201401081044000804CK5861 00Twist,Oliver AT&T 20121208 ]
at org.springframework.batch.
You can add the FlatFileParseException class on your batch job config, for example:
<batch:chunk reader="customImportReader" writer="customImporter" processor="customProcessor" commit-interval="1" skip-limit="10">
<batch:skippable-exception-classes>
<batch:include class="org.springframework.batch.item.file.FlatFileParseException" />
</batch:skippable-exception-classes>
</batch:chunk>
As per Spring Batch Documentation some of the exceptions are not qualified as skippable.
In your case its clear from logs that org.springframework.batch.item.file.FlatFileParseException is not a skippable excepotion hence re throwing org.springframework.batch.core.step.skip.NonSkippableReadException.
Read more about Configuring Skip Logic section that says:
For any exception encountered, the skippability will be determined by the nearest superclass in the class hierarchy. Any unclassifed exception will be treated as 'fatal'.
Read more about NonSkippableReadException that says:
Fatal exception to be thrown when a read operation could not be skipped.
Create a custom fileReader and Override the doRead() method to always throw you CustomException.
public class CustomFlatFileItemReader extends FlatFileItemReader {
#Override
protected T doRead() throws Exception {
T itemRead=null;
try {
itemRead= super.doRead();
} catch (FlatFileParseException e) {
throw new MyException(e.getMessage(), e);
}
return itemRead;}
}
Override your job skip policy to always skip your custom exception as below:
.skipPolicy((Throwable T, int skipCount) -> {
if (T instanceof BatchServiceException)
return true;
else
return false;
I am making a simple spring MVC application that takes data from a view and sends it to a PostgreSQL database on my machine. I have been following tutorials and am not completely familiar with the bean configuration style of handling connection settings in the Data Access Objects. The error that returns when I attempt to post to the database is as follows:
HTTP Status 500 - Request processing failed; nested exception is java.lang.RuntimeException: org.postgresql.util.PSQLException: ERROR: relation "userid" does not exist
"userid" is my simple postgre table I'm using for testing.
My spring bean configuration file is this:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.postgresql.Driver" />
<property name="url" value="jdbc:postgresql://localhost:5432/postgres" />
<property name="username" value="postgres" />
<property name="password" value="kittens" />
</bean>
This is the DAO handling the connection to the DB:
package com.ARSmvcTest.dao.impl;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import javax.sql.DataSource;
import com.ARSmvcTest.dao.arsDAO;
import com.ARSmvcTest.models.ARS;
public class JDBCarsDAO implements arsDAO {
private DataSource dataSource;
public void setDataSource(DataSource dataSource) {
this.dataSource = dataSource;
}
public void insert(ARS ars){
String sql = "INSERT INTO UserID " +
"(ID_, First_, Last_, Email_, Pwd_) VALUES (?, ?, ?, ?, ?)";
Connection conn = null;
try {
conn = dataSource.getConnection();
PreparedStatement ps = conn.prepareStatement(sql);
ps.setString(1, ars.getID_());
ps.setString(2, ars.getFirst_());
ps.setString(3, ars.getLast_());
ps.setString(5, ars.getPwd_());
ps.setString(4, ars.getEmail_());
ps.executeUpdate();
ps.close();
} catch (SQLException e) {
throw new RuntimeException(e);
} finally {
if (conn != null) {
try {
conn.close();
} catch (SQLException e) {}
}
}
}
// FIND ARS BY ID WILL GO HERE EVENTUALLY
}
The spring bean is passing data to the connection successfully as evidenced by this screenshot below:
http://i.imgur.com/Hb7O5Qi.png (apologies, new user and can't embed images)
Despite the dataSource object receiving connection properties from my above spring bean, I notice that the ConnectionProperties field is still null.
Is this what is causing my exception on the Insert attempt?
Lastly I will include a screenshot of the Exception and stack being displayed in browser at the moment of failure:
http://i.imgur.com/prj1HtY.png (apologies, can't embed images)
this exception is directly triggered from the database(-driver).
It tells you that the table named "userid" is not existing.
Please check:
the table name on database and your insert-statement
if the table is read and writabled for the user you use for your connection; you need to have grated the correct right to be able to see and write to the table
I hope this will help you as this seems to be the typical error for this kind of exception.