How to specify fragmentRootElementNames in StaxEventItemReader for nested elements? - spring-batch

Working with Spring-batch 4.1.2 to consume a complex XML and write to database.
Using StaxEventItemReader for reading XML data.
I am facing an issue for nested elements, while specifying fragmentRootElementNames.
<bean id="productFileItemReader" class="org.springframework.batch.item.xml.StaxEventItemReader" scope="step">
<property name="resource" ref="inputResource" />
<property name="fragmentRootElementNames" value="ProductFeatures,ProductFeaturesDetail,SomeOtherNon-NestedFragmentNames..." />
<property name="unmarshaller" ref="productMarshaller" />
</bean>
<bean id="productMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
<property name="classesToBeBound">
<list>
<value>com.xyz.jaxb.ProductFeaturesType</value>
<value>com.xyz.jaxb.ProductFeaturesDetailType</value>
...Some more types
</list>
</property>
</bean>
<bean id="productFileItemProcessor" class="org.springframework.batch.item.support.ClassifierCompositeItemProcessor"
scope="step">
<property name="classifier" ref="prodItemProcessorclassifier" />
</bean>
<bean id="prodItemProcessorclassifier" class="org.springframework.classify.BackToBackPatternClassifier">
<property name="routerDelegate">
<bean class="com.xyz.batch.CustomProdTypeClassifier" />
</property>
<property name="matcherMap">
<map>
<entry key="ProductFeatures" value-ref="prodFeaturesProcessor" />
<entry key="ProductFeaturesDetail" value-ref="prodFeaturesDetailProcessor" />
</map>
</property>
</bean>
<bean id="prodItemWriter" class="org.springframework.batch.item.support.ClassifierCompositeItemWriter"
scope="step">
<property name="classifier" ref="prodItemWriterclassifier" />
</bean>
<bean id="prodItemWriterclassifier" class="org.springframework.classify.BackToBackPatternClassifier">
<property name="routerDelegate">
<bean class="com.xyz.batch.CustomProdTypeClassifier" />
</property>
<property name="matcherMap">
<map>
<entry key="ProductFeaturesDetail" value-ref="prodHibernateItemWriter" />
<entry key="ProductParameters" value-ref="myListItemWriter" />
</map>
</property>
</bean>
<bean id="myListItemWriter" class="com.xyz.writer.ListDelegateItemWriter">
<property name="delegate" ref="prodHibernateItemWriter" />
</bean>
<bean id="prodHibernateItemWriter" class="org.springframework.batch.item.database.HibernateItemWriter">
<property name="sessionFactory" ref="mySessionFactory" />
</bean>
Here is the signature of the ItemProcessors
public class ProductFeaturesProcessor implements ItemProcessor<ProductFeaturesType, List<ProductParamEntity>>{}
public class ProductFeaturesDetailProcessor implements ItemProcessor<ProductFeaturesDetailType, ProductFeaturesDetailEntity> {}
XML structure is as follows -
<ProductFeatures>
<Version>12</Version>
<MessageEN>Welcome</MessageEN>
<MessageFR>Bienvenue</MessageFR>
<ProductFeaturesDetail>
<!-- Some elements here -->
</ProductFeaturesDetail>
<ProductFeaturesDetail>
<!-- Some elements here -->
</ProductFeaturesDetail>
<ProductFeaturesDetail>
<!-- Some elements here -->
</ProductFeaturesDetail>
<ProductFeaturesDetail>
<!-- Some elements here -->
</ProductFeaturesDetail>
</ProductFeatures>
Here, ProductFeatures has minOccurs=1, maxOccurs=1.
ProductFeaturesDetail is unbounded.
Each of Version, MessageEN, MessageFR need to be persisted as a record in Table-A which has 2 columns ParamName|ParamValue Each ProductFeaturesDetail is a record to be persisted in Table-B.
i.e. The 3 elements are each mapped to ProductParamEntity (i.e. with
ParamName=Version, ParamValue=12 ; ParamName=MessageEN,
ParamValue=Welcome etc.)
Logically, ProductFeatures is NOT mapped to a table directly but data
is stored in 2 tables as List and
List into tables PRODUCT_PARAM and
PRODUCT_FEATURES_DETAIL respectively.
Here are the Objects - JAXB and Corresponding Entities
#XmlRootElement(name = "ProductFeatures")
public class ProductFeaturesType {
#XmlElement(name = "Version")
protected String version;
#XmlElement(name = "MessageEN")
protected String messageEN;
#XmlElement(name = "MessageFR")
protected String messageFR;
#XmlElement(name = "ProductFeaturesDetail")
protected List<ProductFeaturesDetailType> prodFeaturesDetail;
}
#Entity
#Table(name="PRODUCT_PARAM")
public class ProductParamsEntity extends BaseEntity implements Serializable {
#Id
#Column(name="PARAM_NAME")
private String paramName;
#Column(name="PARAM_VALUE")
private String paramValue;
}
#Entity
#Table(name="PRODUCT_FEATURES_DETAIL")
public class ProductFeaturesDetailEntity extends BaseEntity implements Serializable {
#Id
#Column(name="PROD_CATEGORY")
private String prodCategory;
//---More attributes ---
}
Here is my Job Config :
<batch:job id="consumeProductFileJob" job-repository="jobRepository" restartable="true">
<batch:step id="validateFile" parent="validateXMLSchema">
<batch:next on="COMPLETED" to="persistData" />
<batch:next on="FAILED" to="notifyException" />
</batch:step>
<batch:step id="persistData">
<batch:tasklet transaction-manager="prodHibernateTransactionManager">
<batch:chunk reader="productFileItemReader" processor="productFileItemProcessor" writer="prodItemWriter" commit-interval="500" >
</batch:chunk>
<batch:transaction-attributes isolation="DEFAULT" propagation="REQUIRED" />
</batch:tasklet>
</batch:step>
<batch:step id="notifyException">
-- Do something---
</batch:step>
</batch:job>
Issue : If I specify both as fragmentRootElementNames, only ProductFeatures elements Version, MessageEN, MessageFR are persisted
in Table-A. ProductFeaturesDetail is being ignored.
If I specify just ProductFeaturesDetail, it works fine. But I dont get
the individual elements within ProductFeatures.
I want the data from both elements. What is the way to achieve this?
P.S. I am using HibernateItemWriter for persistence.

Related

How to run spring batch job through CommandLineRunner, if i am using xml based configuration?

I am reading txt file and writing csv file with itemprocessor
Below my xml
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:batch="http://www.springframework.org/schema/batch" `xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd">
<!-- JobRepository and JobLauncher are configuration/setup classes -->
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<!-- ItemReader reads a complete line one by one from input file -->
<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="resource" value="classpath:Test.txt" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="fieldSetMapper">
<!-- Mapper which maps each individual items in a record to properties in POJO -->
<bean class="com.chaman.springbatch.ResultFieldSetMapper" />
</property>
<property name="lineTokenizer">
<!-- A tokenizer class to be used when items in input record are separated by specific characters -->
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<!-- <property name="delimiter" value="|" /> -->
</bean>
</property>
</bean>
</property>
</bean>
<bean id="flatFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">`
`
<property name="resource" value="file:csv/Result.csv" />
<property name="lineAggregator">
<!-- An Aggregator which converts an object into delimited list of strings -->
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<!-- <property name="delimiter" value="|" /> -->
<property name="fieldExtractor">
<!-- Extractor which returns the value of beans property through reflection -->
<bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="number" />
</bean>
</property>
</bean>
</property>
</bean>
<!-- XML ItemWriter which writes the data in XML format -->
<!-- <bean id="xmlItemWriter" class="org.springframework.batch.item.xml.StaxEventItemWriter">
<property name="resource" value="file:xml/examResult.xml" />
<property name="rootTagName" value="UniversityExamResultList" />
<property name="marshaller">
<bean class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
<property name="classesToBeBound">
<list>
<value>com.websystique.springbatch.model.ExamResult</value>
</list>
</property>
</bean>
</property>
</bean> -->
<!-- Optional ItemProcessor to perform business logic/filtering on the input records -->
<bean id="itemProcessor" class="com.chaman.springbatch.ResultItemProcessor" />
<!-- Optional JobExecutionListener to perform business logic before and after the job -->
<bean id="jobListener" class="com.chaman.springbatch.ResultJobListener" />
<!-- Step will need a transaction manager -->
<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<!-- Actual Job -->
<batch:job id="ResultJob">
<batch:step id="step1">
<batch:tasklet transaction-manager="transactionManager">
<batch:chunk reader="flatFileItemReader" writer="flatFileItemWriter" processor="itemProcessor" commit-interval="10" />
</batch:tasklet>
</batch:step>
<batch:listeners>
<batch:listener ref="jobListener" />
</batch:listeners>
</batch:job>
How to run spring batch job through CommandLineRunner, if i am using xml based configuration?
This is explained in the documentation, see Running Jobs from the Command Line. Here is an example:
java CommandLineJobRunner myJob-configuration.xml myJob param=value
package com.paul.testspringbatch;
import java.util.Date;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.boot.CommandLineRunner;
public class MyCommandLineRunner implements CommandLineRunner {
private JobLauncher jobLauncher;
private Job resultJob;
#Override
public void run(String... args) throws Exception {
JobParameters jobParameters = new JobParametersBuilder().addDate("start-date", new Date()).toJobParameters();
this.jobLauncher.run(resultJob, jobParameters);
}
public void setJobLauncher(JobLauncher jobLauncher) {
this.jobLauncher = jobLauncher;
}
public void setResultJob(Job resultJob) {
this.resultJob = resultJob;
}
}
add the bean to your xml:
<bean id="jobLauncher" class="com.paul.testspringbatch.MyCommandLineRunner">
<property name="jobLauncher" ref="jobLauncher" />
<property name="resultJob" ref="ResultJob" />
</bean>
when the application starts up, the job will be excuted.

Using one of job parameter in Item Processor in Spring Batch application

I am trying get job-parameteres-in-to-item-processor-using-spring-batch-annotation.
I implemented it by referring below link; but for me the variable (batchRunName) is coming as null when I try to access it in my Processor class.
Can any one please look at it. I am sure; I am missing some small thing.
How to get Job parameteres in to item processor using spring Batch annotation
public static void main(String[] args) {
contextObj = new ClassPathXmlApplicationContext(springConfig);
jobObj = (Job) contextObj.getBean("XYZ-1001-DD-01");
JobParametersBuilder jobBuilder = new JobParametersBuilder();
System.out.println("args[0] is " + args[0] );
jobBuilder.addString("batchRunName", args[0]);
public class TimeProcessor implements ItemProcessor<Time, TimeMetric> {
private DataSource dataSource;
#Value("#{jobParameters['batchRunNumber']}")
private String batchRunNumber;
public void setBatchRunNumber(String batchRunNumber) {
this.batchRunNumber = batchRunNumber;
}
<bean id="timeProcessor"
class="com.xyz.processor.TimeProcessor" scope="step">
<property name="dataSource" ref="oracledataSource" />
</bean>
=================FULL XML CONFIGURATION========================
<import resource="classpath:/batch/utility/skip/batch_skip.xml" />
<import resource="classpath:/batch/config/context-postgres.xml" />
<import resource="classpath:/batch/config/oracle-database.xml" />
<context:property-placeholder
location="classpath:/batch/jobs/TPF-1001-DD-01/TPF-1001-DD-01.properties" />
<bean id="gridSizePartitioner"
class="com.tpf.partitioner.GridSizePartitioner" />
<task:executor id="taskExecutor" pool-size="${pool.size}" />
<batch:job id="XYZJob" job-repository="jobRepository"
restartable="true">
<batch:step id="XYZSTEP">
<batch:description>Convert TIF files to PDF</batch:description>
<batch:partition partitioner="gridSizePartitioner">
<batch:handler task-executor="taskExecutor"
grid-size="${pool.size}" />
<batch:step>
<batch:tasklet allow-start-if-complete="true">
<batch:chunk commit-interval="${commit.interval}"
skip-limit="${job.skip.limit}">
<batch:reader>
<bean id="timeReader"
class="org.springframework.batch.item.database.JdbcCursorItemReader"
scope="step">
<property name="dataSource" ref="oracledataSource" />
<property name="sql">
<value>
select TIME_ID as timesheetId,count(*),max(CREATION_DATETIME) as creationDateTime , ILN_NUMBER as ilnNumber
from TS_FAKE_NAME
where creation_datetime >= '#{jobParameters['creation_start_date1']} 12.00.00.000000000 AM'
and creation_datetime < '#{jobParameters['creation_start_date2']} 11.59.59.999999999 PM'
and mod(time_id,${pool.size})=#{stepExecutionContext['partition.id']}
group by time_id ,ILN_NUMBER
</value>
</property>
<property name="rowMapper">
<bean
class="org.springframework.jdbc.core.BeanPropertyRowMapper">
<property name="mappedClass"
value="com.tpf.model.Time" />
</bean>
</property>
</bean>
</batch:reader>
<batch:processor>
<bean id="compositeItemProcessor"
class="org.springframework.batch.item.support.CompositeItemProcessor">
<property name="delegates">
<list>
<ref bean="timeProcessor" />
</list>
</property>
</bean>
</batch:processor>
<batch:writer>
<bean id="compositeItemWriter"
class="org.springframework.batch.item.support.CompositeItemWriter">
<property name="delegates">
<list>
<ref bean="timeWriter" />
</list>
</property>
</bean>
</batch:writer>
<batch:skippable-exception-classes>
<batch:include
class="com.utility.skip.BatchSkipException" />
</batch:skippable-exception-classes>
<batch:listeners>
<batch:listener ref="batchSkipListener" />
</batch:listeners>
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:partition>
</batch:step>
<batch:validator>
<bean
class="org.springframework.batch.core.job.DefaultJobParametersValidator">
<property name="requiredKeys">
<list>
<value>batchRunNumber</value>
<value>creation_start_date1</value>
<value>creation_start_date2</value>
</list>
</property>
</bean>
</batch:validator>
</batch:job>
<bean id="timesheetWriter" class="com.tpf.writer.TimeWriter"
scope="step">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="timeProcessor"
class="com.tpf.processor.TimeProcessor" scope="step">
<property name="dataSource" ref="oracledataSource" />
</bean>
I think you are facing the issue reported in BATCH-2351.
You can try to provide the job parameter via XML instead of annotation (Since the majority of your config is XML based):
<bean id="timeProcessor" class="com.xyz.processor.TimeProcessor" scope="step">
<property name="dataSource" ref="oracledataSource" />
<property name="batchRunNumber" value="#{jobParameters['batchRunNumber']}" />
</bean>
Hope this helps.

Spring Batch With Schedular

enter image description hereI am new to Spring Batch with Scheduler. Here my task is to read the data from one table and write it into another table.
I am randomly going through the blogs and different tutorials.
I don't know whether there is any direct approach read from database and write into database. I took this approach like
Job 1 : Reads the data from the db using JdbcCursorItemReader, writing the data into a txt file using FlatFileItemWriter.
Job 2: Read the data from the txt file using FlatFileItemReader, multiResourceItemReader and writing the data into another table using HibernateItemWriter.
I am using a scheduler and it is going to run the batch for every 20 sec.
In this approach for the first run it is working fine. For the second run(after 20 sec), I am updating the data in the database(base table) but it is not writing updated data into the file and database.
Here is my configuration & code`package com.cg.schedulers;
public class UserScheduler {
#Autowired
private JobLauncher launcher;
#Autowired
private Job userJob;
#Autowired
private Job userJob2;
private JobExecution execution1,execution2;
public void run() {
try {
execution1 = launcher.run(userJob, new JobParameters());
execution2 = launcher.run(userJob2, new JobParameters());
System.out.println("Execution status: " + execution1.getStatus());
System.out.println("Execution status: " + execution2.getStatus());
} catch (JobExecutionAlreadyRunningException e) {
e.printStackTrace();
} catch (JobRestartException e) {
e.printStackTrace();
} catch (JobInstanceAlreadyCompleteException e) {
e.printStackTrace();
} catch (JobParametersInvalidException e) {
e.printStackTrace();
}
}
}
Xml Configuration
<import resource="spring-batch1.xml" />
<import resource="springbatch-database.xml" />
<context:annotation-config/>
<context:component-scan base-package="com.cg"/>
<!-- Reading data from -->
<bean id="itemReader"
class="org.springframework.batch.item.database.JdbcCursorItemReader"
scope="step">
<property name="dataSource" ref="dataSource" />
<property name="sql" value="select UserId, UserName, Password from USER" />
<property name="rowMapper">
<bean class="com.cg.mapper.UserRowMapper" />
</property>
</bean>
<!-- ItemWriter writes a line into output flat file -->
<bean id="flatFileItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter"
scope="step">
<property name="resource" value="file:csv/User.txt" />
<property name="lineAggregator">
<!-- An Aggregator which converts an object into delimited list of strings -->
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="," />
<property name="fieldExtractor">
<!-- Extractor which returns the value of beans property through reflection -->
<bean
class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<property name="names" value="userId, username, password" />
</bean>
</property>
</bean>
</property>
</bean>
<!-- ItemReader reads a complete line one by one from input file -->
<bean id="flatFileItemReader" class="org.springframework.batch.item.file.FlatFileItemReader"
scope="step">
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="fieldSetMapper">
<!-- Mapper which maps each individual items in a record to properties
in POJO -->
<bean class="com.cg.mapper.UserFieldSetMapper" />
</property>
<property name="lineTokenizer">
<!-- A tokenizer class to be used when items in input record are separated
by specific characters -->
<bean
class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value="," />
</bean>
</property>
</bean>
</property>
</bean>
<bean id="multiResourceItemReader"
class="org.springframework.batch.item.file.MultiResourceItemReader">
<property name="resources" value="classpath:csv/User.txt" />
<property name="delegate" ref="flatFileItemReader" />
</bean>
<!-- Optional JobExecutionListener to perform business logic before and
after the job -->
<bean id="jobListener" class="com.cg.support.UserItemListener" />
<!-- Optional ItemProcessor to perform business logic/filtering on the input
records -->
<bean id="itemProcessor1" class="com.cg.support.UserItemProcessor" />
<bean id="itemProcessor2" class="com.cg.support.UserItemProcessor2" />
<!-- ItemWriter which writes data to database -->
<bean id="databaseItemWriter"
class="org.springframework.batch.item.database.HibernateItemWriter">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<!-- Actual Job -->
<batch:job id="userJob">
<batch:step id="step1">
<batch:tasklet transaction-manager="transactionManager">
<batch:chunk reader="itemReader" writer="flatFileItemWriter"
processor="itemProcessor1" commit-interval="10" />
</batch:tasklet>
</batch:step>
<batch:listeners>
<batch:listener ref="jobListener" />
</batch:listeners>
</batch:job>
<batch:job id="userJob2">
<batch:step id="step2">
<batch:tasklet transaction-manager="transactionManager">
<batch:chunk reader="multiResourceItemReader" writer="databaseItemWriter"
processor="itemProcessor2" commit-interval="10" />
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="myScheduler" class="com.cg.schedulers.UserScheduler"/>
<task:scheduled-tasks>
<task:scheduled ref="myScheduler" method="run" cron="*/20 * * * * *" />
</task:scheduled-tasks>
Please provide the direct approach if possible using hibernate.
[enter image description here][2]
Execution status: COMPLETED
Thanks,
Vamshi.

Spring Batch Database Dependency pass parameters to SQL statement

I want separate the sql from the batch.xml file, so I defined the sql statement into a properties file. Inside the batch.xml I bind the property-placeholder bean then point to the properties file.
For simple select statement should not be a problem. But if I want to pass the parameter as where clause condition is it possible to do that?
<context:property-placeholder
location="classpath:batch-sql.properties/>
<bean id="secondReader"
class="org.springframework.batch.item.database.JdbcCursorItemReader"
scope="step">
<property name="dataSource" ref="dataSource" />
<property name="sql" value="${sql1}" />
<property name="rowMapper">
<bean class="com.test.batchjob.process.TestPersonMapper" />
</property>
</bean>
This is my sql statment in properties file:
SELECT * FROM Person WHERE id = ?
Can the id pass from jobparameter?
To set the parameters of the query in a JdbcPagingItemReader, you have to use the property parametersValue. This property takes a Map<String,Object> where the key is either the named parameter or the index of the parameter (if you use ?).
<bean id="secondReader"
class="org.springframework.batch.item.database.JdbcPagingItemReader"
scope="step">
<property name="queryProvider">
<bean class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="selectClause" value="select *" />
<property name="fromClause" value="from persons" />
<property name="whereClause" value="where id = ?" />
</bean>
</property>
<property name="parametersValue">
<map>
<entry key="1" value="#{jobParameters['id']}" />
</map>
</property>
<property name="rowMapper">
<bean class="com.test.batchjob.process.TestPersonMapper" />
</property>
</bean>
See documentation : JdbcPagingItemReader
UPDATE
You have to use a QueryProvider instead of sql and datasource properties.
You can replace the text of the query by values of the properties file.
To set the parameters of the query in a JdbcCursorItemReader, you have to use the property preparedStatementSetter. This property takes a PreparedStatementSetter which you have to implement yourself to set either named or index-based parameters.
<bean id="secondReader"
class="org.springframework.batch.item.database.JdbcCursorItemReader"
scope="step">
<property name="sql" value="${sql1}" />
<property name="itemPreparedStatementSetter">
<bean class="xx.xx.xx.YourPreparedStatementSetter">
<property name="id" value="#{jobParameters['id']}" />
</bean>
</property>
<property name="rowMapper">
<bean class="com.test.batchjob.process.TestPersonMapper" />
</property>
An example implementation of a PreparedStatementSetter :
public class YourPreparedStatementSetter implements PreparedStatementSetter {
private String id;
#Override
public void setValues(PreparedStatement ps) throws SQLException {
ps.setString(1, this.id);
}
public setId(String id) {
this.id = id;
}
}
For JdbcCursorItemReader, you can take a look at answer here : Using Spring Batch JdbcCursorItemReader with NamedParameters

How to invoke sftp:outbound-gateway from spring batch?

I would like to invoke sftp:outbound-gateway from batch tasklet in order to download a file from sftp server.
I've seen other posts related to this subject but I'm not sure what am I doing wrong. Could anybody give me a hint based on my configuration? My batch works so the problem is just to ivoke the sftp component in batch step. I've marked the Spring Integration section with comment so it is easier to read just a relevant configuration.
I can see in my logs: DEBUG [o.s.i.e.SourcePollingChannelAdapter] Received no Message during the poll, returning 'false'. So I am not receiving a file but why?
Thanks in advance for your time spend on analysis!
<bean id="ftsSftpClientFactory" class="org.springframework.integration.sftp.session.DefaultSftpSessionFactory">
<property name="host" value="${my.import.sftp.localhost}"/>
<property name="user" value="${my.import.sftp.username}"/>
<property name="password" value="${my.import.sftp.passwort}"/>
</bean>
<!-- Start: Spring Integration -->
<int:channel id="replyChannel" >
<int:queue/>
</int:channel>
<int:channel id="requestChannel" />
<int-sftp:outbound-gateway id="sftpGateway"
session-factory="ftsSftpClientFactory"
request-channel="requestChannel"
reply-channel="replyChannel"
auto-startup="true"
command="get"
command-options="-P"
expression="payload"
remote-directory="."
local-directory="${my.import.sftp.copy.file.destinationpath}">
</int-sftp:outbound-gateway>
<bean name="copyFileTasklet" class="com.mydomain.CopyFileTasklet">
<property name="channel" ref="replyChannel" />
<property name="pollableChannel" ref="requestChannel" />
</bean>
<!-- Start: Spring Batch -->
<bean name="myImportTask" class="com.mydomain.MyImportTask">
<property name="job" ref="unternehmungImportJob"/>
<property name="jobLauncher" ref="jobLauncher"/>
</bean>
<bean id="jobDetail"
class="com.mydomain.MyImportJob">
<property name="myImportTask" ref="myImportTask" />
</bean>
<!--suppress SpringBatchModel -->
<batch:job id="myImportJob">
<batch:step id="copy-file-step" next="my-import-step">
<batch:tasklet ref="copyFileTasklet"/>
</batch:step>
<batch:step id="my-import-step">
<batch:tasklet>
<batch:chunk reader="myItemReader"
writer="myItemWriter"
commit-interval="10000">
<!--
skip-limit="10000"
<batch:skippable-exception-classes>
<batch:include class="java.lang.Exception"/>
<batch:exclude class="java.io.FileNotFoundException"/>
</batch:skippable-exception-classes> -->
</batch:chunk>
<batch:transaction-attributes isolation="DEFAULT" propagation="REQUIRED"/>
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="myItemReader" scope="step" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="linesToSkip" value="1"/>
<property name="encoding" value="${my.import.batch.encoding}" />
<property name="resource" value="${my.import.batch.input.resource}"/>
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer" ref="lineTokenizer"/>
<property name="fieldSetMapper">
<bean class="com.mydomain.MyImportMapper"/>
</property>
</bean>
</property>
</bean>
<bean id="myItemWriter" class="com.mydomain.MyItemWriter">
<property name="myApplicationService" ref="defaultmyApplicationService" />
</bean>
<bean id="lineTokenizer" class="com.mydomain.DelimitedLineTokenizerWithEOF">
<property name="delimiter" value="${my.import.batch.delimiter}" />
<property name="eofMarker" value="${my.import.batch.eof.marker}" />
</bean>
public class CopyFileTasklet implements Tasklet {
private MessageChannel requestChannel;
private PollableChannel replyChannel;
public void setRequestChannel(MessageChannel requestChannel) {
this.requestChannel = requestChannel;
}
public void setReplyChannel(PollableChannel replyChannel) {
this.replyChannel = replyChannel;
}
#Override
public RepeatStatus execute(StepContribution stepContribution, ChunkContext chunkContext) throws Exception {
Message<?> result = replyChannel.receive(10000);
Object file = result.getPayload();
return RepeatStatus.FINISHED;
}
}
Your issue that you don't inititate Integration Flow from your custom Tasklet. Of course you can't receive anything from the replyChannel, if you haven't sent request before.
If you just need to process Integration Flow and get result from it, it would be better to use POJI <gateway> from that Tasklet:
public interface SftpGateway {
File download(String fileName);
}
<gateway id="sftpGateway" service-interface="com.my.proj.SftpGateway"
default-request-channel="requestChannel"/>
<bean name="copyFileTasklet" class="com.mydomain.CopyFileTasklet">
<property name="sftpGateway" ref="sftpGateway" />
</bean>
Something like that.