Spring Integration poller starting same Spring Batch job multiple times - spring-batch

Our application uses Spring Integration to kick off a Spring Batch job.
It works as follows:
1) The main application class is run, and this loads the Spring application context.
2) A Spring Integration bean is configured to read a file from the file system, and to place the file on a channel.
<int-file:inbound-channel-adapter
directory="${...}" channel="channel"
filename-pattern="*.csv" auto-create-directory="false" prevent-duplicates="true">
<int:poller fixed-delay="${...}"></int:poller>
</int-file:inbound-channel-adapter>
3) The channel connects to a #ServiceActivator bean.
<int:service-activator input-channel="channel" ref="launcher" />
4) The Launcher bean gets the file from the message payload, and launches the Spring Batch job.
The problem is that after the poller fixed-delay time has elapsed, the launcher bean will be called again, and a new Job will be started with the same parameters.
This throws a JobInstanceAlreadyCompleteException.
I do not see anyway to tell the poller to only run once.
What is the recommended way to use Spring Integration and Spring Batch together to avoid this issue?
Thanks

Related

Spring batch integration using OutBoundGateway and ReplyingKafkaTemplate

My Goal
I need to read a file and divide each line as a message and send to kafka from a spring batch project and another spring integration project will be receiving the messages to process it in a async way. I want to return those messages after processing to the batch project and create 4 different files out of those messages.
Here I am trying to use OutBoundGateway and ReplyingKafkaTemplate. I am unable to configure it properly... Is there any example or reference guide to configure it.
I have checked spring batch integration samples github repository... There is no sample for outBoundGateway or ReplyingKafkaTemplate.
Thanks in Advance.
For ReplyingKafkaTemplate logic in Spring Integration there is a dedicated KafkaProducerMessageHandler which can be configured with a ReplyingKafkaTemplate.
See more info in docs:
https://docs.spring.io/spring-integration/docs/current/reference/html/kafka.html#kafka-outbound-gateway
And more about ReplyingKafkaTemplate:
https://docs.spring.io/spring-kafka/reference/html/#replying-template
Probably on the other a KafkaInboundGateway must be configured, respectively:
https://docs.spring.io/spring-integration/docs/current/reference/html/kafka.html#kafka-inbound-gateway

spring cloud stream app starter File Source to Spring Batch Cloud Task

I have a spring batch boot app which takes a flat file as input . I converted the app into cloud task and deployed in spring local data flow server. Next , I created a stream starting with File Source -> tasklaunchrequest-transform -> task-launcher-local which starts my batch cloud task app .
It looks like that the File does not come into the batch app . I do not see anything in the logs to indicate that.
I checked the docs at https://github.com/spring-cloud-stream-app-starters/tasklaunchrequest-transform/tree/master/spring-cloud-starter-stream-processor-tasklaunchrequest-transform
It says
Any input type. (payload and header are discarded)
My question is how do I pass the file as payload from File Source to the Batch app which seems to be a very basic feature.
any help is very much appreciated.
You'll need to write your own transformer that takes the data from the source and packages it up so your task can consume it.

springboot causing 2phase transaction commit when not needed

I have a standalone application A that invokes a webservice B deployed in jboss eap 6.2.3. the application in jboss uses a mysql datasource. This application integration is working very well.
I needed to webify the standalone application itself into a spring rest data jpa microservice
So I wrote a spring boot wrapper for that standalone application A. It runs in an embedded tomcat and invokes the aplpication B running in jboss.
I also ported some configurations logic from properties files into a embedded h2 database.
Now at places I am seeing this exception shown below for insert into the mysql by the application B inside jboss. My guess is what was a simple transaction earlier and working well has now become a 2 phase commit that sometimes fails.
How to prevent this?
Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction.
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177)
at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:92) [jboss-as-ejb3-7.3.3.Final-redhat-SNAPSHOT.jar:7.3.3.Final-redhat-SNAPSHOT]
... 67 more
Caused by: org.infinispan.CacheException: Could not prepare.
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70) [infinispan-core-5.2.7.Final-redhat-2.jar:5.2.7.Final-redhat-2]
at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273)
at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93)
at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
... 70 more
Caused by: javax.transaction.xa.XAException
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161) [infinispan-core-5.2.7.Final-redhat-2.jar:5.2.7.Final-redhat-2]
at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123) [infinispan-core-5.2.7.Final-redhat-2.jar:5.2.7.Final-redhat-2]
at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68) [infinispan-core-5.2.7.Final-redhat-2.jar:5.2.7.Final-redhat-2]
... 75 more
Spring Boot has out-of-the-box support for JTA. When it runs in an environment where a JTA transaction manager is available, it uses that rather than creating a local transaction manager for your data store. If that's not what you want, you can disable jta support by adding spring.jta.enabled=false to your configuration.
Thanks for the answer.Will be useful in some scenarios.
However after posting I realised the flaw in the question.
standalone application A
webified application(even if a microservice runninging embeddedtomcat ) say A1
Both are actually just rest clients to rest services running in a webservice B deployed in jboss eap 6.2.3..
A and A1 might as well be replaced by just a simple human visitor visting from a browser the application B.
I dont think A or A1 can influence the transactions in any ways for B.
Right?
Anyways the problem was due to the closeness to B of the application A1.
The actual problem was related to a cluster event for the cluster in which B was deployed and triggering of it by A1. A bit complicated scenario. The only problem was that the insert was being attempted too soon before the cluster change event could even reach.

Configuring Spring Batch jobs via database instead of xml

i am new to spring batch,i need few clarifications regarding spring batch admin.
can i do job configuration related information in database instead of uploading XML file based configuration???

sftp channel outbound adapter with retry

I am using spring batch and spring integration where once my batch job is completed,it creates texts files and those needs to be uploaded to some ftp server. Sometimes we noticed that those connections drops and it needs to be retried. Is there anyway we can use spring retry project to try few sec later to see if it can upload those files. we want it to be configurable.
If so is there any example out there.
Thanks
Yes, Spring Integration provide retry component for you. It is called RequestHandlerRetryAdvice:
<int-sftp:outbound-channel-adapter>
<int-sftp:request-handler-advice-chain>
<bean class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice" />
</int-sftp:request-handler-advice-chain>
</int-sftp:outbound-channel-adapter>
Please, find more info in the Reference Manual.
Consider to you use RequestHandlerCircuitBreakerAdvice also for your "connections drops" cases.
And here you are the sample.