JPA transaction handling between #Stateless and #Asynchronous EJBs - jpa

I have a stateless EJB which inserts data into database, sends a response immediately and in the last step calls an asynchronous EJB. Asynchronous EJB can run for long (I mean 5-10 mins which is longer then JPA transaction timeout). The asynchronous ejb needs to read (and work on it) the same record tree (only read) as the one persisted by stateless EJB.
Is seems that the asynchronous bean tries to read the record tree before it was commited or inserted (JPA) by the statelsss EJB so record tree is not visible by async bean.
Stateless EJB:
#Stateless
public class ReceiverBean {
public void receiverOfIncomingRequest(data) {
long id = persistRequest(data);
sendResponseToJmsBasedOnIncomingData(data);
processorAsyncBean.calculate(id);
}
}
}
Asynchronous EJB:
#Stateless
public class ProcessorAsyncBean {
#Asynchronous
public void calculate(id) {
Data data = dao.getById(id); <- DATA IS ALLWAYS NULL HERE!
// the following method going to send
// data to external system via internet (TCP/IP)
Result result = doSomethingForLongWithData(data);
updateData(id, result);
}
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void updateData(id, result) {
dao.update(id, result);
}
Maybe I can use a JMS queue to send a signal with ID to the processor bean instead of calling asyc ejb (and message driven bean read data from database) but I want to avoid that if possible.
Another solution can be to pass the whole record tree as a detached JPA object to the processor async EJB instead of reading data back from database.
Can I make async EJB work well in this structure somehow?
-- UPDATE --
I was thinking about using Weblogic JMS. There is another issue here. In case of big load, when there are 100 000 or more data in queue (that will be normal) and there is no internet connection then all of my data in the queue will fail. In case of that exception (or any) appears during sending data via internet (by doSomethingForLongWithData method) the data will be rollbacked to the original queue based on the redelivery-limit and repetitaion settings of Weblogic. This rollback event will generate 100 000 or more threads on Weblogic in the managed server to manage redelivery. That new tons of background processes can kill or at least slow down the server.
I can use IBM MQ as well because we have MQ infrastructure. MQ does not have this kind of affect on Weblogic server but MQ does not have redelivery-limit and delay function. So in case of error (rollback) the message will appear immediately on the MQ again, without delay and I built a hand mill. Thread.sleep() in the catch condition is not a solution in EE application I guess...

Is seems that the asynchronous bean tries to read the record tree before it was commited or inserted (JPA) by the statelsss EJB so record tree is not visible by async bean.
This is expected behavior with bean managed transactions. Your are starting the asynchronous EJB from the EJB with its own transaction context. The asynchronous EJB never uses the callers transaction context (see EJB spec 4.5.3).
As long as you are not using transaction isolation level "read uncommited" with your persistence, you won't see the still not commited data from the caller.
You must think about the case, when the asynch job won't commit (e.g. applicationserver shutdown or abnormal abortion). Is the following calculation and update critical? Is the asynchronous process recoverable if not executed successfully or not even called?
You can think about using bean managed transactions, commiting before calling the asynchronous EJB. Or you can delegate the data update to another EJB with a new transactin context. This will be commited before the call of the asynchronous EJB. This is usally ok for uncritical stuff, missing or failing.
Using persistent and transactional JMS messages along with a dead letter queue has the advantage of a reliable processing of your caclulation and update, even with stopping / starting application server in between or with temporal errors during processing.

You just need to call async method next to the one with transaction markup, so when transaction is committed.
For example, caller of receiverOfIncomingRequest() method, could add
processorAsyncBean.calculate(id);
call next to it.
UPDATE : extended example
CallerMDB
#TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)
public void onMessage(Message message) {
long id = receiverBean.receiverOfIncomingRequest(data);
processorAsyncBean.calculate(id);
}
ReceiverBean
#TransactionAttribute(TransactionAttributeType.REQUIRED)
public long receiverOfIncomingRequest(data) {
long id = persistRequest(data);
sendResponseToJmsBasedOnIncomingData(data);
return id;
}

Related

Spring Batch partitioned job JMS acknowledgement

Let's say I have a Spring Batch remote partitioned job, i.e. I have a manager application instance which starts the job and partitions the work and I have multiple workers who are executing individual partitions.
The message channel where the partitions are sent to the workers is an ActiveMQ queue and the Spring Integration configuration is based on JMS.
Assume that I wanna make sure that in case of a worker crashing in the middle of the partition execution, I want to make sure that another worker will pick up the same partition.
I think here's where acknowledging JMS messages would come in handy to only acknowledge a message in case a worker has fully completed its work on a particular partition but it seems as soon as the message is received by a worker, the message is acknowledged right away and in case of failures in the worker Spring Batch steps, the message won't reappear (obviously).
Is this even possible with Spring Batch? I've tried transacted sessions too but it doesn't really work either.
I know how to achieve this with JMS API. The difficulty comes from the fact that there is a lot of abstraction with Spring Batch in terms of messaging, and I'm unable to figure it out.
I know how to achieve this with JMS API. The difficulty comes from the fact that there is a lot of abstraction with Spring Batch in terms of messaging, and I'm unable to figure it out.
In this case, I think the best way to answer this question is to remove all these abstractions coming from Spring Batch (as well as Spring Integration), and try to see where the acknowledgment can be configured.
In a remote partitioning setup, workers are listeners on a queue in which messages coming from the manager are of type StepExecutionRequest. The most basic code form of a worker in this setup is something like the following (simplified version of StepExecutionRequestHandler, which is configured as a Spring Integration service activator when using the RemotePartitioningWorkerStepBuilder):
#Component
public class BatchWorkerStep {
#Autowired
private JobRepository jobRepository;
#Autowired
private StepLocator stepLocator;
#JmsListener(destination = "requests")
public void receiveMessage(final Message<StepExecutionRequest> message) throws JMSException {
StepExecutionRequest request = message.getObject();
Long jobExecutionId = request.getJobExecutionId();
Long stepExecutionId = request.getStepExecutionId();
String stepName = request.getStepName();
StepExecution stepExecution = jobRepository.getStepExecution(jobExecutionId, stepExecutionId);
Step step = stepLocator.getStep(stepName);
try {
step.execute(stepExecution);
stepExecution.setStatus(BatchStatus.COMPLETED);
} catch (Throwable e) {
stepExecution.addFailureException(e);
stepExecution.setStatus(BatchStatus.FAILED);
} finally {
jobRepository.update(stepExecution); // this is needed in a setup where the manager polls the job repository
}
}
}
As you can see, the JMS message acknowledgment cannot be configured on the worker side (there is no way to do it with attributes of JmsListener, so it has to be done somewhere else. And this is actually at the message listener container level with DefaultJmsListenerContainerFactory#setSessionAcknowledgeMode.
Now if you are using Spring Integration to configure the messaging middleware, you can configure the acknowledgment mode in Spring Integration .

JPA locks PESSEMISTIC_WRITE and FEW transactions

I'm using PESSEMSTIC_WRITE lock on my repository method. So that is locks my object till end of transaction. However, I've got a problem, within one endpoint, controller -> service -> I start transaction then I need to update my object and send message to kafka, after that I need within this method again update my object and send to kafka. So because it's one transaction, changes works only local in cache. But I need to save in database then send to kafka, then again change my object and save to database and send to kafka message, I can't use REQUIRES_NEW and create a new transaction in any way, because my object is locked. So how I can deal with it?
This lock is used in many parts of my project to fix parallel transactions.
You should create new service which will orchestrate the flow. That way you will be able to obtain the same pessimistic lock again in the second operation.
#Service
class OrchestratorService {
...
void executeFlow() {
someService.executeFirstOperationAndSendKafkaEvent()
someService.executeSecondOperationAndSendKafkaEvent()
}
}
#Service
class SomeService {
#Transactional(REQUIRES_NEW)
void executeFirstOperationAndSendKafkaEvent() {
// any lock which obtained inside this method will be released once this method finishes
...
}
#Transactional(REQUIRES_NEW)
void executeSecondOperationAndSendKafkaEvent() {
// any lock which obtained inside this method will be released once this method finishes
...
}
}
There is one more important aspect worth to mention - sending kafka event is not transactional. #Transactional guarantees only that changes made to datasource will be transactional (in this case DB). Hence following scenarios are possible:
if event is sent inside transaction scope, transaction can be rollbacked after succesfull sending kafka event
if event is sent outside transaction commit, event sending may fail after succesful commiting transaction
Due to this nature it's good to split the process into few phases:
apply business changes in DB and store a flag in DB that kafka event should be sent, but it hasn't been done yet,
outside TX scope send event to kafka
in new TX change the flag that event has been sent, or schedule retry if there was error during sending event.

Vertx EventBus blocked

I have a small vertx application. A http verticle gets a request and sends it over eventbus with request-response pattern. So something like:
vertx.eventBus().request(queue, request, options, reply -> {
if (reply.succeeded()) {
JsonObject body = (JsonObject) reply.result().body();
context.response().end(body.encode());
} else {
JsonObject result = new JsonObject().put("errorMessage", reply.cause().getMessage());
context.response().end(result.encode());
}
});
In the DB Vertical i use the consumer to get a message to go to DB, do some changes and send back to HTTP verticle.
My problem is, i have a delete action that must do a lot of checks, so this process can take up to 10 seconds. In this moment HTTP verticle can still get some new requests, but DB consumer does not receive anything until the delete action is done. So no requests are processed. The only thing that helps is setmultithreaded to DB verticle and that is depricated. Vertx.executeBlocking or JAVA Thread pool around DB execution also does not help, as consumer just does not get anything until it replies.
Do i miss something?
Thank you
I take from your question that the DB verticle is deployed with one instance. The DB verticle needs to be deployed as a worker. You can also deploy multiple instances of this verticle so that you always have one DB verticle that can take the next request.
Suggestion for optimization: If only the delete action is taking up so much time, separate this action in a special DB verticle. In this way your system is more responsive and you are able to control how many of the DB-"delecte-action"-Verticles are deployed and thus how many connections to the database are at may blocked for a longer time.

On Partitions Assignment and ChainedKafkaTransactionManager at startup with JPA

I have many transactional consumers with a ChainedKafkaTransactionManager based on a JpaTransactionManager and a KafkaTransactionManager (all #KafkaListener's).
The JPA one needs a ThreadLocal variable to be set, to be able to know to which DB to connect to (is the tenant id).
When starting the application, in the onPartitionsAssigned listener, spring-kafka is trying to create a chained txn, hence trying to create a JPA txn, but there's no tenant set, then it fails.
That tenant is set through a http filter and/or kafka interceptors (through event headers).
I tried using the auto-wired KafkaListenerEndpointRegistry with setAutoStartup(false), but I see that the consumers don't receive any events, probably because they aren't initialized yet (I thought they were initialized on-demand).
If I set a mock tenant id and call registry.start() when the application is ready, the initializations seem to be done in other threads (probably because I'm using a ConcurrentKafkaListenerContainerFactory), so it doesn't work.
Is there a way to avoid the JPA transaction on that initial onPartitionsAssigned listener, that is part of the consumer initialization?
If your chained TM has the KafkaTM first, followed by JPA TM (which would be the normal case), you can achieve similar functionality by just injecting the Kafka TM into the container and using #Transactional (with just the JPA TM on the listener) to start the JPA transaction when the listener is called.
The time between the transaction commits will be marginally increased but it would provide similar functionality.
If that won't work for you, open a GitHub issue; we can either disable the initial commit on assignment, or do it without a transaction at all (optionally).

JPA EntityManager - when the transaction starts?

I am confused about the life cycle of the transactions, the entitymanagers and the persistence context in the EJB container.
I use the entitymanager this way:
#PersistenceContext(unitName = "..")
private EntityManager em;
in every stateless ejb.
My question is as simple as:
When the transaction starts ?
How the transaction is propagated ? ie when stateless ejbs call each others, does they keep using the same transaction ?
When the transaction is committed ?
For container-managed transactions:
The transaction (TX) starts when the first transactional method is invoked. Per default, all EJB methods are transactional ( equivalent to TransactionAttributeType.REQUIRED, which is the default setting).
The default TX propagation keeps the same TX over all local EJB calls. This is equivalent to an explicit TrasactionAttributeType.REQUIRED on all invoked methods
The transaction is committed when the first method in the invocation chain (the one the TX has been created for) returns.
You can have a fine-grained control over the TX propagation by annotating your EJB methods with different TransactionAttributeTypes.