JBPM - Catch Intermediate Signal fails sometime - drools

I have added "Signal Intermediate Event" to a human-task as a boundary-event as I have given in previous question.
Sometimes the signal is processed successfully and sometimes it is NOT.
JBPM runtime just updates the process-instance-info and doesn't process the signal.
I am using StatefulKnowledgeSession.signalEvent() , it is just updating the ProcessInstanceInfo at the backend and the event doesn't cancel the current activity in progress
What could be the problem ? Any bug related to this 'Signal Intermediate Event'? .
LOG:
08:34:38,955 INFO [stdout] (http--0.0.0.0-8280-20) 2013-03-13 08:34:38,954 [http--0.0.0.0-8280-20] DEBUG web.mvc.controller.SignalController - A new PROCESS signal recieved ..putProcessOnHOLD
08:34:38,966 INFO [stdout] (http--0.0.0.0-8280-20) 2013-03-13 08:34:38,966 [http--0.0.0.0-8280-20] DEBUG org.drools.container.spring.beans.persistence.DroolsSpringTransactionManager - Current TX name (According to TransactionSynchronizationManager) : core.service.impl.event.ExternalEventManagerImpl.dispatchSignal
08:34:38,978 INFO [stdout] (http--0.0.0.0-8280-20) 2013-03-13 08:34:38,978 [http--0.0.0.0-8280-20] DEBUG org.drools.container.spring.beans.persistence.DroolsSpringTransactionManager - Current TX: org.springframework.transaction.support.DefaultTransactionStatus#3dda5edd
08:34:38,987 INFO [stdout] (http--0.0.0.0-8280-20) Hibernate: select processins0_.InstanceId as InstanceId1_0_, processins0_.id as id1_0_, processins0_.lastModificationDate as lastModi3_1_0_, processins0_.lastReadDate as lastRead4_1_0_, processins0_.processId as processId1_0_, processins0_.processInstanceByteArray as processI6_1_0_, processins0_.startDate as startDate1_0_, processins0_.state as state1_0_, processins0_.OPTLOCK as OPTLOCK1_0_ from ProcessInstanceInfo processins0_ where processins0_.InstanceId=?
08:34:39,014 INFO [stdout] (http--0.0.0.0-8280-20) Hibernate: update ProcessInstanceInfo set id=?, lastModificationDate=?, lastReadDate=?, processId=?, processInstanceByteArray=?, startDate=?, state=?, OPTLOCK=? where InstanceId=? and OPTLOCK=?
Environment : JBPM 5.4.0.Final, Jboss 7.1.0.Final

When you say that the engine updates the ProcessInstanceInfo, I assume you are referring to the last read date only (in case the process instance is not moving forward as expected)? Or other fields as well?
The engine should process each request the same way. So I assume that the process instance itself might not always be in the same state? If the signal for example arrives before or after the human task is active, it will not cause any changes to the process instance itself.

Related

Thread blocked while try closing a mongoclient

I have an app that use multiple databases, so I create multiple connections and keep them in a concurrenthashmap.
If a connection doesn't exist I simply create a new one and put it into that map, and we will close and remove some of the client if that task get done.
After many rounds of the create-close-remove operations, exception happens while close some of that client, it says thread has been blocked.
Why close a mongo client end up with thread block for that long period of time, how can I avoid this problem from happen?
THX!
Vertx version: 4.2.3
Java version : 1.8.0_201
02:10:05.626 WARN i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-27,5,main] has been blocked for 5058 ms, time limit is 2000 ms
io.vertx.core.VertxException: Thread blocked
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at com.mongodb.internal.connection.AsynchronousChannelStream$FutureAsyncCompletionHandler.get(AsynchronousChannelStream.java:317)
at com.mongodb.internal.connection.AsynchronousChannelStream$FutureAsyncCompletionHandler.getRead(AsynchronousChannelStream.java:312)
at com.mongodb.internal.connection.AsynchronousChannelStream.read(AsynchronousChannelStream.java:151)
at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:648)
at com.mongodb.internal.connection.InternalStreamConnection.receiveMessageWithAdditionalTimeout(InternalStreamConnection.java:513)
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:356)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:280)
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:107)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:62)
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:144)
at com.mongodb.internal.connection.UsageTrackingInternalConnection.open(UsageTrackingInternalConnection.java:51)
at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.open(DefaultConnectionPool.java:431)
at com.mongodb.internal.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:115)
at com.mongodb.internal.connection.DefaultConnectionPool.get(DefaultConnectionPool.java:100)
at com.mongodb.internal.connection.DefaultServer.getConnection(DefaultServer.java:92)
at com.mongodb.internal.session.ServerSessionPool.endClosedSessions(ServerSessionPool.java:156)
at com.mongodb.internal.session.ServerSessionPool.close(ServerSessionPool.java:103)
at com.mongodb.internal.async.client.AsyncMongoClientImpl.close(AsyncMongoClientImpl.java:119)
at com.mongodb.reactivestreams.client.internal.MongoClientImpl.close(MongoClientImpl.java:65)
at io.vertx.ext.mongo.impl.MongoClientImpl$MongoHolder.close(MongoClientImpl.java:1283)
at io.vertx.ext.mongo.impl.MongoClientImpl.close(MongoClientImpl.java:132)
at io.vertx.ext.mongo.impl.MongoClientImpl.close(MongoClientImpl.java:1243)
at io.vertx.reactivex.ext.mongo.MongoClient.close(MongoClient.java:1921)
at io.vertx.reactivex.ext.mongo.MongoClient.close(MongoClient.java:1928)

How to setup Narayana ConnectionManager so it doesn't stop after some transactions

I'm using Spring Boot, Spring Session and JTA Narayana (arjuna), I'm sending select and insert statements in a loop using two different threads.
The application runs correctly for some time but after some number of transactions, the Arjuna ConnectionManager fails to get a connection and generates the following exception:
2019-10-05 22:48:20.724 INFO 27032 --- [o-auto-1-exec-4] c.m.m.db.PrepareStatementExec : START select
2019-10-05 22:49:20.225 WARN 27032 --- [nsaction Reaper] com.arjuna.ats.arjuna : ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffc0a82101:c116:5d989ef0:6e in state RUN
2019-10-05 22:49:20.228 WARN 27032 --- [Reaper Worker 0] com.arjuna.ats.arjuna : ARJUNA012095: Abort of action id 0:ffffc0a82101:c116:5d989ef0:6e invoked while multiple threads active within it.
2019-10-05 22:49:20.234 WARN 27032 --- [Reaper Worker 0] com.arjuna.ats.arjuna : ARJUNA012381: Action id 0:ffffc0a82101:c116:5d989ef0:6e completed with multiple threads - thread http-nio-auto-1-exec-10 was in progress with java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
com.arjuna.ats.internal.jdbc.ConnectionManager.create(ConnectionManager.java:134)
com.arjuna.ats.jdbc.TransactionalDriver.connect(TransactionalDriver.java:89)
java.sql.DriverManager.getConnection(DriverManager.java:664)
java.sql.DriverManager.getConnection(DriverManager.java:208)
com.mono.multidatasourcetest.db.PrepareStatementExec.executeUpdate(PrepareStatementExec.java:51)
Source code is in github https://github.com/saavedrah/multidataset-test
I'm wondering if the connection should be closed or if I should change some settings in Arjuna to make the ConnectionManager work.
although what you are showing is a stack trace being printed by the Narayana BasicAction class (rather than an exception) the result for you is ultimately the same and you need to close your connections.
You should most likely look to add it in close to the same place you are doing the getConnection calls within https://github.com/saavedrah/multidataset-test/blob/cf910c345db079a4e910a071ac0690af28bd3f81/src/main/java/com/mono/multidatasourcetest/db/PrepareStatementExec.java#L38
e.g.
//connection = getConnection
//do something with it
//connection.close()
But as Connection is AutoCloseable you could just do:
try (Connection connection = DriverManager.getConnection) {
connnection.doSomething();
}

Getting exception while doing block() on Mono object I got back from ReactiveMongoRepository object

I have a service that streams data to a second service that receives stream of objects and saves them to my MongoDB.
inside my subscribe function on the Flux object that I get from the streaming service I use the save method from the ReactiveMongoRepository interface.
when I try to use the block function and get the data I get the following error :
2019-10-11 13:30:38.559 INFO 19584 --- [localhost:27017] org.mongodb.driver.connection : Opened connection [connectionId{localValue:1, serverValue:25}] to localhost:27017
2019-10-11 13:30:38.566 INFO 19584 --- [localhost:27017] org.mongodb.driver.cluster : Monitor thread successfully connected to server with description ServerDescription{address=localhost:27017, type=STANDALONE, state=CONNECTED, ok=true, version=ServerVersion{versionList=[4, 0, 1]}, minWireVersion=0, maxWireVersion=7, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=6218300}
2019-10-11 13:30:39.158 INFO 19584 --- [ctor-http-nio-4] quote-monitor-service : onNext(Quote(id=null, ticker=AAPL, price=164.8, instant=2019-10-11T10:30:38.800Z))
2019-10-11 13:30:39.411 INFO 19584 --- [ctor-http-nio-4] quote-monitor-service : cancel()
2019-10-11 13:30:39.429 INFO 19584 --- [ntLoopGroup-2-2] org.mongodb.driver.connection : Opened connection [connectionId{localValue:3, serverValue:26}] to localhost:27017
2019-10-11 13:30:39.437 WARN 19584 --- [ctor-http-nio-4] io.netty.util.ReferenceCountUtil : Failed to release a message: DefaultHttpContent(data: PooledSlicedByteBuf(freed), decoderResult: success)
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
at io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[netty-common-4.1.39.Final.jar:4.1.39.Final]
at io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) ~[netty-common-4.1.39.Final.jar:4.1.39.Final]
at
reactor.core.Exceptions$ErrorCallbackNotImplemented: java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-http-nio-4
Caused by: java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-http-nio-4
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:77) ~[reactor-core-3.2.12.RELEASE.jar:3.2.12.RELEASE]
at reactor.core.publisher.Mono.block(Mono.java:1494) ~[reactor-core-3.2.12.RELEASE.jar:3.2.12.RELEASE]
at
my code:
stockQuoteClient.getQuoteStream()
.log("quote-monitor-service")
.subscribe(quote -> {
Mono<Quote> savedQuote = quoteRepository.save(quote);
System.out.println("I saved a quote! Id: " +savedQuote.block().getId());
});
after some digging, I manage to get it to work but I don't understand why it works now.
the new code:
stockQuoteClient.getQuoteStream()
.log("quote-monitor-service")
.subscribe(quote -> {
Mono<Quote> savedQuote = quoteRepository.insert(quote);
savedQuote.subscribe(result ->
System.out.println("I saved a quote! Id :: " + result.getId()));
});
the definition of block(): Subscribe to this Mono and block indefinitely until a next signal is received.
the definition of subscribe(): Subscribe to this Mono and request unbounded demand.
can someone help me understand why the block didn't work and the subscribe worked?
what am I missing here?
Blocking is bad, since it ties up a thread waiting for a response. It's very bad in a reactive framework which has few threads at its disposal, and is designed so that none of them should be unnecessarily blocked.
This is the very thing that reactive frameworks are designed to avoid, so in this case it simply stops you doing it:
block()/blockFirst()/blockLast() are blocking, which is not supported in thread reactor-http-nio-4
Your new code, in contrast, works asynchronously. The thread isn't blocked, as nothing actually happens until the repository returns a value (and then the lambda that you passed to savedQuote.subscribe() is executed, printing out you result to the console.)
However, the new code still isn't optimal / normal from a reactive streams perspective, as you're doing all your logic in your subscribe method. The normal thing to do is to us a series of flatMap/map calls to transform the items in the stream, and use doOnNext() for side effects (such as printing out a value):
stockQuoteClient.getQuoteStream()
.log("quote-monitor-service")
.flatMap(quoteRepository::insert)
.doOnNext(result -> System.out.println("I saved a quote! Id :: " + result.getId())))
.subscribe();
If you're doing any serious amount of work with reactor / reactive streams, it would be worth reading up on them in general. They're very powerful for non-blocking work, but they do require a different way of thinking (and coding) than more "standard" Java.

JBoss 5.1 cluster fails with EAR

We have 2 servers that cluster just fine when we do not deploy any EAR files.
Server 1:
2015-03-26 08:23:00,339 INFO [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-13,10.200.51.14:62610) org.jboss.messaging.core.impl.postoffice.GroupMember$ControlMembershipListener#1f15227c got new view [10.200.51.14:62610|1] [10.200.51.14:62610, 10.200.51.16:58992], old view is [10.200.51.14:62610|0] [10.200.51.14:62610]
2015-03-26 08:23:00,339 INFO [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-13,10.200.51.14:62610) I am (10.200.51.14:62610)
2015-03-26 08:23:00,339 INFO [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-13,10.200.51.14:62610) New Members : 1 ([10.200.51.16:58992])
2015-03-26 08:23:00,355 INFO [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-13,10.200.51.14:62610) All Members : 2 ([10.200.51.14:62610, 10.200.51.16:58992])
2015-03-26 08:24:32,140 INFO [org.jboss.cache.RPCManagerImpl] (Incoming-16,10.200.51.14:62610) Received new cluster view: [10.200.51.14:62610|2] [10.200.51.14:62610]
Server2:
2015-03-26 08:23:00,011 INFO [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) All Members : 2 ([10.200.51.14:62610, 10.200.51.16:58992])
Multicast is successfully configured and working.
The clustering does NOT occur when EAR files are included at JBoss startup.
We see NAKACK messages (on Server1) when Server2 starts... but clustering does not complete.
2015-03-26 14:28:41,105 ERROR [org.jgroups.protocols.pbcast.NAKACK] (Incoming-2,10.200.51.14:7900) sender 10.200.51.16:7900 not found in xmit_table
2015-03-26 14:28:41,105 ERROR [org.jgroups.protocols.pbcast.NAKACK] (Incoming-2,10.200.51.14:7900) range is null
We see multiple NAKACK messages (on Server2) when it starts:
2015-03-26 14:27:47,488 WARN [org.jgroups.protocols.pbcast.NAKACK] (OOB-4,10.200.51.16:50648) 10.200.51.16:50648] discarded message from non-member 10.200.51.14:59139, my view is [10.200.51.16:50648|0] [10.200.51.16:50648]
2015-03-26 14:27:47,675 WARN [org.jgroups.protocols.pbcast.NAKACK] (OOB-4,10.200.51.16:50648) 10.200.51.16:50648] discarded message from non-member 10.200.51.14:59139, my view is [10.200.51.16:50648|0] [10.200.51.16:50648]
and
2015-03-26 14:28:34,038 WARN [org.jgroups.protocols.pbcast.NAKACK] (Incoming-4,10.200.51.16:50648) 10.200.51.16:50648] discarded message from non-member 10.200.51.14:59139, my view is [10.200.51.16:50648|0] [10.200.51.16:50648]
2015-03-26 14:28:34,038 ERROR [org.jgroups.protocols.pbcast.NAKACK] (Incoming-9,10.200.51.16:50648) sender 10.200.51.14:59139 not found in xmit_table
2015-03-26 14:28:34,038 ERROR [org.jgroups.protocols.pbcast.NAKACK] (Incoming-9,10.200.51.16:50648) range is null
2015-03-26 14:28:40,356 ERROR [org.jgroups.protocols.pbcast.NAKACK] (Incoming-2,10.200.51.16:7900) sender 10.200.51.14:7900 not found in xmit_table
2015-03-26 14:28:40,356 ERROR [org.jgroups.protocols.pbcast.NAKACK] (Incoming-2,10.200.51.16:7900) range is null
We also see a JBoss messaging error on Server2 after the EAR file completes its deployment:
2015-03-26 14:32:38,557 ERROR [org.jboss.messaging.util.ExceptionUtil] (WorkManager(2)-7) SessionEndpoint[ej-53z3kq7i-1-gg3wjq7i-1d1bnk-gf1k5a] createConsumerDelegate [4k-dcm4kq7i-1-gg3wjq7i-1d1bnk-gf1k5a]
java.lang.IllegalStateException: org.jboss.messaging.core.impl.postoffice.GroupMember#692dba20 response not received from 10.200.51.14:59139 - there may be others
at org.jboss.messaging.core.impl.postoffice.GroupMember.multicastControl(GroupMember.java:262)
This application is CA Identity Manager R12.6 SP4, with 2 EAR files involved.
We have discussed this clustering issue with CA, and they indicate something is mis-configured within the JBoss AS.
Does anyone have any idea how we might troubleshoot and resolve this problem???

How to enable JBoss server TRACE log?

I am having web application running in JBOSS AS 4.2.2.
Observed that jboss server automatically shuts down, and the following exception is observed in server.log
14:20:38,048 INFO [Server] Runtime shutdown hook called, forceHalt: true
14:20:38,049 INFO [Server] JBoss SHUTDOWN: Undeploying all packages
I want to enable TRACE for org.jboss.system.server.Server in jboss-log4j.xml, to hopefully get some more info when the server shuts down.
Please let me know how to enable TRACE for org.jboss.system.server.Server in jboss-log4j.xml.
I was able to add trace for server log and i could see the following output when JBOSS AS shuts down automatically:
2010-06-09 19:07:46,631 DEBUG [org.jboss.wsf.stack.jbws.RequestHandlerImpl] END handleRequest: jboss.ws:context=hpnp_lqs,endpoint=APIWebService
2010-06-09 19:07:46,631 DEBUG [org.jboss.ws.core.soap.MessageContextAssociation] popMessageContext: org.jboss.ws.core.jaxws.handler.SOAPMessageContextJAXWS#3290a11e (Thread http-0.0.0.0-8080-1)
2010-06-09 19:07:55,895 INFO [org.jboss.system.server.Server] Runtime shutdown hook called, forceHalt: true
2010-06-09 19:07:55,895 TRACE [org.jboss.system.server.Server] Shutdown caller:
java.lang.Throwable: Here
at org.jboss.system.server.ServerImpl$ShutdownHook.shutdown(ServerImpl.java:1017)
at org.jboss.system.server.ServerImpl$ShutdownHook.run(ServerImpl.java:996)
2010-06-09 19:07:55,895 INFO [org.jboss.system.server.Server] JBoss SHUTDOWN: Undeploying all packages
If anybody, has any clue, on what might be cause for automatic shutdown, pls help me.
Thanks!
There's a JBoss wiki page listing log output for various shutdown causes. It looks like yours was caused by a Ctrl-C. I assume you would have known if you hit Ctrl-C, though.
On unix-type servers, Ctrl-C generates a TERM signal, which could also come from someone or some script running as your jboss user or as root executing "kill <jboss pid>". If you're on linux I'd take a look at this question about the OOM killer.
One possible cause for this behaviour is console logout. We have observed this with our own server.
In brief, by default the Sun JVM listens to the event of the console user logging out, and shuts itself down automatically when that happens. To disable this, start the JVM with the -Xrs parameter.
See here for more details (look for Mysterious shutdowns).
One possible cause for a forced shutdown is if the virtual machine is out of memory.
I had this problem several years ago when a colleague implemented some very nasty bulk loading of objects from a database which caused jboss to shutdown on certain requests.
Try searching for "memory" or similar keywords in the log file and/or monitor the memory usage of the server.