Corrupted KieBase object? - drools

Due to performance consideration, we try to re-use the same KieBase object to spawn new KieSession to for each rule invocation against the same ruleset. Everthingyth works well until after a period of time when all of a sudden, the newly created kieSession from the cached kieBase stops firing the rules that it was supposed to.
But as soon as we get rid of the cached kieBase and re-create a new kieBase and new a kieSession with it, it starts working again.
Our understanding is that kieBase object does not hold session-specific data. But the behavior seems to indicate that the cached kieBase is subject to being tampered over time.
The version we are using is 6.3.0.Final.
Any hints on this would be highly appreciated.

I had this issue, too. After creating 20 sessions in rapid succession, the 21st session would throw a ClassCastException on a cast to BigDecimal in this rule even though runs on the previous 20 sessions did not:
rule IsMaterialChange_TotalMonthlyIncomeAmountUpdate when
Data("TOT_MO_INCM_AMT-ORIGINAL_VALUE"; $originalTotalMonthlyIncomeAmount : value)
Data("TOTALMONTHLYINCOMEAMOUNT"; (BigDecimal) value != (BigDecimal) $originalTotalMonthlyIncomeAmount)
then
insert(new IsMaterialChange("TotalMonthlyIncomeAmountUpdate", true));
end
After digging around for a bit, I found this answer that pointed to the JIT compiler as the cause. It has a default threshold of 20 which lined up exactly with my issue. For some reason the JIT compiler has problems casting this value.
I disabled the JIT compiler like this:
KieContainer kieContainer = kieServices.newKieContainer(kieServices.getRepository().getDefaultReleaseId());
KieBaseConfiguration kieBaseConfiguration = kieServices.newKieBaseConfiguration();
// Disable JIT Compiler to prevent ClassCastException BigDecimal
kieBaseConfiguration.setOption(ConstraintJittingThresholdOption.get(-1));
KieBase kieBase = kieContainer.newKieBase(loaderKey, kieBaseConfiguration);

Related

How to reduce the time cost when calling getKieContainer() in Drools?

I'm using the Drools API to build a kieContainer when my application starts. But I noticed that it cost a lot of time when calling getKieContainer().
I am searching for a method to reduce the time cost to get a reusable KieContainer.
KieHelper kieHelper = new KieHelper();
kieHelper.addContent(getContent(), ResourceType.GDST);
KieContainer kieContainer = kieHelper.getKieContainer();
Do it only twice. You're supposed to call this only once when you initialise the application.
Otherwise you can consider using Drools' executable model

Exception handling in drools statefullSession

I'm not sure what is the correct way of handling exceptions in RHS of rules.
I've a statefullSession (drools version 5.5) that keeps firing until halt gets called i.e.
Runnable firingTask = new Runnable() {
public void run() {
ksession.fireUntilHalt();
};
taskExecutor.execute(firingTask);
The problem is that some parts of my code that gets called as a part of consequence section might throw an exception. If that happens no furher activations happen
My question is:
Is there any correct way how application exception should be propagated back to drools? Or is it that any RHS code has to be run inside try/catch block and these exceptions should be propagated as some additional facts into session?
The general strategies for exception handling apply here as well: just consider a consequence as being a static method that is being called due to the fireUntilHalt.
If the exception should or must be handled in the context of the RHS code, you'll have to use a try-catch statement. Otherwise, the exception is propagated and the remainder of the RHS code will not be executed. (This may include the omission of some essential statement such as a modify or insert or retract, which may affect the progress of the session.)
If you catch the exception org.drools.runtime.rule.ConsequenceException in a try around the fireUntilHalt, you can call fireUntilHalt again. The remarks from the previous paragraph apply here as well.
How you log an exception is completely up to you, but inserting Exception objects as facts (and writing rules to reason over them?) seems rather unconventional.

potential memory leak using TriMap in Scala and Tomcat

I am using a scala.collection.concurrent.TriMap wrapped in an object to store configuration values that are fetched remotely.
object persistentMemoryMap {
val storage: TrieMap[String, CacheEntry] = TrieMap[String, CacheEntry]()
}
It works just fine but I have noticed that when Tomcat is shut down it logs some alarming messages about potential memory leaks
2013-jun-27 08:58:22 org.apache.catalina.loader.WebappClassLoader checkThreadLocalMapForLeaks
ALLVARLIG: The web application [] created a ThreadLocal with key of type [scala.concurrent.forkjoin.ThreadLocalRandom$1] (value [scala.concurrent.forkjoin.ThreadLocalRandom$1#5d529976]) and a value of type [scala.concurrent.forkjoin.ThreadLocalRandom] (value [scala.concurrent.forkjoin.ThreadLocalRandom#59d941d7]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak
I am guessing this thread will terminate on it's own eventually but I am wondering if there is some way to kill it or should I just leave it alone?
The scala.concurrent.forkjoin.ThreadLocalRandom's value is created only once per thread. It does not hold any references to objects other than the random value generator used by that thread -- the memory it consumes has a fixed size. Once the thread is garbage collected, its thread local random value will be collected as well -- you should just let the GC do its work.
You could still remove it manually by using Java reflection to remove the private modifier on the static field localRandom in the ThreadLocalRandom class:
https://github.com/scala/scala/blob/master/src/forkjoin/scala/concurrent/forkjoin/ThreadLocalRandom.java#L62
You could then call localRandom.set(null) to null out the reference to the random number generator. You should also then ensure that TrieMap is no longer used from that thread, otherwise ThreadLocalRandom will break by assuming that the random number generator is different than null.
Seems hacky to me, and I think you should just stick to letting the GC collect the thread local value.

Accessing the process instance from Rule Tasks in JBPM 5

The short version: How do I get JBPM5 Rule Nodes to use a DRL file which reads and updates process variables?
The long version:
I have a process definition, being run under JBPM5. The start of this process looks something like this:
[Start] ---> [Rule Node] ---> [Gateway (Diverge)] ... etc
The gateway uses constraints on a variable named 'isValid'.
My Rule Node is pointing to the RuleFlowGroup 'validate', which contains only one rule:
rule "Example validation rule"
ruleflow-group "validate"
when
processInstance : WorkflowProcessInstance()
then
processInstance.setVariable("isValid", new Boolean(false));
end
So, by my logic, if this is getting correctly processed then the gateway should always follow the "false" path.
In my Java code, I have something like the following:
KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();
kbuilder.add(ResourceFactory.newClassPathResource("myProcess.bpmn"), ResourceType.BPMN2);
kbuilder.add(ResourceFactory.newClassPathResource("myRules.drl"), ResourceType.DRL);
KnowledgeBase kbase = kbuilder.newKnowledgeBase();
StatefulKnowledgeSession ksession = kbase.newStatefulKnowledgeSession();
new Thread(new Runnable()
{
public void run()
{
ksession.fireUntilHalt();
}
}).start();
// start a new process instance
Map<String, Object> params = new HashMap<String, Object>();
params.put("isValid", true);
ksession.startProcess("test.processesdefinition.myProcess", params);
I can confirm the following:
The drl file is getting loaded into working memory, because when I put syntax errors in the file then I get errors.
If I include a value for "isValid" in the Java params map, the process only ever follows the path specified by Java, apparently ignoring the drools rule.
If I take the "isValid" parameter out of the params map, I get a runtime error.
From this I assume that the final "setVariable" line in the rule is either not executing, or is updating the wrong thing.
I think my issue is related to this statement in the official documentation:
Rule constraints do not have direct access to variables defined inside the process. It is
however possible to refer to the current process instance inside a rule constraint, by adding
the process instance to the Working Memory and matching for the process instance in your
rule constraint. We have added special logic to make sure that a variable processInstance of
type WorkflowProcessInstance will only match to the current process instance and not to other
process instances in the Working Memory. Note that you are however responsible yourself to
insert the process instance into the session and, possibly, to update it, for example, using Java
code or an on-entry or on-exit or explicit action in your process.
However I cannot figure out how to do what is described here. How do I add the process instance into working memory in a way that would make it accessible to this first Rule Node? Rule Nodes do not seem to support on-entry behaviors, and I can't add it to the Java code because the process could very easily complete execution of the rules node before the working memory has been updated to include the process.
As you mentioned, there are several options to inserting the process instance into the working memory:
- inserting it after calling startProcess()
- using an action script to insert it (using "insert(kcontext.getProcessInstance()")
If calling startProcess() might already have gone over the rule task (which is probably the case in your example), and you don't have another node in front of your rule task where you could just use an on-entry/exit script to do this (so that's is hidden), I would recommend using an explicit script task before your rule task to do this.
Kris

mvc-mini-profiler slows down Entity Framework

I've set up mvc-mini-profiler against my Entity Framework-powered MVC 3 site. Everything is duly configured; Starting profiling in Application_Start, ending it in Application_End and so on. The profiling part works just fine.
However, when I try to swap my data model object generation to providing profilable versions, performance slows to a grind. Not every SQL query, but some queries take about 5x the entire page load. (The very first page load after firing up IIS Express takes a bit longer, but this is sustained.)
Negligible time (~2ms tops) is spent querying, executing and "data reading" the SQL, while this line:
var person = dataContext.People.FirstOrDefault(p => p.PersonID == id);
...when wrapped in using(profiler.Step()) is recorded as taking 300-400 ms. I profiled with dotTrace, which confirmed that the time is actually spent in EF as usual (the profilable components do make very brief appearances), only it is taking much longer.
This leads me to believe that the connection or some of its constituent parts are missing sufficient data, making EF perform far worse.
This is what I'm using to make the context object (my edmx model's class is called DataContext):
var conn = ProfiledDbConnection.Get(
/* returns an SqlConnection */CreateConnection());
return CreateObjectContext<DataContext>(conn);
I originally used the mvc-mini-profiler provided ObjectContextUtils.CreateObjectContext method. I dove into it and noticed that it set a wildcard metadata workspace path string. Since I have the database layer isolated to one project and several MVC sites as other projects using the code, those paths have changed and I'd rather be more specific. Also, I thought this was the cause of the performance issue. I duplicated the CreateObjectContext functionality into my own project to provide this, as such:
public static T CreateObjectContext<T>(DbConnection connection) where T : System.Data.Objects.ObjectContext {
var workspace = new System.Data.Metadata.Edm.MetadataWorkspace(
GetMetadataPathsString().Split('|'),
// ^-- returns
// "res://*/Redacted.csdl|res://*/Redacted.ssdl|res://*/Redacted.msl"
new Assembly[] { typeof(T).Assembly });
// The remainder of the method is copied straight from the original,
// and I carried over a duplicate CtorCache too to make this work.
var factory = DbProviderServices.GetProviderFactory(connection);
var itemCollection = workspace.GetItemCollection(System.Data.Metadata.Edm.DataSpace.SSpace);
itemCollection.GetType().GetField("_providerFactory", // <==== big fat ugly hack
BindingFlags.NonPublic | BindingFlags.Instance).SetValue(itemCollection, factory);
var ec = new System.Data.EntityClient.EntityConnection(workspace, connection);
return CtorCache<T, System.Data.EntityClient.EntityConnection>.Ctor(ec);
}
...but it doesn't seem to make much of a difference. The problem still exists whether I use the above hacked version that's more specific with metadata workspace paths or the mvc-mini-profiler provided version. I just thought I'd mention that I've tried this too.
Having exhausted all this, I'm at my wits' end. Once again: when I just provide my data context as usual, no performance is lost. When I provide a "profilable" data context, performance plummets for certain queries (I don't know what influences this either). What could mvc-mini-profiler do that's wrong? Am I still feeding it the wrong data?
I think this is the same problem as this person ran into.
I just resolved this issue today.
see: http://code.google.com/p/mvc-mini-profiler/issues/detail?id=43
It happened cause some of our fancy hacks were not cached well enough. In particular:
var workspace = new System.Data.Metadata.Edm.MetadataWorkspace(
new string[] { "res://*/" },
new Assembly[] { typeof(T).Assembly });
Is a very expensive call, so we need to cache the workspace.
Profiling, by definition, will effect performance of the application being profiled. The profiler needs to insert it's own method calls throughout the application, intercept low level system calls, and record all that data someplace (meaning writes to disk). All of those tasks take up precious CPU cycles, memory, and disk access.