Insertions/evaluations are progressively slower when using temporal rule - drools

I have a rule that looks for 2 consecutive events for the same entity. To stress test it, I inserted 10K consecutive events. I'm calling fireAllRules() after each event is inserted. I'm printing out timestamps after every 100 events. I'm noticing that insertions/evaluations are increasingly slower as events are added. Here's the rule :
rule "Consecutive events"
when
$latest : Event($id : id) // newest event
not Event(id == $id, this after $latest) // no events after the newest
$previous : Event(id == $id, this before $latest) // event before the newest
not Event(id == $id, this before $latest, this after $previous) // no events between $latest and $previous
then
//System.out.println($latest.toString());
end
It is my understanding that the above rule should only match the latest 2 events and automatic memory management should remove older events. If so, why are insertions progressively slower? Interestingly, ksession.getObjects() returns all the events inserted and not just the latest 2 events. Is my understanding of the rule incorrect? Does the rule somehow force all events to stay in memory? I'm using v6.0.1.Final.

Your rule does not define a time limit for automatic retraction as the before and after are not limited.
There are several ways to keep the number of events low, which is the reason for inserts getting progressively slower. Here is a simple technique:
declare Pair
one : Event
two : Event
id : String
end
rule "create Pair"
when
$e: Event( $id: id )
not Pair( id == id )
then
insert( new Pair( null, $e, $id ) );
end
rule "another Pair"
when
$e2: Event( $id: id )
$p: Pair( $e0: one, $e1: two != $e2, id == $id )
then
modify( $p ){
setOne( $e1 ),
setTwo( $e2 ) }
retract( $e0 ); // optional
// process pair $e1, $e2
end

Related

Drools compare if one list contains element from another list

I have got below structure in Java:
public class Request {
List<Product> product;
List<Account> accounts;
}
public class Product {
String productIdOne;
String productIdTwo;
String productTax;
}
public class Account {
List<ProductRelationship> productsRelationship;
}
public class ProductRelationship {
String productIdOne;
String productIdTwo;
}
And the request is the fact object send to drools. I am wondering how I can check if there is at least one product that productTax is set to 'true' and there is a relationship between one account and one product. In other words, if there is a product with tax set to true and at least one account contains a relationship with this product (by productIdOne and productIdTwo) then the rule result should pass;
The main issue is that the list of the product relationship is inside the account list.
Thanks for any advice
You have a rather straight-forward set of conditions, so it is possible to write a relatively simple rule to check them. I will consider each condition separately and then combine them into a final rule.
As you wrote:
there is at least one product that productTax is set to 'true'
Now, as you mentioned, your rule inputs are the Request instance which contains two lists (products, accounts.) We'll start by declaring that:
rule "Account exists with taxed product"
when
Request( $products: product != null,
$accounts: accounts != null )
Next, we want to find the taxed product. If we only wanted to prove the existence of the product, we could use an exists( ... ) condition, which is extremely fast. However since we want to do further comparisons, we'll want to actuall find the product with this condition and save a reference to it.
$taxedProduct: Product( productTax == "true" ) from $products
I've assumed here that any value other than exactly "true" is indicative of an untaxed product. You should adjust as needed (and possibly consider changing this type to a boolean.)
The next condition is to find the account:
there is a relationship between one account and [the taxed] product
First, we'll need to update our $taxedProduct declaration and get references to its ids:
$taxedProduct: Product( productTax == "true",
$id1: productIdOne,
$id2: productIdTwo ) from $products
Now we need to find an account with a matching relationship.
$account: Account( $relationships: productsRelationship != null ) from $accounts
exists( ProductRelationship( productIdOne == $id1,
productIdTwo == $id2 ) from $relationships )
Here, I used an exists condition for the relationship because we don't need to refer to the relationship itself ever again, just verify that the relationship exists. I did declare a variable $account to point to the account that has the product relationship.
Putting it all together, we have:
rule "Account exists with taxed product"
when
Request( $products: product != null,
$accounts: accounts != null )
$taxedProduct: Product( productTax == "true",
$id1: productIdOne,
$id2: productIdTwo ) from $products
$account: Account( $relationships: productsRelationship != null ) from $accounts
exists( ProductRelationship( productIdOne == $id1,
productIdTwo == $id2 ) from $relationships )
then
// We have a taxed product $taxedProduct
// and an associated account $account
end
When I first started with Drools I found it difficult to wrap my head around the way it treated objects in lists, which is why the ProductRelationship sub-list seems like a tricky issue on its face. What Drools is going to do is iterate through the $products list and find those Product instances that meet our criteria (namely, have productTax == "true".) Once it has found these taxed products, it then similarly goes through the $accounts list and finds all Accounts that meet the criteria (which have a productsRelationship list.) Then for each of those accounts, it is going to test that there exists a relationship as we've defined.
This is a simplified explanation, of course, but it helps to form a mental model of roughly what Drools is doing here. In reality Drools is much more efficient than then roughly O(n^3) workflow I've just described.
An interesting thing you should keep in mind is that this rule is not going to "stop" as soon as it finds a match. If you have two taxed products that have a relationship to a single account, this rule will fire twice -- once for each taxed product. Or, alternatively, if you have one taxed product and two accounts that have a relationship to it, the rule will fire twice (once for each account.) Basically, the rule will fire once for each "match" it finds in the given request.
can you try this following
rule "sample"
no-loop
when
request:Request(accountList: accounts)
request1:Request(productList: product)
Account(productsRelationshipList:ProductRelationship) from accountList
Product(productId contains productsRelationshipList, productTax = true ) from productList
then
System.out.println("Rule fired satisfied");
end

Adding rule condition based on two objects in drools

> { "batch-execution":{
> "lookup":"defaultKieSession",
> "commands":[
> {
> "insert":{
> "out-identifier":"FieldData1",
> "object":{
> "FieldData":{
> "name":"abc",
> "value":"111"
> }
> }
> }
> },
> {
> "insert":{
> "out-identifier":"FieldData2",
> "object":{
> "FieldData":{
> "name":"xyz",
> "value":"222"
> }
> }
> }
> },
> {
> "fire-all-rules":{
>
> }
> }
> ] } }
Now i want to write a condition in drl similar to this:
rule "testrule"
when
fieldData(name == "abc" , value == "111") && fieldData(name == "xyz",value = "222")
then
System.out.println("Condition executed")
Can someone help on how this can be done in drools ?
Of course you can! Your "example" rule is nearly perfect as-is.
The way drool works is that it evaluates the conditions of all of the objects in working memory, and only if all conditions are met will it trigger that rule.
So you could write a rule that looks like this:
rule "Test Rule"
when
exists( FieldData( name == "abc", value == "111") )
exists( FieldData( name == "xyz", value == "222") )
then
System.out.println("Condition Executed")
end
This rule will trigger if there exists an object in working memory that has a name of 'abc' and a value of '111', and there also exists an object in working memory with the name of 'xyz' and value of '222'.
In the example above, I used the 'exists' predicate because we weren't going to actually be doing anything with those values, and we just wanted to confirm that there is such a FieldData object in memory that matches the required conditions.
Note that this assumes that you've entered your FieldData objects directly into the working memory as standalone objects. (Eg you fired the rules and passed it a List of FieldData.) If you're working with bigger structures, you'll have to extract the FieldData objects, and then do an exists check like I had in my previous example.
For example, let's say you had a set of classes like this (which mimic your example JSON; getters and setters omitted for brevity):
class FieldData {
String name;
String value;
}
class Command {
String outIdentified;
FieldData object;
}
class BatchExecution {
String action; // eg "insert"
String lookup;
List<Command> commands;
}
If you passed a BatchExecution into the rules, you'll need to pull the field data out of the commands before you can check that two FieldData exist with the conditions you want. Your rule would therefore look more like this:
rule "Test Rule 2"
when
// Get the BatchExecution in working memory and pull out the Commands
BatchExecution( $commands: commands != null )
// Get the FieldData from each of the commands
$data: ArrayList( size >= 2) from
accumulate( Command( $fd: object != null ) from $commands,
init( ArrayList fieldDatas = new ArrayList() ),
action( fieldDatas.add( $fd ) ),
reverse( fieldDatas.remove( $fd ) ),
result( fieldDatas ))
// Confirm there exist FieldData with our criteria inside of the Commands
exists( FieldData( name == "abc", value == "111" ) from $data
exists( FieldData( name == "xyz", value == "222" ) from $data
then
System.out.println("Condition executed, 2");
end
Basically what we have to do is drill down from the object actually inserted into working memory until we can get to the objects that we need to be doing work against, in this case the FieldData. I used the accumulate aggregate to pull all of the FieldData into a List that we then check for the presence of the two FieldData that we're looking for in this particular rule.
For more information, you should consider reading the Drools documentation, specifically the part on the "Rule Language Reference" (section 4) which is very well written and contains plenty of examples that you can adapt or expand upon.
The rule firing in drools only happens on the occurrence of some event. Read about sessions and execution of rules here.
Coming to the your question of writing the above rule. I am not sure how you want the rule to be executed. As per my understanding if you want to write a separate rule each to check name as xyz and abc, then you can write the rule as below:
rule "testrule1"
when
fieldData(name == "abc" , value == "111")
then
System.out.println("Condition executed 1")
end
rule "testrule2"
when
fieldData(name == "xyz" , value == "222")
then
System.out.println("Condition executed2")
end
If you want to combine the rule then you can write it as:
rule "testrule"
when
fieldData(name == "abc" || name == "xyz" && value == "111" || value == "222")
then
System.out.println("Condition executed")
end
Note: You cannot define a rule like above if you want to work on the occurrence of multiple events. If you want to work on multiple events you can read about windowing in drools. Add more information on your use case to get better answers.

Using collect in drools to get similar objects

I am trying to collect some objects in Drools, but I want to only collect objects which have the same attribute. To wit, imagine a TestData class:
public class TestData {
int number;
String name;
//Getters, setters, rest of class
}
I want to use collect to get all TestDatas which have the same name. For the following data set, I want a rule which can collect the first two (both having the name 'Test1') and the second two ('Test2') as separate collections.
custom.package.TestData< number: 1, name:'Test1' >
custom.package.TestData< number: 2, name:'Test1' >
custom.package.TestData< number: 3, name:'Test2' >
custom.package.TestData< number: 4, name:'Test2' >
Something like
rule "Test Rule"
when
$matchingTestDatas : ArrayList( size > 1 ) from collect ( TestData( *magic* ) )
then
//Code
end
But obviously that won't work without the magic- it gets me an array of all the TestData entries with every name. I can use that as the basis for a rule, and do extended processing in the right hand side iterating over all the test data entries, but it feels like there should be something in drools which is smart enough to match this way.
Presumably the "magic" is just:
TestData( name == 'Test1' )
or
TestData( name == 'Test2' )
... for each of the collections. But that seems too obvious. Am I missing something?
Based on the clarification from the OP in the comments on this answer, it would appear that a Map of collections is required, keyed from the name. To support this, accumulate is required, rather than collect.
$tests: HashMap()
from accumulate( $test : TestData( name == $name ),
init( HashMap tests = new HashMap(); ),
action(
if (tests.get($name) == null) tests.put($name, new ArrayList();
tests.get($name).add($test);
),
reverse(
tests.get($name).remove($test);
),
result( tests )
);

Account Trigger update all contacts - Too many SOQL queries: 101

I have a trigger that runs after updating any account and it actually just updates a field (Relationship_category_r__c) in all the related contacts after few conditions.
Condition1: If we update the account type to "Member"
Condition2: If the contact doesn't have "Member" already in the (Relationship_category_r__c) field
ACTION: Update the contact Relationship_Category_r__c field to "Member - staff"
Condition2: If we update the account type to "Member - past"
ACTION: Update all the contacts Relationship_Category_r__c field to "Member - past"
The trigger works absolutely find when the account has less than 25 to 50 contacts but it generates an error when we have an account more than 55 or so contacts
ERROR: Apex trigger UpdateAllContacts caused an unexpected exception, contact your administrator: UpdateAllContacts: System.LimitException: Too many SOQL queries: 101
======================================= TRIGGER ==============================
trigger UpdateAllContacts on Account (after update) {
for ( Account acc : Trigger.New ) {
List<Contact> listCon = [Select id, Relationship_Category_r__c from Contact where AccountId =: acc.id];
for ( Contact con : listCon ) {
if ( acc.Type=='Member' ) {
if ( con.Relationship_Category_r__c != 'Member' ) {
con.Relationship_Category_r__c = 'Member - staff';
}
} else if ( acc.Type=='Member - past ' ) {
con.Relationship_Category_r__c = 'Member - past';
}
}
try {
update listCon;
}
catch (DmlException e) {}
}
}
Any help will be greatly appreciated
Thanks
Couple of things that jump out at me here:
First and foremost is the main cause of your error: a SOQL query within a loop. You need to bulkify your trigger to perform a single query, rather than querying once for each Account record in your update batch.
You have a DML operation (update listCon;) within that same loop, which will exacerbate the problem if you happen to have any SOQL queries in your Contact triggers. This update should also be bulkified, so that you are making as few inserts/updates as possible (in this case you can limit it to a single update call).
You are updating all Contact records queried regardless of whether or not they've been updated by your code. This can lead to unnecessarily long processing times, and can easily be prevented by keeping track of which records should be updated in a second list.
This may be by design, but you're suppressing any errors in your update call without even attempting to deal with them. There are many different ways to go about this depending on your implementation so I won't delve into that, but in most circumstances you would probably want to at least be aware that your update to those records was failing so you can correct the issue.
After making a few corrections to your code we are left with the following which uses a single SOQL query, single DML statement, and won't update any Contact records that haven't been modified by the trigger:
trigger UpdateAllContacts on Account (after update) {
List<Contact> updateCons = new List<Contact>();
for ( Contact con : [select Id, AccountId, Relationship_Category_r__c from Contact where AccountId in :Trigger.NewMap.keySet()] ) {
Account parentAcc = Trigger.NewMap.get(con.AccountId);
if ( parentAcc.Type == 'Member' ) {
if ( con.Relationship_Category_r__c != 'Member' ) {
con.Relationship_Category_r__c = 'Member - staff';
updateCons.add(con);
}
} else if ( parentAcc.Type == 'Member - past ' ) {
con.Relationship_Category_r__c = 'Member - past';
updateCons.add(con);
}
}
update updateCons;
}

Drools object graph rule definition

I have an object graph that I am trying to generate Fulfillment object from in Drools. Specifically, Fulfillment objects represent a rule that is either satisfied, or unsatisfied. My object graph looks like the following:
Users ---> many Requirements --> Event
`--> many Records ----^
Records can fulfill Requirements if they both point at the same Event. This produces a Fulfillment object in Drools.
A reduce down rule to produce Fulfillments is the following:
rule "fulfils"
when
$u : User()
$rec : Record() from $u.records
$r : Requirement(event contains $rec.event) from $u.requirements
then
insertLogical( new Fulfillment($u, $rec, $r, true));
System.out.println("Inserting logical");
end
rule "unfulfils"
when
$u : User()
$rec : Record() from $u.records
$r : Requirement(event not contains $rec.event) from $u.requirements
then
insertLogical( new Fulfillment($u, $rec, $r, false));
System.out.println("Inserting logical");
end
query "fulfillment"
$fulfillment : Fulfillment()
end
The problem I run into here is if the user has no records, there is no Fulfillment inserted for the requirement. I believe this is because there is no Record() to search on to satisfy my graph.
Is there a way to use the records without requiring more than zero to exist?
Also, do I need two rules here to insert both true and false Fulfillments or is there a better way to do this?
Edit
Another problem I am facing with these rules is the Requirement(event contains $rec.event) does not accomplish the task of finding if any records satisfy the given collection of events. Is there a better way to find if there exists an overlap between the many record's single events, and the single requirements multiple events?
Another Edit
Here's another approach I thought up. Instead of inserting Fulfillments if a requirement/record pair is not found, why not just insertLogical Fullfillments for all Requirements that have no matching positive Fullfillment:
rule "unfulfils"
when
$u : User()
$r : Requirement() from $u.requirements
not(Fulfillment(user == $u, requirement == $r, fulfilled == true))
then
insertLogical( new Fulfillment($u, null, $r, false));
System.out.println("Inserting logical");
end
query "fulfillment"
$fulfillment : Fulfillment()
end
This takes care of the issue of comparing the overlap of two collections, and the case where a user has no records. (Would appreciate some validation on this).
Using 2 different rules for your situation is a common pattern. It makes your rule base easier to read (and in a way to maintain too).
Regarding your question about no Record(), I think you could write something like this (If I understood your question correctly):
rule "unfulfils because of no Record"
when
$u : User(records == null || records.empty == true) //A user without records
$r : Requirement() from $u.requirements // but with Requirements
then
//You don't have a record to set in your Fulfillment object
insertLogical( new Fulfillment($u, $rec, null, false));
System.out.println("Inserting logical");
end