Drools Fusion SessionPseudoClock Not working as expected - drools

I am trying to set KieSession date to same date with the date variable in my object before running the rules. I use this configuration to create my KieSession
KieSessionConfiguration configuration = KieServices.Factory.get().newKieSessionConfiguration();
configuration.setOption(ClockTypeOption.get("pseudo"));
Before running the rules I use advanceTime() to set the date of the session to desired date.
final SessionPseudoClock clock = kSession.getSessionClock();
clock.advanceTime(object.getDate().getTime() - System.currentTimeMillis(), TimeUnit.MILLISECONDS);
final List<Command<?>> commands = new ArrayList<>();
commands.add(CommandFactory.newInsertElements(objects)
commands.add(CommandFactory.newInsert(object, "object"));
final ExecutionResults results = kSession.execute(CommandFactory.newBatchExecution(commands));
This resulted misfires in the rules that uses sliding windows. Lets say checking the objects that passed in 1 hour and I don't have any in the last hour. I only have 3 objects day before. Here is an example dataset of objects.
objects: [
{
clientId: "id",
date: 2021-02-09T12:00:38.249Z,
...
}
{
clientId: "id",
date: 2021-02-09T13:00:38.249Z,
...
}
{
clientId: "id",
date: 2021-02-09T14:00:38.249Z,
...
}
]
I have a rule which checks if there are more than 2 objects with the same clientId over 1 hour.
$object : Object($clientId : clientId)
List( size > 2 ) from collect ( Object( this before $object, clientId == $clientId ) over window:time(1h))
When I pass an object with these values. The rule above returns true but we clearly don't have any objects that has a date within last hour.
{ clientId: "id", date: 2021-02-10T14:00:38.249Z, ... }
I believe this is broken because of the new configuration as it was working previously (when I did not try to change session clock) but I want session date to be equal to object date. Anyone has ideas what is the problem here and how to fix it?

As Roddy of the Frozen Peas made the remark, you need to manipulate SessionPseudoClock in between every insert of your events. I'd implement a new Command, extend InsertElementsCommand and #Override it's execute method:
#Override
public Collection<FactHandle> execute(Context context) {
KieSession kSession = ((RegistryContext) context).lookup(KieSession.class);
List<FactHandle> handles = new ArrayList<>();
Object workingMemory = StringUtils.isEmpty(super.getEntryPoint()) ?
kSession :
kSession.getEntryPoint(super.getEntryPoint());
SessionPseudoClock clock = kSession.getSessionClock();
super.objects.forEach(event -> {
clock.advanceTime(((YourObject) event).getDate().getTime() - clock.getCurrentTime(), TimeUnit.MILLISECONDS);
handles.add(((EntryPoint) workingMemory).insert(event));
});
...
return handles;
}
and instead of:
commands.add(CommandFactory.newInsertElements(objects));
I'd:
commands.add(new YourInsertElementsCommand(objects));

Related

Spark mapPartitionsToPai execution time

In the current project I am working, we are using spark as computation engine for one of workflows.
Workflow is as follows
We have product catalog being served from several pincodes. User logged in from any particular pin code should be able to see least available cost from all available serving pincodes.
Least cost is calculated as follows
product price+dist(pincode1,pincode2) -
pincode2 being user pincode and pincode1 being source pincode. Apply the above formula for all source pincodes and identify the least available one.
My Core spark logic looks like this
pincodes.javaRDD().cartesian(pincodePrices.javaRDD()).mapPartitionsToPair(new PairFlatMapFunction<Iterator<Tuple2<Row,Row>>, Row, Row>() {
#Override
public Iterator<Tuple2<Row, Row>> call(Iterator<Tuple2<Row, Row>> t)
throws Exception {
MongoClient mongoclient = MongoClients.create("mongodb://localhost");
MongoDatabase database = mongoclient.getDatabase("catalogue");
MongoCollection<Document>pincodeCollection = database.getCollection("pincodedistances");
List<Tuple2<Row,Row>> list =new LinkedList<>();
while (t.hasNext()) {
Tuple2<Row, Row>tuple2 = t.next();
Row pinRow = tuple2._1;
Integer srcPincode = pinRow.getAs("pincode");
Row pricesRow = tuple2._2;
Row pricesRow1 = (Row)pricesRow.getAs("leastPrice");
Integer buyingPrice = pricesRow1.getAs("buyingPrice");
Integer quantity = pricesRow1.getAs("quantity");
Integer destPincode = pricesRow1.getAs("pincodeNum");
if(buyingPrice!=null && quantity>0) {
BasicDBObject dbObject = new BasicDBObject();
dbObject.append("sourcePincode", srcPincode);
dbObject.append("destPincode", destPincode);
//System.out.println(srcPincode+","+destPincode);
Number distance;
if(srcPincode.intValue()==destPincode.intValue()) {
distance = 0;
}else {
Document document = pincodeCollection.find(dbObject).first();
distance = document.get("distance", Number.class);
}
double margin = 0.02;
Long finalPrice = Math.round(buyingPrice+(margin*buyingPrice)+distance.doubleValue());
//Row finalPriceRow = RowFactory.create(finalPrice,quantity);
StructType structType = new StructType();
structType = structType.add("finalPrice", DataTypes.LongType, false);
structType = structType.add("quantity", DataTypes.LongType, false);
Object values[] = {finalPrice,quantity};
Row finalPriceRow = new GenericRowWithSchema(values, structType);
list.add(new Tuple2<Row, Row>(pinRow, finalPriceRow));
}
}
mongoclient.close();
return list.iterator();
}
}).reduceByKey((priceRow1,priceRow2)->{
Long finalPrice1 = priceRow1.getAs("finalPrice");
Long finalPrice2 = priceRow2.getAs("finalPrice");
if(finalPrice1.longValue()<finalPrice2.longValue())return priceRow1;
return priceRow2;
}).collect().forEach(tuple2->{
// Business logic to push computed price to mongodb
}
I am able to get the answer correctly, however mapPartitionsToPair is taking a bit of time(~22 secs for just 12k records).
After browsing internet I found that mapPartitions performs better than mapPartitionsToPair, but I am not sure how to emit (key,value) from mapPartitions and then sort it.
Is there any alternative for above transformations or any better approach is highly appreciated.
Spark Cluster: Standalone(1 executor, 6 cores)

How to log new records during SaveChanges

I want to log new and modified records. This code works just fine for Modified Records.
But with Added records, there is an issue. Since it is new to the Database, there is not yet a primary key for it. So there is no way to log which record was added.
However, if I try to log the records after the save, the EntityState is no longer Added. So I don't know what was added.
The only solution I have been able to come up with is to save a list of the new records, and then after the save, then Log the changes. But that seems like a workaround.
Is there some way to resolve this?
private List<Event> LogChanges(EntityEntry entityEntry, Enums.TableNames tableName)
{
List<Event> result = new List<Event>();
var databaseValues = entityEntry.GetDatabaseValues();
foreach (var property in entityEntry.CurrentValues.Properties.Where(a=> a.Name !="TenantId"))
{
string original = databaseValues[property]?.ToString();
string current = entityEntry.CurrentValues[property]?.ToString();
if(!object.Equals(original,current))
{
result.Add(
new Event()
{
AppUserId = this._appUserProvider.CurrentAppUserId,
EventDate = DateTimeOffset.UtcNow,
EventTypeId = (int)Enums.EventTypes.Modified,
TenantId = databaseValues.GetValue<int>("TenantId"),
RecordId = databaseValues.GetValue<int>("Id"),
ColumnName = property.Name,
OriginalValue = original,
NewValue = current,
TableId = (int)tableName
});
}
}
return result;
}
This library adds triggers to EntityFrameworkCore. Using the Triggers it provides is a much cleaner way to accomplish the above.

please help me to convert trigger to batch apex

please help me in converting my after trigger to batch apex.
This trigger fires when opportunity stage changes to won.
It runs through line items and checks if forecast(custom objet) exists with that acunt.if yes,iit links to them..if no,itt will create a new forecat.
my trigger works fine forr some records.but to mass update i am getting timed out error.So opting batch apex but i had never written it.pls help me.
trigger Accountforecast on Opportunity (after insert,after update) {
List<Acc_c> AccproductList =new List<Acc_c>();
List<Opportunitylineitem> opplinitemlist =new List<Opportunitylineitem>();
list<opportunitylineitem > oppdate= new list<opportunitylineitem >();
List<Acc__c> accquery =new List<Acc__c>();
List<date> dt =new List<date>();
Set<Id> sProductIds = new Set<Id>();
Set<Id> sAccountIds = new Set<Id>();
Set<id> saccprodfcstids =new set<Id>();
Acc__c accpro =new Acc__c();
string aname;
Integer i;
Integer myIntMonth;
Integer myIntyear;
Integer myIntdate;
opplinitemlist=[select Id,PricebookEntry.Product2.Name,opp_account__c,Opp_account_name__c,PricebookEntry.Product2.id, quantity,ServiceDate,Acc_Product_Fcst__c from Opportunitylineitem WHERE Opportunityid IN :Trigger.newMap.keySet() AND Acc__c=''];
for(OpportunityLineItem oli:opplinitemlist) {
sProductIds.add(oli.PricebookEntry.Product2.id);
sAccountIds.add(oli.opp_account__c);
}
accquery=[select id,Total_Qty_Ordered__c,Last_Order_Qty__c,Last_Order_Date__c,Fcst_Days_Period__c from Acc__c where Acc__c.product__c In :sproductids and Acc__c.Account__c in :saccountids];
for(Acc__c apf1 :accquery){
saccprodfcstids.add(apf1.id);
}
if(saccprodfcstids!=null){
oppdate=[select servicedate from opportunitylineitem where Acc__c IN :saccprodfcstids ];
i =[select count() from Opportunitylineitem where acc_product_fcst__c in :saccprodfcstids];
}
for(Opportunity opp :trigger.new)
{
if(opp.Stagename=='Closed Won')
{
for(opportunitylineitem opplist:opplinitemlist)
{
if(!accquery.isempty())
{
for(opportunitylineitem opldt :oppdate)
{
string myDate = String.valueOf(opldt);
myDate = myDate.substring(myDate.indexof('ServiceDate=')+12);
myDate = myDate.substring(0,10);
String[] strDate = myDate.split('-');
myIntMonth = integer.valueOf(strDate[1]);
myIntYear = integer.valueOf(strDate[0]);
myIntDate = integer.valueOf(strDate[2]);
Date d = Date.newInstance(myIntYear, myIntMonth, myIntDate);
dt.add(d);
}
dt.add(opp.closedate);
dt.sort();
integer TDays=0;
system.debug('*************dt:'+dt.size());
for(integer c=0;c<dt.size()-1;c++)
{
TDays=TDays+dt[c].daysBetween(dt[c+1]);
}
for(Acc_product_fcst__c apf:accquery)
{
apf.Fcst_Days_Period__c = TDays/i;
apf.Total_Qty_Ordered__c =apf.Total_Qty_Ordered__c +opplist.quantity;
apf.Last_Order_Qty__c=opplist.quantity;
apf.Last_Order_Date__c=opp.CloseDate ;
apf.Fcst_Qty_Avg__c=apf.Total_Qty_Ordered__c/(i+1);
Opplist.Acc__c =apf.Id;
}
}
else{
accpro.Account__c=opplist.opp_account__c;
accpro.product__c=opplist.PricebookEntry.Product2.Id;
accpro.opplineitemid__c=opplist.id;
accpro.Total_Qty_Ordered__c =opplist.quantity;
accpro.Last_Order_Qty__c=opplist.quantity;
accpro.Last_Order_Date__c=opp.CloseDate;
accpro.Fcst_Qty_Avg__c=opplist.quantity;
accpro.Fcst_Days_Period__c=7;
accproductList.add(accpro);
}
}
}
}
if(!accproductlist.isempty()){
insert accproductlist;
}
update opplinitemlist;
update accquery;
}
First of all, you should take a look at this: Apex Batch Processing
Once you get a better idea on how batches work, we need to take into account the following points:
Identify the object that requires more processing. Account? Opportunity?
Should the data be maintained across batch calls? Stateful?
Use correct data structure in terms of performance. Map, List?
From your code, we can see you have three objects: OpportunityLineItems, Accounts, and Opportunities. It seems that your account object is using the most processing here.
It seems you're just keeping track of dates and not doing any aggregations. Thus, you don't need to maintain state across batch calls.
Your code has a potential of hitting governor limits, especially memory limits on the heap. You have a four-nested loop. Our suggestion would be to maintain opportunity line items related to Opportunities in a Map rather than in a List. Plus, we can get rid of those unnecessary for loops by refactoring the code as follows:
Note: This is just a template for the batch you will need to construct.
globalglobal Database.QueryLocator start(Database.BatchableContext BC) class AccountforecastBatch implements Database.Batchable<sObject>
{
global Database.QueryLocator start(Database.BatchableContext BC)
{
// 1. Do some initialization here: (i.e. for(OpportunityLineItem oli:opplinitemlist) {sProductIds.add(oli.PricebookEntry.Product2.id)..}
// 2. return Opportunity object here: return Database.getQueryLocator([select id,Total_Qty_Ordered__c,Last_Order_Qty ....]);
}
global void execute(Database.BatchableContext BC, List<sObject> scope)
{
// 1. Traverse your scope which at this point will be a list of Accounts
// 2. You're adding dates inside the process for Opportunity Line Items. See if you can isolate this process outside the for loops with a Map data structure.
// 3. You have 3 potential database transactions here (insert accproductlist;update opplinitemlist; update accquery; ). Ideally, you will only need one DB transaction per batch.If you can complete step 2 above, you might only need to update your opportunity line items. Otherwise, you're trying to do more than one thing in a method and you will need to redesign your solution
}
global void finish(Database.BatchableContext BC)
{
// send email or do some other tasks here
}
}

How to get a Fact creating BRL in Guvnor and querying Drools Server

I'm stuck with BRL rules in Guvnor.. I'm trying to execute rules from my application using Drools Server (this solution because in production I can use more server and maybe improve performance.. Not sure about this as it's the first time that in my company we are using Drools)..
So basically the rule is .. Given an object Route setting the property "selectedOutboundJourney" that I uploaded in guvnor in a jar, I'd like to get another object with the property "selectedReturnJourney" set.. (but Is it possible to get the same object??)
Actually I get a Route object where the selectedReturnJourney is null.
I'm not sure if using BRL is a good solution given the troubles that I'm having.. It seems easy to use for non technical people that may want to change the rules or creating new ones.
Anyway..
This is the BRL that I created in Guvnor:
rule "Selected Return for Dover - Calais"
dialect "mvel"
when
Route( selectedOutboundJourney == "DOCA" )
then
Route fact0 = new Route();
fact0.setSelectedReturnJourney( "CADO" );
insertLogical( fact0 );
end
This is the code I'm using:
final List<Command> commands = new ArrayList<Command>();
final Command insertObjectCommand = CommandFactory.newInsert(input, RESULT, true, "default");
final Command getObjectCommand = CommandFactory.newGetObjects();
final Command fireAllRulesCommand = CommandFactory.newFireAllRules();
commands.add(insertObjectCommand);
commands.add(getObjectCommand);
commands.add(fireAllRulesCommand);
final ExecutionResults executionResults = droolsHttpClient.callDroolsServer(commands);
return executionResults.getValue(RESULT);
The class DroolsHttpClient is:
public ExecutionResults callDroolsServer(final List<Command> commands) throws DroolsException
{
PostMethod postMethod = null;
try
{
final HttpClient httpClient = new HttpClient();
final String droolsServerHost = Config.getString(PoferriesrulesengineConstants.DROOLS_SERVER_HOST, "");
final int droolsServerPort = Config.getInt(PoferriesrulesengineConstants.DROOLS_SERVER_PORT, 0);
httpClient.getHostConfiguration().setHost(droolsServerHost, droolsServerPort);
final String droolsServerUrl = Config.getString(PoferriesrulesengineConstants.DROOLS_SERVER_URL, "");
postMethod = new PostMethod(droolsServerUrl);
final BatchExecutionCommand command = CommandFactory.newBatchExecution(commands, PoferriesrulesengineConstants.DROOLS_SESSION);
final XStream xStreamMarshaller = BatchExecutionHelper.newXStreamMarshaller();
final String xmlCommand = xStreamMarshaller.toXML(command);
final StringRequestEntity request = new StringRequestEntity(xmlCommand, MediaType.TEXT_PLAIN_VALUE, CharEncoding.UTF_8);
postMethod.setRequestEntity(request);
httpClient.executeMethod(postMethod);
if (postMethod.getStatusCode() != 200)
{
throw new RuntimeException("Drools Communication Error, code: " + postMethod.getStatusCode());
}
final String response = postMethod.getResponseBodyAsString();
final ExecutionResults executionResults = (ExecutionResults) xStreamMarshaller.fromXML(response);
return executionResults;
}
catch (final Exception e)
{
throw new DroolsException(e.getMessage());
}
finally
{
postMethod.releaseConnection();
}
}
If I use a DRL like below it words perfectly without using the getObjectCommand:
rule "Selected Return Routes for Dover Calais"
when
r : Route(selectedOutboundJourney == "DOCA")
then
r.setSelectedReturnJourney("CADO")
end
Can anyone help me out, please?
Assuming you only have one fact in your Knowledge Session at the start, after the execution of the following rule
rule "Selected Return for Dover - Calais"
dialect "mvel"
when
Route( selectedOutboundJourney == "DOCA" )
then
Route fact0 = new Route()
fact0.setSelectedReturnJourney( "CADO" )
insert( fact0 )
end
you will have two Route facts in your session, since you just have inserted the second one.
Route: selectedOutboundJourney = "DOCA", selectedReturnJourney=null
Route: selectedOutboundJourney = null, selectedReturnJourney="DACO"
If you want to modify the original fact use the following rule:
rule "Selected Return Routes for Dover Calais"
when
$r : Route(selectedOutboundJourney == "DOCA")
then
modify ($r) {
selectedReturnJourney = "CADO"
}
end

Drools: Having trouble with drools event processing

I am quite new to drools.
I am working on an application where my drools engine will get a series of event every second. I need to see if all the events in last 10 seconds has attribute value below 10, if the condition is true, I have to do some processing. Here is the example code which I tried, Please help me understand and resolve the issue.
My Rule file.....
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
declare Employee
#role (event)
#expires(10s)
end
// Using timer to ensure rule processing starts only after 10 secs,
//else processing starts as soon as first event comes in
rule "Test Timer"
no-loop true
10timer(int: 5s)
when
$E : Employee()
$total : Number( doubleValue < 1 )
from accumulate( Employee( Age > 10 ), count() )
then
System.out.println( $E.getName() + " is crossing the threshold of 20");
retract($E);
end
And Main class
// import classes removed from here...
public class MainClass {
/**
* #param args
*/
public static void main(String[] args){
//Create KnowledgeBase...
KnowledgeBase knowledgeBase = createKnowledgeBase();
//Create a stateful session
StatefulKnowledgeSession session = knowledgeBase.newStatefulKnowledgeSession();
// KnowledgeRuntimeLogger logger = KnowledgeRuntimeLoggerFactory.newConsoleLogger(session);
try {
// Using random generator to simulate the data.
int randomInt=0;
Random randomGenerator = new Random();
DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
Date date = null;
while (true) {
Thread.sleep(1000);
date = new Date();
randomInt = randomGenerator.nextInt(12);
//Create Facts and insert them
Employee emp = new Employee();
emp.setName("Anurag" + randomInt);
emp.setAge(randomInt);
//LOAD THE FACT AND FIREEEEEEEEEEEEEEEEEEE............
System.out.println(dateFormat.format(date)+ " => Random no " + randomInt);
session.insert(emp);
session.fireAllRules();
}
} catch (Exception e) {
e.printStackTrace();
}finally {
session.dispose();
}
}
/**
* Create new knowledge base
*/
private static KnowledgeBase createKnowledgeBase() {
KnowledgeBuilder builder = KnowledgeBuilderFactory.newKnowledgeBuilder();
//Add drl file into builder
File drl = new File("D:\\eclipse\\worspace\\Research\\misc\\testforall.drl");
builder.add(ResourceFactory.newFileResource(drl), ResourceType.DRL);
if (builder.hasErrors()) {
System.out.println(builder.getErrors().toString());
//throw new RuntimeException(builder.getErrors().toString());
}
KnowledgeBase knowledgeBase = KnowledgeBaseFactory.newKnowledgeBase();
//Add to Knowledge Base packages from the builder which are actually the rules from the drl file.
knowledgeBase.addKnowledgePackages(builder.getKnowledgePackages());
KnowledgeBaseConfiguration config = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
config.setOption( EventProcessingOption.STREAM );
return knowledgeBase;
}
}
public class Employee {
private String Name;
private int Age;
// getter - setters
}
did you check the Drools Fusion documentation?
First of all, Employee doesn't sounds as a good idea for an Event. Events are meaningful changes of states of something related with your domain. So, an event could be the time of arrival of an Employee to the office, or the time of departure, but the Employee itself is a domain entity (or a fact for the rule engine) more than an event.
If you are interested in using Drools fusion temporal operators I recommend you to read about sliding windows (temporal ones) which will allow you to see what happen in the last ten seconds all the time. You don't need to use timers for that.
Cheers
You forgot telling what happened when you ran it, if you did.
If your entities set is not very large, I think this problem can be solved very easily with the base Drools distribution.
I have a similar app to yours and works for me:
rule "Clear only auxiliar fact"
salience 1
when
af: AuxFact()
then
DroolsRepository.retractFact(af);
end
rule "Clear auxiliar fact and an old meaningful fact"
salience 1000
when
af: AuxFact()
mf: MeaningfulFact()
then
if(DroolsRepository.getCurrentTimeMillis() - tmf.getCreationDate().getTime() > 5000){
DroolsRepository.retractFact(af);
DroolsRepository.retractFact(mf);
// YOUR MEANINGFUL CODE
}
else{
DroolsRepository.retractFact(af);}
end
query "getAllFacts"
$result: Fact()
end
and
// Boot rules re-executing thread.
new Thread(new Runnable(){
public void run(){
do{
kSession.insert(new AuxFact());
try{
Thread.sleep(99);}
catch(InterruptedException e){
e.printStackTrace();}}
while(true);}
}).start();
A similar approach could be simpler and effective.