I am working on a multi-client web-based application that analyses sensor data and shall invoke actions based on this data with a rule engine.
Every client of this application has a set of environmental sensors (10s - 100s) and a set of rules to be evaluated every time the sensor values change (the sensor values are copied into a database).
A basic set of rules will often be reused by different clients but the rules are individually parameterized (e.g. time dependant) for each client and every client has a different amount of sensors and rules, which can be configured individually. Some rules might even be specific to single clients.
I believe that drools might be a good choice for such an implementation - using drools guvnor to manage the rules for each client. Every client would have his own knowledge base and rule execution session.
I wonder if such an environment would scale and if there is a benchmark or real-world example where someone has used drools for such scenario.
Most benchmarks I could find assess different rule engines by their ability to perform rules on a growing number of facts. The amount of facts in my scenario would be relatively stable (per client) and scalability would rather be limited by the amount of clients and concurrent application of many knowledge bases and sessions.
Any comment about benchmarks or rule engine comparison regarding this scalability problem is welcome. I'd also be glad to hear about real-world implementations where every client has his own rules and dataset to work on.
The main problem with benchmarks is that they will vary a lot depending on the specific rules that you write for your own domain. Most of the benchmarks are tweaked to perform better in the rule engine that is testing. If you have a session per client and you have a stable number of client you will face no problem. Once you get the initial version of your project you can fine tune the engine to improve the performance.
The most "difficult" thing in my opinion is to get the infrastructure right, with that I mean, when to create the sessions and how to select the rules for each of the clients. Because that's part of your specific domain, you will need to code it and manage all the sessions.
Hope it helps
Acting on sensor data is one of the examples given for "Complex Event Processing". The following link may give a deeper insight on this subject.
Drools Fusion is also capable of CEP.
Related
I have been asked to start exploring a Activiti tool for some client demo.
The demo will also have JBoss Drools with which Activiti will be integrated.
I am new to both of these tools and business process world, so excuse me if the question is dumb.
The question is why do you need Drools? Isn't Activiti enough for the job?
Both of them have conditional elements so why do you need Activiti on top of drools?
This question doesn't quite fit the purpose of StackOverflow, so don't be surprised if you get a few flags. But I'll try to give a short answer.
Activity is a workflow engine, Drools is a business rules engine. They serve two different purposes.
Workflow engines are useful when you have a flow of actions of different actors that need to be controlled programmatically.
Rules engines are useful when you have business rules for executing some task automatically that you want to describe in a declarative way.
Both purposes are orthogonal to each other, meaning that the problem you have to solve may require none, just one, or both of them.
Imagine a workflow where a customer reports an incident, some experts have to work on it, and finally a bill gets produced, but no heavy algorithms are behind those tasks. That might be supported by a workflow engine without a rules engine.
Imagine a complex price model for a product, like cars having all sorts of special features that may be ordered. (Hifi speakers cost 400 €, except if the executive version of the car is ordered, where they only cost 200 € if ordered in combination with smartphone adapter...) Here a rules engine may be useful, although nobody talked about a workflow, so no workflow engine is needed.
Imagine the first example (incident workflow) together with a complex billing scheme. Here both tools may be used.
I wonder why these two types of tools are in some places described as perfectly fitting together. (Maybe this kind of claim motivated your question.) They serve two different purposes, and whether you need them both depends on the problem you have to solve.
I'm trying to build an claim processing system. There will be multiple variations of insurance policies (based on the negotiations with individual clients). Aim is to keep a base policies per provider and then apply variations to them per client to ensure easy maintenance of top level policies (like damage due to fire covered or not). The policies should be easy to be created by non-technical business users.
What is the best approach for this? I'm thinking on the lines of using Drools to come up with basic rules and then create jBPM processes per policy provider that will consume the rules. Guvnor for authoring and maintenance of rules and processes.
Assuming no human tasks (its going to be just a set of rules that need to be fired and results be thrown out), is using jBPM going to be an overkill? Are there better alternatives in the open source world?
Drools is already closely integrated with jBPM for use cases like this, so it definitely won't be overkill, they will work very nicely together. jBPM is not only about human interactions, it can just as well be used for automatic processing.
One remark, it might even be possible to not have one process per provider but have only one (or a small set of) process(es) and use rules to handle the variations.
I am about to use Apache Hadoop, the headlines read:
The Apache Hadoop project develops
open-source software for reliable,
scalable, distributed computing.
I can relate "scalability" to programming, but I just don't know how this "distributing" can help me in my development. According to wikipedia:
A distributed system consists of
multiple autonomous computers that
communicate through a computer
network. The computers interact with
each other in order to achieve a
common goal
So does this mean I can deploy my web apps across multiple computers and do some sort of "intense computing"? The terms that come into my mind are Content Delivery Networks and Cloud Computing.
Web development has always been about distributed computing, since clients have been on different machines to the servers they talk to, web pages can pull in resources from many servers to build a page's content, and servers may talk to other machines to achieve their goals. CDNs make this more obvious than before, but really they're just an evolution, an introduction of a virtualization/indirection layer between what you ask for and the hardware used to provide it.
Clouds are about taking the concepts of virtualization and applying them to remote hosting, both of low-level OSes and higher-level software platforms. The really interesting thing about them is that this enables different business models on the part of customers (and with different risks too, but that's mostly not related to the fact that it's distributed computing but rather that it is not wholly under your control in your own jurisdiction).
I've found that the most effective use of distributed computing is when you think in terms of connecting together distinct services, each of which with different capabilities (which might be for technical reasons, or might not; sometimes, it's for business or legal reasons that things have to be divided up) and where each of those services may be provided by many components in multiple locations. There are, and continue to remain, issues with balancing the need for performance (which is a force that brings components together) and the need for robustness (which tends to lead to distribution and replication) within the overall context of the general capabilities map.
My goodness! That paragraph sounds like terrible piffle! What I'm trying to say is that it's all trade-offs, and you should be prepared for not getting it right first time.
(Hadoop is a mechanism for doing a distributed file store, and for efficiently applying certain classes of operation – those that fit well with MapReduce or other similar scatter-gather algorithms – across that whole dataset. If that shoe fits, use it. But it doesn't solve all problems, and thank goodness for that! Things that can do everything tend to look very much like things that can't actually do anything at all, and usefulness and comprehensibility come in the restrictions.)
Hadoop is typically used to process massive data sets by distributing the processing of that data set across multiple machines.
What this means is you probably don't want to use it to "deploy an application". You might use it to process stats on your application, however. For instance, you might have very large logs of user data. This would happen if your user data grows to become too large to fit on a single hard drive, and/or would take too long for one machine to process stats on (using standard methods like an SQL query).
Ygam. While the traditional role of "clients" and "servers" have been pretty stable from 1960 till about 2005.
I believe with every fiber of my being, that distributed computing is that we all carry processors around in our pockets.
Phones do computing work. Phones do NOT need centralized servers, but they DO benefit from them.
Phones , Smartphones, tablets are an example of where distributed computation is going.
You can make a wifi base-station out of an Android device now. So now a phone becomes a server of sorts for just that instant in the coffee shop that you turn it on for that cute person next to you without internet ....and now I digress.......
What goals can be accomplished using a Rules Engine?
A Rules Engine is typically a component in software that is used to validate business rules at runtime.
They're often used in order to be able to easily change business rules without having to recompile/redeploy software. Rules can easily be stored in a corporate database, and sometimes even edited by less technical users who understand the business rules much more effectively.
For example, a mortgage company may need to change its criteria for approving a mortgage every week. By using a rules engine, you can easily create very complex criteria that can be modified without having to "program" the logic into the software. Say, this week, the base credit score shifts, you just adjust that rule in the chain of rules. Then, tomorrow, when the minimum income percentage for requiring PMI given a 90% equity rate shifts, you can just put that "rule" in place.
To add to #Reed Copsey's answer. Most rule engines keep their rules in external files that can be modified by business analysts or end users. Drools is one such solution.
In most cases the Rules being processed are at the business layer, it is possible to look at a firewall as a rule engine of sorts, another common type of rule engine is based on high level domain specific languages for transaction processing and decision making.
There are a number of companies that produce appliances, and web transactions as a service to. These give the analysts tools to form complex decision trees and expert systems without having to do the grunt work of the developer, thanks to the magic of EGL most of the translation is done by modeling tools that can stub and generate code and valid end points to meet the "business" needs automagically.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have a pretty decent list of the advantages of using a Rules Engine, as well as some reasons to use them, what I need is a list of the reasons why you should NOT use a Rules Engine
The best I have so far is this:
Rules engines are not really intended to handle workflow or process
executions nor are workflow engines or process management tools
designed to do rules.
Any other big reasons why you should not use them?
I will give 2 examples from personal experience where using a Rules Engine was a bad idea, maybe that will help:-
On a past project, I noticed that the rules files (the project used Drools) contained a lot of java code, including loops, functions etc. They were essentially java files masquerading as rules file. When I asked the architect on his reasoning for the design I was told that the "Rules were never intended to be maintained by business users".
Lesson: They are called "Business Rules" for a reason, do not use rules when you cannot design a system that can be easily maintained/understood by Business users.
Another case; The project used rules because requirements were poorly defined/understood and changed often. The development team's solution was to use rules extensively to avoid frequent code deploys.
Lesson: Requirements tend to change a lot during initial release changes and do not warrant usage of rules. Use rules when your business changes often (not requirements). Eg:- A software that does your taxes will change every year as taxation laws change and usage of rules is an excellent idea. Release 1.0 of an web app will change often as users identify new requirements but will stabilize over time. Do not use rules as an alternative to code deploy.
I get very nervous when I see people using very large rule sets (e.g., on the order of thousands of rules in a single rule set). This often happens when the rules engine is a singleton sitting in the center of the enterprise in the hope that keeping rules DRY will make them accessible to many apps that require them. I would defy anyone to tell me that a Rete rules engine with that many rules is well-understood. I'm not aware of any tools that can check to ensure that conflicts don't exist.
I think partitioning rules sets to keep them small is a better option. Aspects can be a way to share a common rule set among many objects.
I prefer a simpler, more data driven approach wherever possible.
The one poit I've noticed to be "the double edged sword" is:
placing the logic in hands of non technical staff
I've seen this work great, when you have one or two multidisciplinary geniuses on the non technical side, but I've also seen the lack of technicity leading to bloat, more bugs, and in general 4x the development/maintenance cost.
Thus you need to consider your user-base seriously.
I'm a big fan of Business Rules Engines, since it can help you make your life much easier as a programmer. One of the first experiences I've had while working on a Data Warehouse project was to find Stored Procedures containing complicated CASE structures stretching over entire pages. It was a nightmare to debug, since it was very difficult to understand the logic applied in such long CASE structures, and to determine if you have an overlapping between a rule at page 1 of the code and another from page 5. Overall, we had more than 300 such rules embedded in the code.
When we've received a new development requirement, for something called Accounting Destination, which was involving treating more than 3000 rules, i knew something had to change. Back then I've been working on a prototype which later on become the parent of what now is a Custom Business Rule engine, capable of handling all SQL standard operators. Initially we've been using Excel as an authoring tool and , later on, we've created an ASP.net application which will allow the Business Users to define their own business rules, without the need of writing code. Now the system works fine, with very few bugs, and contains over 7000 rules for calculating this Accounting Destination. I don't think such scenario would have been possible by just hard-coding. And the users are pretty happy that they can define their own rules without IT becoming their bottleneck.
Still, there are limits to such approach:
You need to have capable business users which have an excellent understanding of the company business.
There is a significant workload on searching the entire system (in
our case a Data Warehouse), in order to determine all hard-coded
conditions which make sense to translate into rules to be handled by
a Business Rule Engine. We've also had to take good care that these
initial templates to be fully understandable by Business Users.
You need to have an application used for rules authoring, in which
algorithms for detection of overlapping business rules is implemented. Otherwise you'll end up with a big mess, where no one understands anymore the results they get.
When you have a bug in a generic component like a Custom Business Rule Engine, it can be very difficult to debug and involve extensive tests to make sure that things that worked before also work now.
More details on this topic can be found on a post I've written: http://dwhbp.com/post/2011/10/30/Implementing-a-Business-Rule-Engine.aspx
Overall, the biggest advantage of using a Business Rule Engines is that it allows the users to take back control over the Business Rule definitions and authoring, without the need of going to the IT department each time they need to modify something. It also the reduces the workload over IT development teams, which can now focus on building stuff with more added value.
Cheers,
Nicolae
GREAT article on when not to use a rules Engine...(as well as when to use one)....
http://www.jessrules.com/guidelines.shtml
Another option is if you have a linear set of rules that only apply once in any order to get an outcome is to create a groovy interface and have developers write and deploy these new rules. The advantage is that it is wickedly fast because normally you would pass the hibernate session OR jdbc session as well as any parameters so you have access to all your apps data but in an efficient manner. With a fact list, there can be alot of looping/matching that really can slow the system down.....It's another way to avoid a rules engine and be able to be deployed dynamically(yes, our groovy rules were deployed in a database and we had no recursion....it either met the rule or it didn't). It is just another option.....oh and one more benefit is not learning rules syntax for incoming developers. They have to learn some groovy but that is very close to java so the learning curve is much better.
It really depends on your context. Rules engine have their place and the above is just another option if you have rules on a project that you may want to deploy dynamically for very simplified situations that don't require a rules engine.
BASICALLY do NOT use a rules engine if you have a simple ruleset and can have a groovy interface instead.....just as dynamically deployable and new developers joining your team can learn it faster than the drools language.(but that's my opinion)
In my experience, rules engines work best when the following are true:
Well-defined doctrine for your problem domain
High quality (preferably automated) data to help drive most of your inputs
Access to subject matter experts
Software developers with experience creating expert systems
If any of these four traits are missing, you still might find a rules engine works for you, but every time I've tried it with even 1 missing, I've run into trouble.
That's certainly a good start. The other thing with rules engines is that some things are well-understood, deterministic, and straight-forward. Payroll withholding is (or use to be) like that. You could express it as rules that would be resolved by a rules engine, but you could express the same rules as a fairly simple table of values.
So, workflow engines are good when you're expressing a longer-term process that will have persistent data. Rules engines can do a similar thing, but you have to do a lot of added complexity.
Rules engines are good when you have complicated knowledge bases and need search. Rules engines can resolve complicated issues, and can be adapted quickly to changing situations, but impose a lot of complexity on the base implementation.
Many decision algorithms are simple enough to express as a simple table-driven program without the complexity implied by a real rules engine.
I would strongly recommend business rules engines like Drools as open source or Commercial Rules Engine such as LiveRules.
When you have a lot of business policies which are volatile in nature, it is very hard to maintain that part of the core technology code.
The rules engine provides a great flexibility of the framework and easy to change and deploy.
Rules engines are not to be used everywhere but need to used when you have lot of policies where changes are inevitable on a regular basis.
I don't really understand some points such as :
a) business people needs to understand business very well, or;
b) disagreement on business people don't need to know the rule.
For me, as a people just touching BRE， the benefit of BRE is so called to let system adapt to business change, hence it's focused on adaptive of change.
Does it matter if the rule set up at time x is different from the rule set up at time y because of:
a) business people don't understand business, or;
b) business people don't understand rules?