Related
thanks you very much in advance.
First of all, I conceive scalability as the ability to design a system that doest not change when the demand of its services, whatever they are, increases considerably. May you need more hardware (vertically or horizontally0? Fine, add it at your leisure because the system is prepared and has been designed to cope with it.
My question is simple to ask but presumably very complex to answer. I would like to know what you I look at in a framework to make sure it will scale accordingly, both in number of hits and number of sessions running simultaneously.
This question is not about technology nor a particular framework at all, it is more a theoretical question.
I know that depend very much on having a good database design and a proper hardware behind with replication, etc... Let's assume that this all exists, however yet my framework must meet some criteria, what?
Provide a memcache?
Ability to run across multiple machines (at the web server level) and use many replicated databases? But what is in the software that makes that possible?
etc...
Please, let's not relate the answers with any particular programming language or technology behind.
Thanks again,
D.
I think scalability depends most of all on the use case: do you expect huge amounts of data, then you should focus on the database, if it's about traffic, focus on the server, is it about adding new features, focus on your data-model and the framework you are using...
Comparing a microposts-service like Twitter to a university website or a webservice like GoogleDocs you will find quite different requirements.
First of all the common notion of scalability is the ability of a software to improve in throughput or capacity if more hardware resources are added (CPUs, memory, bandwidth etc).
Software that does not improve in increased resources is not scalable.
Getting out of the definitions, I think your question is related to evaluation of frameworks you are planning to introduce to your implementation that may affect your software's ability to scale.
IMHO the most important factor to evaluate when introducing a framework is to see if there is hidden serialization in it (that serialization in effects transfers to/affects your software)
So if you introduce a framework that introduces serialization in your application that can affect your ability to scale.
How to evaluate?
Careful source code inspection (if open source)
Are there any performance guarantees offered by those that build the
framework?
Do measurements yourself to see how introducing this framework
affects your performance and replace if not satisfied
Our company plans to auto-generate our projects from the domain area up-to the presentation layer so that we can mass produce software. The idea is we can produce a project within 24 hours. I think this is possible but that's not my concern.
What are the ramifications of such plan? I just think that the quality of software produced from such grandeur idea is below good quality. First, clients have varying requirements. Assuming we can standardize what's common among them, there are still requirements that will be beyond our original template.
Second, how can such software be reliable if not fully tested? Does a 24 hour period can cover a full unit/integration/other types of test?
At the end, it appears we won't be able to hit the 24 hour target thereby defeating our original purpose.
I just believe it's better to build quality software than mass producing them. How would I tell my boss that their idea is wrong?
Sorry, but I don't think this is sensible.
In order to build a system that can auto-generate any kind of software that would fill any kind of requirement, you will kind of have to implement more then the softwares you plan to generate.
Auto-generated code is great, when you have some repetitive tasks, information, or components, that are similar enough to enable you to make a one time effort to generate all repetitions.
But trying to write a system to produce any kind of project is not feasible. In order to meet a wide enough range of supported projects, your system will have to allow a very wide set of capabilities to describe project configuration and behavior, and the time required to describe each project's behavior will not necessarily be shorter than the time it would have taken to implement the project in the first place. You will just end up developing a development environment, and implementing projects in your own language.
So, instead, why not just take an existing development environment that is already available? Like Visual Studio?
Maybe you should have a big library reprository and stuff everything that could be reusable in it. This way, the software you have to write would be very small. There could be documentation templates associated with the library and you would just have to C&P them.
However, it takes time to build such a library.
Or do it this way.
I'm at a point in my freelance career where I've developed several web applications for small to medium sized businesses that support things such as project management, booking/reservations, and email management.
I like the work but find that eventually my applications get to a point where the overhear for maintenance is very high. I look back at code I wrote 6 months ago and find I have to spend a while just relearning how I originally coded it before I can make a fix or feature additions. I do try to practice using frameworks (I've used Zend Framework before, and am considering Django for my next project)
What techniques or strategies do you use to plan out an application that is capable of handling a lot of users without breaking and still keeping the code clean enough to maintain easily?
If anyone has any books or articles they could recommend, that would be greatly appreciated as well.
Although there are certainly good articles on that topic, none of them is a substitute of real-world experience.
Maintainability is nothing you can plan straight ahead, except on very small projects. It is something you need to take care of during the whole project. In fact, creating loads of classes and infrastructure code in advance can produce code which is even harder to understand than naive spaghetti code.
So my advise is to clean up your existing projects, by continuously refactoring them. Look at the parts which were a pain to change, and strive for simpler solutions that are easier to understand and to adjust. If the code is even too bad for that, consider rewriting it from scratch.
Don't start new projects and expect them to succeed, just because your read some more articles or used a new framework. Instead, identify the failures of your existing projects and fix their specific problems. Whenever you need to change your code, ask yourself how to restructure it to support similar changes in the future. This is what you need to do anyway, because there will be similar changes in the future.
By doing those refactorings you'll stumble across various specific questions you can ask and read articles about. That way you'll learn more than by just asking general questions and reading general articles about maintenance and frameworks.
Start cleaning up your code today. Don't defer it to your future projects.
(The same is true for documentation. Everyone's first docs were very bad. After several months they turn out to be too verbose and filled with unimportant stuff. So complement the documentation with solutions to the problems you really had, because chances are good that next year you'll be confronted with a similar problem. Those experiences will improve your writing style more than any "how to write good" style guide.)
I'd honestly recommend looking at Martin Fowlers Patterns of Enterprise Application Architecture. It discusses a lot of ways to make your application more organized and maintainable. In addition, I would recommend using unit testing to give you better comprehension of your code. Kent Beck's book on Test Driven Development is a great resource for learning how to address change to your code through unit tests.
To improve the maintainability you could:
If you are the sole developer then adopt a coding style and stick to it. That will give you confidence later when navigating through your own code about things you could have possibly done and the things that you absolutely wouldn't. Being confident where to look and what to look for and what not to look for will save you a lot of time.
Always take time to bring documentation up to date. Include the task into development plan; include that time into the plan as part any of change or new feature.
Keep documentation balanced: some high level diagrams, meaningful comments. Best comments tell that cannot be read from the code itself. Like business reasons or "whys" behind certain chunks of code.
Include into the plan the effort to keep code structure, folder names, namespaces, object, variable and routine names up to date and reflective of what they actually do. This will go a long way in improving maintainability. Always call a spade "spade". Avoid large chunks of code, structure it by means available within your language of choice, give chunks meaningful names.
Low coupling and high coherency. Make sure you up to date with techniques of achieving these: design by contract, dependency injection, aspects, design patterns etc.
From task management point of view you should estimate more time and charge higher rate for non-continuous pieces of work. Do not hesitate to make customer aware that you need extra time to do small non-continuous changes spread over time as opposed to bigger continuous projects and ongoing maintenance since the administration and analysis overhead is greater (you need to manage and analyse each change including impact on the existing system separately). One benefit your customer is going to get is greater life expectancy of the system. The other is accurate documentation that will preserve their option to seek someone else's help should they decide to do so. Both protect customer investment and are strong selling points.
Use source control if you don't do that already
Keep a detailed log of everything done for the customer plus any important communication (a simple computer or paper based CMS). Refresh your memory before each assignment.
Keep a log of issues left open, ideas, suggestions per customer; again refresh your memory before beginning an assignment.
Plan ahead how the post-implementation support is going to be conducted, discuss with the customer. Make your systems are easy to maintain. Plan for parameterisation, monitoring tools, in-build sanity checks. Sell post-implementation support to customer as part of the initial contract.
Expand by hiring, even if you need someone just to provide that post-implementation support, do the admin bits.
Recommended reading:
"Code Complete" by Steve Mcconnell
Anything on design patterns are included into the list of recommended reading.
The most important advice I can give having helped grow an old web application into an extremely high available, high demand web application is to encapsulate everything. - in particular
Use good MVC principles and frameworks to separate your view layer from your business logic and data model.
Use a robust persistance layer to not couple your business logic to your data model
Plan for statelessness and asynchronous behaviour.
Here is an excellent article on how eBay tackles these problems
http://www.infoq.com/articles/ebay-scalability-best-practices
Use a framework / MVC system. The more organised and centralized your code is the better.
Try using Memcache. PHP has a built in extension for it, it takes about ten minutes to set up and another twenty to put in your application. You can cache whatever you want to it - I cache all my database records in it - for every application. It does wanders.
I would recommend using a source control system such as Subversion if you aren't already.
You should consider maybe using SharePoint. It's an environment that is already designed to do all you have mentioned, and has many other features you maybe haven't thought about (but maybe you will need in the future :-) )
Here's some information from the official site.
There are 2 different SharePoint environments you can use: Windows Sharepoint Services (WSS) or Microsoft Office Sharepoint Server (MOSS). WSS is free and ships with Windows Server 2003, while MOSS isn't free, but has much more features and covers almost all you enterprise's needs.
I'm auditing a project that uses what is called a Rules Engine. In short, it's a way to externalize business logic from application code.
This concept is entirely new to me and I'm pretty skeptical about it. After hearing people talk about Anemic Domain Models for the past few years, I'm questioning the Rules Engine Approach. To me they seem like a great way to WEAKEN a domain model. For example say I'm doing a java webapp interacting with a Rules Engine. Then I decide I want to have an Android app based on the same domain. Unless I want the Android app to interact with the Rules Engine as well, I'm going to have to miss out on whatever business logic was already written.
As I don't have any experience with them yet, just curiosity, I was interested to hear about the pros and cons are in using a Rules Engine? The only pro that I can think of is that you don't need to rebuild your entire Application just to change some business rule (but really, how many apps really have that many changes?). But using a Rules Engine to solve that problem kind of sounds to me like putting a band-aid over a shotgun wound.
UPDATE - since writing this, the god himself, Martin Fowler, has blogged about using a Rules engine.
Most rule engines that I have seen are viewed as a black box by system code. If I were to build a domain model, I would probably want certain business rules to be intrinsic to the domain model, e.g. business rules that tell me when an object has invalid values. This allows multiple systems to share the domain model without duplicating business logic. I could have each system use the same rule service to validate my domain model, but this appears to weaken my domain model (as was pointed out in the question). Why? Because instead of consistently enforcing my business rules across all systems at all times, I am relying on system programmers to determine when the business rules should be enforced (by calling the rule service). This may not be a problem if the domain model comes to you completely populated, but can be problematic if you're dealing with a user interface or system that changes values in the domain model over its lifetime.
There is another class of business rules: decision making. For example, an insurance company may need to classify the risk of underwriting an applicant and arrive at a premium. You could place these types of business rules in your domain model, but a centralized decision for scenarios like this are usually desirable and, actually, fit quite well into a service-oriented architecture. This does beg the question of why a rule engine and not system code. The place where a rule engine may be a better choice is where business rules responsible for the decision change over time (as some other answers have pointed out).
Rule engines usually allow you to change rules without restarting your system or deploying new executable code (regardless of what promises you receive from a vendor, do make sure you test your changes in a non-production environment because, even if the rule engine is flawless, humans are still changing the rules). If you're thinking, "I can do that by using a database to store values that change", you're right. A rule engine is not a magical box that does something new . It is intended to be a tool that provides a higher level of abstraction so you can focus less on reinventing the wheel. Many vendors take this a step further by letting you create templates so that business users can fill in the blanks instead of learning a rule language.
One parting caution about templates: templates can never take less time than writing a rule without a template because the template must, at the bare minimum, describe the rule. Plan for a higher initial cost (the same as if you were to build a system that used a database to store values that change vs. writing the rules in directly in system code) - the ROI is because you save on future maintenance of system code.
I think your concerns about anemic domain models are valid.
I've seen two applications of a well-known commercial Rete rules engine running in production where I work. I'd consider one a success and the other a failure.
The successful application is a decision tree app, consisting of ~10 trees of ~30 branch points each. The rules engine has a UI that does allow business folks to maintain the rules.
The less successful application has ~3000 rules slammed into a rules database. No one has any idea if there are conflicting rules when a new one is added. There is little understanding of the Rete algorithm, and the expertise with the product has left the firm, so it's become a black box that's untouchable and unrefactorable. The deployment cycle is still affected by rules changes - a complete regression test must be done when rules are changed. Memory was an issue, too.
I'd tread lightly. When a rule set is modest in size it's easy to understand changes, like the simplistic e-mail sample given above. Once the number of rules climbs into the hundreds I think you might have a problem.
I'd also worry about a rules engine becoming a singleton bottleneck in your application.
I see nothing wrong with using objects as a way to partition that rules engine space. Embedding behavior in objects that defer to a private rules engine seems okay to me. Problems will hit you when the rules engine requires state that isn't part of its object to fire properly. But that's just another example of design being difficult.
Rule Engines can offer a lot of value, in certain instances.
First, many rule engines work in a more declarative way. A very crude example would be AWK, where you can assign regexes to blocks of code. When the regex is seen by the file scanner, the block of code is executed.
You can see that in this case, if you had, say, a large AWK file and you wanted to add Yet Another "rule", you can readily go to the bottom of the file, add you regex and logic, and be done with it. Specifically, for many applications, you're not particularly concerned with what the other rules are doing, and the rules don't really interoperate with each other.
Thus the AWK file becomes more like a "rule soup". This "rule soup" nature lets folks focus very tightly on their domain with little concern for all of the other rules that may be in the system.
For example, Frank is interested in orders with a total of more than $1000, so he puts in to the rule system that he's interested. "IF order.total > 1000 THEN email Frank".
Meanwhile, Sally wants all orders from the west coast: "IF order.source == 'WEST_COAST' THEN email Sally".
So, you can see in this trivial, contrived case, that an order can satisfy both rules, yet both rules are independent of each other. A $1200 order from the West Coast notifies both Frank and Sally. When Frank is no longer concerned, he'll simply yank his rule out from the soup.
For many situations, this flexibility can be very powerful. It can also, like this case, be exposed to end users for simple rules. Using high level expressions and perhaps lightweight scripting.
Now, clearly, in a complicated system there are all sorts of interrelationships that can happen, an this is why the entire system is not "Done with rules". Someone, somewhere is going to be in charge of the rules not getting out of hand. But that doesn't necessarily lessen the value such a system can provide.
Mind this doesn't even go in to things like expert systems, where rules fire on data that rules can create, but a simpler rules system.
Anyway, I hope this example shows how a rules system can help augment a larger application.
The biggest pro that I've seen for rules engines is that it lets the Business Rule owners implement the business rules, instead of putting the onus on programmers. Even if you have an agile process where you are constantly getting feedback from stakeholders and going through rapid iterations, it's still not going to achieve the level of efficiency that can be achieved by having the people making the business rules implement them as well.
Also, you can't under-emphasize the value in removing the recompile-retest-redeploy cycle that can result from a simple rule change, if the rules are embedded in code. There are often several teams that are involved in putting the blessing on a build, and using a Rules Engine can make much of that unnecessary.
I've written a rules engine for a client. The biggest win was including all the stakeholders. The engine could run (or replay) a query and explain what was happening in text. The business people could look at the text description and quickly point out nuances in rules, exceptions, and other special cases. Once the business side was involved, the validation got much better because it was easy to get their input. Additionally, the rules engine can live separately from other parts of an application code base so you can use it across applications.
The con is that some programmers don't like to learn too much. Rule engines and the rules you put into them, along with the stuff that implements them, can be a bit hairy. Although a good system can easily handle sick and twisted webs of logic (or illogic often ;), it's not as simple as coding a bunch of if statements (no matter what some of the simple-minded rule engines do). The rules engine gives you the tools to handle rule relationships, but you still have to be able to imagine all of that in your mind. Sometimes it's like living in the movie Brazil. :)
It (as everything else) depends on your application. For some applications (usually the ones that never change or the rules are best on real life constants, i.e. won't change noticeably in eons, for instance physical properties and formulae) it doesn't make sense to use a rule engine, it just introduces additional complexity and requires the developer to have a larger skill set.
For other applications it's really a good idea. Take for instance order processing (orders being anything from invoicing to processing currency transactions), every now and then there's a minute change to some relevant law or code (in the judicial sense) that requires you to fulfil a new requirement (for instance sales tax, a classic).
Rather than trying to force your old application into this new situation where all the sudden you have to think about sales tax, where as before you didn't, it is easier to adapt your rule set rather than having to meddle about in potentially a large set of your code.
Then the next amendment from your local government requires reporting of all sales within a certain criteria, rather than you have to go in and add that, too. In the end you'll end up with very complex code that will prove pretty difficult to manage when you turn around and want to revert effect of one of the rules, without influencing all the others...
Everybody thus far has been very positive about rules engines, but I advise the reader to be wary. When a problem becomes a little bit more complicated, you may suddenly find that an entire rules engine has been rendered unsuitable, or much more complicated than in a more powerful language. Also, for many problems, rules engines will not be able to easily detect properties that greatly reduce the runtime and memory footprint of evaluating the condition. There are relatively few situations in which I would prefer a rule engine to a dependency injection framework or a more dynamic programming language.
"but really, how many apps really have that many changes?"
Honestly, every app I have worked on has gone through serious workflow and/or logic changes from concept until well after deployment. It's the number one reason for "maintenance" programming...
The reality is that you can't think of everything up front, hence the reason for Agile processes. Further, the BA's always seem to miss something vital until it's found in testing.
Rule Engines force you to truly separate business logic from presentation and storage. Further, if using the right engine, your BA's can add and remove logic as necessary. As Chris Marasti-Georg said, it puts the onus on the BA. But more than that, it allows the BA to get exactly what they are asking for.
A rules engine is a win on a configurable application where you don't want to have to do custom builds if it can be avoided. They are also good at centralising large bases of rules and algorithms like Rete are efficient for quickly matching against large rule sets.
Lots of good answers already but wanted to add a couple of things:
in automating a decision of any complexity the critical thing rapidly becomes your ability to manage rather than execute the logic involved. A rules engine will not help with this - you need to think about the rule management capabilities that a business rules management system has. Most commercial and open source rules engines have evolved into rule management systems with repositories, reporting on rule usage, versioning etc. A repository of rules, structured into coherent rule sets that can be orchestrated to make business decisions is a lot easier to manage than either thousands of lines of code or a rule soup.
There are many ways to use a declarative, rules-based approach. Using rules to manage a UI or as part of defining a process can be very effective. The most valuable use of a rules approach, however, is to automate business decisions and to deliver this as loosely coupled decision services that take input, execute rules and return an answer - a decision. Thing of these as services that answer questions for other services like "is this customer a good credit risk" or "what discount should I give this customer for this order or "what's the best cross sell for this customer at this time. These decision services can be very effectively built using a rules management system and allow for easy integration of analytics over time, something many decisions benefit from.
I see rule, process and data engines (a.k.a. databases) as essentially being similar. But, for some reason, we never say that blackboxing the persistence subsystem is bad.
Secondly, from my POV, an anemic model is not one that is light in implementation of behaviors, it is one that is light in behaviors itself. The actual method that describes an available behavior in a domain model object does not have to be done by the object itself.
The biggest complexity from my experience in Rule Engines is that:
from OOP POV it's a real pain to refactor and test rules written in a declarative language while you are refactoring code that affects them.
Often we should always think about the execution order of rules which turns into a mess when there are lots of them.
Some minor changes may trigger incorrect behaviour of rules leading to production bugs. In practice it's not always possible to cover all cases with tests up front.
Rules mutating objects used in other ones also increase complexity causing developers to break them into stages.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 7 years ago.
Improve this question
Our system (exotic commodity derivative trade capture and risk management) is being redeveloped shortly. One proposal that I have heard is that a rule engine will be incorporated to make it easier for the end-users (commodities traders, so fairly sophisticated) to make certain changes to the business logic.
I am a little skeptical of rules engines. The agilist in me wonders if they are just a technical solution to a process problem... ie. it takes too long for our developers to respond to the business's need for change. The solution to that problem should be a more collaborative approach to development, better test coverage, more agile practices all around.
Hearing about situations where a rule engine was truly a boon (especially in a trading environment) would be certainly helpful.
I've seen two applications that used the Blaze Rete engine from Fair Issac.
One application slammed thousands of rules into a single knowledge base, had terrible memory problems, has become a black box that few understand. I would not call that a success, but it is running in production.
Another application used decision trees to represent on the order of hundreds of questions on a medical form to disposition clients. It was done so elegantly that business people can update the rules as needed, without having to involve a developer. (Still has to be deployed by one, though.) I'd call that a great success.
So it depends on how well focused the problem is, the size of the rule set, the knowledge of the developers. My prejudice is that simply making a rules engine a single point of failure and dumping rules into it probably isn't a good approach. I'd start with a data-driven or table-driven approach and grow that until a rules engine was needed. I'd also strive to encapsulate the rules engine as part of the behavior of an object. I'd hide the rules engine from users and try to partition the rules space into the domain model.
I don't know if I'd say they're ever truly a boon, but I think they can certainly be valuable. I worked on a system for a few years in the insurance industry, where a rules engine was employed quite successfully to allow the business users to create rules that determined what policies were legal, depending on the state.
For instance, if you had to have a copay in certain states, or certain combinations of deductible and copay were not allowed, either because of product considerations, or because it was simply illegal due to state law.
The number of states that the company operated in, along with the constant change in rules (quarterly) would make this a dizzying coding practice. More importantly, it's not in the expertise of a programmer. It adds extra pointless communication where the end user is describing the rule to be put in effect to a programmer who is not an insurance industry expert like they are.
Designed correctly, a rules engine can still enable a workflow system that allows for good testing. In this case, the rules were stored in a database, and there were QA and PROD databases. So the BA's could test their rules in QA, and then promote them to PROD.
As with anything, its usually about the implementation, and not the actual technique.
Yes, Microsoft has a Business Rule Engine (BRE) in BizTalk that has been used successfully for years. I've heard that they've had clients buy BizTalk (very expensive) just for the BRE.
In my experience, the practicality of having a business user update the rules is slim to none. It usually takes a technical person to work the business rules editor.
A rule engine is little more than something that executes declarative statements. They come with two primary advantages (that I see):
Your business logic is maintained from a single place instead of being sprinkled throughout application code. Technically, a well-designed application should already do this with the architecture, regardless of a rule engine being present or not.
You need to worry [less] about dependencies between declarative statements. The rule engine should be smart enough to decide the order to run rules based on dependencies. You may find that some rule engines support a sequential ordering of rules within a ruleset or calling rulesets (groups of rules) in a particular order, but this isn't really in the spirit of declarative programming. Many rule engines use Rete (an algorithm) to decide when to schedule the execution of declarative statements.
I suspect most, if not all, rule engines add more overhead than if you were to write the best possible program that doesn't use a rule engine. This is similar to how writing code in assembly is generally faster than a compiler (but you usually don't write assembly because it's more convenient and productive to use higher-level abstractions).
If you were to stop here, then you would probably use programmers to maintain rules and use a rule engine as a convenient way to build a business logic tier in your application. Some rule engines offer something called templates that let you define templates for rules. The advantage here is that non-technical users are supposed to be able to write their own rules and modify existing rules.
A rule engine is one more tool in your tool chest that, when used properly, can be valuable.
The problem with many of these rule engines are the lack of speed and the fact that replacing or augmenting rules can break existing working rules in subtle ways. So you still have to re-test the system thoroughly after each rule change. So you're basically just exchanging one computer language for another one - one with a much smaller base of users. As another poster mentioned, I've yet to see a business analyst successfully use a rule engine. You need a programmer anyway.
I certainly have, but can't publicly talk about them, but its likely you have interacted with one several times this year ;)
I see it in 2 camps: the logic programmers and the business users. Different tools target different sets, some both. The successful cases of business users have only worked when it was a subset of the logic, and they also had a way to define test cases and run them themselves (and they are prepared to think logically).
Logic programmers are rarer, but can often be found coming from non imperative programming backgrounds (they are also the sort of people that find functional programming intuitive).
Keep in mind at the end of the day even with visual tools, if you are telling a computer to do something it is still programming.
I work with lots of vendors in the space and one of the great things about this is that I get to talk to lots of their customers. So, yes, hundreds of companies have got exactly the benefits they were promised - increased agility, better business/IT collaboration, easier regulatory compliance, better consistency of decision making, lower maintenance costs, faster times to market etc.
Over and over again, across all the major vendors and the open source players, I see that used correctly - to automate and improve high-volume operational decisions with many rules, rules that change a lot, rules that interact in complex ways or rules with a high business domain content - business rules management systems work.
Really.
My experience is limited to (i)not much and (ii)prolog; but I can safely say that a rule engine can help you express propositional concepts much cleaner than procedural code.
Rules engines are routinely used in the insurance business. I've worked on systems with hundreds (600ish) rules that were implemented in a rules engine. It worked very well.
Do you have a credit rating? A FICO score, perhaps? That's Fair Isaac COrporation, the developers of the Blaze rules engine.
For a while I worked for the PEATE distributed computing project which was developing a system for calculation of large scale, high volume atmospheric data. The system had three parts to it: the data manager, the scheduler, and the algorithm execution component. There could be any number of any of these components, all done through web services, but what it allowed for was for different researchers to execute arbitrary jobs against arbitrary data, and also allowed for different scheduling mechanisms to be plugged in as requirements changed.
I left the project before it got too far off the ground, but this seems like it could potentially fit the scenario, and serve as another example for some kind of rule engine. That being said, however, if the original developers are still going to be the one's making the algorithms to run, I can't see too much benefit in having a rule engine unless it handled a substantial overhead that each rule or algorithm would incur on it's own.
This sounds like a bit more involved than a simple rule engine, but such an architecture could feasibly apply to a rule engine as well.