I've recently been learning about inversion of control through dependency injection, and using Castle Windsor. I like it. I get it. The lightbulb over my head is burning brightly. But I have a nagging concern about the logging facility and the ILogger interface. What is that really doing there? Is it for me, or just for Windsor itself?
Since ILogger is intended to abstract away the differences between log4net and Nlog and whatever other logging frameworks it supports, it has to represent the lowest common denominator between them. The various frameworks are similar, but not necessarily identical. If there were some fantastic feature in one, but not in the other, it would either have to be left out of ILogger, or the ILogger implementations for the other logging frameworks would have to have no-op implementations of it, or something else that's not very satisfying.
Long before Windsor, I was a fan of log4net, and a lot of my favorite libraries use it, like NHibernate. So if I'm building a new application, I'll use log4net. I'm willing to commit to it, and I consider it a stable dependency -- as stable a dependency as needing System.Web for example. I would not write my components to use ILogger, I would write them to use ILog. But I get the impression that Windsor expects me to use ILogger for my own logging. Isn't that saddling my project with a dependency on Windsor, when I shouldn't have any dependency on my IoC container?
I see the point of Windsor having the logging facility so it can log its own operations using whichever logging framework the project wants to use. That seems perfectly sensible. But if I don't use ILogger for my own code, and just go straight to log4net's ILog, what am I giving up? Will I regret this?
The obvious response is that I might want to change logging frameworks in six months. But I won't. log4net is mature and stable. It's a project with a limited and very well-defined scope, which it implements nearly perfectly. It can be considered "finished". At most, I might need to write a custom appender to handle messages. (Maybe I want to write them onto a postcard and drop them in the mail for some reason.) But that's easily done within the log4net framework, and I would use it just like any other log4net appender. I would be no more likely to change logging frameworks than I would be to change web platforms.
It seems as if you have already made your mind up to use log4net directly. This is perfectly reasonable, as you consider it to be a stable dependency. I have just switched from log4net to NLog for our projects, so it does happen that you may change logging frameworks in future, which is where an abstraction has its advantages.
Another consideration when thinking about using a logging abstraction (other than losing functionality specific to a particular logging framework), is the extra overhead of learning the abstraction. Does this make the code more or less complex for developers to pick up?
In our case, we found NLog was so easy to install and configure directly, that we decided to lose our custom logging abstraction and switch from log4net (which we found a bit verbose in its xml configuration compared to NLog).
Related
Now firstly I realise the title is extremely broad, so let me describe the use case.
Background:
I'm currently teaching myself Scala+Gradle (because I like the flexibility and power of gradle and the much more legible build files)
As such with learning new languages its often best to make applications that you can actually use, and being primarily a PHP (with Symfony) programmer and formerly a Java programmer, there are many patterns that could carry across from both paradigms.
Use Case:
I'm writing an application where I am experimenting with a Provider+Interface(trait) layout, the goal is to define traits that encompass all the expected functionality for any particular type of component e.g. a ConfigReaderTrait and a YamlConfigReager as a provider. Theoretically the advantage of this would be to allow me to switch out core mechanisms or even architectural components with minimal effort, this allows for a great deal of R&D and experimenting.
PHP Symfony Influence
Now currently I work as a pure PHP dev, and as such has been introduced to Symfony, which has a brilliant Dependency Injection framework where the dependencies are defined in yaml files, and can be delegated to sub directories. I like this, because unlike with SBT I am unphased by using different languages for different purposes (eg groovy with gradle for build scripts) and I want to maintain a separation of concerns.
Such that each type of interface/trait or bundle of related functionality should be able to have its own DI config, and I would prefer it separate from the scala code itself.
Now for Scala....
Obviously things are not the same across different languages, and if you don't embrace the differences you may aswell go back to the previous language and leave things at that.
That said, I am not yet convinced by the DI frameworks I see for scala.
Guice for example is really a modified java framework (which is fine
because scala can use java libs, but because they don't function in
the entirely same paradigm of coding languages it feels as though
scala's capabilities are not leveraged)
MacWire annoyed me a bit,because you had to define the dependencies
in the files where you used them. Which does not assist in my
interface/provider concept.
SubCut so far seems to be the best suited to what I would expect.
But while going through all of this (and bare in mind this is all in the research phase, I havent used any of them yet) it seemed that DI in Scala is still very scattered, and in its infancy, by that I mean that there are different implementations with different applications, but not one flexible enough or power enough to compare to Symfonys DI. particularly not for my application.
Comments? Thoughts?
My 5 cents:
I have actually stopped using dependency injection frameworks after switching to Scala from Java.
The language allows for a few nice ways of doing it without a framework (multiple parameter lists and currying as well as the mixins for doing injection the way the 'cake pattern' does)
and I find myself more and more just using constructor or method parameter based injection as it clearly documents what dependencies a given piece of logic has and where it got those dependencies from.
It's also fairly easy to create different modules sets of implementations or factories for implementations using Scala objects and then selecting between those at runtime. This will give you the guarantee that it wont compile unless there is an implementation available, as opposed to the big ones in Java-land that will fail in runtime, effectively pushing a compile time problem into runtime.
This also removes the 'magic' of how dependencies are created and wired (reflection, runtime weaving, macros, XML, binding context to thread local etc). I think this makes it much easier for new developers to jump into a project and understand how the codebase is interconnected.
Regarding declaring implementations in non-code like XML I have found that projects rarely or never change those files without making a new release so then they might as well be code with all the benefits that bring (IDE support, performance, type checking).
I've been a heavy Windsor users for the last several years. Prior to the Fluent Registration API, I would toggle between Xml Registration and huge piles of AddComponent() code. We've been happily using the Fluent Registration API and Installers specifically for quite some time now. I've gotten the impression from various writings like this:
http://docs.castleproject.org/Windsor.XML-Registration-Reference.ashx
That the Xml Registration approach has fallen out of favor and it wouldn't surprise me if it were marked for deprecation at some point in the near future.
Now, for my question: The Fluent Registration API and Installers work swimmingly for auto-wiring scenarios (i.e. when I want Windsor to just figure out how to construct my object graphs). Auto-wiring is the vast majority of IoC use cases out there, but what about when auto-wiring isn't possible? In other words I have multiple implementations of a service and I need to tell Windsor how to construct parts of my object graph. I've done this many times with the Xml Registration approach, but is there a more preferred approach these days? I'm hesitant to go the Xml Registration approach as its future seems uncertain, but I don't know how else to accomplish this with Windsor.
My uses cases are:
System needs to be able to swap implementations at QA-test (i.e.
credit checks and fraud detection processing where we want to test
without a dependency on a credit bureau API)
Provider patterns in our
system where we need to conditionally turn on and off different
implementations at deploy-time.
This all seems very well suited for IoC and we have all the building blocks in place, but want to make sure I'm taking the most future-proof approach with Windsor.
UPDATE:
While I like the feature toggle approach, I recently discovered a Windsor feature that is very useful on this front - Fallback Components. I'm leaving this edit here for anyone that might stumbled across this later.
Configuring your DI container completely through XML is error prone, verbose, and just too painful. The XML configuration possibilities are always a subset of what you can do with code based configuration; code is always more expressive.
Sometimes though your DI configuration depends on deploy-time configurations, but since the number of knobs you need are often fairly small, using a configuration flag is often a much better approach than polluting your configuration file with fully qualified type names.
Or let me put it differently, when you have large amounts of your DI configuration placed in your configuration file because your might want to change them at deploy time, please think again. Many of the changes need testing (by a developer) anyway, so there is no way you want someone from your operations team to fiddle around with that. And when you need a developer to look at it and verify it, what's the advantage of not having to recompile the project? Is this actually any quicker? A developer would still have to start the application anyway.
It is a false sense of flexibility and in fact a poor interface design (xml is the interface for your maintenance and operations department). BTW, are you the person that needs to document how the configuration file should be changed?
Instead of describing the list of fully qualified type names that are valid somewhere in the middle of the xml file, wouldn't it be much easier of all you have to write is "place 'false' in this field to disable ..."?
Here is an example of how to use a configuration switch:
bool detectFraught =
ConfigurationManager.AppSettings["DetectFraud"] != "false";
container.Register(
Component.For(typeof(IFraughtDetector)).ImplementedBy(
detectFraught ? typeof(RealDectector) : typeof(FakeDetector));
See how the configuration switch is now simply a boolean flag. This makes the configuration file much more maintainable, since the configuration is now a simple boolean switch instead of a complete type name (that can be misspelled).
Of course doing the ["DetectFraud"] != "false" isn't that nice by itself, but this can simply be solved by creating a strongly-typed configuration helper.
This answer might help as well. Allows you to dynamically, at runtime, provide an implementation. Though, sounds like you don't need it that dynamically and it's a little less obvious what's going on.
There are no plans to obsolete or remove the XML config support in Windsor.
Yes, you are right, it isn't a preferred approach due to its numerous drawbacks.
Anything you can do in XML can be done in code (note that inverse is not true).
Also keep in mind XML is not all-or-nothing. There are many ways to achieve the scenarios you gave as examples without resorting to registration in XML.
Feature toggles
Conditional compilation
if/else in your installer based on a appSettings flag
others...
I've used each of them in different projects in the past.
My company needs to rewrite a large monolithic program, and I would want it written using a plugin type architecture. Currently the best solution appears to be MEF, but as it is a fairly 'new' thing I am warey of betting the future of my company (and my reputation) on it.
Does anyone have a feeling on how mature a solution MEF is ?
Thanks
Visual Studio's entire extension system is now built on MEF.
That is to say that Microsoft is Dog-fooding it (like they are doing with WPF).
Given that the framework developers themselves will be working with it, you can feel pretty confident that it is here to stay. However, as with any first release, you are almost guaranteed to have some growing pains when the next release comes around.
Personally, I would go for it. It is certainly better than the tightly-coupled-reflection-based alternative.
I don't think it is necessary to "bet on MEF". Your code should have very little dependencies on MEF.
You can use the technique of dependency injection to break up your monolithic application into components which have only a single responsibility, and which limit their knowledge of other components to abstractions. See this blog post by Nicholas Blumhardt for a nice overview of the type of relations that can exist between components.
Wiring the components together into an application can then be done with any dependency injection framework, or even manually. The component logic shouldn't need to be aware of the container - there might not even be a container.
In the case of MEF, you do need to add import/export attributes to your classes. However, you can still ignore those attributes and reuse those components without MEF, e.g. by using another DI framework like AutoFac.
It's a relatively new technology, so I'm not sure if it's exactly mature. I'm sure it will change quite a bit over the next several years, perhaps merging with other frameworks to better support IoC. That said, MS has a pretty good history of preserving backwards compatibility, so now that MEF is actually part of the Framework, I would consider the public interfaces stable.
That said, MEF might not actually be the right solution for your project. It depends on your extensibility needs and how large is 'large'. If you want to support true extensibility, including the possibility for third-party plugins, it has an enormous impact on your design responsibilities. It's much harder to make changes to the infrastructure as you now need to maintain very stable public interfaces. If you're really only after the IoC features, you're probably better off with a true IoC framework, which more clearly limits your design responsibility to support of your internal dependencies. If you're betting the future of the company, this is the bigger question, in my mind.
A repeating theme in my development work has been the use of or creation of an in-house plug-in architecture. I've seen it approached many ways - configuration files (XML, .conf, and so on), inheritance frameworks, database information, libraries, and others. In my experience:
A database isn't a great place to store your configuration information, especially co-mingled with data
Attempting this with an inheritance hierarchy requires knowledge about the plug-ins to be coded in, meaning the plug-in architecture isn't all that dynamic
Configuration files work well for providing simple information, but can't handle more complex behaviors
Libraries seem to work well, but the one-way dependencies have to be carefully created.
As I seek to learn from the various architectures I've worked with, I'm also looking to the community for suggestions. How have you implemented a SOLID plug-in architecture? What was your worst failure (or the worst failure you've seen)? What would you do if you were going to implement a new plug-in architecture? What SDK or open source project that you've worked with has the best example of a good architecture?
A few examples I've been finding on my own:
Perl's Module::Plugable and IOC for dependency injection in Perl
The various Spring frameworks (Java, .NET, Python) for dependency injection.
An SO question with a list for Java (including Service Provider Interfaces)
An SO question for C++ pointing to a Dr. Dobbs article
An SO question regarding a specific plugin idea for ASP.NET MVC
These examples seem to play to various language strengths. Is a good plugin architecture necessarily tied to the language? Is it best to use tools to create a plugin architecture, or to do it on one's own following models?
This is not an answer as much as a bunch of potentially useful remarks/examples.
One effective way to make your application extensible is to expose its internals as a scripting language and write all the top level stuff in that language. This makes it quite modifiable and practically future proof (if your primitives are well chosen and implemented). A success story of this kind of thing is Emacs. I prefer this to the eclipse style plugin system because if I want to extend functionality, I don't have to learn the API and write/compile a separate plugin. I can write a 3 line snippet in the current buffer itself, evaluate it and use it. Very smooth learning curve and very pleasing results.
One application which I've extended a little is Trac. It has a component architecture which in this situation means that tasks are delegated to modules that advertise extension points. You can then implement other components which would fit into these points and change the flow. It's a little like Kalkie's suggestion above.
Another one that's good is py.test. It follows the "best API is no API" philosophy and relies purely on hooks being called at every level. You can override these hooks in files/functions named according to a convention and alter the behaviour. You can see the list of plugins on the site to see how quickly/easily they can be implemented.
A few general points.
Try to keep your non-extensible/non-user-modifiable core as small as possible. Delegate everything you can to a higher layer so that the extensibility increases. Less stuff to correct in the core then in case of bad choices.
Related to the above point is that you shouldn't make too many decisions about the direction of your project at the outset. Implement the smallest needed subset and then start writing plugins.
If you are embedding a scripting language, make sure it's a full one in which you can write general programs and not a toy language just for your application.
Reduce boilerplate as much as you can. Don't bother with subclassing, complex APIs, plugin registration and stuff like that. Try to keep it simple so that it's easy and not just possible to extend. This will let your plugin API be used more and will encourage end users to write plugins. Not just plugin developers. py.test does this well. Eclipse as far as I know, does not.
In my experience I've found there are really two types of plug-in Architectures.
One follows the Eclipse model which is meant to allow for freedom and is open-ended.
The other usually requires plugins to follow a narrow API because the plugin will fill a specific function.
To state this in a different way, one allows plugins to access your application while the other allows your application to access plugins.
The distinction is subtle, and sometimes there is no distiction... you want both for your application.
I do not have a ton of experience with Eclipse/Opening up your App to plugins model (the article in Kalkie's post is great). I've read a bit on the way eclipse does things, but nothing more than that.
Yegge's properties blog talks a bit about how the use of the properties pattern allows for plugins and extensibility.
Most of the work I've done has used a plugin architecture to allow my app to access plugins, things like time/display/map data, etc.
Years ago I would create factories, plugin managers and config files to manage all of it and let me determine which plugin to use at runtime.
Now I usually just have a DI framework do most of that work.
I still have to write adapters to use third party libraries, but they usually aren't that bad.
One of the best plug-in architectures that I have seen is implemented in Eclipse. Instead of having an application with a plug-in model, everything is a plug-in. The base application itself is the plug-in framework.
http://www.eclipse.org/articles/Article-Plug-in-architecture/plugin_architecture.html
I'll describe a fairly simple technique that I have use in the past. This approach uses C# reflection to help in the plugin loading process. This technique can be modified so it is applicable to C++ but you lose the convenience of being able to use reflection.
An IPlugin interface is used to identify classes that implement plugins. Methods are added to the interface to allow the application to communicate with the plugin. For example the Init method that the application will use to instruct the plugin to initialize.
To find plugins the application scans a plugin folder for .Net assemblies. Each assembly is loaded. Reflection is used to scan for classes that implement IPlugin. An instance of each plugin class is created.
(Alternatively, an Xml file might list the assemblies and classes to load. This might help performance but I never found an issue with performance).
The Init method is called for each plugin object. It is passed a reference to an object that implements the application interface: IApplication (or something else named specific to your app, eg ITextEditorApplication).
IApplication contains methods that allows the plugin to communicate with the application. For instance if you are writing a text editor this interface would have an OpenDocuments property that allows plugins to enumerate the collection of currently open documents.
This plugin system can be extended to scripting languages, eg Lua, by creating a derived plugin class, eg LuaPlugin that forwards IPlugin functions and the application interface to a Lua script.
This technique allows you to iteratively implement your IPlugin, IApplication and other application-specific interfaces during development. When the application is complete and nicely refactored you can document your exposed interfaces and you should have a nice system for which users can write their own plugins.
I once worked on a project that had to be so flexible in the way each customer could setup the system, which the only good design we found was to ship the customer a C# compiler!
If the spec is filled with words like:
Flexible
Plug-In
Customisable
Ask lots of questions about how you will support the system (and how support will be charged for, as each customer will think their case is the normal case and should not need any plug-ins.), as in my experience
The support of customers (or
fount-line support people) writing
Plug-Ins is a lot harder than the
Architecture
Usualy I use MEF. The Managed Extensibility Framework (or MEF for short) simplifies the creation of extensible applications. MEF offers discovery and composition capabilities that you can leverage to load application extensions.
If you are interested read more...
In my experience, the two best ways to create a flexible plugin architecture are scripting languages and libraries. These two concepts are in my mind orthogonal; the two can be mixed in any proportion, rather like functional and object-oriented programming, but find their greatest strengths when balanced. A library is typically responsible for fulfilling a specific interface with dynamic functionality, whereas scripts tend to emphasise functionality with a dynamic interface.
I have found that an architecture based on scripts managing libraries seems to work the best. The scripting language allows high-level manipulation of lower-level libraries, and the libraries are thus freed from any specific interface, leaving all of the application-level interaction in the more flexible hands of the scripting system.
For this to work, the scripting system must have a fairly robust API, with hooks to the application data, logic, and GUI, as well as the base functionality of importing and executing code from libraries. Further, scripts are usually required to be safe in the sense that the application can gracefully recover from a poorly-written script. Using a scripting system as a layer of indirection means that the application can more easily detach itself in case of Something Bad™.
The means of packaging plugins depends largely on personal preference, but you can never go wrong with a compressed archive with a simple interface, say PluginName.ext in the root directory.
I think you need to first answer the question: "What components are expected to be plugins?"
You want to keep this number to an absolute minimum or the number of combinations which you must test explodes. Try to separate your core product (which should not have too much flexibility) from plugin functionality.
I've found that the IOC (Inversion of Control) principal (read springframework) works well for providing a flexible base, which you can add specialization to to make plugin development simpler.
You can scan the container for the "interface as a plugin type advertisement" mechanism.
You can use the container to inject common dependencies which plugins may require (i.e. ResourceLoaderAware or MessageSourceAware).
The Plug-in Pattern is a software pattern for extending the behaviour of a class with a clean interface. Often behaviour of classes is extended by class inheritance, where the derived class overwrites some of the virtual methods of the class. A problem with this solution is that it conflicts with implementation hiding. It also leads to situations where derived class become a gathering places of unrelated behaviour extensions. Also, scripting is used to implement this pattern as mentioned above "Make internals as a scripting language and write all the top level stuff in that language. This makes it quite modifiable and practically future proof". Libraries use script managing libraries. The scripting language allows high-level manipulation of lower level libraries. (Also as mentioned above)
Looking at Microsoft's Managed Extensibility Framework (MEF) and various IoC containers (such as Unity), I am failing to see when to use one type of solution over the other. More specifically, it seems like MEF handles most IoC type patterns and that an IoC container like Unity would not be as necessary.
Ideally, I would like to see a good use case where an IoC container would be used instead of, or in addition to, MEF.
When boiled down, the main difference is that IoC containers are generally most useful with static dependencies (known at compile-time), and MEF is generally most useful with dynamic dependencies (known only at run-time).
As such, they are both composition engines, but the emphasis is very different for each pattern. Design decisions thus vary wildly, as MEF is optimized around discovery of unknown parts, rather than registrations of known parts.
Think about it this way: if you are developing your entire application, an IoC container is probably best. If you are writing for extensibility, such that 3rd-party developers will be extending your system, MEF is probably best.
Also, the article in #Pavel Nikolov's answer provides some great direction (it is written by Glenn Block, MEF's program manager).
I've been using MEF for a while and the key factor for when we use it instead of IOC products is that we regularly have 3-5 implementations of a given interface sitting in our plugins directory at a given time. Which one of those implementations should be used is actually something that can only be decided at runtime.
MEF is good at letting you do just that. Typically, IOC is geared toward making sure you could swap out, for a cononical example, an IUserRepository based on ORM Product 1 for ORM Product 2 at some point in the future. However, most IOC solutions assume that there will only be one IUserRepository in effect at a given time.
If, however, you need to choose one based on the input data for a given page request, IOC containers are typically at a loss.
As an example, we do our permission checking and our validation via MEF plugins for a big web app I've been working on for a while. Using MEF, we can look at when the record's CreatedOn date and go digging for the validation plugin that was actually in effect when the record was created and run the record BOTH through that plugin AND the validator that's currently in effect and compare the record's validity over time.
This kind of power also lets us define fallthrough overrides for plugins. The apps I'm working on are actually the same codebase deployed for 30+ implementations. So, we've typically go looking for plugins by asking for:
An interface implementation that is specific to the current site and the specific record type in question.
An interface implementation that is specific to the current site, but works with any kind of record.
An interface that works for any site and any record.
That lets us bundle a set of default plugins that will kick in, but only if that specific implementation doesn't override it with customer specific rules.
IOC is a great technology, but really seems to be more about making it easy to code to interfaces instead of concrete implementations. However, swapping those implementations out is more of a project shift kind of event in IOC. In MEF, you take the flexibility of interfaces and concrete implementations and make it a runtime decision between many available options.
I am apologizing for being off-topic. I simply wanted to say that there are 2 flaws that render MEF an unnecessary complication:
it is attribute based which doesn't do any good to helping you figuring out why things work as they do. There's no way to get to the details burred in the internals of the framework to see what exactly is going on there. There is no way to get a tracing log or hook up to the resolving mechanisms and handle unresolved situations manually
it doesn't have any troubleshooting mechanism to figure out the reasons for why some parts get rejected. Despite pointing at a failing part it doesn't tell you why that part has failed.
So I am very disappointed with it. I spent too much time fighting windmills trying to bootstrap a few classes instead of working on the real problems. I convinced there is nothing better than the old-school dependency injection technique when you have full control over what is created, when, and can trace anything in the VS debugger. I wish somebody who advocates MEF presented a bunch of good reasons as to why would I choose it over plain DI.
I agree that MEF can be a fully capable IoC framework. In fact I'm writing an application right now based on using MEF for both extensibility and IoC. I took the generic parts of it and made it into a "framework" and open sourced it as its own framework called SoapBox Core in case people want to see how it works.
In particular, take a look at how the Host works if you want to see MEF in action.