We're building an application that supports plug-ins (which must import/export certain interfaces defined by us) written by third party developers. We're planning to use MEF to detect the presence of these plug-ins in a specified directory.
What level of control can I impose on plug-in developers to minimise the possibility that badly written components can cause memory leaks (affecting stability) and/or hog the processor (affecting performance) of our host application?
I have used tools such as ANTS memory and performance profiling applications to track down and diagnose problems when they occur, but it may not always be possible to do this given that the application may be running in environments over which we have no control.
What level of control can I impose on plug-in developers to minimise the possibility that badly written components can cause memory leaks (affecting stability) and/or hog the processor (affecting performance) of our host application?
You would have to load the plugins in a separate child process, and then communicate with that process via WCF or some other form of IPC. That way you could monitor resource usage and kill the plugin if necessary.
MEF doesn't help you with this. But perhaps it might be possible to write your own ExportProvider implementation which handles some or all of that.
edit: The IsolatingCatalog by Piotr Włodek look promising.
Related
I'm in the process of making an app which I was hoping could have optional modules/plugins depending on what the user needs. Concretely, the host application would be lightweight (mostly a text/markdown editor) and I'd add the ability to use plugins. However, some plugins could be fairly heavy (for example a 3D viewer).
Would it be better to have plugins loaded at runtime at the cost of performance or to directly integrate those with the main code with an ability to turn them off at the cost of space? Ideally I'd want both high performance and low volume, but if I had to pick one I'd choose performance.
Feel free to suggest alternatives! I'm not too familiar with modular programming :)
In our firm we run complex simulations using our own software developed in .NET. These simulations are well-suited to parallel computation and we currently make much use of the various multi-threading features native to .NET. Even so, simulations often take hours or days.
We'd like to explore the potential of distributing computation over our local network of high-performance (24 core) workstations to access more CPU power. However we have no experience in this area.
Searching on Google reveals a few MPI-based options such as Pure MPI, MPI.NET, plus some commercial software such as Frontier.
Which solution should we consider for something that is ideally well-suited to a .NET environment and is relatively easy to set up?
Thanks!
Multithreading != grid computing, so you will need to rewrite some parts of your application regardless of what you will choose in the end.
I don’t know your network infrastructure but it sounded to me, like you would want to use normal desktop workstations to run distribute the code. I wouldn’t use MPI for that. MPI was rather developed for clusters and supercomputers where the network supports high bandwidth and low latency. Those aren’t the properties of a traditional office network (unless I understood something wrong).
The next thing you have to deal with is the fact that users shouldn’t turn off their machines if computations are performed on them. No grid computing platform (including MPI) deals with these kind of issues, as it is usually running on server hardware which has little failures and are running 24/7.
I don’t think there is a simple and inexpensive solution to this. You could have a service running on each machine which could execute code from DLLs with predefined parameters and send responses. Those assemblies could be downloadable from some windowsshare. But you want to have really huge peaces of work to be distributed like this. You wouldn’t get almost any improvements if the application runs only for a minute or less.
In the end you’d need also a service to find those services which are online or not, some kind of in memory DB where every service could write the IP address and that it’s online so that the clients would know to whom they can distribute the work. This could be done using RavenDB (as you said you are working with .Net), Redis or an application which was actually written for these kind of problems, Zookeeper.
I have a small Akka application that passes many messages between its actors and each actor does some calculations on the data it receives. What I want is to profile this application in order to see which parts of the code take up most time and so on.
I tried VisualVM but I cannot really understand what's going on. I added a picture of the profiler output.
My questions are
What for example is this first line and why does it take up so much time? (scala.concurrent.forkjoin.ForkJoinPool.scan())
Can Akka applications because of their asynchronous behaviour be profiled well at all?
Can I see for instance how long one specific actor(-type) works for one specific message(-type) it receives?
Are there other best-practices for profiling Akka applications?
There are packages not profiled by default and it is their time that is accounted in the profile of scala.concurrent.forkjoin.ForkJoinPool.scan(). If all the hidden packages are allowed to be sampled, the true CPU time consumers will be revealed. For example, the following before/after illustrative profiles uncover that threads are put to sleep most of the time by sun.misc.Unsafe.park waiting to be unparked.
Akka applications can be profiled quite well with proper instrumentation and call tracing. Google's prominent Dapper, a Large-Scale Distributed Systems Tracing Infrastructure paper contains detailed explanation of the technique. Twitter created Zipkin based on that. It is open sourced and has an extension for distributed tracing of Akka. Follow its wiki for a good explanation of how to set up a system that allows to
trace call hierarchies inside an actor system;
debug request processing pipelines (you can log to traces, annotate them with custom key-value pairs);
see dependencies between derived requests and their contribution to resulting response time;
find and analyse slowest requests in your system.
There is also a new kid on the block, Kamon. It is a reactive-friendly toolkit for monitoring applications that run on top of the JVM, which is specially enthusiastic to applications built with the Typesafe Reactive Platform. That definitely means yes for Akka and the integration comes in the form of the kamon-akka and kamon-akka-remote modules that bring bytecode instrumentation to gather metrics and perform automatic trace context propagation on your behalf. Explore the documentation starting from Akka Integration Overview to understand what it can and how to achieve that.
Just a couple of days ago TypeSafe announced that TypeSafe console now is free. I don't know what can be better for profiling Scala/Akka applications. Of cause you can try JProfiler for JVM languages, I've used it with Java projects, but it's not free and for Java.
I was thinking about profiling/metrics in code since I also use Akka/Scala a lot for building production applications, but I also eager to hear alternative ways to make sure that application is healthy.
Metrics (like Dropwizard)
Very good tool for collecting metrics in the code, with good documentation and embedded support for Graphite, Ganglia, Logback, etc.
It has verbose tools for collecting in-app statistics like gauges, counter histograms, timings - information to figure out what is the current state of your app, how many actors were created, etc, if they are alive, what the current state is of majority of actors, etc.
Agree, it's a bit different from profiling but helps a lot to find roots of the problem, especially if integrated with some char building tool.
Profilers like (VisualVM, XRebel)
Since I'm a big fun of doing monitoring, it still answers a slightly different question - what are current insights of my application right now?
But there is quite another matter may disturb us - what of my code is slow (or sloppy)?
For that reason, we have VisualVM and another answers to this question - how to profile Akka actors with VisualVM.
Also, I'd suggest trying XRebel profiler that just adds a bit more firepower to process of figuring out what code makes app slower. It's also paid but on my project it saved a lot of time dealing with sloppy code.
New Relic
I'd suggest it for some playground projects since you can get some monitoring/profiling solutions for free, but on more serious projects I'd go for things I highlighted above.
So I hope, that my overview was helpful.
We are evaluating few options for developing a telecommunication related application platform (and migrating/consolidating some of the standalone apps into the new platform). One of our main concerns is the ability to handle high volume of requests during peak hours.
We feel the TorqueBox seems to an interesting solution worthy of consideration because:
Speed (Next to pure Java performance)
Faster development time over Java
Maintainability
Support for threads/concurrency even though it's Ruby
Faster/Easier front end development with Rails
...
RedHat supported and runs on JBoss (scalability, future development and ability to call Java if necessary)
Has anyone developed/deployed similar application(s) with JRuby/TorqueBox?
Any serious performance bottlenecks ahead? (or why we shouldn't use JRuby and should stick with Java?)
The answer is YES but be aware of memory leaks (gems, threadsafety issues, etc). You have to be familiar with tools like VisualVM, Eclipse MAT and/or NewRelic.
We're successfully using Torquebox on production for some clients on amazon EC2 handling 60k-80k visits per day (new c3 instances are great for Java).
Deployment is also an issue. We're unable to setup any kind of rolling restart because of memory consumption. So every time we deploy using Capistrano a full JBoss restart is needed (no big issue for us).
Bests,
Antonio
yes any mature Java web-server with JRuby is a valuable option. the details of handling high-loads on peak hours will really depend on what kind of app you'll be running, how much "hardware" can you afford to use but in general it's achievable but be aware there might still be some "gotchas" e.g. Ruby libraries (gems) that do not handle thread-safety well. you simply need to understand how to proceed than - which seems you do since you're want to use 'Celluloid.IO' :)
Can you give examples how you used gSOAP and how well it was integrated in your existing architecture? Have you found development bottlenecks with gSOAP?
We used gSOAP for a bunch of ARM clients to communicate with an AXIS Web Service server. Pros of gSOAP:
very powerful, supports nearly all Web Service constructs
easy to use, its abstraction of WS calls into functions removes all Web Service complexity to the programmer
elegant interfaces in both C and C++
However, we ran into several development bottlenecks:
when using custom datatypes like maps or sets, it takes quite some hacking to get the gSOAP compiler to handle them (marshal/unmarshalling). Especially bad with dynamic data structures.
debugging is hard because of its intrinsic complex network, parsing and memory allocation parts. Do everything possible to stick with static memory allocation.
the mailing list is alive, but developers are not very active on it. Simple questions can get answered quickly, but the toughest problems often go unanswered
forget about optimization. Linking in gSOAP eats about 1MB of memory at runtime (-Os). Runtime performance is fine on our 32MB linux-based ARM board, but there really is little to do about optimization if you need it.
We used gSOAP in a C++-based web server about 4 years back. Overall it worked fine. The only major issue was that the interface was in C and procedural (I understand it is difficult to design a good non-procedural interface). There may be a lot of repeated code while implementing the interface, for which you might have to use macros (we didn't explore the templates option too far then).
We are using gSoap to deploy a web service onto an embedde linux device running an ARM MX processor.
We are using gSOAP to consume a WCF based webservice from an application deployed on a linux device running on ARM processor. The experience is good to a large extent.
We used gSOAP in a web server on ARM ARM9 400MHz device.
gSOAP daemon connected to a database daemon through zeromq library, which is run on the same device.
It supports more than 1000 basic requests wich does not requre connection to database.
Disabling support for multi-referenced SOAP option by the WITH_NOIDREF define helped to decrease serialization time about 4 times faster on big requests with large number of serialization nodes.