Is there a way to use the Optaplanner benchmarker using Optapy? If not, will this be added? What do you suggest to do in order to facilitate systematic benchmarking of Optapy solvers in the meantime?
Currently there is no OptaPy Benchmarker yet.
It will likely be added in the future as OptaPy grows, but there are more priority issues in OptaPy, such as full features support, more quickstarts and bringing the performance closer to OptaPlanner's performance.
Related
I have a performance problem on a server that contains the Alfresco document manager. From one day to another, the use of the CPU is 95% fixed, with the command that Alfresco launches to execute. I have tried to change the garbage collector, change the JVM, optimize the threads, but I do not get any improvement.
The alfresco version is the Community
Has anyone had a similar problem?
This question is very hard to answer with the little information that is provided. CPU problems could be caused by a bad transformation, but it can also be caused by several other factors. Luis Colorado did a nice presentation on performance at DevCon 2019. You can check out his presentation for some tips.
A general piece of advice is to make sure you have Alfresco's Tomcat, SOLR, and the database all running on different machines. Those all have very different performance profiles, so keeping them separate allows you to tune them individually.
thanks you very much in advance.
First of all, I conceive scalability as the ability to design a system that doest not change when the demand of its services, whatever they are, increases considerably. May you need more hardware (vertically or horizontally0? Fine, add it at your leisure because the system is prepared and has been designed to cope with it.
My question is simple to ask but presumably very complex to answer. I would like to know what you I look at in a framework to make sure it will scale accordingly, both in number of hits and number of sessions running simultaneously.
This question is not about technology nor a particular framework at all, it is more a theoretical question.
I know that depend very much on having a good database design and a proper hardware behind with replication, etc... Let's assume that this all exists, however yet my framework must meet some criteria, what?
Provide a memcache?
Ability to run across multiple machines (at the web server level) and use many replicated databases? But what is in the software that makes that possible?
etc...
Please, let's not relate the answers with any particular programming language or technology behind.
Thanks again,
D.
I think scalability depends most of all on the use case: do you expect huge amounts of data, then you should focus on the database, if it's about traffic, focus on the server, is it about adding new features, focus on your data-model and the framework you are using...
Comparing a microposts-service like Twitter to a university website or a webservice like GoogleDocs you will find quite different requirements.
First of all the common notion of scalability is the ability of a software to improve in throughput or capacity if more hardware resources are added (CPUs, memory, bandwidth etc).
Software that does not improve in increased resources is not scalable.
Getting out of the definitions, I think your question is related to evaluation of frameworks you are planning to introduce to your implementation that may affect your software's ability to scale.
IMHO the most important factor to evaluate when introducing a framework is to see if there is hidden serialization in it (that serialization in effects transfers to/affects your software)
So if you introduce a framework that introduces serialization in your application that can affect your ability to scale.
How to evaluate?
Careful source code inspection (if open source)
Are there any performance guarantees offered by those that build the
framework?
Do measurements yourself to see how introducing this framework
affects your performance and replace if not satisfied
I am building an enhancement to the Spark framework (http://www.spark-project.org/). Spark is a project out of UC Berkeley that does MapReduce quickly in RAM. Spark is built in Scala.
The enhancement I'm building allows some data to be shared between the mappers while they are computing. This can be useful, for example, if each of the mappers is looking for an optimal solution, and they all want to share the current best solution (to prune out bad solutions early). The solution may be slightly out of date as it propagates, but this should still speed up the solution. In general, this is called the branch-and-bound approach.
We can share monotonically increasing numbers, but also we can share arrays, and dictionaries.
We are also looking at machine learning applications where the mappers describe local natural gradient information, and then a new best current optimal solution is shared among all nodes.
What are some other good real-world applications of this kind of enhancement? What kinds of real, useful applications might benefit from a Map Reduce computation with just a little bit of information-sharing between mappers. What applications use MapReduce or Hadoop right now but are just a little too slow because of the independence restriction of the Map phase?
The benefit can be to either speed up the map phase, or improve the solution.
The enhancement I'm building allows some data to be shared between the mappers while they are computing.
Apache Giraph is based on Google Pregel which is based on BSP and is used for graph processing. In BSP, there is data sharing between the processes in the communication phase.
Giraph depends on Hadoop for implementation. In general there is no communication between the mappers in MapReduce, but in Giraph the mappers communicate with each other during the communication phase of BSP.
You might be also interested in Apache Hama which implements BSP and can be used for more than graph processing.
There might be some reason why mappers don't communicate in the MR. Have you considered these factors in your enhancement?
What are some other good real-world applications of this kind of enhancement?
Graph processing is one thing I can think of, similar to Giraph. Checkout the different use cases for BSP, some might be applicable for this kind of enhancement. I am also very interested what other have to say on this.
What are your criteria for selection a (open source) library (or framework) for enterprise usage?
Some libraries are pretty small and can be easily checked for security flaws or tested for performance. But most libraries are too big to be reviewed before you can start to use them.
When I think of me selecting a library, most if the selection process is just gut feeling. When I try to be more specific, these are the first criteria which come to my mind:
How many developers are working on the project? My feeling is that more developers will find more bugs and security issues. In addition it will be harder to introduce security issues intentionally.
How good is the support? Compared to closed source libraries, I've got the feeling that the support of open source is often much better since you have a community around the globe which will be available whenever you need them.
How wide spread is the library? Are there any books about it on the market? Which other projects are using the library?
What are your criteria? Feel free to edit this note as community wiki.
For me, it depends on whether or not it is paid for or not. In your case, you give the impression you are looking at open source libraries.
In that specific case, I'll look at test coverage. Regardless of the number of contributors, if there aren't any unit tests that I can run myself (as well as enhance and test my use cases for if they fall outside the coverage of the unit tests provided), then that's a massive issue for me.
It's not that I don't appreciate the work that is done already in providing the library, but code in projects like this should have unit tests already with good coverage in order to gain traction.
If there are no libraries that have unit tests, then I would start searching for the library on search engines, actively seeking out negative replies. People who have negative feelings about the code and can crystalize the objective basis for those feelings in terms of how the code failed them will provide more valuable feedback than the masses that say "it works great".
Now for a commercial piece of code, it's completely different. At that point, I'd start looking at the company and it's support staff as a whole, and using that as a determination (as well as tests of your own to see if the library is right for you) as to whether or not to use that company's offering.
Quite often in open source libraries you cannot get reliable support. In such situations your best bet is to fix it yourself, which involves the following requirements.
You need to have the ability to read often messy and undocumented code.
The technical ability to ask the right questions from the right people -- i.e., these people aren't being paid to fix problems and they will only answer you if you make it easy enough for them.
Then you need the ability to fix the bug and get the patch accepted -- because if the patch isn't accepted .....
With this in mind I would be inclined to get a commercial library, or dual licensed library so that I could pay to get a competent engineer (motivated by the money I pay his company) to fix my problem.
QVT (Query View Transformation) is a OMG specification of a Model-to-Model transformation language. Some tools already implement it (Eclipse, androMDA). I'm wondering whether it is really used in real-world cases. Will it ever take off and be used to tackle real-world problems? Is anybody using the QVT language?
From observing the MDD community for our own projects, I'd guess that QVT will eventually pick up. Currently ATL and Kermeta seem to be very popular, and from looking at the postings in the groups not only in academia.
There's an implementation of Declarative QVT now (see the M2M Eclipse group for the announcement), that'll be very interesting for us. We've been using the ModelMorf prototype, but it was a prototype and had a very huge turnaround time. I hope that with the integration of dQVT into the Eclipse tool chain we'll be able to use it for our own projects (a SoftEng tool, see http://rcos.iist.unu.edu, sorry, academic of course :).
I guess the pain of doing Model-driven development by hand/with man-power is not high enough yet...once the tools really increase the order of magnitude of productivity, that'll change.
Seems like QVT is beeing used for Model Driven Security applications. It is a good choice because of clearly defined semantics and provability. This still is reasearch however. France Telecom is experimenting with QVT. They want to use it for database migrations and a generative approach for applications.
http://smartqvt.elibel.tm.fr/events/QVT%20Experimentations%20at%20France%20Telecom.pdf
http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4159881