I'm planning to carry out a curve fitting task using genetic algorithms. For this purpose, I'm looking for an out of the box tool in python. Can you recommend such libraries? So far, I've come across scipy's optimize.differential_evolution. It looks promising, but before I dive into its specifics, I'd like to get a good sense of what other methods are out there, if any.
Thanks
Related
I have been recently trying to use svm for feature classification. While i was doing so, a question came to my mind.
Which would be a better method to use, LIBSVM or svmclassify? What I mean by svmclassify is to use in-built functions in MATLAB such as svmtrain and svmclassify. In that sense, I was interested to know which method would be more accurate and which would be easier to use.
Since MATLAB has already the Bioinformatics toolbox already, why would you use LIBSVM? Aren't the functions like svmtrain and svmclassify already built in.. what additional benefits does LIBSVM bring about?
I would like to hear some of your opinions. Please Pardon me if the question is stupid..
I expect you would get very similar result using each library.
They are both very easy to use. The only big difference is that one comes with the MATLAB Bioinformatics toolbox and the other one you need to obtain from the authors web site and install by hand. If to you this is an issue I would recommend you stick to what is already installed in your computer. If not consider using LIBSVM, as it is a very well tested and well regarded library.
Also, from personal experience on playing with both, libSVM is much faster than MATLAB svm routines for obvious reasons. Last but not the least, libSVM has MATLAB plugins which can be called from MATLAB if you are more comfortable within a MATLAB environment.
I have also the same question, but I think that Libsvm is very useful and very easy in the case of multi-classes classification , but the matlab toolbox is designed for only two classes classification.
In my experience the libsvm performed giving cross validaion results as 45% where matlab code did 90%. So I looked up the explanation of matlab function for svm where they had such options related with perceptrones, I wonder if they are using pure svm or not but will write again in my case matlab was much better. (multiclass svm)
I'm doing some numerical estimation and correction with the Kalman filter, and would like to better estimate my parameters of Q and R, preferably dynamically.
http://en.wikipedia.org/wiki/Kalman_filter#Estimation_of_the_noise_covariances_Qk_and_Rk
That article mentions that GNU Octave is currently the best way of determining these parameters from data:
http://en.wikipedia.org/wiki/GNU_Octave#C.2B.2B_integration
Unfortunately it is written for Matlab, and there's supposedly a C++ implementation. I'm very weak in C++ and would not even know how to import a C++ library and link it properly in XCode. All of my C++ libraries to date have been wrapped in 3rd party Objective-C classes.
Has anyone used the C++ implementation for scientific computing or engineering applications on iPhone? I'd appreciate any pointers or tutorials on how to do this kind of analysis with Objective-C.
Additional keywords:
estimating covariance from data
Autocovariance Least-Squares (ALS) technique
noise covariance
Thank you!
I do not know of any such C++ library, if you fancy doing numerical analysis on iOS, the best way to go is the accelerate framework, specifically (from this description):
Linear Algebra: LAPACK and BLAS
The Basic Linear Algebra Subprograms (BLAS) and Linear Algebra Package
(LAPACK) libraries contain—as you would expect—functions to perform
linear algebra computations such as solving simultaneous linear
equations, least squares solutions of linear equations, and eigenvalue
problems. The BLAS library serves as a building block for the LAPACK
library. The BLAS and LAPACK libraries are widely distributed and
industry standard computational libraries. They are available on a
number of different platforms and architectures. So, if you are
already using these libraries you should feel right at home, as the
APIs are exactly the same on Mac OS X.
You'll need a fairly good grounding in C, pointers, arrays and such though, no way around it I feel. There is a detailed description of how to use these linear algebra primitives to implement kalman filtering (although this is using R, so probably not of mush use to you).
This is a SO post on Kalman Filtering which expressed my opinion quite well. I'm afraid I think the chances of finding a magic Objective-C wrapper for Kalman Filtering are fairly low, though I would be very happy to be proven wrong!
I would like to learn how does the Wavelet transform works from a practical point of view. I have read the theory regarding it and I think that I have understood the main idea behind it, but I would like to have some practice with it.
Can you please recommend me some tutorial and some data which I can use for learning the Wavelet Transform by using Matlab environment?
I tried to search for audio signal or practical tutorial on which I can work on but I had no results.
The Mathworks site has some information on their wavelet toolbox and some simple examples of continuous 1D wavelet transforms and discrete 2D wavelet transforms.
Since you have studied and understood the theory behind wavelet transforms, the best way to learn is to go through the source code for various algorithms that have been used by others. For starters looking at the core of the various functions provided in the toolbox above (just enter type functionname at the command line in MATLAB. Unless if it's a built-in function, you'll see the file contents). By core of the function, I mean the main algorithm without all the various input checks that are common.
The Wavelab toolbox from Stanford university is also a good resource to learn from (and later use in your applications when you're comfortable with it).
Lastly, this is a resource I found by Googling and it looks like they have some examples that you can try out.
Do you know if anyone has tried to compile high level programming languages (java, c#, etc') into a recurrent neural network and then evolve them?
I mean that the whole process including memory usage is stored in a graph of a neural net, and I'm talking about complex programs (thinking about natural language processing problems).
When I say neural net I mean a directed weighted graphs that spreads activation, and the nodes are functions of their inputs (linear, sigmoid and multiplicative to keep it simple).
Furthermore, is that what people mean in genetic programming or is there a difference?
Neural networks are not particularly well suited for evolving programs; their strength tends to be in classification. If anyone has tried, I haven't heard about it (which considering I barely touch neural networks is not a surprise, but I am active in the general AI field at the moment).
The main reason why neural networks aren't useful for generating programs is that they basically represent a mathematical equation (numeric, rather than functional). Given some numeric input, you get a numeric output. It is difficult to interpret these in the context of a program any more complicated than simple arithmetic.
Genetic Programming traditionally uses Lisp, which is a pure functional language, and often programs are often shown as tree diagrams (which occasionally look similar to some neural network diagrams - is this the source of your confusion?). The programs are evolved by exchanging entire branches of a tree (a function and all its parameters) between programs or regenerating an entire branch randomly.
There are certainly a lot of good (and a lot of bad) references on both of these topics out there - I refrain from listing them because it isn't clear what you are actually interested in. Wikipedia covers each of these techniques, and is a good starting point.
Genetic programming is very different from Neural networks. What you are suggesting is more along the lines of genetic programming - making small random changes to a program, possibly "breeding" successful programs. It is not easy, and I have my doubts that it can be done successfully across a large program.
You may have more luck extracting a small but critical part of your program, one which has a few particular "aspects" (such as parameter values) that you can try to evolve.
Google is your friend.
Some sophisticated anti-virus programs as well as sophisticated malware use formal grammar and genetic operators to evolve against each other using neural networks.
Here is an example paper on the topic: http://nexginrc.org/nexginrcAdmin/PublicationsFiles/raid09-sadia.pdf
Sources: A class on Artificial Intelligence I took a couple years ago.
With regards to your main question, no one has ever tried that on programming languages to the best of my knowledge, but there is some research in the field of evolutionary computation that could be compared to something like that (but it's obviously a far-fetched comparison). As a matter of possible interest, I asked a similar question about sel-improving compilers a while ago.
For a difference between genetic algorithms and genetic programming, have a look at this question.
Neural networks have nothing to do with genetic algorithms or genetic programming, but you can obviously use either to evolve neural nets (as any other thing for that matters).
You could have look at genetic-programming.org where they claim that they have found some near human competitive results produced by genetic programming.
I have not heard of self-evolving and self-imrpvoing programs before. They may exist as special research tools like genetic-programming.org have but nothing solid for generic use. And even if they exist they are very limited to special purpose operations like malware detection as Alain mentioned.
I am writing programs that are based on robots navigating through mazes (would involve stochastic programming).
Since it will involve heavy matrix handling (plus point for MATLAB) and simulating a robot (plus point for Prolog), I am in a dilemma between the choice of MATLAB and Prolog.
Note: I do have MATLAB at my work environment, hence cost is not an issue.
As mentioned previously, I am not sure if you are looking for comparisons between MATLAB and Python or MATLAB and Prolog. I can speak to the former, at least: MATLAB provides fast linear algebraic computation and a great IDE... and that's about it. Python will cost you much fewer headaches (and dollars), and you can manage "heavy matrix handling" nearly as easily if you tack on Numpy in particular, or SciPy in general.
Also, VPython (Visual Python) is a great 3D visualization tool that uses Numpy under the hood. I developed a robot simulator using VPython; you can see screenshots and example code (for simple wall-following maze navigation) that you can check out in a recent blog post.