I am trying to implement bayesian optimization using gauss process regression, and I want to try the multiple output GP firstly.
There are many softwares that implemented GP, like the fitrgp function in MATLAB and the ooDACE toolbox.
But I didn't find any available softwares that implementd the so called multiple output GP, that is, the Gauss Process Model that predict vector valued functions.
So, Are there any softwares that implemented the multiple output gauss process that I can use directly?
I am not sure my answer will help you as you seem to search matlab libraries.
However, you can do co-kriging in R with gstat. See http://www.css.cornell.edu/faculty/dgr2/teach/R/R_ck.pdf or https://github.com/cran/gstat/blob/master/demo/cokriging.R for more details about usage.
The lack of tools to do cokriging is partly due to the relative difficulty to use it. You need more assumptions than for simple kriging: in particular, modelling the dependence between in of the cokriged outputs via a cross-covariance function (https://stsda.kaust.edu.sa/Documents/2012.AGS.JASA.pdf). The covariance matrix is much bigger and you still need to make sure that it is positive definite, which can become quite hard depending on your covariance functions...
Related
I am trying to fit a custom distribution to a large (~O(500,000) measurements) dataset using scipy. I have derived a theoretical PDF based on some other factors, but both by hand and using symbolic integration software I cannot find an exact form of the CDF.
Currently, simply evaluating 1000 random samples from my custom distribution is expensive, which I believe is due to the need to invert an unknown CDF. If I cannot find an explicit form of the CDF and it's inverse, is there anything else I can do to speed up usage of this distribution?
I've used maple, matlab and Sympy to try and determine a CDF, yet none give a result. I also tried down-sampling my data whilst still retaining the tail attributes, but this still required so much data that doing anything with the distribution was slow.
My distribution is a sub-class of SciPy's rv_continuous class.
Thanks for any advice.
This sounds like you want to sample from a Kernel Density Estimation of the probability distribution. While Scipy does offer a Gaussian Kernel package, for that many measurements you would be much better off using sklearn's implementation. A good resource with code examples can be found on Jake VanderPlas's blog.
I have a probability distribution of some datasets. I have drawn this from the datasets in matlab. The question is that I want to generate similar data with these. I want to get the approximate function in this graph so that I can use it to generate new numbers. Is there a way to make this? either getting the nearest function of that plot OR some other suggestion to make this is appreciated.
EDIT: I have added the plot
A more general question that may be little more useful can be that
Given that there are some random variables I need an algorithm, software, technique to get similar random variable numbers.
I can understand that artificial intelligence techniques can be useful in this but I am wondering if there is a simple question. Possibly (my ai knowledge is not that good) some neural network or markov chain model can make this job but I believe that could be done easier. The function that I want is probability distribution function. I have 3 datasets and they are consistent with each other. They have very similar distributions.
I have a function fun(x,y,z), such that say, x=1:10, y=50:60, z=100:105. Which optimization method (and how) I can use to get the minimum of this function, for example, where (x,y,z)=(3,52,101). I am working in Matlab.
Thank you for any help
Algorithms
There are many many algorithms out there that you can use for direct search optimization such as Nelder-Mead, Particle Swarm, Genetic Algorithm, etc.
I believe Nelder-Mead is a simplex optimization method which is used by fminsearch function in MATLAB.
Also, there is Genetic Algorithm which comes with MATLAB Global Optimization toolbox. You may want to give that a try as well.
Particle Swarm Optimization (PSO) is another direct search method that you can use. However, there is no official toolbox for Particle Swarm method built by Mathworks. The good news is there is quite a few PSO toolbox developed by other people. I personally have used this one and am quite happy with the performance. Its syntax is similar to Genetic Algorithm functions that come with Global Optimization Toolbox.
Discrete Optimization
Regarding your question that you are looking for a set of integer values namely x,y, and z corresponding to the minimum objective function value, I would add a part at the beginning of the objective function that rounds the variables to the closest integers and then feeds them to your main function fun(x,y,z). This way you would discretize your function space.
I hope my answer helps.
I am trying to learn how to use support vector machines in matlab. I have the bioinformatics toolbox, which has SVM functions svmtrain and svmclassify.
I managed to successfully use it for some reference data sets, with some nice accuracy. When I try to use the svm on my actual data the training never stops. My data set is 400 instances in 25 dimensions, so it should not take very long?!
Can I use other solvers in matlab? I dont want to buy new toolbox please ...
There are several things that may cause problems for training, but it should not run infinitely. Do you get any errors when using the solver?
With regard to alternatives: LIBSVM has an interface to matlab. This is a state-of-the-art library with thousands of users. I highly recommend it, because it is easy to install/use and offers additional functionality for parameter tuning and more.
I need to construct an interpolating function from a 2D array of data. The reason I need something that returns an actual function is, that I need to be able to evaluate the function as part of an expression that I need to numerically integrate.
For that reason, "interp2" doesn't cut it: it does not return a function.
I could use "TriScatteredInterp", but that's heavy-weight: my grid is equally spaced (and big); so I don't need the delaunay triangularisation.
Are there any alternatives?
(Apologies for the 'late' answer, but I have some suggestions that might help others if the existing answer doesn't help them)
It's not clear from your question how accurate the resulting function needs to be (or how big, 'big' is), but one approach that you could adopt is to regress the data points that you have using a least-squares or Kalman filter-based method. You'd need to do this with a number of candidate function forms and then choose the one that is 'best', for example by using an measure such as MAE or MSE.
Of course this requires some idea of what the form underlying function could be, but your question isn't clear as to whether you have this kind of information.
Another approach that could work (and requires no knowledge of what the underlying function might be) is the use of the fuzzy transform (F-transform) to generate line segments that provide local approximations to the surface.
The method for this would be:
Define a 2D universe that includes the x and y domains of your input data
Create a 2D fuzzy partition of this universe - chosing partition sizes that give the accuracy you require
Apply the discrete F-transform using your input data to generate fuzzy data points in a 3D fuzzy space
Pass the inverse F-transform as a function handle (along with the fuzzy data points) to your integration function
If you're not familiar with the F-transform then I posted a blog a while ago about how the F-transform can be used as a universal approximator in a 1D case: http://iainism-blogism.blogspot.co.uk/2012/01/fuzzy-wuzzy-was.html
To see the mathematics behind the method and extend it to a multidimensional case then the University of Ostravia has published a PhD thesis that explains its application to various engineering problems and also provides an example of how it is constructed for the case of a 2D universe: http://irafm.osu.cz/f/PhD_theses/Stepnicka.pdf
If you want a function handle, why not define f=#(xi,yi)interp2(X,Y,Z,xi,yi) ?
It might be a little slow, but I think it should work.
If I understand you correctly, you want to perform a surface/line integral of 2-D data. There are ways to do it but maybe not the way you want it. I had the exact same problem and it's annoying! The only way I solved it was using the Surface Fitting Tool (sftool) to create a surface then integrating it.
After you create your fit using the tool (it has a GUI as well), it will generate an sftool object which you can then integrate in (2-D) using quad2d
I also tried your method of using interp2 and got the results (which were similar to the sfobject) but I had no idea how to do a numerical integration (line/surface) with the data. Creating thesfobject and then integrating it was much faster.
It was the first time I do something like this so I confirmed it using a numerically evaluated line integral. According to Stoke's theorem, the surface integral and the line integral should be the same and it did turn out to be the same.
I asked this question in the mathematics stackexchange, wanted to do a line integral of 2-d data, ended up doing a surface integral and then confirming the answer using a line integral!