I want to optimize a multi-variable function with the patternsearch function in MATLAB. The function requires a lower and upper boundary and looks within the boundaries in a continuous domain.
I however have a discrete set of values in an excel file and would like the algorithm to search within this discrete domain instead of in the continuous domain.
Is this possible with patternsearch?
Maybe I don't understand correctly your question but if you have a (discret and finite) set of values, why don't you compute the function's value at these points and return the minium?
In short, no. That is not what patternsearch is intended for. Optimization techniques for discrete and continuous search spaces are quite expectedly different.
If you're looking for an approximate answer however, it is possible to use spline, polyfit, etc. to arrive at an approximate continuous function for your data and then apply patternsearch on it.
If you provide greater detail about your problem, I or someone else may be able to suggest a more suitable way of working with your data.
The best optimization tool for this is the Genetic Algorithm. This optimization tool comes with Matlab's global optimization toolbox and allows for optimization of both continuous and discrete variables at the same time.
In the genetic algorithm variables that are integers have to be declared as such. Non-declared variables are continuous by default.
Check the Global Optimization Toolbox guide for information on how it works: http://it.mathworks.com/help/pdf_doc/gads/gads_tb.pdf.
Related
I am performing a numerical optimization where I try to find the parameters of a statistical model that best match certain moments of the data. I have 6 parameters in total I need to find. I have written a matlab function which takes the parameters as input and gives the sum of squared deviations from the empirical moments as output. I use the fminsearch function to find the parameters and it gives me a solution.
However, I am unsure if this is really a global minimum. What type of checks I could do to ensure the numerical solution is correct? Plotting the function is challenging due to high dimensionality. Any general advice in solving this type of problem is also appreciated.
You are describing the difficulties of a global optimization problem.
As mentioned in one of the comments, fminsearch() and related function fminunc() will return a local minimum. It provides no guarantee that you will get a global minimum.
A simple way to check if the answer you get really is a global minimum, would be to run the function multiple times from various starting points. If the answer all converges to the same value, it might be a global minimum. If you find an answer with lower error values, then the last answer was not the global minimum.
The only way to be perfectly sure that you have the global minima, is to know whether or not your function is convex (i.e. your function has only a single minima.) This will have to be done analytically.
If it is not possible to be done analytically, there are many global optimization methods you may want to consider, including some available as this MATLAB toolbox.
I was wondering if there exists a technical way to choose initial parameters to these kind of problems (as they can take virtually any form). My question arises from the fact that my solution depends a little on initial parameters (as usual). My fit consists of 10 parameters and approximately 5120 data points (x,y,z) and has non linear constraints. I have been doing this by brute force, that is, trying parameters randomly and trying to observe a pattern but it led me nowhere.
I also have tried using MATLAB's Genetic Algorithm (to find a global optimum) but with no success as it seems my function has a ton of local minima.
For the purpose of my problem, I need justfy in some manner the reasons behind choosing initial parameters.
Without any insight on the model and likely values of the parameters, the search space is too large for anything feasible. Think that just trying ten values for every parameter corresponds to ten billion combinations.
There is no magical black box.
You can try Bayesian Optimization to find a global optimum for expensive black box functions. Matlab describes it's implementation [bayesopt][2] as
Select optimal machine learning hyperparameters using Bayesian optimization
but you can use it to optimize any function. Bayesian Optimization works by updating a prior belief over a distribution of functions with the observed data.
To speed up the optimization I would recommend adding your existing data via the InitialX and InitialObjective input arguments.
I have a function fun(x,y,z), such that say, x=1:10, y=50:60, z=100:105. Which optimization method (and how) I can use to get the minimum of this function, for example, where (x,y,z)=(3,52,101). I am working in Matlab.
Thank you for any help
Algorithms
There are many many algorithms out there that you can use for direct search optimization such as Nelder-Mead, Particle Swarm, Genetic Algorithm, etc.
I believe Nelder-Mead is a simplex optimization method which is used by fminsearch function in MATLAB.
Also, there is Genetic Algorithm which comes with MATLAB Global Optimization toolbox. You may want to give that a try as well.
Particle Swarm Optimization (PSO) is another direct search method that you can use. However, there is no official toolbox for Particle Swarm method built by Mathworks. The good news is there is quite a few PSO toolbox developed by other people. I personally have used this one and am quite happy with the performance. Its syntax is similar to Genetic Algorithm functions that come with Global Optimization Toolbox.
Discrete Optimization
Regarding your question that you are looking for a set of integer values namely x,y, and z corresponding to the minimum objective function value, I would add a part at the beginning of the objective function that rounds the variables to the closest integers and then feeds them to your main function fun(x,y,z). This way you would discretize your function space.
I hope my answer helps.
I'm using the genetic algorithm from the MATLAB Global Optimization Toolbox with SimEvents, in order to implement a mixed integer optimization making use of simulation outputs to evaluate the fitness function. My model is pretty similar to the one described in this video from MathWorks website:
http://www.mathworks.it/videos/optimizing-manufacturing-production-processes-68961.html
Reading the documentation, I found that ga can solve constrained problems only if such constraints are linear inequalities. The constraints are supposed to be written as functions of the problem's variables, that in this case are the number of resources used during the simulation.
I would like, instead, to set a constraint that takes into account another simulation output (e.g. the drain utilization), i.e. minimize
objfun = backlog*10000 + cost
where backlog is a simulation output (obtained using simOut.get), considering the following constraint:
drain_utilization > 0.7
where drain_ utilization is another simulation output (again, obtained using simOut.get).
Is it possible or this feature is not supported by the Global Optimization Toolbox?
Thank you in advance and forgive me for any improper term, but I'm new to the Global Optimization Toolbox.
I have some data for which I have a set of numerically determined model curves. Now I would like to find the one with least square deviation, I only need to vary one parameter, which is the amplitude of these model curves.
I used fitting with analytic functions, but I did not find a way to handle such a problem.
Is there any solution?
Thanks a lot!
One of the optimize functions should do the trick. You can also read the section on optimization in the manual. Without any specifics on the data or the model you wish to match, it's hard to recommend anything more specific. For example, if your cost function has many maxima and minima or is not differentiable, you'll have to choose some of the more expensive routines.