I remember reading once in the Matlab documentation about an optimisation algorithm which allowed the user to specify the "scale" of variation expected for each parameter during the search (at least initially).
I can't remember what this function is, but now I am using fminsearch and there is no such option. In fact, I can't even specify parameter bounds, and the documentation states that it takes 5% of the initial guess as a default step (or 25e-5 if 0). Because this seems to be a relative choice to the initial guess, it makes me think that perhaps I should re-normalise my parameters to a suitable scale, in order to indirectly define a suitable step for my optimisation problem.
For example, if I have a parameter which value is on the order of 10e5 but that I would like steps on the order of 100, then I should "divide it" by 500 during optimisation (obviously I would then multiply it when computing the objective function). However this becomes trickier if a parameter range is centred around 0 for example; then I can rescale it and offset it.
My question is; is it effectively what people usually do when using the downhill-simplex method, and is there a "standard" or "better" way to do it?
Related
I am writing a matlab code where i calculate the max-min.
I am using matlab's "fminimax" to solve the following problem:
ki=G(i,:);
ki(i)=0;
fs(i)=-((G(i,i)*pt(i)+sum(ki.*pt)+C1)-(C2*(sum(ki.*pt)+C1)));
G: is a system matrix. pt: is the optimization variable.
When the actual system matrix is used, the "fminimax" stops after one iteration and returns the initial value of "pt", no matter what the initial value for "pt", i.e. no solution is found. (the initial value is defined as X0 in the documentation). The system has the following parameters: G is in the order of e-11, pt is in the order of e-1, and c1 is in the order of e-14.
when i try a randomly generated test matrix and different parameters, the "fminimax" finds a solution for the problem, and everything works fine. G in order of e-2, pt in order of e-2, c1 is in the order of e-7.
I tried to scale the actual system: "fminimax" lasted more than one iteration, however, it still returned the initial value of pt, i.e. it couldn't find a solution.
I tried to change the tolerance of the "fminmax", using "options" [StepTolerance, OptimalityTolerance, ConstraintTolerance, and functiontolerance]. There were no impact at all. still no solution.
I thought that the problem might be that the precision of "fminimax" is not that high, or it is not suitable to solve the problem. i think it is also slow.
i downloaded CPLX, and i wanted to transform the max-min problem into linear programing, using a method i found in a book. However, when i tried my code on a simple minimax it didn't give the same solution.
I thought of using CVX for example, but the problem is not convex.
What might be the problem?
P.S. the system matrix, G, has different realizations, i tried some of them. However, the "fminimax" responds in the same way for all of them, i.e. it wasn't able to find an adequate solution.
I am not convinced that the optimization solvers are broken. If the problem is nonconvex, then there can be multiple local minimizers. Given the information you have provided, we have no way of knowing whether you started at an initial condition.
The first place you need to start is by getting more information from the optimization exit condition... Did it finish because it hit the iteration limit? (I hope not since it isn't doing many iterations)... Did it finish because a tolerance was hit (e.g. the function did not change by more than xxxx)? Or perhaps it could not find a feasible solution? (I don't know if you have any constraints that need to be met).
More than likely, I wold guess that you are starting at a local minimizer without realizing it. So you need to determine whether you are indeed at a local minimizer by looking at the jacobian of the function evaluated at your initial guess. Either calculate it analytically or use a finite step approximation....
My initial problem is that I have a continuous transfer function which coefficients change with time.
Currently the TF's coefficients are expressed in function of the block mask parameters. These parameters are tunable, and if I change the value in the mask parameters dialog during a simulation the response seems to react appropriately.
However how can I do just that in the code/block flow? Basically, I
have the block parameter 'maskParam' which is set using the mask
parameters dialog, and in the mask initialization commands:
'param=maskParam'. 'param' is used in the transfer function and I
would like to change it in real time (as param=maskParam*f(t)).
I have already looked around and found relevant solutions but either it's unbelievably complicated; or the only transfer function which we are allowed to modify at runtime is discrete and 1) I would like to avoid z-transforming my quite complex TF (I don't have the control toolbox) 2) The sampling time seems to be fixed.. None uses this "dirty" technique of updating parameters, maybe that's the way around?
To illustrate:
I am assuming that you want to change your sim parameters whilst the simulation is running?
A solution is that you run your simulation for inf period and use/change a workspace variable during the simulation period to make the changes take effect.
for Example:
If you look at the w block, you can set it's value in runtime, by doing this:
set_param('my_model_name/w', 'value', 100); % Will change to 100 immediately
You can do similar things with arrays (i.e. a list of coefficients in your case).
HINT FOR YOU
You are using discrete transfer function block. Try the following:
1) Give your block a name e.g. fcn_1
2) In your script, type set_param('your_model_name/fcn_1', 'numerator', '[1 2]'); This will set the numerator value to [1 2]. Do the same for denominator.
3) You should be able to understand, through this exercise, how to handle the property names etc. so that you can change/get them using set_param/get_param.
I leave you to investigate further.
The short answer is that Simulink blocks are not really designed to do this. By definition, a transfer function is Liner-Time Invariant, meaning its characteristics (read coefficients) do not vary with time.
Having said that, there are some workarounds, such as the ones you mentioned in your question. These are the correct way to approach the problem I'm afraid, other than the set_param method suggested by #ha9u63ar. See also this blog on the subject on the MathWorks web site.
I need to implement an anti-windup (output limitation) for my PID controller. Simulink is offering two options: back calculation and clamping (documentation) which seem to deliver equal results. I know what back calculation is doing mathematically. It requires to define the back-calculation gain Kb. This gain is dependent on how long my controller is saturated, therefore it is actually a dynamic value (because I may have a high variation of saturation times). Do you see a way to control this value? (in this case it probably would be necessary to build my own PID Controller as shown in the documentation above or in the picture below.
Which brings me to the question, what is clamping actually doing? And what are other differences? Which one is faster, which one is more robust against stiff slopes? Does anybody has experiences using both?
Not sure if this fully answers the question, but the PID Controller documentation page, explains a bit more about clamping:
clamping
Stops integration when the sum of the block components
exceeds the output limits and the integrator output and block input
have the same sign. Resumes integration when the sum of the block
components exceeds the output limits and the integrator output and
block input have opposite sign. The integrator portion of the block
is:
The clamping circuit implements the logic necessary to determine whether integration continues.
If you select the clamping option and look under the mask, you can probably see the details of the clamping circuit.
Additionally to am304's answer there are some more things to consider.
Clamping
Clamping will always work. It detects when there is integrator overflow and sets the integral path of the PID-controller to zero to avoid windup by using a simple switch.
Clamping is a commmonly used anti windup method, especially in case of digital control systems. In serious applications however, there is also forward clamping involved - evaluating the controller input as well. This mechanism must me implemented manually.
Back Calculation
Back Calculation highly depends on the back calculation coefficient Kb. If you don't know how to actually calculate the parameter Kb don't use back-calculation. This method calculates the difference between the actual controller output and the saturated output and subtracts it from the I-Gain path, amplified by Kb.
In most of cases the default value Kb = 1 will lead to worse results than clamping, it is even possible that it has no effect at all. Kb should be calculated based on the sampling time or
in case a D-Gain is involded, based on D- and I-Gain. Appropriate literatur should be consulted to calculate the coefficient. Back calculation with a properly set coeffient enables better dynamics than clamping!
I used Matlab-fminsearch for a negativ max likelihood model for a binomial distributed function. I don't get any error notice, but the parameter which I want to estimate, take always the start value. Apparently, there is a mistake. I know that I ask a totally general question. But is it possible that anybody had the same mistake and know how to deal with it?
Thanks a lot,
#woodchips, thank you a lot. Step by step, I've tried to do what you advised me. First of all, I actually maximized (-log(likelihood)) and this is not the problem. I think I found out the problem but I still have some questions, if I don't bother you. I have a model(param) to maximize in paramstart=p1. This model is built for (-log(likelihood(F))) and my F is a vectorized function like F(t,Z,X,T,param,m2,m3,k,l). I have a data like (tdata,kdata,ldata),X,T are grids and Z is a function on this grid and (m1,m2,m3) are given parameters.When I want to see the value of F(tdata,Z,X,T,m1,m2,m3,kdata,ldata), I get a good output. But I think fminsearch accept that F(tdata,Z,X,T,p,m2,m3,kdata,ldata) like a constant and thatswhy I always have as estimated parameter the start value. I will be happy, if you have any advise to tweak that.
You have some options you can try to tweak. I'd start with algorithm.
When the function value practically doesn't change around your startpoint it's also problematic. Maybe switching to log-likelyhood helps.
I always use fminunc or fmincon. They allow also providing the Hessian (typically better than "estimated") or 'typical values' so the algorithm doesn't spend time in unfeasible regions.
It is virtually always true that you should NEVER maximize a likelihood function, but ALWAYS maximize the log of that function. Floating point issues will almost always corrupt the problem otherwise. That your optimization starts and stops at the same point is a good indicator this is the problem.
You may well need to dig a little deeper than the above, but even so, this next test is the test I recommend that all users of optimization tools do for every one of their problems, BEFORE they throw a function into an optimizer. Evaluate your objective for several points in the vicinity. Does it yield significantly different values? If not, then look to see why not. Are you creating a non-smooth objective to optimize, or a zero objective? I.e., zero to within the supplied tolerances?
If it does yield different values but still not converge, then make sure you know how to call the optimizer correctly. Yeah, right, like nobody has ever made this mistake before. This is actually a very common cause of failure of optimizers.
If it does yield good values that vary, and you ARE calling the optimizer correctly, then think if there are regions into which the optimizer is trying to diverge that yield garbage results. Is the objective generating complex or imaginary results?
If I'm getting a anonymous operator from a user I would like to test (extremely quickly) if the operator is linear. Is there a standard way to do this? I have no way of doing symbolic operators, or parsing the function. Is the only way trying some random functions (what random functions do I choose) and seeing if they satisfy linearity??
Thanks in advance.
Context:
User supplies a black box operator, that is a function which takes functions to functions.
I can give the operator a function and I get a function back. I want to determine if the operator is linear? Is there a standard fast method which gives me high confidence?
No, not without sweeping the entire parameter space. Imagine the following:
#(f) #(x) f(x) + (x == 1e6)
This operator is non-linear, but there's no way of knowing that unless you happen to test at x == 1e6.
UPDATE
As others have suggested, it may be "good enough" to determine a domain of interest, and check for linearity at periodic intervals across the domain. This may give you false positives (i.e. suggest an operator is linear when in fact it's non-linear), but never false negatives.
This is information the user should supply. Add a parameter linear true/false, default to false (I'm assuming that the code for non-linear will work for linear too, just taking more time).
The problem with random testing is that you will classify a non-linear function as linear sooner or later, and then the user has a problem, because your function unpredictably produces wrong results (depending on which points you pick randomly), that may be reasonably close to the correct results, ie people may not notice for a loooong time --> this is a recipe for disaster.
Really, the user should know this in the first place, its very important to avoid false positives and as said before there is no completely reliable way to test this. Save yourself the trouble and add an additional parameter.