I'm using scipy.optimize.minimize with the Newton-CG (Newton Conjugate Gradient) method since I have an objective function for which I know the analytical Jacobian and Hessian. However, I need to add a regularization term R=exp(max(s)) based on the maximum value inside the array parameter "s" that being fit. It isn't entirely obvious to me how to implement derivatives for R. Letting the minimization algorithm do numeric derivatives for the whole objective function isn't an option, by the way, because it is far too complex. Any thoughts, oh wise people of the web?
I need to take derivatives in Matlab of a lot of equations w.r.t. generic functions, which will provide me with generic derivatives, of the type:
diff(f(x,y),x)
or
D([1],f(x,y)).
What I need is to transform these derivatives into actual symbolic variables, in order to be able to use solve, etc. What I am doing now, but which is highly inefficient, is brute force string replacement. Here is a minimal working example of what I am doing:
syms x y
f(x,y) = sym('f(x,y)')
jacobian(f)
first_d = jacobian(f)
strrep(char(first_d),'D([1], f)(x, y)','fx')
In my real application, I have lots of derivatives to take from lots of equations, so looping such replacements is not the smartest thing to do. Can anybody shed some light into a more efficient solution?
Note: I'm using R2014b. Symbolic Math functionality has changed greatly in recent versions and continues to do so. Users on different versions may need to do slightly different things to achieve the results below, which relies on accessing undocumented functionality.
First, since this is about performance, it is sufficient to simply declare
syms f(x,y)
which also defines x and y as symbolic variables.
As I mention in my comments above, Matlab/MuPAD's symbolic math is all about manipulating strings. Doing this more directly and and adding in you own knowledge of the problem can help speed things up. You want to avoid unnecessary conversions between strings and the sym/symfun types.
1. The first thing to do is investigate how a particular symbolic math function is handling input and output and what lower level private functions it is calling. In the case of your jacobian function example, type edit jacobian in your command window to view the code in the Editor. Much of what you see may be confusing, but you should see this line:
res = mupadmex('symobj::jacobian',Fsym.s,v.s);
This calls the low level 'symobj::jacobian' function and passes in string versions of the function and variables. To call this yourself, you can do (this also assumes you know your variables are x and y):
syms f(x,y)
first_d = mupadmex('symobj::jacobian',char(f),char([x,y]))
This returns [ diff(f(x, y), x), diff(f(x, y), y)]. The undocumented mupadmex function is a direct way of calling MuPAD function from within Matlab – there are others, which are documented.
2. You'll notice that that the first_d output above is symfun class. We actually don't want want the output to be converted back to a symbolic function. To avoid this, we can pass an addition argument to mupadmex:
syms f(x,y)
first_d = mupadmex('symobj::jacobian',char(f),char([x,y]),0)
with now returns the string matrix([[diff(f(x, y), x), diff(f(x, y), y)]]). (I only know this trick of adding the additional 0 argument from having browsed through a lot of Symbolic Math toolbox code.)
3. From this string, we can now find and replace various patterns for partial derivatives with simple variables. The strrep function that you're using is generally a good choice for this. It is much faster than regexprep. However, if you have a large number of different, but similar, patterns to replace, you might do a performance comparison between the two. That would probably be the subject of a separate question.
I'm not sure what your overall goal is or the full extent of your problem, but here is my final code for your example:
syms f(x,y)
first_d = mupadmex('symobj::jacobian',char(f),char([x,y]),0)
first_d = strrep(first_d(9:end-2),'diff(f(x, y), x)','fx');
first_d = sym(strrep(first_d,'diff(f(x, y), y)','fy'));
This returns the symbolic vector [ fx, fy]. If you want a symfun, you'll need to modify the last line slightly. In some simple testing, this basic example is about 10% faster than calling jacobian and converting the result back to a string. If you directly specify the inputs as strings instead of allocating a symbolic function, the result is about 30% faster then your original:
first_d = mupadmex('symobj::jacobian','f(x,y)','[x,y]',0)
first_d = strrep(first_d(9:end-2),'diff(f(x, y), x)','fx');
first_d = sym(strrep(first_d,'diff(f(x, y), y)','fy'));
Using subs, as in this answer, while convenient, is the slowest approach. Converting back and forth to and from strings is costly.
I am using the Opti Toolbox, a free optimization toolbox for Matlab. I am solving a Mixed Integer Nonlinear Program, a MINLP. Inside the Opti Toolbox, the MINLP solver used is SCIP.
I define my own objective as a separate function (fun argument in Opti), and this function needs to call other matlab functions which take double arguments.
The problem is that whenever Opti invokes my function to evaluate the objective, it first calls it using a vector of 'scipvar' objects and then it calls it again using a vector of 'double' objects. My obj function does not work with the scipvar objects, it returns an error.
I tried (just for testing) setting the output of my function for something fixed when the type is 'scipvar', and for the actual real thing when the type is 'double', and this doesn't work, changing the fixed value actually changes the final optimal value.
I basically need to convert a scipvar object to double, is this possible? Or is there any other alternative?
Thank you.
Ok, so after enlightenment by J. Currie, an Opti toolbox developer, I understood the cause of the problem above.
The first call to the objective with a vector of scipvar variables is actually a parser sweeping the objective function to see if it can be properly mapped to something that can be handled by SCIP. I reimplemented the objective function to use only methods allowed by scip - obtained by typing methods(scipvar) in matlab:
abs dot log minus mrdivide norm power rdivide sqrt times
display exp log10 mpower mtimes plus prod scipvar sum uminus
Once the objective could be parsed by scip my problem worked fine.
I want to differentiate the following function wrt in MATLAB:
T(e(x(t),t)⁄p(t))
My problem is that I know the derivatives of x numerically (I am inside a kind of odefun).
I want to use diff to make my code generalizeable for high order derivatives,but the derivatives of x are now constant. I would also like all this to be in an anonymous function where I can make the differentiation and substitute accordingly the time and the derivative of x needed,so that I don't have to write multiple functions for every state of my system.
My code is as follows:
syms q x star;
qd=symfun(90*pi/180+30*pi/180*cos(q),[q]);
p=symfun(79*pi/180*exp(-1.25*q)+pi/180,[q]);
T=log(-(1+star)/star);
e=symfun(x-qd,[x,q]);
and I want to write for example a function in the form
#(t,y)(d^2⁄dt^2 T(e(x(t),t)⁄p(t))+d⁄dt T(e(x(t),t)⁄p(t))+T(e(x(t),t)⁄p(t)))
I am not sure of the implementation details but in general this is one approach that you could take. It involves two steps.
in the T(.) function replace x with exp(t) this way when you do the differentiation exp(t) always stays there for the higher order derivatives to be taken and the outer functions will be differentiated with respect to x at the same time. After you do diff you should receive an expression that contains exp(t) (not tested so hopefully it is the case). At this point exp(t) is your time derivative of x. Now you only need to evaluate this expression in t. When doing so you need to replace exp(t) by the derivative of x. I do not know if this can be done, if not then perhaps using y instead of exp(t) with the constraint y=exp(t) would do it but you need to figure the correct implementation out yourself.
Here you need to substitute the derivative of x at the right t. If you do not have the value at the particular point t then do what I suggested in the comment. Pre-calculate it beforehand in many points and interpolate in this step.
This approach relies on swapping x(t) with exp(t) if that does not work then I would do what I suggested in the comment. Approximate x(t) by a known function and use that instead of x in your code.
I am attempting to use InteriorPointSolver to solve a standard Quadratic Programming problem with linear constraints (per the definition that can be found here). My problem has no linear term (the "c" vector in the definition). I am setting up the "Q" matrix by using SetCoefficient(Int32, Rational, Int32, Int32) across all my variables (passing the "goal" row as the vidRow). Am I correct in assuming that the InteriorPointSolver is minimizing the objective function as defined in the standard definition of the quadratic programming problem?
I ask this because when I calculate x^T * Q * x myself (using the optimal solution for x that I get from the solver), I get a value that is substantially different than what the solver claims the optimal objective function value is (via Statistics.Primal or GetValue(goal)). The only time my calculation and the solver's optimal value agree is when I use an identity matrix for Q. I am guessing that I am setting something up wrong or am not understanding exactly what function is being minimized.
I have consulted all the documentation I can find and cannot find a good explanation of exactly what function the interior point solver is minimizing. Can anyone guide me in the right direction?
As it turns out,
SetCoefficient(goal, 2.0, x, y)
Has exactly the same effect as
SetCoefficient(goal, 2.0, y, x)
The effect of both calls is to set the coefficient of the x*y term in your objective function, and the second call simply overwrites the coefficient that you set in the first call. The solver does not treat the xy term as distinct from the yx term, and does not add the coefficients (as I had expected). So, if your goal is to have a 4xy term in your objective function, you must make the following call:
SetCoefficient(goal, 4.0, x, y)
instead of the two calls listed above.