I've got a porblem. I'm new in Octave need to solve these equations in this format:
-397.95 = min(k1*rate + k2);
776.37 = max(k1*rate + k2);
where rate is my row vector at size 10000. All I need is octave function which can deal with roots which are in other function (in my max and min). I know, that this question is a little bit mathematical, but I can't get the right easy function for solving this...
Thank you for answer
It looks like you need to use an optimization to minimise a cost function which would look like:
function y = f(x)
% k1 is x(1), k2 is x(2)
rate = ...
y = [min(x(1)*rate + x(2))+397.95; max(x(1)*rate+x(2))-776.37]
end
You can then use optimization functions such as fminsearch or other (from the optim package). The idea is to try and minimize your cost function, i.e. get y towards 0. It might be a good idea to use abs in your function to avoid issue with negative numbers.
It's easy to see that the constraints for this problem are:
k1 * rate + k2 >= -397.95
and
k1 * rate + k2 <= 776.37
Since larger values of k1 give a larger variance in the result of this equation, your objective is to maximize k1 subject to these constraints (equivalently minimize -k1).
You can now run this as a simple linear program:
height = size(rate,1);
c = [-1;0];
A = [rate',ones(height,1); rate',ones(height,1)];
b = [-397.95*ones(height,1); 766.37*ones(height,1)];
lb = [0;-Inf];
ub = [Inf; Inf];
ctype = [repmat("L",height,1); repmat("U",height,1)];
k = glpk (c,A,b,lb,ub,ctype)
k1 = k(1);
k2 = k(2);
EDIT : I missed that rate was a row vector. I've transposed it appropriately
Actually since there are only two variables, this can be solved directly. Assuming k1 is positive (No reason not to make k1 positive since flipping the sign doesn't really change the problem since k2 can be shifted appropriately), then we have k1*max(rate) + k2 = 776.37 and k1*min(rate) + k2 = -397.95
So
k1*(max(rate) - min(rate)) = 776.37 - (-397.95)
Then we can solve for k1 as
k1 = (776.37 - (-397.95))/(max(rate) - min(rate))
And then k2 can be found as
k2 = 776.37 - k1 * max(rate)
Related
Finding m and c for an equation y = mx + c, with the help of math and plots.
y is data_model_1, x is time.
Avoid other MATLAB functions like fitlm as it defeats the purpose.
I am having trouble finding the constants m and c. I am trying to find both m and c by limiting them to a range (based on smart guess) and I need to deduce the m and c values based on the mean error range. The point where mean error range is closest to 0 should be my m and c values.
load(file)
figure
plot(time,data_model_1,'bo')
hold on
for a = 0.11:0.01:0.13
c = -13:0.1:-10
data_a = a * time + c ;
plot(time,data_a,'r');
end
figure
hold on
for a = 0.11:0.01:0.13
c = -13:0.1:-10
data_a = a * time + c ;
mean_range = mean(abs(data_a - data_model_1));
plot(a,mean_range,'b.')
end
A quick & dirty approach
You can quickly get m and c using fminsearch(). In the first example below, the error function is the sum of squared error (SSE). The second example uses the sum of absolute error. The key here is ensuring the error function is convex.
Note that c = Beta(1) and m = Beta(2).
Reproducible example (MATLAB code[1]):
% Generate some example data
N = 50;
X = 2 + 13*random(makedist('Beta',.7,.8),N,1);
Y = 5 + 1.5.*X + randn(N,1);
% Example 1
SSEh =#(Beta) sum((Y - (Beta(1) + (Beta(2).*X))).^2);
Beta0 = [0.5 0.5]; % Initial Guess
[Beta SSE] = fminsearch(SSEh,Beta0)
% Example 2
SAEh =#(Beta) sum(abs(Y-(Beta(1) + Beta(2).*X)));
[Beta SumAbsErr] = fminsearch(SAEh,Beta0)
This is a quick & dirty approach that can work for many applications.
#Wolfie's comment directs you to the analytical approach to solve a system of linear equations with the \ operator or mldivide(). This is the more correct approach (though it will get a similar answer). One caveat is this approach gets the SSE answer.
[1] Tested with MATLAB R2018a
I wanted to compute a finite difference with respect to the change of the function in Matlab. In other words
f(x+e_i) - f(x)
is what I want to compute. Note that its very similar to the first order numerical partial differentiation (forward differentiation in this case) :
(f(x+e_i) - f(x)) / (e_i)
Currently I am using for loops to compute it but it seems that Matlab is much slower than I thought. I am doing it as follows:
function [ dU ] = numerical_gradient(W,f,eps)
%compute gradient or finite difference update numerically
[D1, D2] = size(W);
dU = zeros(D1, D2);
for d1=1:D1
for d2=1:D2
e = zeros([D1,D2]);
e(d1,d2) = eps;
f_e1 = f(W+e);
f_e2 = f(W-e);
%numerical_derivative = (f_e1 - f_e2)/(2*eps);
%dU(d1,d2) = numerical_derivative
numerical_difference = f_e1 - f_e2;
dU(d1,d2) = numerical_difference;
end
end
it seems that its really difficult to vectorize the above code because for numerical differences follow the definition of the gradient and partial derivatives which is:
df_dW = [ ..., df_dWi, ...]
where df_dWi assumes the other coordinates are fixed and it only worries about the change of the variable Wi. Thus, I can't just change all the coordinates at once.
Is there a better way to do this? My intuition tells me that the best way to do this is to implement this not in matlab but in some other language, say C and then have matlab call that library. Is that true? Does it mean that the best solution is some Matlab library that does this for me?
I did see:
https://www.mathworks.com/matlabcentral/answers/332414-what-is-the-quickest-way-to-find-a-gradient-or-finite-difference-in-matlab-of-a-real-function-in-hig
but unfortunately, it computes exact derivatives, which isn't what I am looking for. I am explicitly looking for differences or "bad approximation" to the gradient.
Since it seems this code is not easy to vectorize (in fact my intuition tells me its not possible to do so) my only other idea is to implement this finite difference function in C and then have C call the function. Is this a good idea? Anyone know how to do this?
I did try reading the following:
https://www.mathworks.com/help/matlab/matlab_external/standalone-example.html
but it was too difficult to understand for me because I have no idea what a mex file is, if I need to have a arrayProduct.c file as well as a mex.h file, if I also needed a matlab file, etc. If there just existed a way to simply download a working example with all the functions they suggest there and some instructions to compile it, then it would be super helpful. But just reading the hmtl/article like that its impossible for me to infer what they want me to do.
For the sake of completness it seems reddit has some comments in its discussion of this:
https://www.reddit.com/r/matlab/comments/623m7i/how_does_one_compute_a_single_finite_differences/
Here is a more efficient doing so:
function [ vNumericalGrad ] = CalcNumericalGradient( hInputFunc, vInputPoint, epsVal )
numElmnts = size(vInputPoint, 1);
vNumericalGrad = zeros([numElmnts, 1]);
refVal = hInputFunc(vInputPoint);
for ii = 1:numElmnts
% Set the perturbation vector
refInVal = vInputPoint(ii);
vInputPoint(ii) = refInVal + epsVal;
% Compute Numerical Gradient
vNumericalGrad(ii) = (hInputFunc(vInputPoint) - refVal) / epsVal;
% Reset the perturbation vector
vInputPoint(ii) = refInVal;
end
end
This code allocate less memory.
The above code performance will be totally controlled by the speed of the hInputFunction.
The small tricks compared to original code are:
No memory reallocation of e each iteration.
Instead of addition of vectors W + e there are 2 assignments to the array.
Decreasing the calls to hInputFunction() by half by defining the reference value outside the loop (This only works for Forward / Backward difference).
Probably this will be very close to C code unless you can code in C more efficiently the function which computes the value (hInputFunction).
A full implementation can be found in StackOverflow Q44984132 Repository (It was Posted in StackOverflow Q44984132).
See CalcFunGrad( vX, hObjFun, difMode, epsVal ).
A way better approach (numerically more stable, no issue of choosing the perturbation hyperparameter, accurate up to machine precision) is to use algorithmic/automatic differentiation. For this you need the Matlab Deep Learning Toolbox. Then you can use dlgradient to compute the gradient. Below you find the source code attached corresponding to your example.
Most importantly, you can examine the error and observe that the deviation of the automatic approach from the analytical solution is indeed machine precision, while for the finite difference approach (I choose second order central differences) the error is orders of magnitude higher. For 100 points and a range of $[-10, 10]$ this errors are somewhat tolerable, but if you play a bit with Rand_Max and n_points you observe that the errors become larger and larger.
Error of algorithmic / automatic diff. is: 1.4755528111219851e-14
Error of finite difference diff. is: 1.9999999999348703e-01 for perturbation 1.0000000000000001e-01
Error of finite difference diff. is: 1.9999999632850161e-03 for perturbation 1.0000000000000000e-02
Error of finite difference diff. is: 1.9999905867860374e-05 for perturbation 1.0000000000000000e-03
Error of finite difference diff. is: 1.9664569947425062e-07 for perturbation 1.0000000000000000e-04
Error of finite difference diff. is: 1.0537897883625319e-07 for perturbation 1.0000000000000001e-05
Error of finite difference diff. is: 1.5469326944467290e-06 for perturbation 9.9999999999999995e-07
Error of finite difference diff. is: 1.3322061696937969e-05 for perturbation 9.9999999999999995e-08
Error of finite difference diff. is: 1.7059535957436630e-04 for perturbation 1.0000000000000000e-08
Error of finite difference diff. is: 4.9702408787320664e-04 for perturbation 1.0000000000000001e-09
Source Code:
f2.m
function y = f2(x)
x1 = x(:, 1);
x2 = x(:, 2);
x3 = x(:, 3);
y = x1.^2 + 2*x2.^2 + 2*x3.^3 + 2*x1.*x2 + 2*x2.*x3;
f2_grad_analytic.m:
function grad = f2_grad_analytic(x)
x1 = x(:, 1);
x2 = x(:, 2);
x3 = x(:, 3);
grad(:, 1) = 2*x1 + 2*x2;
grad(:, 2) = 4*x2 + 2*x1 + 2 * x3;
grad(:, 3) = 6*x3.^2 + 2*x2;
f2_grad_AD.m:
function grad = f2_grad_AD(x)
x1 = x(:, 1);
x2 = x(:, 2);
x3 = x(:, 3);
y = x1.^2 + 2*x2.^2 + 2*x3.^3 + 2*x1.*x2 + 2*x2.*x3;
grad = dlgradient(y, x);
CalcNumericalGradient.m:
function NumericalGrad = CalcNumericalGradient(InputPoints, eps)
% (Central, second order accurate FD)
NumericalGrad = zeros(size(InputPoints) );
for i = 1:size(InputPoints, 2)
perturb = zeros(size(InputPoints));
perturb(:, i) = eps;
NumericalGrad(:, i) = (f2(InputPoints + perturb) - f2(InputPoints - perturb)) / (2 * eps);
end
main.m:
clear;
close all;
clc;
n_points = 100;
Rand_Max = 20;
x_test_FD = rand(n_points, 3) * Rand_Max - Rand_Max/2;
% Calculate analytical solution
grad_analytic = f2_grad_analytic(x_test_FD);
grad_AD = zeros(n_points, 3);
for i = 1:n_points
x_test_dl = dlarray(x_test_FD(i,:) );
grad_AD(i,:) = dlfeval(#f2_grad_AD, x_test_dl);
end
Err_AD = norm(grad_AD - grad_analytic);
fprintf("Error of algorithmic / automatic diff. is: %.16e\n", Err_AD);
eps_range = [1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-7, 1e-8, 1e-9];
for i = 1:length(eps_range)
eps = eps_range(i);
grad_FD = CalcNumericalGradient(x_test_FD, eps);
Err_FD = norm(grad_FD - grad_analytic);
fprintf("Error of finite difference diff. is: %.16e for perturbation %.16e\n", Err_FD, eps);
end
I have a 30x30 matrix as a base matrix (OD_b1), I also have two base vectors (bg and Ag). My aim is to optimize a matrix (X) who's dimensions are 30X30 such that:
1) the squared difference between vector (bg) and vector of sum of all the columns is minimized.
2)the squared difference between vector (Ag) and vector of sum of all rows is minimized.
3)the squared difference between the elements of matrix (X) and matrix (OD_b1) is minimized.
The mathematical form of the equation is as follows:
I have tried this:
fun=#(X)transpose(bg-sum(X,2))*(bg-sum(X,2))+ (Ag-sum(X,1))*transpose(Ag-sum(X,1))+sumsqr(X_b-X);
[val,X]=fmincon(fun,OD_b1,AA,BB,Aeq,beq,LB,UB)
I don't get errors but it seems like it's stuck.
Is it because I have too many variables or is there another reason?
Thanks in advance
This is a simple, unconstrained least squares problem and hence has a simple solution that can be expressed as the solution to a linear system.
I will show you (1) the precise and efficient way to solve this and (2) how to solve with fmincon.
The precise, efficient solution:
Problem setup
Just so we're on the same page, I initialize the variables as follows:
n = 30;
Ag = randn(n, 1); % observe the dimensions
X_b = randn(n, n);
bg = randn(n, 1);
The code:
A1 = kron(ones(1,n), eye(n));
A2 = kron(eye(n), ones(1,n));
A = (A1'*A1 + A2'*A2 + eye(n^2));
b = A1'*bg + A2'*Ag + X_b(:);
x = A \ b; % solves A*x = b
Xstar = reshape(x, n, n);
Why it works:
I first reformulated your problem so the objective is a vector x, not a matrix X. Observe that z = bg - sum(X,2) is equivalent to:
x = X(:) % vectorize X
A1 = kron(ones(1,n), eye(n)); % creates a special matrix that sums up
% stuff appropriately
z = A1*x;
Similarly, A2 is setup so that A2*x is equivalent to Ag'-sum(X,1). Your problem is then equivalent to:
minimize (over x) (bg - A1*x)'*(bg - A1*x) + (Ag - A2*x)'*(Ag - A2*x) + (y - x)'*(y-x) where y = Xb(:). That is, y is a vectorized version of Xb.
This problem is convex and the first order condition is a necessary and sufficient condition for the optimum. Take the derivative with respect to x and that equation will define your solution! Sample example math for almost equivalent (but slightly simpler problem is below):
minimize(over x) (b - A*x)'*(b - A*x) + (y - x)' * (y - x)
rewriting the objective:
b'b- b'Ax - x'A'b + x'A'Ax +y'y - 2y'x+x'x
Is equivalent to:
minimize(over x) (-2 b'A - 2y'*I) x + x' ( A'A + I) * x
the first order condition is:
(A'A+I+(A'A+I)')x -2A'b-2I'y = 0
(A'A+I) x = A'b+I'y
Your problem is essentially the same. It has the first order condition:
(A1'*A1 + A2'*A2 + I)*x = A1'*bg + A2'*Ag + y
How to solve with fmincon
You can do the following:
f = #(X) transpose(bg-sum(X,2))*(bg-sum(X,2)) + (Ag'-sum(X,1))*transpose(Ag'-sum(X,1))+sum(sum((X_b-X).^2));
o = optimoptions('fmincon');%MaxFunEvals',30000);
o.MaxFunEvals = 30000;
Xstar2 = fmincon(f,zeros(n,n),[],[],[],[],[],[],[],o);
You can then check the answers are about the same with:
normdif = norm(Xstar - Xstar2)
And you can see that gap is small, but that the linear algebra based solution is somewhat more precise:
gap = f(Xstar2) - f(Xstar)
If the fmincon approach hangs, try it with a smaller n just to gain confidence that my linear algebra based solution is more precise, way way faster etc... n = 30 is solving a 30^2 = 900 variable optimization problem: not easy. With the linear algebra approach, you can go up to n = 100 (i.e. 10000 variable problem) or even larger.
I would probably solve this as a QP using quadprog using the following reformulation (keeping the objective as simple as possible to make the problem "less nonlinear"):
min sum(i,v(i)^2)+sum(i,w(i)^2)+sum((i,j),z(i,j)^2)
v = bg - sum(c,x)
w = ag - sum(r,x)
Z = xbase-x
The QP solver is more precise (no gradients using finite differences). This approach also allows you to add additional bounds and linear equality and inequality constraints.
The other suggestion to form the first order conditions explicitly is also a good one: it also has no issue with imprecise gradients (the first order conditions are linear). I usually prefer a quadratic model because of its flexibility.
I ran through the algebra which I had previously done for the Verlet method without the force - this lead to the same code as you see below, but the "+(2*F/D)" term was missing when I ignored the external force. The algorithm worked accurately, as expected, however for the following parameters:
m = 7 ; k = 8 ; b = 0.1 ;
params = [m,k,b];
(and step size h = 0.001)
a force far above something like 0.00001 is much too big. I suspect I've missed a trick with the algebra.
My question is whether someone can spot the flaw in my addition of a force term in my Verlet method
% verlet.m
% uses the verlet step algorithm to integrate the simple harmonic
% oscillator.
% stepsize h, for a second-order ODE
function vout = verlet(vinverletx,h,params,F)
% vin is the particle vector (xn,yn)
x0 = vinverletx(1);
x1 = vinverletx(2);
% find the verlet coefficients
D = (2*params(1))+(params(3)*h);
A = (2/D)*((2*params(1))-(params(2)*h^2));
B=(1/D)*((params(3)*h)-(2*params(1)));
x2 = (A*x1)+(B*x0)+(2*F/D);
vout = x2;
% vout is the particle vector (xn+1,yn+1)
end
As written in the answer to the previous question, the moment friction enters the equation, the system is no longer conservative and the name "Verlet" does no longer apply. It is still a valid discretization of
m*x''+b*x'+k*x = F
(with some slight error with large consequences).
The discretization employs the central difference quotients of first and second order
x'[k] = (x[k+1]-x[k-1])/(2*h) + O(h^2)
x''[k] = (x[k+1]-2*x[k]+x[k-1])/(h^2) + O(h^2)
resulting in
(2*m+b*h)*x[k+1] - 2*(2*m+h^2*k) * x[k] + (2*m-b*h)*x[k-1] = 2*h^2 * F[k] + O(h^4)
Error: As you can see, you are missing a factor h^2 in the term with F.
I've got a question about the implementation of a double integral in MATLAB.
It's known that
Making use of
k1 = 1E-04:0.001:1E+04;
k2 = 1E-04:0.001:1E+04;
k3 = 1E-04:0.001:1E+04;
the above procedure(calling the formula of F11,F22 and F33) leads to the results shown below:
Now comes the question:
I would like to achieve the same results by means of a double integral (or nested single integral in k2 and k3) involving only phi11,phi22 and phi33, hence without calling directly the formula for F11, F22 and F33.
Up to now I'm using this code (restricted to F11 calculation):
clc,clear all,close all
k1 = (1E-04:0.01:10000);
k2 = (1E-04:0.01:10000);
k3 = (1E-04:0.01:10000);
F_11_benchmark = ((9/55)*1.453)*(1./(1+k1.^2).^(5/6));
count = 0;
F_11 = zeros(1,numel(k1));
for i = 2:numel(k1)
count = count + 1;
phi_11 = #(k2,k3) (1.453./(4.*pi)).*((k2.^2+k3.^2)./((1 + k1(count).^2 + k2.^2+k3.^2).^(17/6)));
F_11(count) = dblquad(phi_11,k2(i-1),k2(i),k3(i-1),k3(i));
end
not leading to the same benchmark results shown in the last figure.
How would you suggest to tackle this problem? I thank you in advance.
Best regards,
FPE
You are using dblquad wrong. The way you are doing it calculates for each k1 the integral over the little box [k1(i-1) k1(i)]x[k1(i-1) k1(i)], which in no way approximates the whole domain of integration. Use the integral2 command instead: try just using
F_11(count) = integral2(phi_11,-100,100,-100,100);
or
F_11(count) = integral2(phi_11,-Inf,Inf,-Inf,Inf);
which should be more accurate but will be slower. You can probably also get away with using a much larger resolution for k1. I used
k1=10.^(-4:.01:3);
to sort of match your graph.