I am testing MATLAB capabilities in solving equations for a project that I intend to do, so I gave it a test run with something simple, but the results that it gives me are incorrect. I tried to solve two non-linear equations with two unknowns, one of the solutions is correct the other is not.
syms theta d x y
eq1 = d * cos(theta) == x;
eq2 = d * sin(theta) == y;
sol = solve(eq1, eq2, theta, d)
sol.theta
sol.d
The solutions for d are correct, but for theta I get:
-2*atan((x - (x^2 + y^2)^(1/2))/y)
-2*atan((x + (x^2 + y^2)^(1/2))/y)
And the correct answer for theta is simply atan(y/x)
Then when I evaluate these solutions with x = 1, y = 0, I get:
eval(sol.d)
eval(sol.theta)
d = 1, -1
theta = NaN, -3.1416
Solutions for d are correct, but theta in that scenario should be 0.
What am I doing wrong?
EDIT: solving it by hand it looks like this: Divide the y equation by the x equation
y/x = (d * sin(theta)) / (d * cos(theta))
y/x = sin(theta)/cos(theta)
y/x = tan(theta)
theta = atan(y/x)
Even if matlab solves it in some other way and gets a different expression, it should still yield the same final result when I use numbers and it PARTIALLY does.
For x = 1 and y = 0, theta should be 0, => this doesnt work, it gives NaN (explanation bellow)
for x = 1 and y = 1, theta should be 45 degrees => this works
for x = 0 and y = 1 theta should be 90 degrees => this works
And I just checked it again with the 45 and 90 degree values for x and y and it works, but for x = 1 and y = 0 it still gives NaN as one of the answers and that is because it gets a 0/0 from the way it is expressing it
-2*atan((x - (x^2 + y^2)^(1/2))/y)
-2*(1 - (1^2 + 0^2))^(1/2)/0
-2*(1 - 1)^(1/2)/0
0/0
but if its in the form of atan(y/x) the result is
theta = atan(0/1)
theta = atan(0)
theta = 0
Did you mean to solve this:
syms a b theta d real
eq1 = a==d * cos(theta) ;
eq2 = b==d * sin(theta) ;
[sol] = solve([eq1 eq2],[d theta] ,'IgnoreAnalyticConstraints', true,'Real',true,'ReturnConditions',true);
When solving the equations with symbolic x and y, the solver will find a solution with a certain condition, which can be obtained using the argument 'ReturnCondition':
syms x y theta d real
eq1 = d*cos(theta) == x;
eq2 = d*sin(theta) == y;
sol = solve([eq1; eq2],[d theta],'ReturnConditions',true);
This gives the following result for sol
>> sol.d
(x^2 + y^2)^(1/2)
-(x^2 + y^2)^(1/2)
>> sol.theta
2*pi*k - 2*atan((x - (x^2 + y^2)^(1/2))/y)
2*pi*k - 2*atan((x + (x^2 + y^2)^(1/2))/y)
>> sol.parameters
k
>> sol.conditions
y ~= 0 & in(k, 'integer')
y ~= 0 & in(k, 'integer')
As you can see, y = 0 does not fulfill this general solution given by the solver, resulting in your problem for y = 0. You can find solutions for y = 0 by either making y numeric instead of symbolic, or by adding an assumption:
syms x y theta d real
assume(y==0)
sol = solve([eq1; eq2],[d theta],'ReturnConditions',true);
I guess its easier to just set y=0 numeric, for this one condition, since there are already 4 possible solutions and conditions for the three lines above.
A paper I'm reading contains the following theorem.
I wrote some MATLAB code to try and reproduce results that appear later in the paper, and initially it seemed to work well.
M = 6;
Sigma = [1 .5 .15 .15 0 0;
.5 1 .15 .15 0 0;
.15 .15 1 .25 0 0;
.15 .15 .25 1 0 0;
0 0 0 0 1 .1;
0 0 0 0 .1 1];
Delta = [0 0 .2 .2 .5 .5]';
cov_vect = [.3 .3 .35 .35 .25 .25];
u = ones(M,1);
lastcol = [u' 0];
First = Sigma+(Delta*Delta');
First(M+1,:) = u;
First(:,M+1) = lastcol;
Third = [cov_vect 1]';
X = linsolve(First,Third);
This code creates results that match those from the paper.
I want to use my code with other data sets, but when I try to do that I encounter a problem. M, Sigma, Delta, and cov_vect will vary from data set to data set, but the rest of the code should stay the same.
When I use my code on new data sets, then although the vector w sums to 1 (as it should) it sometimes contains negative values. According to the paper, this shouldn't happen. It's fine for lambda to be negative, but none of the values in the w vector can be negative.
How can I get MATLAB to constrain the results so that all the values in w must be positive, while maintaining the requirement that the vector w sum to 1?
Your question appears to reference this paper.
Theorem 2 you reference is the solution to the following optimization problem (see error/typos section, I've had to make at least one correction).
minimize (over w) w' * (Sigma + delta * delta') * w - 2 * cov_vect' * w
subject to: w'*ones(n, 1) = 1
This can be solved using Matlab function quadprog with:
H = 2 * (Sigma + delta * Delta'); % see quadprog docs, it solves 1/2 so we need 2
f = - 2 * cov_vect;
A = [];
b = [];
Aeq = ones(1,6);
beq = 1;
w = quadprog(H, f, A, b, Aeq, beq);
You can add the lower bound constraint of 0 with:
lb = zeros(6, 1);
ub = [];
w2 = quadprog(H, f, A, b, Aeq, beq, lb, ub);
How to solve this in CVX (awesome optimization package)
cvx_begin
variables y(n);
minimize(y' * (Sigma + Delta * Delta') * y - 2 * cov_vect * y)
subject to:
y'*ones(n,1) == 1;
y >= 0;
cvx_end
Link to cvx.
Typo in appendix of paper as posted on researchgate:
(typo) Their proof of theorem 2 omits the 2*w in term 2*cov_vect' * w of thier objective function. The minimization problem should be:
minimize (over w) w' * (Sigma + delta * delta') * w - 2*cov_vect' * w
Which indeed gives solution:
0.1596 0.1596 0.2090 0.2090 0.1314 0.1314
Or equivalently:
minimize (over w) .5 * w' * (Sigma + delta * delta') * w - cov_vect' * w
I want to write my own 2 Dimensional DFT function with reduced loops.
What I try to implement is Discrete Fourier Transform:
Using the separability property of transform (actually exponential function), we can write this as multiplication of two 1 dimensional DFT. Then, we can calculate the exponential terms for rows (the matrix wM below) and columns (the matrix wN below) of transform. Then, for summation process we can multiply them as "F = wM * original_matrix * wN"
Here is the code I wrote:
f = imread('cameraman.tif');
[M, N, ~] = size(f);
wM = zeros(M, M);
wN = zeros(N, N);
for u = 0 : (M - 1)
for x = 0 : (M - 1)
wM(u+1, x+1) = exp(-2 * pi * 1i / M * x * u);
end
end
for v = 0 : (N - 1)
for y = 0 : (N - 1)
wN(y+1, v+1) = exp(-2 * pi * 1i / N * y * v);
end
end
F = wM * im2double(f) * wN;
The first thing is I dont want to use 2 loops which are MxM and NxN times running. If I used a huge matrix (or image), that would be a problem. Is there any chance to make this code faster (for example eliminating the loops)?
The second thing is displaying the Fourier Transform result. I use the codes below to display the transform:
% // "log" method
fl = log(1 + abs(F));
fm = max(fl(:));
imshow(im2uint8(fl / fm))
and
% // "abs" method
fa = abs(F);
fm = max(fa(:));
imshow(fa / fm)
When I use the "abs" method, I see only black figure, nothing else. What is wrong with "abs" method you think?
And the last thing is when I compare the transform result of my own function with MATLAB' s fft2() function', mine displays darker figure than MATLAB' s result. What am I missing here? Implementation misktake?
The transform result of my own function:
The transform result of MATLAB fft2() function:
I am happy you solved your problem but unfortunately you answer is not completely right. Indeed it does the job, but as I commented, im2double will normalize everything to 1, therefore showing the scaled result you have. What you want (if you are looking for performance) is not doing im2doubleand then multiply by 255, but directly casting to double().
You can eliminate loops by using meshgrid.
For example:
M = 1024;
tic
[ mX, mY ] = meshgrid( 0 : M - 1, 0 : M - 1 );
wM1 = exp( -2 * pi * 1i / M .* mX .* mY );
toc
tic
for u = 0 : (M - 1)
for x = 0 : (M - 1)
wM2( u + 1, x + 1 ) = exp( -2 * pi * 1i / M * x * u );
end
end
toc
all( wM1( : ) == wM2( : ) )
The timing on my system was:
Elapsed time is 0.130923 seconds.
Elapsed time is 0.493163 seconds.
I'm using octave's ezmesh to plot a linear regression defined as follows:
f = #(x,y) 1 * theta(1) + x * theta(2) + y * theta(3) + x * y * theta(4)
For some fixed vector theta:
octave:275> theta
theta =
9.4350e+00
1.7410e-04
3.3702e-02
1.6498e-07
I'm using a domain of [0 120000 0 1400], and can evaluate:
octave:276> f(0, 0)
ans = 9.4350
octave:277> f(120000, 1400)
ans = 105.23
However, if I run:
octave:278> ezmesh(f, [0 120000 0 1400])
The resulting mesh has a z value of around 570 for (0, 0) and just under 640 for (120000, 1400). I'm baffled. What could be causing this?
EDIT: Even if I simplify f to the following, similar behavior occurs:
octave:308> f = #(x, y) (x * y)
Why is ezmesh not handling multiplication as expected (by me), so that the function evaluates as I expect, and the values change when the function is used inside of ezmesh?
ezmesh invokes the function handle on a matrix of values (to benefit from vectorization performance). Use .* for multiplication.
Suppose we are given a training dataset {yᵢ, xᵢ}, for i = 1, ..., n, where yᵢ can either be -1 or 1 and xᵢ can be e.g. a 2D or 3D point.
In general, when the input points are linearly separable, the SVM model can be defined as follows
min 1/2*||w||²
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1
This is often called the hard-margin SVM model, which is thus a constrained minimization problem, where the unknowns are w and b. We can also omit 1/2 in the function to be minimized, given it's just a constant.
Now, the documentation about Matlab's quadprog states
x = quadprog(H, f, A, b) minimizes 1/2*x'*H*x + f'*x subject to the restrictions A*x ≤ b. A is a matrix of doubles, and b is a vector of doubles.
We can implement the hard-margin SVM model using quadprog function, to get the weight vector w, as follows
H becomes an identity matrix.
f' becomes a zeros matrix.
A is the left-hand side of the constraints
b is equal to -1 because the original constraint had >= 1, it becomes <= -1 when we multiply with -1 on both sides.
Now, I am trying to implement a soft-margin SVM model. The minimization equation here is
min (1/2)*||w||² + C*(∑ ζᵢ)
w,b
subject to the constraints (for i = 1, ..., n)
yᵢ*(w*xᵢ - b) >= 1 - ζᵢ
such that ζᵢ >= 0, where ∑ is the summation symbol, ζᵢ = max(0, 1 - yᵢ*(w*xᵢ - b)) and C is a hyper-parameter.
How can this optimization problem be solved using the Matlab's quadprog function? It's not clear to me how the equation should be mapped to the parameters of the quadprog function.
The "primal" form of the soft-margin SVM model (i.e. the definition above) can be converted to a "dual" form. I did that, and I am able to get the Lagrange variable values (in the dual form). However, I would like to know if I can use quadprog to solve directly the primal form without needing to convert it to the dual form.
I don't see how it can be a problem. Let z be our vector of (2n + 1) variables:
z = (w, eps, b)
Then, H becomes diagonal matrix with first n values on the diagonal equal to 1 and the last n + 1 set to zero:
H = diag([ones(1, n), zeros(1, n + 1)])
Vector f can be expressed as:
f = [zeros(1, n), C * ones(1, n), 0]'
First set of constrains becomes:
Aineq = [A1, eye(n), zeros(n, 1)]
bineq = ones(n, 1)
where A1 is a the same matrix as in primal form.
Second set of constraints becomes lower bounds:
lb = (inf(n, 1), zeros(n, 1), inf(n, 1))
Then you can call MATLAB:
z = quadprog(H, f, Aineq, bineq, [], [], lb);
P.S. I can be mistaken in some small details, but the general idea is right.
I wanted to clarify #vharavy answer because you could get lost while trying to deduce what 'n' means in his code. Here is my version according to his answer and SVM wikipedia article. I assume we have a file named "test.dat" which holds coordinates of test points and their class membership in the last column.
Example content of "test.dat" with 3D points:
-3,-3,-2,-1
-1,3,2,1
5,4,1,1
1,1,1,1
-2,5,4,1
6,0,1,1
-5,-5,-3,-1
0,-6,1,-1
-7,-2,-2,-1
Here is the code:
data = readtable("test.dat");
tableSize = size(data);
numOfPoints = tableSize(1);
dimension = tableSize(2) - 1;
PointsCoords = data(:, 1:dimension);
PointsSide = data.(dimension+1);
C = 0.5; %can be changed
n = dimension;
m = numOfPoints; %can be also interpretet as number of constraints
%z = [w, eps, b]; number of variables in 'z' is equal to n + m + 1
H = diag([ones(1, n), zeros(1, m + 1)]);
f = [zeros(1, n), C * ones(1, m), 0];
Aineq = [-diag(PointsSide)*table2array(PointsCoords), -eye(m), PointsSide];
bineq = -ones(m, 1);
lb = [-inf(1, n), zeros(1, m), -inf];
z = quadprog(H, f, Aineq, bineq, [], [], lb);
If let z = (w; w0; eps)T be a the long vector with n+1+m elements.(m the number of points)
Then,
H= diag([ones(1,n),zeros(1,m+1)]).
f = [zeros(1; n + 1); ones(1;m)]
The inequality constraints can be specified as :
A = -diag(y)[X; ones(m; 1); zeroes(m;m)] -[zeros(m,n+1),eye(m)],
where X is the n x m input matrix in the primal form.Out of the 2 parts for A, the first part is for w0 and the second part is for eps.
b = ones(m,1)
The equality constraints :
Aeq = zeros(1,n+1 +m)
beq = 0
Bounds:
lb = [-inf*ones(n+1,1); zeros(m,1)]
ub = [inf*ones(n+1+m,1)]
Now, z=quadprog(H,f,A,b,Aeq,beq,lb,ub)
Complete code. The idea is the same as above.
n = size(X,1);
m = size(X,2);
H = diag([ones(1, m), zeros(1, n + 1)]);
f = [zeros(1,m+1) c*ones(1,n)]';
p = diag(Y) * X;
A = -[p Y eye(n)];
B = -ones(n,1);
lb = [-inf * ones(m+1,1) ;zeros(n,1)];
z = quadprog(H,f,A,B,[],[],lb);
w = z(1:m,:);
b = z(m+1:m+1,:);
eps = z(m+2:m+n+1,:);