I am looking for some help on determining the linear decision boundary between two classes.
I've taken a look at the search results with no luck.
Implementing a linear, binary SVM (support vector machine) is similar but not quite on the mark.
My question comes down to how to pull the correct line equation out of the weight vector.
Given a matrix of test data X=[Xa Xb], where Xa=[Nx2] && Xb=[Nx2] data samples.
These are in two classes (-1,1) saved in [Nx1] vector Y=[1 1 ... 1 -1 -1... -1]'
I use MATLAB's quadprog.m to solve the quadratic program...
I understand that we want to solve w1x1 + w2x2 + wo=0, so in my code I solve for W=[w1 w2]; and I solve for wo=1/Y1 - WX1^T.
When implemented my decision boundary plots as:
Clearly this is not what I want. The slope of the line looks legit, but I think I want to translate it north a little bit to optimize. In this picture the yellow dots are the support vectors.
It appears that I used the first data point, X(1,:) to be precise, and my line sucks. If I use different points it draws the line in different places, this makes sense algebraically, but how do i get the optimal boundary, not just a parallel boundary orthogonal to the weighting vector?
Thanks!
If you're interested in the code, here it is:
function [alph,w,wo,sv]=svm_binary(Class1,Class2)
x1=Class1;
x2=Class2;
% Combine data into one set
xt=[x1;x2];
% Create class labels
y=[ones(length(x1),1); -1.*ones(length(x2),1)];
N=length(xt);
% Scatter plot of original data class data points
figure
scatter(x1(:,1),x1(:,2));
hold on
scatter(x2(:,1),x2(:,2));
legend('Class1','Class2')
xlabel('x1')
ylabel('x2')
title('Class Data Scatter Plot')
% Data component of Langrangian dual
H=(xt*xt').*(y*y');
% Vector to flip signs
f=-ones(N,1);
%Constraint 1) a(i)>=0
A= -eye(N);
a=zeros(N,1);
% Constraint 2) sum[a(i)y(i)]=0
B=[y';zeros(N-1,N)];
b=zeros(N,1);
%Solve Quadratic Programming optimization for alpha
alph=quadprog(H+eye(N)*.001,f,A,a,B,b);
%Solve for W
w=(alph.*y)'*xt;
sv=[];
for i=1:length(xt)
if abs(alph(i))>=.0000001
sv=[sv i];
end
end
xtsv=xt(sv,:);
wo=1/y(1)-w*xt(1,:)';
if abs(w(1))<=.000001
y=-wo/w(2).*ones(round(max(xt(:,1))-min(xt(:,1))),1);
x=min(xt(:,2)):(max(xt(:,2))-min(xt(:,2)))/(length(y)-1):max(xt(:,2));
elseif abs(w(2))<=.000001
x=-wo/w(1).*ones(round(max(xt(:,2))-min(xt(:,2))),1);
y=min(xt(:,1)):(max(xt(:,1))-min(xt(:,1)))/(length(x)-1):max(xt(:,1));
else
x=round(min(xt(:,1))):round(max(xt(:,1)))
y=(w(1)/w(2)).*-x-wo/(w(2));
end
sv=[];
for i=1:length(xt)
if abs(alph(i))>=.0000001
sv=[sv i];
end
end
xtsv=xt(sv,:);
scatter(xtsv(:,1),xtsv(:,2),'fillled','markeredgecolor','black','markerfacecolor','yellow');
% y=-(w(1).*x)-wo
length(x)
length(y)
hold on
plot(x,y)
Related
I have the following memory-speed problem in Matlab and I would like your help to understand whether there may be a solution.
Consider the following 4 big column vectors X1, X2, Y1, Y2.
clear
rng default
P=10^8;
X1=rand(1,P)*5;
X2=rand(1,P)*5;
Y1=rand(1,P)*5;
Y2=rand(1,P)*5;
What I would like to do is a scatter plot where on the x-axis I have the sum between any possible two elements of X1 and X2 and on the y-axis I have the sum between any possible two elements of Y1 and Y2.
I post here three options I thought about that do not work mainly because of memory and speed issues.
Option 1 (issues: too slow when doing the loop, out of memory when doing vertcat)
Xtemp=cell(P,1);
Ytemp=cell(P,1);
for i=1:P
tic
Xtemp{i}=X1(i)+X2(:);
Ytemp{i}=Y1(i)+Y2(:);
toc
end
X=vertcat(Xtemp{:});
Y=vertcat(Ytemp{:});
scatter(X,Y)
Option 2 (issues: too slow when doing the loop, time increasing as the loop proceeds, Matlab going crazy and unable to produce the scatter even if I stop the loop after 5 iterations)
for i=1:P
tic
scatter(X1(i)+X2(:), Y1(i)+Y2(:))
hold on
toc
end
Option 3 (sort of giving up) (issues: as I increase T the scatter gets closer and closer to a square which is correct; I am wondering though whether this is caused by the fact that I generated the data using rand and in option 3 I use randi; maybe with my real data the scatter does not "converge" to the true plot as I increase T; also, what is the "optimal" T and R?).
T=20;
R=500;
for t=1:T
tic
%select R points at random from X1,X2,Y1,Y2
X1sel=(X1(randi(R,R,1)));
X2sel=(X2(randi(R,R,1)));
Y1sel=(Y1(randi(R,R,1)));
Y2sel=(Y2(randi(R,R,1)));
%do option 1 among those points and plot
Xtempsel=cell(R,1);
Ytempsel=cell(R,1);
for r=1:R
Xtempsel{r}=X1sel(r)+X2sel(:);
Ytempsel{r}=Y1sel(r)+Y2sel(:);
end
Xsel=vertcat(Xtempsel{:});
Ysel=vertcat(Ytempsel{:});
scatter(Xsel,Ysel, 'b', 'filled')
hold on
toc
end
Is there a way to do what I want or is simply impossible?
You are trying to build a vector with P^2 elements, i.e. 10^16. This is many order of magnitude more that what would fit into the memory of a standard computer (10GB is 10^10 bytes or 1.2 billion double precision floats).
For smaller vectors (i.e. P<1e4), try:
Xsum=bsxfun(#plus,X1,X2.'); %Matrix with the sum of any two elements from X1 and X2
X=X(:); %Reshape to vector
Ysum=bsxfun(#plus,Y1,Y2.');
Y=Y(:);
plot(X,Y,'.') %Plot as small dots, likely to take forever if there are too many points
To build a figure with a more reasonable number of pairs picked randomly from these large vectors:
Npick=1e4;
sel1=randi(P,[Npick,1]);
sel2=randi(P,[Npick,1]);
Xsel=X1(sel1)+X2(sel2);
Ysel=Y1(sel1)+Y2(sel2);
plot(Xsel,Ysel,'.'); %Plot as small dots
I would like to plot some figures like this one:
-axis being real and imag part of some complex valued vector(usually either pure real or imag)
-have some 3D visualization like in the given case
First, define your complex function as a function of (Re(x), Im(x)). In complex analysis, you can decompose any complex function into its real parts and imaginary parts. In other words:
F(x) = Re(x) + i*Im(x)
In the case of a two-dimensional grid, you can obviously extend to defining the function in terms of (x,y). In other words:
F(x,y) = Re(x,y) + i*Im(x,y)
In your case, I'm assuming you'd want the 2D approach. As such, let's use I and J to represent the real parts and imaginary parts separately. Also, let's start off with a simple example, like cos(x) + i*sin(y) which is based on the very popular Euler exponential function. It isn't exact, but I modified it slightly as the plot looks nice.
Here are the steps you would do in MATLAB:
Define your function in terms of I and J
Make a set of points in both domains - something like meshgrid will work
Use a 3D visualization plot - You can plot the individual points, or plot it on a surface (like surf, or mesh).
NB: Because this is a complex valued function, let's plot the magnitude of the output. You were pretty ambiguous with your details, so let's assume we are plotting the magnitude.
Let's do this in code line by line:
% // Step #1
F = #(I,J) cos(I) + i*sin(J);
% // Step #2
[I,J] = meshgrid(-4:0.01:4, -4:0.01:4);
% // Step #3
K = F(I,J);
% // Let's make it look nice!
mesh(I,J,abs(K));
xlabel('Real');
ylabel('Imaginary');
zlabel('Magnitude');
colorbar;
This is the resultant plot that you get:
Let's step through this code slowly. Step #1 is an anonymous function that is defined in terms of I and J. Step #2 defines I and J as matrices where each location in I and J gives you the real and imaginary co-ordinates at their matching spatial locations to be evaluated in the complex function. I have defined both of the domains to be between [-4,4]. The first parameter spans the real axis while the second parameter spans the imaginary axis. Obviously change the limits as you see fit. Make sure the step size is small enough so that the plot is smooth. Step #3 will take each complex value and evaluate what the resultant is. After, you create a 3D mesh plot that will plot the real and imaginary axis in the first two dimensions and the magnitude of the complex number in the third dimension. abs() takes the absolute value in MATLAB. If the contents within the matrix are real, then it simply returns the positive of the number. If the contents within the matrix are complex, then it returns the magnitude / length of the complex value.
I have labeled the axes as well as placed a colorbar on the side to visualize the heights of the surface plot as colours. It also gives you an idea of how high and how long the values are in a more pleasing and visual way.
As a gentle push in your direction, let's take a slice out of this complex function. Let's make the real component equal to 0, while the imaginary components span between [-4,4]. Instead of using mesh or surf, you can use plot3 to plot your points. As such, try something like this:
F = #(I,J) cos(I) + i*sin(J);
J = -4:0.01:4;
I = zeros(1,length(J));
K = F(I,J);
plot3(I, J, abs(K));
xlabel('Real');
ylabel('Imaginary');
zlabel('Magnitude');
grid;
plot3 does not provide a grid by default, which is why the grid command is there. This is what I get:
As expected, if the function is purely imaginary, there should only be a sinusoidal contribution (i*sin(y)).
You can play around with this and add more traces if you need to.
Hope this helps!
I am trying to fit a line to some data without using polyfit and polyval. I got some good help already on how to implement this and I have gotten it to work with a simple sin function. However, when applied to the function I am trying to fit, it does not work. Here is my code:
clear all
clc
lb=0.001; %lowerbound of data
ub=10; %upperbound of data
step=.1; %step-size through data
a=.03;
la=1482/120000; %1482 is speed of sound in water and 120kHz
ep1=.02;
ep2=.1;
x=lb:step:ub;
r_sq_des=0.90; %desired value of r^2 for the fit of data without noise present
i=1;
for x=lb:step:ub
G(i,1)= abs(sin((a/la)*pi*x*(sqrt(1+(1/x)^2)-1)));
N(i,1)=2*rand()-1;
Ghat(i,1)=(1+ep1*N(i,1))*G(i,1)+ep2*N(i,1);
r(i,1)=x;
i=i+1;
end
x=r;
y=G;
V=[x.^0];
Vfit=[x.^0];
for i=1:1:1000
V = [x.^i V];
c = V \ y;
Vfit = [x.^i Vfit];
yFit=Vfit*c;
plot(x,y,'o',x,yFit,'--')
drawnow
pause
end
The first two sections are just defining variables and the function. The second for loop is where I am making the fit. As you can see, I have it pause after every nth order in order to see the fit.
I changed your fit formula a bit, I got the same answers but quickly got
a warning that the matrix was singular. No sense in continuing past
the point that the inversion is singular.
Depending on what you are doing you can usually change out variables or change domains.
This doesn't do a lot better, but it seemed to help a little bit.
I increased the number of samples by a factor of 10 since the initial part of the curve
didn't look sampled highly enough.
I added a weighting variable but it is set to equal weight in the code below. Attempts
to deweight the tail didn't help as much as I hoped.
Probably not really a solution, but perhaps will help with a few more knobs/variables.
...
step=.01; %step-size through data
...
x=r;
y=G;
t=x.*sqrt(1+x.^(-2));
t=log(t);
V=[ t.^0];
w=ones(size(t));
for i=1:1:1000
% Trying to solve for value of c
% c that
% yhat = V*c approximates y
% or y = V*c
% V'*y = V'*V * c
% c = (V'*V) \ V'*y
V = [t.^i V];
c = (V'*diag(w.^2)*V ) \ (V'*diag(w.^2)*y) ;
yFit=V*c;
subplot(211)
plot(t,y,'o',t,yFit,'--')
subplot(212)
plot(x,y,'o',x,yFit,'--')
drawnow
pause
end
It looks like more of a frequency estimation problem, and trying to fit a unknown frequency
with polynomial tends to be touch and go. Replacing the polynomial basis with a quick
sin/cos basis didn't seem to do to bad.
V = [sin(t*i) cos(t*i) V];
Unless you specifically need a polynomial basis, you can apply your knowledge of the problem domain to find other potential basis functions for your fit, or to attempt to make the domain in which you are performing the fit more linear.
As dennis mentioned, a different set of basis functions might do better. However you can improve the polynomial fit with QR factorisation, rather than just \ to solve the matrix equation. It is a badly conditioned problem no matter what you do however, and using smooth basis functions wont allow you to accurately reproduce the sharp corners in the actual function.
clear all
close all
clc
lb=0.001; %lowerbound of data
ub=10; %upperbound of data
step=.1; %step-size through data
a=.03;
la=1482/120000; %1482 is speed of sound in water and 120kHz
ep1=.02;
ep2=.1;
x=logspace(log10(lb),log10(ub),100)';
r_sq_des=0.90; %desired value of r^2 for the fit of data without noise present
y=abs(sin(a/la*pi*x.*(sqrt(1+(1./x).^2)-1)));
N=2*rand(size(x))-1;
Ghat=(1+ep1*N).*y+ep2*N;
V=[x.^0];
xs=(lb:.01:ub)';
Vfit=[xs.^0];
for i=1:1:20%length(x)-1
V = [x.^i V];
Vfit = [xs.^i Vfit];
[Q,R]=qr(V,0);
c = R\(Q'*y);
yFit=Vfit*c;
plot(x,y,'o',xs,yFit)
axis([0 10 0 1])
drawnow
pause
end
1. What I WANT to do:
(i) Use input n to generate an n*n cartesian grid
[x y] = meshgrid(linspace(-1,1,n));
(ii) Generate polar coordinates
[theta r] = cart2pol(x,y);
(iii) Evaluate a function in cylindrical coordinates
z = f(theta,r);
(iv) Plot the result using (say) pcolor (or surf, or anything)
pcolor(x,y,abs(z).^2) %The function is complex, a Laguerre-Gauss to be exact.
2. What I CAN do... The only way I can get the plots to work is by starting with my polar parameters and working back to cartesian from there:
(i) Define parameters
r=linspace(0,1,n); theta=linspace(0,2*pi,n);
(ii) Create both grids and evaluate f
[theta r]=meshgrid(theta,r);
[x y]=pol2cart(theta,r);
z=f(theta,r);
(iii) Plot
pcolor(x,y,abs(z).^2)
The PROBLEM is that now my grid is circular, and I would like to evaluate the function everywhere ON A RECTANGULAR grid (because my analysis depends on having square pixel arrays). The reiterate, using method 2 above, I get a circular plot circumscribed in a square; imagine a black circle with white along the edges... but I WANT to evaluate the function in this "white" region. HOWEVER, using method 1 does NOT work -- the function is all messed up when I plot (Just google Laguerre-Gauss modes to see what the plots should look like).
I want to be able to start with a rect grid and assign every point a polar coordinate, instead of start with polar coordinates and assign them all cartesian points.
I've been messing with this on an off for a long time, and I can't figure out how to get around this seemingly simple issue.
Edit 1
It seems that the problem lies in how the coordinate matrices are generated. Below I've posted screen-shots of a simple 3by3 example illustrating how approach 1 and approach 2 generate different numbers.
How to make these numbers compatible?
I have no reputation points so I cannot upload the images directly... links below show the 3by3 example... see comments for links to actual images of the Laguerre-Gauss plots I'm trying to make...
apply cart2pol
apply pol2cart
Edit 2
Currently, the result of approach (1.) gives wrong results, as shown here:
desired approach, wrong result
The second approach gives the right images, unfortunately it's only a circle and not the entire square. It is shown here:
implemented approach, limited result
3D plots of both approaches are shown here - only the colorful part of the top figure is correct.
Edit 3
Here is a screenshot of the function f which is being used above. Note, that it asks for more input parameters than just r,theta. Typical values are:
w0 = 0.5;
p = 0;
l = 5;
The function C gives a normalization and L are Laguerre polynomials. Both of these functions have been thoroughly tested and yield the expected results.
Edit 4
Here is enough code to run my example z=U(0,5,r,phi,w0)+U(0,-5,r,phi,w0); explicitly. The plot itself is given by pcolor(x,y,abs(z).^2).
Note that the Lpl() function is inserted as a comment. This will have to be saved as its own m-file for the U function to run properly.
%% Laguerre-Gauss Modes U = U(p,l,r,phi,w0)
% Source: OAM theory paper section 2.A eqn 1.
% Assuming POLAR coordinates and evaluating AT beam waist.
% -- That is, z=0 for w(z)=w0(sqrt(1+z/zR))
% ---- ie, w(0) = w0
% Assuming z=0 also renders the Gouy phase arctan(z/zR) irrelevant.
% Note: Rayleigh Range zR is not explicitly defined because z=0 --> it is irrelevant too.
% Since zR is the only wavelength dependent term, wavelength also doesn't
% matter.
function out = U(p,l,r,phi,w0)
%Function handles for clarity
e = #(x) exp(x);
C = #(p,l) sqrt((2*factorial(p))/(pi*factorial(p+abs(l))));
L = #(p,l,z) Lpl(p,l,z);
%% Lpl() FUNCTION
% function out = Lpl(p,l,z)
%
% l=abs(l);
% LL=0;
% for mm=1:p+1
% m=mm-1;
% L=LL;
% LL= L+((-1)^m)*(factorial(p+l)/(factorial(p-m)*factorial(l+m)*factorial(m)))*(z.^m);
% end
% out = LL;
%%
out = (C(p,l)/w0)*...
(((sqrt(2).*r)/w0)^abs(l))*...
(e((-r.^2)/w0^2))*...
(L(p,l,((2.*r.^2)/w0^2)))*...
(e((-1)*1i*l.*phi)); ``
Edit
The answer was rewritten based on the code provided in Edit 4 of the question.
I think the trouble stems from the function U. You don't apply element wise operations to all parts of the equation. If you change it to:
out = (C(p,l)./w0).* ... % here it's a .* instead of *
(((sqrt(2).*r)./w0).^abs(l)).* ... % here it's a .* instead of *
(e((-r.^2)./w0.^2)).* ... % here it's a .* instead of *
(L(p,l,((2.*r.^2)./w0.^2))).* ... % here it's a .* instead of *
(e((-1)*1i*l.*phi));
You get the following two results, shown below.
This figure used an input of cartesian coordinates:
And this figure used the polar coordinates:
The "coarser" resolution in the second figure is due to the less suitable resolution of the grid. But in essence you resolve the same features.
I have various plots (with hold on) as show in the following figure:
I would like to know how to find equations of these six curves in Matlab. Thanks.
I found interactive fitting tool in Matlab simple and helpful, though somewhat limited in scope:
The graph above seems to be linear interpolation. Given vectors X and Y of data, where X contains the arguments and Y the function points, you could do
f = interp1(X, Y, x)
to get the linearly interpolated value f(x). For example if the data is
X = [0 1 2 3 4 5];
Y = [0 1 4 9 16 25];
then
y = interp1(X, Y, 1.5)
should give you a very rough approximation to 1.5^2. interp1 will match the graph exactly, but you might be interested in fancier curve-fitting operations, like spline approximations etc.
Does rxns stand for reactions? In that case, your curves are most likely exponential. An exponential function has the form: y = a*exp(b * x) . In your case, y is the width of mixing zone, and x is the time in years. Now, all you need to do is run exponential regression in Matlab to find the optimal values of parameters a and b, and you'll have your equations.
The advice, though there might be better answer, from me is: try to see the rate of increase in the curve. For example, cubic is more representative than quadratic if the rate of increase seems fast and find the polynomial and compute the deviation error. For irregular curves, you might try spline fitting. I guess there is also a toolbox in matlab for spline fitting.
There is a way to extract information with the current figure handle (gcf) from you graph.
For example, you can get the series that were plotted in a graph:
% Some figure is created and data are plotted on it
figure;
hold on;
A = [ 1 2 3 4 5 7] % Dummy data
B = A.*A % Some other dummy data
plot(A,B);
plot(A.*3,B-1);
% Those three lines of code will get you series that were plotted on your graph
lh=findall(gcf,'type','line'); % Extract the plotted line from the figure handle
xp=get(lh,'xdata'); % Extract the Xs
yp=get(lh,'ydata'); % Extract the Ys
There must be other informations that you can get from the "findall(gcf,...)" methods.