the following matlab code is a regression loop:
for j=1:size(X,2)
IdentityVector=ones((size(t,1)-1),1);
Y=X((2:end),j);
if j==1
X2=[IdentityVector,X((2:end),((j+1):end)),Diff1X];
elseif j>1 & j<size(X,2)
X2=[IdentityVector,X((2:end),(1:(j-1))),X((2:end),((j+1):end)),Diff1X];
elseif j==size(X,2)
X2=[IdentityVector,X((2:end),(1:(j-1))),Diff1X];
end
[b(:,j)]= regress(Y,X2);
end
this works fine for the beta estimate as it for each j the dimensions of the beta are adjusting accordingly, although if I request from the estimation some additional features/statistics e.g. [b,bint,r,rint,stats] = regress(y,X) the adjustments i have tried for each j do not work. Any help?
My best guess is that you are treating all the outputs as vectors, which they are not. If you read doc regress you will see that bint is a p-by-2, r is n-by-2, so on and so forth. That means you can't assign bint(:,j) because bint itself is a matrix. Instead try cells.
[b{j}, bint{j}, r{j}, rint{j}, stats{j}]= regress(Y,X2);
Related
This is my first go with ML (and Matlab) and I'm following "Learning From Data" by Yaser S. Abu-Mostafa.
I'm trying to implement the Perceptron algorithm, after trying to go through the pseudocode, using other people's solutions I can't seem to fix my problem (I went through other threads too).
The algorithm separates the data fine, it works. However, I want to plot a single line, but it seems as it separates them in a way so the '-1' cluster is divided to a second cluster or more.
This is the code:
iterations = 100;
dim = 3;
X1=[rand(1,dim);rand(1,dim);ones(1,dim)]; % class '+1'
X2=[rand(1,dim);1+rand(1,dim);ones(1,dim)]; % class '-1'
X=[X1,X2];
Y=[-ones(1,dim),ones(1,dim)];
w=[0,0,0]';
% call perceptron
wtag=weight(X,Y,w,iterations);
% predict
ytag=wtag'*X;
% plot prediction over origianl data
figure;hold on
plot(X1(1,:),X1(2,:),'b.')
plot(X2(1,:),X2(2,:),'r.')
plot(X(1,ytag<0),X(2,ytag<0),'bo')
plot(X(1,ytag>0),X(2,ytag>0),'ro')
legend('class -1','class +1','pred -1','pred +1')
%Why don't I get just one line?
plot(X,Y);
The weight function (Perceptron):
function [w] = weight(X,Y,w_init,iterations)
%WEIGHT Summary of this function goes here
% Detailed explanation goes here
w = w_init;
for iteration = 1 : iterations %<- was 100!
for ii = 1 : size(X,2) %cycle through training set
if sign(w'*X(:,ii)) ~= Y(ii) %wrong decision?
w = w + X(:,ii) * Y(ii); %then add (or subtract) this point to w
end
end
sum(sign(w'*X)~=Y)/size(X,2); %show misclassification rate
end
I don't think the problem is in the second function but I added it regardless
I'm pretty sure the algorithm separates it to more than one cluster but I can't tell why most of the learning I've done so far was math and theory and not actual coding so I'm probably missing something obvious..
I've calculated coefficients an, bn (100, T=2*pi) in c++ and checked that they are correct using few sources. Now i try to generate Fourier series graph for given example function in Scilab:
(x+2)*abs(cos(2*x*(x-pi/6)))
M=csvRead(filename, ";", [], 'double')
n=size(M,1)
for i = 1:n
A(i)=M(i)
B(i)=M(i + n)
end
function series=solution(x)
series=A(1)/2;
for i = 2:n
series=series+(A(i)*cos(i*x)+B(i)*sin(i*x));
end
endfunction
function series=solution2(x)
series=(x+2).*abs(cos(2.*x.*(x-%pi/6)));
endfunction
x = -%pi:%pi/100:%pi
plot2d(x, solution(x), 3)
x2 = -%pi:%pi/100:%pi
plot2d(x2, solution2(x2), 4)
Here is the result:
It clearly looks that tendency is ok but the beginning and the end of the period are wrong (reversed?). Do you see any issues in Scilab code? What could cause the problem - values in sin/cos in function solution(x)? Should i provide an, bn values and check for miscalculation there?
I don't know how did you calculated your A & B coefficients, but I assume that you used the usual notations to get the first line of the below formula:
Thus n starts from 1. Since Scilab starts vector indexing from 1, you correctly made your loop from 2, but forgot to compensate for this "offset".
Your function should be something like this:
function series=solution(x)
series=A(1)/2;
for i = 2:n
series=series+(A(i)*cos((i-1)*x)+B(i)*sin((i-1)*x));
end
endfunction
Since you didn't provided A & B, I can not check the result.
Additional note: Syntactically more correct if you explicitly define all input variables in a function, like this:
function series=solution(x,A,B)
This way you may be sure that your input is not changed somewhere else in the code.
I am trying to solve
argmin||W_vector*FT^-1(Ax)-W_vector*B||
using lsqr with a function handle and without any initial guess (W_vector is a weighting vector). I thought that lsqr would have performed the first step computing x by C'b (call C=W*FT^-1(A)) using the vector b=W_vector.*B I passed to the function. This is the code:
b=W_vector.*B;
[x, flag]=lsqr(#(x1,modo)FUNC(x1,W_vector,A,modo),b,tol,maxit);
function [result, modo]=FUNC(x1,W_vector,A,modo)
%Computes y=A*x for modo='notransp'
%Computes y=A'*x for modo='transp'
switch modo
case 'notransp'
res=A*x1;
R1=reshape(res,norient,dim(1)*dim(2)*dim(3));
for co=1:norient
R2(:,:,:,co)=reshape(R1(co,:),dim(1),dim(2),dim(3));
FR(:,:,:,co)=ifftn(R2(:,:,:,co));
aux=FR(:,:,:,co);
R3(co,:)=aux(:).';
end
result=W_vector.*R3(:);
case 'transp'
R1=reshape(x1./(W_vector+eps),norient,dim(1)*dim(2)*dim(3));
for co=1:norient
R2(:,:,:,co)=reshape(R1(co,:),dim(1),dim(2),dim(3));
FR(:,:,:,co)=fftn(R2(:,:,:,co));
aux=FR(:,:,:,co);
R3(co,:)=aux(:).';
end
result=A'*R3(:);
end
end
When I checked the result of R1=reshape(x1./(W_vector+eps),norient,dim(1)*dim(2)*dim(3)); in case 'transp' in the first iteration, I found out that x1 is not equal to b (after reshape, they are images, so I can see that look qualitatively the same but with very different values). I would like to understand how lsqr works because I am having some problems with weightings. Thanks for your help.
Im trying to divide a Gaussian distribution into equiprobable parts. Im using the following code:
function main
mu=100;
sigma=2;
n=100;
k=3;
samp = mu + sigma.*randi([20,100],1,n);
%hist(samp)
v=optim_m2(samp,k)
end
function v=optim_m(d,k)
v=-inf;
mu=mean(d);
sigma=var(d);
for i=1:k
[x, ~] = fminbnd(#(x) (0.5*( 1+erf( (x-mu)/(((sigma^2) )^0.5 ) ) ) -i/k )^2 ,mu-3*sigma,mu+3*sigma);
v=[v,x];
end
end
I get some pretty strange results such as negative values and so on. If I use the same function but with small natural numbers everything seems to work fine.
PS. Im a NOOB so ...:) dont judge too harsh
If you can use Matlab commands, you can simply use:
boundaries = norminv(1/8:1/8:7/8,mu,sigma);
Once again I have a problem with the Gauss-Seidel Method in Matlab. Here it is:
function [x] = ex1_3(A,b)
format long
sizeA=size(A,1);
x=zeros(sizeA,1);
%Just a check for the conditions of the Gauss-Seidel Method (if it has dominant diagonal)
for i=1:sizeA
sum=0;
for j=1:sizeA
if i~=j
sum=sum+abs(A(i,j));
end
end
if abs(A(i,i))<sum
fprintf('\nGauss-Seidel''s conditions not met!\n');
return
end
end
%Actual Gauss-Seidel Method
max_temp=10^(-6); %Pass first iteration
while max_temp>(0.5*10^(-6))
xprevious=x;
for i=1:sizeA
x(i,1)=b(i,1);
for j=1:sizeA
if i~=j
x(i,1)=x(i,1)-A(i,j)*x(j,1);
end
end
x(i,1)=x(i,1)/A(i,i);
end
x
%Calculating infinite norm of vector x-xprevious
temp=x-xprevious;
max_temp=temp(1,1);
for i=2:sizeA
if abs(temp(i,1))>max_temp
max_temp=abs(temp(i,1));
end
end
end
It actually works fine for a 100x100 matrix or smaller. However, my tutor wants it to work for 100000x100000 matrices. At first it was difficult to even create the matrix itself, but I managed to do it with a little help from here:
Matlab Help Center
Now, I call the ex1_3 function with A as a parameter, but it goes really slow. Actually it never ends. How can I make it work?
Here's my code for creating the specific matrix my tutor wanted:
The important part is just that it meets these conditions:
A(i; i) = 3, A(i - 1; i) = A(i; i + 1) = -1 n=100000
b=ones(100000,1);
b(1,1)=2;
b(100000,1)=2;
i=zeros(299998,1); %Matrix with the lines that we want to put nonzero elements
j=zeros(299998,1); %Matrix with the columns that we want to put nonzero elements
s=zeros(299998,1); %Matrix with the nonzero elements.
number=1;
previousNumberJ=0;
numberJ=0;
for k=1:299998 %Our index in i and j matrices
if mod((k-1),3)==0
s(k,1)=3;
else
s(k,1)=-1;
end
if k==1 || k==2
i(k,1)=1;
j(k,1)=k;
elseif k==299997 || k==299998
i(k,1)=100000;
j(k,1)=(k-200000)+2;
else
if mod(k,3)==0
number=number+1;
numberJ=previousNumberJ+1;
previousNumberJ=numberJ;
end
i(k,1)=number;
j(k,1)=numberJ;
numberJ=numberJ+1;
end
end
A=sparse(i,j,s); %Creating the sparse array
x=ex1_3(A,b);
the for loop works very slowly in Matlab, perhaps you may want to try the matrix form of the iteration:
function x=gseidel(A,b)
max_temp=10^(-6); %Pass first iteration
x=b;
Q=tril(A);
r=b-A*x;
for i=1:100
dx=Q\r;
x=x+1*dx;
r=b-A*x;
% convergence check
if all(abs(r)<max_temp) && all(abs(dx)<max_temp), return; end
end
For your A and b, it only takes 16 steps to converge.
tril extracts the lower triangular part of A, you can also obtain this Q when you build up the matrix. Since Q is already the triangular matrix, you can solve the equation Q*dx=r very easily if you are not allowed to use \ function.