I just want to make some predictions using these calculations but I'm getting this error: Error using * Incorrect dimensions for matrix multiplication. Check that the number of columns in the first matrix matches the number of rows in the second matrix. To operate on each element of the matrix individually, use TIMES (.*) for elementwise multiplication.
Error in main (line 64) r=(pn')*(best);
the dataset
Main driver
clear; clc;
%load dataset
ds = load('ex1data1.txt');
%split x/y
x = ds(:,1); % Examples
y = ds(:,2);
n=1;
m=length(y);
format long;
b1 = x\y; %intercept
%plot lineer regression
% Top plot
yCalc1 = b1*x;
ax1 = nexttile;
scatter(ax1,x,y,'o');
hold on
plot(ax1,x,yCalc1);
title(ax1,'Linear Regression Relation Between Profit & Truck')
ax1.FontSize = 14;
ax1.XColor = 'red';
ylabel('profit of a food truck');
xlabel('population of a city');
grid on
%normalise
[x,maxs,mins]=normalize(x);
%add column with ones - help hyphothesizs
xo=[ones(m,1),x];
%gradient descent
repeat=1500;
lrate=0.01;
thetas=zeros(2,1);
[best,costs] = gradientDescent(repeat,lrate,thetas,xo,y,m);
% plot 𝑱(𝜽) vs iteration
ax2 = nexttile;
scatter(ax2,costs,1:repeat)
title(ax2,' 𝑱(𝜽) vs iteration')
grid(ax2,'on')
ylabel('Iteration');
xlabel('J(𝜽)');
%prediction
p=[7;7];
pn=(p-maxs')./(maxs'-mins');
pn = [1;pn];
r=(pn')*(best);
Gradient Descent
function [thetas,costs] = gradientDescent(repeat,lrate,thetas,xo,y,m)
costs = zeros(repeat,1);
for r = 1:repeat
hc=xo*thetas - y;
tempintercept=sum(hc.*xo);
thetas = thetas - (lrate * (1/m)) * tempintercept';
costs(r)=cost(thetas,xo,y);
end
end
Normalize
function [x,maxs,mins] = normalize (x)
x=(x-max(x))./(max(x)/min(x));
maxs=max(x);
mins=min(x);
end
Related
I'm trying to implement stochastic gradient descent in MATLAB however I am not seeing any convergence. Mini-batch gradient descent worked as expected so I think that the cost function and gradient steps are correct.
The two main issues I am having are:
Randomly shuffling the data in the training set before the
for-loop
Selecting one example at a time
Here is my MATLAB code:
Generating Data
alpha = 0.001;
num_iters = 10;
xrange =(-10:0.1:10); % data lenght
ydata = 5*(xrange)+30; % data with gradient 2, intercept 5
% plot(xrange,ydata); grid on;
noise = (2*randn(1,length(xrange))); % generating noise
target = ydata + noise; % adding noise to data
f1 = figure
subplot(2,2,1);
scatter(xrange,target); grid on; hold on; % plot a scttaer
title('Linear Regression')
xlabel('xrange')
ylabel('ydata')
tita0 = randn(1,1); %intercept (randomised)
tita1 = randn(1,1); %gradient (randomised)
% Initialize Objective Function History
J_history = zeros(num_iters, 1);
% Number of training examples
m = (length(xrange));
Shuffling data, Gradient Descent and Cost Function
% STEP1 : we shuffle the data
data = [ xrange, ydata];
data = data(randperm(size(data,1)),:);
y = data(:,1);
X = data(:,2:end);
for iter = 1:num_iters
for i = 1:m
x = X(:,i); % STEP2 Select one example
h = tita0 + tita1.*x; % building the estimated %Changed to xrange in BGD
%c = (1/(2*length(xrange)))*sum((h-target).^2)
temp0 = tita0 - alpha*((1/m)*sum((h-target)));
temp1 = tita1 - alpha*((1/m)*sum((h-target).*x)); %Changed to xrange in BGD
tita0 = temp0;
tita1 = temp1;
fprintf("here\n %d; %d", i, x)
end
J_history(iter) = (1/(2*m))*sum((h-target).^2); % Calculating cost from data to estimate
fprintf('Iteration #%d - Cost = %d... \r\n',iter, J_history(iter));
end
On plotting the cost vs iterations and linear regression graphs, the MSE settles (local minimum?) at around 420 which is wrong.
On the other hand if I re-run the exact same code however using batch gradient descent I get acceptable results. In batch gradient descent I am changing x to xrange:
Any suggestions on what I am doing wrong?
EDIT:
I also tried selecting random indexes using:
f = round(1+rand(1,1)*201); %generating random indexes
and then selecting one example:
x = xrange(f); % STEP2 Select one example
Proceeding to use x in the hypothesis and GD steps also yield a cost of 420.
First we need to shuffle the data correctly:
data = [ xrange', target'];
data = data(randperm(size(data,1)),:);
Next we need to index X and y correctly:
y = data(:,2);
X = data(:,1);
Then during gradient descent I need to update based on a single value not on target, like so:
tita0 = tita0 - alpha*((1/m)*((h-y(i))));
tita1 = tita1 - alpha*((1/m)*((h-y(i)).*x));
Theta converges to [5, 30] with the changes above.
I have conducted a linear SVM on a large dataset, however in order to reduce the number of dimensions I performed a PCA, than conducted the SVM on a subset of the component scores (the first 650 components which explained 99.5% of the variance). Now I want to plot the decision boundary in the original variable space using the beta weights and bias from the SVM created in PCA space. But I can't figure out how to project the bias term from the SVM into the original variable space. I've written a demo using the fisher iris data to illustrate:
clear; clc; close all
% load data
load fisheriris
inds = ~strcmp(species,'setosa');
X = meas(inds,3:4);
Y = species(inds);
mu = mean(X)
% perform the PCA
[eigenvectors, scores] = pca(X);
% train the svm
SVMModel = fitcsvm(scores,Y);
% plot the result
figure(1)
gscatter(scores(:,1),scores(:,2),Y,'rgb','osd')
title('PCA space')
% now plot the decision boundary
betas = SVMModel.Beta;
m = -betas(1)/betas(2); % my gradient
b = -SVMModel.Bias; % my y-intercept
f = #(x) m.*x + b; % my linear equation
hold on
fplot(f,'k')
hold off
axis equal
xlim([-1.5 2.5])
ylim([-2 2])
% inverse transform the PCA
Xhat = scores * eigenvectors';
Xhat = bsxfun(#plus, Xhat, mu);
% plot the result
figure(2)
hold on
gscatter(Xhat(:,1),Xhat(:,2),Y,'rgb','osd')
% and the decision boundary
betaHat = betas' * eigenvectors';
mHat = -betaHat(1)/betaHat(2);
bHat = b * eigenvectors';
bHat = bHat + mu; % I know I have to add mu somewhere...
bHat = bHat/betaHat(2);
bHat = sum(sum(bHat)); % sum to reduce the matrix to a single value
% the correct value of bHat should be 6.3962
f = #(x) mHat.*x + bHat;
fplot(f,'k')
hold off
axis equal
title('Recovered feature space')
xlim([3 7])
ylim([0 4])
Any guidance on how I'm calculating bHat incorrectly would be much appreciated.
Just in case anyone else comes across this problem, the solution is the bias term can be used to find the y-intercept, b = -SVMModel.Bias/betas(2). And the y-intercept is just another point in space [0 b] which can be recovered/unrotated by inverse transforming it through the PCA. This new point can then be used to solve the linear equation y = mx + b (i.e., b = y - mx). So the code should be:
% and the decision boundary
betaHat = betas' * eigenvectors';
mHat = -betaHat(1)/betaHat(2);
yint = b/betas(2); % y-intercept in PCA space
yintHat = [0 b] * eigenvectors'; % recover in original space
yintHat = yintHat + mu;
bHat = yintHat(2) - mHat*yintHat(1); % solve the linear equation
% the correct value of bHat is now 6.3962
I cannot get fgoalattain to work to save my life. It just comes with the following errors that I have not figured out how to fix despite several hours of messing with the highlighted areas. The issues with interpolation work fine until I use fgolattain and the fgoalattain is used as specified in documentation.
Help?
Error using griddedInterpolant The number of input coordinate arrays
does not equal the number of dimensions (NDIMS) of these arrays.
Error in Scan (line 5) F = griddedInterpolant(xx',yy',a','cubic'); %
Prepare image for interpolation
Error in #(vals)Scan(vals,xx,yy,nr,m,d)
Error in goalcon (line 63)
f = feval(funfcn{3},x,varargin{:});
Error in fgoalattain (line 399)
[ctmp,ceqtmp] = feval(cfun{3},xnew,extravarargin{:});
Error in Cu (line 24) xz =
fgoalattain(#(vals)Scan(vals,xx,yy,nr,m,d),x0,goal,weight);
clear all;
close all;
clc
% Create ring
x = -31:31;
[xx,yy] = meshgrid(x,x); % 2-D arrays with x and y-coord. respectively
r = sqrt(xx.^2+yy.^2)/31; % Distance to center of grid
a = 2*exp(max(-40.0,-6000.0*(r-0.7).^4)).*(1.1-0.75*yy/31.0);
a = a/max(max(a)); % Normalize a so in range [0,1]
% Create the scan. This time specify size as desired
nr = 60; % Number of rays for each angle
m = 66; % Number of angles to use
%scan function
sc = Scan(a,xx,yy,nr,m);
%applying optimization tool
x0 = rand();
goal = sc;
weight = abs(sc);
xz = fgoalattain(#(vals)Scan(vals,xx,yy,nr,m),x0,goal,weight);
figure
mesh(xz)
drawnow
figure
pcolor(xz)
drawnow
with Scan function written as:
function [ sc ] = Scan( a, xx, yy, nr, m)
d = zeros(nr,m); % Array to hold the scan vectors
xt = linspace(-31,31,nr);
[xx1,yy1] = meshgrid(xt,xt);
F = griddedInterpolant(xx',yy',a','cubic'); % Prepare image for interpolation
for j=0:m-1 % Loop over m angles
th = pi*j/m; st = sin(th); ct = cos(th);
xr = xx1*ct-yy1*st; % Generate the x1,x2-coordinates for
yr = xx1*st+yy1*ct; % the grid points in the xp1,xp2 grid.
s = F(xr',yr')'; % Interpolate
d(:,j+1) = sum(s,2); % Sum along each ray
end
sc = d/max(max(d)); % Normalize so max value = 1.
end
I have a problem multiplying a vector times the inverse of a matrix in Matlab. The code I am using is the following:
% Final Time
T = 0.1;
% Number of grid cells
N=20;
%N=40;
L=20;
% Delta x
dx=1/N
% define cell centers
%x = 0+dx*0.5:dx:1-0.5*dx;
x = linspace(-L/2, L/2, N)';
%define number of time steps
NTime = 100; %NB! Stability conditions-dersom NTime var 50 ville en fått helt feil svar pga lambda>0,5
%NTime = 30;
%NTime = 10;
%NTime = 20;
%NTime = 4*21;
%NTime = 4*19;
% Time step dt
dt = T/NTime
% Define a vector that is useful for handling teh different cells
J = 1:N; % number the cells of the domain
J1 = 2:N-1; % the interior cells
J2 = 1:N-1; % numbering of the cell interfaces
%define vector for initial data
u0 = zeros(1,N);
L = x<0.5;
u0(L) = 0;
u0(~L) = 1;
plot(x,u0,'-r')
grid on
hold on
% define vector for solution
u = zeros(1,N);
u_old = zeros(1,N);
% useful quantity for the discrete scheme
r = dt/dx^2
mu = dt/dx;
% calculate the numerical solution u by going through a loop of NTime number
% of time steps
A=zeros(N,N);
alpha(1)=A(1,1);
d(1)=alpha(1);
b(1)=0;
c(1)=b(1);
gamma(1,2)=A(1,2);
% initial state
u_old = u0;
pause
for j = 2:NTime
A(j,j)=1+2*r;
A(j,j-1)=-(1/dx^2);
A(j,j+1)=-(1/dx^2);
u=u_old./A;
% plotting
plot(x,u,'-')
xlabel('X')
ylabel('P(X)')
hold on
grid on
% update "u_old" before you move forward to the next time level
u_old = u;
pause
end
hold off
The error message I get is:
Matrix dimensions must agree.
Error in Implicit_new (line 72)
u=u_old./A;
My question is therefore how it is possible to perform u=u_old*[A^(-1)] in Matlab?
David
As knedlsepp said, v./A is the elementwise division, which is not what you wanted. You can use either
v/A provided that v is a row vector and its length is equal to the number of columns in A. The result is a row vector.
A\v provided that v is a column vector and its length is equal to the number of rows in A
The results differ only in shape: v/A is the transpose of A'\v'
I have following code for calculating PCA in Matlab:
train_out = train';
test_out = test';
% subtract off the mean for each dimension
mn = mean(train_out,2);
train_out = train_out - repmat(mn,1,train_size);
test_out = test_out - repmat(mn,1,test_size);
% calculate the covariance matrix
covariance = 1 / (train_size-1) * train_out * train_out';
% find the eigenvectors and eigenvalues
[PC, V] = eig(covariance);
% extract diagonal of matrix as vector
V = diag(V);
% sort the variances in decreasing order
[junk, rindices] = sort(-1*V);
V = V(rindices);
PC = PC(:,rindices);
% project the original data set
out = PC' * train_out;
train_out = out';
out = PC' * test_out;
test_out = out';
Train and test matrix have observations in rows and feature variables in columns. When I perform classification on original data (without PCA) I get much better results than with PCA, even when I keep all dimensions. When I tried doing PCA directly on the whole dataset (train + test) I noticed correlation between these new principal components and previous ones are either near 1 or near -1 which I find strange. I am probably doing something wrong but just can't figure it out.
The code is correct, however using princomp function my be easier:
train_out=train; % save original data
test_out=test;
mn = mean(train_out);
train_out = bsxfun(#minus,train_out,mn); % substract mean
test_out = bsxfun(#minus,test_out,mn);
[coefs,scores,variances] = princomp(train_out,'econ'); % PCA
pervar = cumsum(variances) / sum(variances);
dims = max(find(pervar < var_frac)); % var_frac - e.g. 0.99 - fraction of variance explained
train_out = train_out*coefs(:,1:dims); % dims - keep this many dimensions
test_out = test_out*coefs(:,1:dims); % result is in train_out and test_out