Directional artifacts in MATLAB randn arrays? - matlab

I'm generating 3d fractal noise in MATLAB using a variety of methods. It's working relatively well, but I'm having an issue where I see vertical striping artifacts in my noise. This happens regardless of what data type or resolution I use.
Edit: I figured it out. The solution is posted as an answer below. Thanks everyone for your thoughts and guidance!
expo = 2^6;
dims = [expo,expo,expo];
beta = -4.5;
render = randnd(beta, dims); % Create volumetric fractal
render = render - min(render); % Set floor to zero
render = render ./ max(render); % Set ceiling to one
%render = imbinarize(render); % BW Threshold option
render = render .* 255; % For greyscale
slicer = 1; % Turn on image slicer/saver
i = 0; % Page counter
format = '.png';
imagename = '___testDump/slice';
imshow(render(:,:,1),[0 255]); %Single test image
if slicer == 1
for c = 1:length(render)
i = i+1;
pagenumber = num2str(i);
filename = [imagename, pagenumber, format];
imwrite(uint8(render(:,:,i)),filename)
end
end
function X = randnd(beta,varargin)
seed = 999;
rng(seed); % Set seed
%% X = randnd(beta,varargin)
% Based on similar functions by Jon Yearsley and Hristo Zhivomirov
% Written by Marcin Konowalczyk
% Timmel Group # Oxford University
%% Parse the input
narginchk(0,Inf); nargoutchk(0,1);
if nargin < 2 || isempty(beta); beta = 0; end % Default to white noise
assert(isnumeric(beta) && isequal(size(beta),[1 1]),'''beta'' must be a number');
assert(-6 <= beta && beta <= 6,'''beta'' out of range'); % Put on reasonable bounds
%% Generate N-dimensional white noise with 'randn'
X = randn(varargin{:});
if isempty(X); return; end; % Usually happens when size vector contains zeros
% Squeeze prevents an error if X has more than one leading singleton dimension
% This is a slight deviation from the pure functionality of 'randn'
X = squeeze(X);
% Return if white noise is requested
if beta == 0; return; end;
%% Generate corresponding N-dimensional matrix of multipliers
N = size(X);
% Create matrix of multipliers (M) of X in the frequency domain
M = [];
for j = 1:length(N)
n = N(j);
if (rem(n,2)~=0) % if n is odd
% Nyquist frequency bin does not show up in odd-numbered fft
k = ifftshift(-(n-1)/2:(n-1)/2);
else
k = ifftshift(-n/2:n/2-1);
end
% Spectral multipliers
m = (k.^2)';
if isempty(M);
M = m;
else
% Create the permutation vector
M_perm = circshift(1:length(size(M))+1,[0 1]);
% Permute a singleton dimension to the beginning of M
M = permute(M,M_perm);
% Add m along the first dimension of M
M = bsxfun(#plus,M,m);
end
end
% Reverse M to match X (since new dimensions were being added form the left)
M = permute(M,length(size(M)):-1:1);
assert(isequal(size(M),size(X)),'Bad programming error'); % This should never occur
% Shape the amplitude multipliers by beta/4 which corresponds to shaping the power by beta
M = M.^(beta/4);
% Set the DC component to zero
M(1,1) = 0;
%% Multiply X by M in frequency domain
Xstd = std(X(:));
Xmean = mean(X(:));
X = real(ifftn(fftn(X).*M));
% Force zero mean unity standard deviation
X = X - mean(X(:));
X = X./std(X(:));
% Restore the standard deviation and mean from before the spectral shaping.
% This ensures the random sample from randn is truly random. After all, if
% the mean was always exactly zero it would not be all that random.
X = X + Xmean;
X = X.*Xstd;
end

Here is my solution:
My "min/max" code (lines 6 and 7) was bad. I wanted to divide all values in the matrix by the single largest value in the matrix so that all values would be between 0 and 1. Because I used max() improperly, I was stepping through the max value of each column and using that as my divisor; thus the vertical stripes.
In the end this is what my code looks like. X is the 3 dimensional matrix:
minVal = min(X,[],'all'); % Get the lowest value in the entire matrix
X = X - minVal; % Set min value to zero
maxVal = max(X,[],'all'); % Get the highest value in the entire matrix
X = X ./ maxVal; % Set max value to one

Related

How to make probability matrix with each row elements sum 1 and each row have uniform probability in matlab

I found matlab file (https://in.mathworks.com/matlabcentral/fileexchange/9700-random-vectors-with-fixed-sum) which generates probability matrix with uniform probability and each column has sum 1. file is as follow
function [x,v] = randfixedsum(n,m,s,a,b)
%[x,v] = randfixedsum(n,m,s,a,b)
%
% This generates an n by m array x, each of whose m columns
% contains n random values lying in the interval [a,b], but
% subject to the condition that their sum be equal to s. The
% scalar value s must accordingly satisfy n*a <= s <= n*b. The
% distribution of values is uniform in the sense that it has the
% conditional probability distribution of a uniform distribution
% over the whole n-cube, given that the sum of the x's is s.
%
% The scalar v, if requested, returns with the total
% n-1 dimensional volume (content) of the subset satisfying
% this condition. Consequently if v, considered as a function
% of s and divided by sqrt(n), is integrated with respect to s
% from s = a to s = b, the result would necessarily be the
% n-dimensional volume of the whole cube, namely (b-a)^n.
%
% This algorithm does no "rejecting" on the sets of x's it
% obtains. It is designed to generate only those that satisfy all
% the above conditions and to do so with a uniform distribution.
% It accomplishes this by decomposing the space of all possible x
% sets (columns) into n-1 dimensional simplexes. (Line segments,
% triangles, and tetrahedra, are one-, two-, and three-dimensional
% examples of simplexes, respectively.) It makes use of three
% different sets of 'rand' variables, one to locate values
% uniformly within each type of simplex, another to randomly
% select representatives of each different type of simplex in
% proportion to their volume, and a third to perform random
% permutations to provide an even distribution of simplex choices
% among like types. For example, with n equal to 3 and s set at,
% say, 40% of the way from a towards b, there will be 2 different
% types of simplex, in this case triangles, each with its own
% area, and 6 different versions of each from permutations, for
% a total of 12 triangles, and these all fit together to form a
% particular planar non-regular hexagon in 3 dimensions, with v
% returned set equal to the hexagon's area.
%
% Roger Stafford - Jan. 19, 2006
% Check the arguments.
if (m~=round(m))|(n~=round(n))|(m<0)|(n<1)
error('n must be a whole number and m a non-negative integer.')
elseif (s<n*a)|(s>n*b)|(a>=b)
error('Inequalities n*a <= s <= n*b and a < b must hold.')
end
% Rescale to a unit cube: 0 <= x(i) <= 1
s = (s-n*a)/(b-a);
% Construct the transition probability table, t.
% t(i,j) will be utilized only in the region where j <= i + 1.
k = max(min(floor(s),n-1),0); % Must have 0 <= k <= n-1
s = max(min(s,k+1),k); % Must have k <= s <= k+1
s1 = s - [k:-1:k-n+1]; % s1 & s2 will never be negative
s2 = [k+n:-1:k+1] - s;
w = zeros(n,n+1); w(1,2) = realmax; % Scale for full 'double' range
t = zeros(n-1,n);
tiny = 2^(-1074); % The smallest positive matlab 'double' no.
for i = 2:n
tmp1 = w(i-1,2:i+1).*s1(1:i)/i;
tmp2 = w(i-1,1:i).*s2(n-i+1:n)/i;
w(i,2:i+1) = tmp1 + tmp2;
tmp3 = w(i,2:i+1) + tiny; % In case tmp1 & tmp2 are both 0,
tmp4 = (s2(n-i+1:n) > s1(1:i)); % then t is 0 on left & 1 on right
t(i-1,1:i) = (tmp2./tmp3).*tmp4 + (1-tmp1./tmp3).*(~tmp4);
end
% Derive the polytope volume v from the appropriate
% element in the bottom row of w.
v = n^(3/2)*(w(n,k+2)/realmax)*(b-a)^(n-1);
% Now compute the matrix x.
x = zeros(n,m);
if m == 0, return, end % If m is zero, quit with x = []
rt = rand(n-1,m); % For random selection of simplex type
rs = rand(n-1,m); % For random location within a simplex
s = repmat(s,1,m);
j = repmat(k+1,1,m); % For indexing in the t table
sm = zeros(1,m); pr = ones(1,m); % Start with sum zero & product 1
for i = n-1:-1:1 % Work backwards in the t table
e = (rt(n-i,:)<=t(i,j)); % Use rt to choose a transition
sx = rs(n-i,:).^(1/i); % Use rs to compute next simplex coord.
sm = sm + (1-sx).*pr.*s/(i+1); % Update sum
pr = sx.*pr; % Update product
x(n-i,:) = sm + pr.*e; % Calculate x using simplex coords.
s = s - e; j = j - e; % Transition adjustment
end
x(n,:) = sm + pr.*s; % Compute the last x
% Randomly permute the order in the columns of x and rescale.
rp = rand(n,m); % Use rp to carry out a matrix 'randperm'
[ig,p] = sort(rp); % The values placed in ig are ignored
x = (b-a)*x(p+repmat([0:n:n*(m-1)],n,1))+a; % Permute & rescale x
return
but I want to generate matrix which has each row elements sum 1 and each row have uniform probability in matlab. how to do this with above programme. to run above programme I am calling it to another file and setting parameters
i.e.
m=4;n=4; a=0; b=1.5;s=1;
[x,v] = randfixedsum(n,m,s,a,b)
create a random matrix and divide each row by sum of elements of that row:
function result = randrowsum(m ,n)
rnd = rand(m,n);
rowsums = sum(rnd,2);
result = bsxfun(#rdivide, rnd, rowsums);
end
to create an m * n random matrix :
a=randrowsum(3,4)
check if sum of each row is 1:
sum(a,2)
I would say the easiest was is to generate the array with the given function.
[x,v] = randfixedsum(n,m,s,a,b);
Then just transport the results.
x = x';

The range of the coefficients of 2 level Wavelet LeGall 5/3 2D transform

As in the title, I am confused about the range of the coefficients of Wavelet LeGall 5/3 (has to be exact this filter) 2D transform (only for a 8*8 block) if the value of the input matrix are within the range from 0-255.
For the formulas, the link is here: Wavelet LeGall 5/3
Here is what I did for now:
Minus 128 for all value (easier to calculate the low frequency values, see later);
Do the transform in horizontal direction. This will generate all coefficients in all lines: the first 4 are low frequency and last 4 are high frequency. It is easy to find the range for high frequency is -255 to +255 (double the range of input). And the range for low frequency is actually -192 to +192 (1.5* the range of input).
Do the transform in vertical direction. This will do the same in vertical directions. And there are four blocks generated: LL (lowlow), LH (low high), HL, HH. It is easy to calculate the range for the HH is largest: -511 to +511 and the range for LL is 1.5*1.5 = 2.25 times of the input range (-128 to +127).
Then, here comes the question. What if I do this wavelet again for the LL block? Theoretically the range of the HH (2nd level coefficients) of the LL block should be 4 times of the LL range which becomes 10 times of the input range (-128 to +127) which is -1280 to +1270.
However, I tried many times random calculation, the max value never exceed -511 to +511 (see code at the end). I guess it is because the theoretical values cannot be reached because all calculations are based on the previous one. But this seemly easy question is difficult for me to prove theoretically. So can someone help me get out please?
Code I used (put two files in one directory and run the test file for any times you want but just the max value will not exceed 512...):
Function of waveletlegall53:
function X = waveletlegall53(X, Level)
%WAVELETLEGALL53 Le Gall 5/3 (Spline 2.2) wavelet transform.
% Y = WAVELETLEGALL53(X, L) decomposes X with L stages of the
% Le Gall 5/3 wavelet. For the inverse transform,
% WAVELETLEGALL53(X, -L) inverts L stages. Filter boundary
% handling is half-sample symmetric.
%
% X may be of any size; it need not have size divisible by 2^L.
% For example, if X has length 9, one stage of decomposition
% produces a lowpass subband of length 5 and a highpass subband
% of length 4. Transforms of any length have perfect
% reconstruction (exact inversion).
%
% If X is a matrix, WAVELETLEGALL53 performs a (tensor) 2D
% wavelet transform. If X has three dimensions, the 2D
% transform is applied along the first two dimensions.
%
% Example:
% Y = waveletlegall53(X, 5); % Transform image X using 5 stages
% R = waveletlegall53(Y, -5); % Reconstruct from Y
X = double(X);
if nargin < 2, error('Not enough input arguments.'); end
if ndims(X) > 3, error('Input must be a 2D or 3D array.'); end
if any(size(Level) ~= 1), error('Invalid transform level.'); end
N1 = size(X,1);
N2 = size(X,2);
% Lifting scheme filter coefficients for Le Gall 5/3
LiftFilter = [-1/2,1/4];
ScaleFactor =1; sqrt(2);
LiftFilter = LiftFilter([1,1],:);
if Level >= 0 % Forward transform
for k = 1:Level
M1 = ceil(N1/2);
M2 = ceil(N2/2);
%%% Transform along columns %%%
if N1 > 1
RightShift = [2:M1,M1];
X0 = X(1:2:N1,1:N2,:);
% Apply lifting stages
if rem(N1,2)
X1 = [X(2:2:N1,1:N2,:);X0(M1,:,:)]...
+floor(filter(LiftFilter(:,1),1,X0(RightShift,:,:),...
X0(1,:,:)*LiftFilter(1,1),1));
else
X1 = X(2:2:N1,1:N2,:) ...
+floor(filter(LiftFilter(:,1),1,X0(RightShift,:,:),...
X0(1,:,:)*LiftFilter(1,1),1));
end
X0 = X0 + floor(filter(LiftFilter(:,2),1,...
X1,X1(1,:,:)*LiftFilter(1,2),1)+0.5);
if rem(N1,2)
X1(M1,:,:) = [];
end
X(1:N1,1:N2,:) = [X0*ScaleFactor;X1/ScaleFactor];
end
%%% Transform along rows %%%
if N2 > 1
RightShift = [2:M2,M2];
X0 = permute(X(1:N1,1:2:N2,:),[2,1,3]);
% Apply lifting stages
if rem(N2,2)
X1 = permute([X(1:N1,2:2:N2,:),X(1:N1,N2,:)],[2,1,3])...
+ floor(filter(LiftFilter(:,1),1,X0(RightShift,:,:),...
X0(1,:,:)*LiftFilter(1,1),1));
else
X1 = permute(X(1:N1,2:2:N2,:),[2,1,3]) ...
+ floor(filter(LiftFilter(:,1),1,X0(RightShift,:,:),...
X0(1,:,:)*LiftFilter(1,1),1));
end
X0 = X0 +floor( filter(LiftFilter(:,2),1,...
X1,X1(1,:,:)*LiftFilter(1,2),1)+0.5);
if rem(N2,2)
X1(M2,:,:) = [];
end
X(1:N1,1:N2,:) = permute([X0*ScaleFactor;X1/ScaleFactor],[2,1,3]);
end
N1 = M1;
N2 = M2;
end
else % Inverse transform
for k = 1+Level:0
M1 = ceil(N1*pow2(k));
M2 = ceil(N2*pow2(k));
%%% Inverse transform along rows %%%
if M2 > 1
Q = ceil(M2/2);
RightShift = [2:Q,Q];
X1 = permute(X(1:M1,Q+1:M2,:)*ScaleFactor,[2,1,3]);
if rem(M2,2)
X1(Q,1,1) = 0;
end
% Undo lifting stages
X0 = permute(X(1:M1,1:Q,:)/ScaleFactor,[2,1,3]) ...
- floor(filter(LiftFilter(:,2),1,X1,X1(1,:,:)*LiftFilter(1,2),1)+0.5);
X1 = X1 - floor(filter(LiftFilter(:,1),1,X0(RightShift,:,:),...
X0(1,:,:)*LiftFilter(1,1),1));
if rem(M2,2)
X1(Q,:,:) = [];
end
X(1:M1,[1:2:M2,2:2:M2],:) = permute([X0;X1],[2,1,3]);
end
%%% Inverse transform along columns %%%
if M1 > 1
Q = ceil(M1/2);
RightShift = [2:Q,Q];
X1 = X(Q+1:M1,1:M2,:)*ScaleFactor;
if rem(M1,2)
X1(Q,1,1) = 0;
end
% Undo lifting stages
X0 = X(1:Q,1:M2,:)/ScaleFactor ...
- floor(filter(LiftFilter(:,2),1,X1,X1(1,:,:)*LiftFilter(1,2),1)+0.5);
X1 = X1 - floor(filter(LiftFilter(:,1),1,X0(RightShift,:,:),...
X0(1,:,:)*LiftFilter(1,1),1));
if rem(M1,2)
X1(Q,:,:) = [];
end
X([1:2:M1,2:2:M1],1:M2,:) = [X0;X1];
end
end
end
The test .m file:
clear all
close all
clc
n=100000;
maxt=zeros(1,n);
maxt2=zeros(1,n);
for it=1:n
X=floor(rand(8,8)*256);
X = X-128;
a = waveletlegall53(X,2);
maxt(it)=max(max(abs(a)));
if max(max(abs(a))) > 470
max(max(abs(a)))
end
end
[fr ind]=hist(maxt,length(unique(maxt)));
pr = length(find(maxt>512))/n
fr=fr/n;
figure()
plot(ind, fr)
grid on
Maxvalue = max(maxt)

Matlab Random Number with Condition [duplicate]

This question already has answers here:
Random numbers that add to 100: Matlab
(4 answers)
Closed 7 years ago.
I am looking how to pick 10 positive non-zero elements in 1x10 array randomly whose sum is 1
Example :
A=[0.0973 0.1071 0.0983 0.0933 0.1110 0.0942 0.1062 0.0970 0.0981 0.0974]
Note: If we sum the elements in above matrix it will be 1. I need matlab to generate a matrix like this randomly
Try using Roger's fex submission: http://www.mathworks.com/matlabcentral/fileexchange/9700-random-vectors-with-fixed-sum
Here is a copy of the content of the file (in case the link dies).
All the credit obviously goes to the original poster Roger Stafford:
function [x,v] = randfixedsum(n,m,s,a,b)
% [x,v] = randfixedsum(n,m,s,a,b)
%
% This generates an n by m array x, each of whose m columns
% contains n random values lying in the interval [a,b], but
% subject to the condition that their sum be equal to s. The
% scalar value s must accordingly satisfy n*a <= s <= n*b. The
% distribution of values is uniform in the sense that it has the
% conditional probability distribution of a uniform distribution
% over the whole n-cube, given that the sum of the x's is s.
%
% The scalar v, if requested, returns with the total
% n-1 dimensional volume (content) of the subset satisfying
% this condition. Consequently if v, considered as a function
% of s and divided by sqrt(n), is integrated with respect to s
% from s = a to s = b, the result would necessarily be the
% n-dimensional volume of the whole cube, namely (b-a)^n.
%
% This algorithm does no "rejecting" on the sets of x's it
% obtains. It is designed to generate only those that satisfy all
% the above conditions and to do so with a uniform distribution.
% It accomplishes this by decomposing the space of all possible x
% sets (columns) into n-1 dimensional simplexes. (Line segments,
% triangles, and tetrahedra, are one-, two-, and three-dimensional
% examples of simplexes, respectively.) It makes use of three
% different sets of 'rand' variables, one to locate values
% uniformly within each type of simplex, another to randomly
% select representatives of each different type of simplex in
% proportion to their volume, and a third to perform random
% permutations to provide an even distribution of simplex choices
% among like types. For example, with n equal to 3 and s set at,
% say, 40% of the way from a towards b, there will be 2 different
% types of simplex, in this case triangles, each with its own
% area, and 6 different versions of each from permutations, for
% a total of 12 triangles, and these all fit together to form a
% particular planar non-regular hexagon in 3 dimensions, with v
% returned set equal to the hexagon's area.
%
% Roger Stafford - Jan. 19, 2006
% Check the arguments.
if (m~=round(m))|(n~=round(n))|(m<0)|(n<1)
error('n must be a whole number and m a non-negative integer.')
elseif (s<n*a)|(s>n*b)|(a>=b)
error('Inequalities n*a <= s <= n*b and a < b must hold.')
end
% Rescale to a unit cube: 0 <= x(i) <= 1
s = (s-n*a)/(b-a);
% Construct the transition probability table, t.
% t(i,j) will be utilized only in the region where j <= i + 1.
k = max(min(floor(s),n-1),0); % Must have 0 <= k <= n-1
s = max(min(s,k+1),k); % Must have k <= s <= k+1
s1 = s - [k:-1:k-n+1]; % s1 & s2 will never be negative
s2 = [k+n:-1:k+1] - s;
w = zeros(n,n+1); w(1,2) = realmax; % Scale for full 'double' range
t = zeros(n-1,n);
tiny = 2^(-1074); % The smallest positive matlab 'double' no.
for i = 2:n
tmp1 = w(i-1,2:i+1).*s1(1:i)/i;
tmp2 = w(i-1,1:i).*s2(n-i+1:n)/i;
w(i,2:i+1) = tmp1 + tmp2;
tmp3 = w(i,2:i+1) + tiny; % In case tmp1 & tmp2 are both 0,
tmp4 = (s2(n-i+1:n) > s1(1:i)); % then t is 0 on left & 1 on right
t(i-1,1:i) = (tmp2./tmp3).*tmp4 + (1-tmp1./tmp3).*(~tmp4);
end
% Derive the polytope volume v from the appropriate
% element in the bottom row of w.
v = n^(3/2)*(w(n,k+2)/realmax)*(b-a)^(n-1);
% Now compute the matrix x.
x = zeros(n,m);
if m == 0, return, end % If m is zero, quit with x = []
rt = rand(n-1,m); % For random selection of simplex type
rs = rand(n-1,m); % For random location within a simplex
s = repmat(s,1,m);
j = repmat(k+1,1,m); % For indexing in the t table
sm = zeros(1,m); pr = ones(1,m); % Start with sum zero & product 1
for i = n-1:-1:1 % Work backwards in the t table
e = (rt(n-i,:)<=t(i,j)); % Use rt to choose a transition
sx = rs(n-i,:).^(1/i); % Use rs to compute next simplex coord.
sm = sm + (1-sx).*pr.*s/(i+1); % Update sum
pr = sx.*pr; % Update product
x(n-i,:) = sm + pr.*e; % Calculate x using simplex coords.
s = s - e; j = j - e; % Transition adjustment
end
x(n,:) = sm + pr.*s; % Compute the last x
% Randomly permute the order in the columns of x and rescale.
rp = rand(n,m); % Use rp to carry out a matrix 'randperm'
[ig,p] = sort(rp); % The values placed in ig are ignored
x = (b-a)*x(p+repmat([0:n:n*(m-1)],n,1))+a; % Permute & rescale x
return

Accuracy issues with multiplication of matrices in Matlab R2012b

I have implemented a script that does constrained optimization for solving the optimal parameters of Support Vector Machines model. I noticed that my script for some reason gives inaccurate results (although very close to the real value). For example the typical situation is that the result of a calculation should be exactly 0, but instead it is something like
-1/18014398509481984 = -5.551115123125783e-17
This situation happens when I multiply matrices with vectors. What makes this also strange is that if I do the multiplications by hand in the command window in Matlab I get exactly 0 result.
Let me give an example: If I take the vectors Aq = [-1 -1 1 1] and x = [12/65 28/65 32/65 8/65]' I get exactly 0 result from their multiplication if I do this in the command window, as you can see in the picture below:
If on the other hand I do this in my function-script I don't get the result being 0 but rather the value -1/18014398509481984.
Here is the part of my script that is responsible for this multiplication (I've added the Aq and x into the script to show the contents of Aq and x as well):
disp('DOT PRODUCT OF ACTIVE SET AND NEW POINT: ')
Aq
x
Aq*x
Here is the result of the code above when run:
As you can see the value isn't exactly 0 even though it really should be. Note that this problem doesn't occur for all possible values of Aq and x. If Aq = [-1 -1 1 1] and x = [4/13 4/13 4/13 4/13] the result is exactly 0 as you can see below:
What is causing this inaccuracy? How can I fix this?
P.S. I didn't include my whole code because it's not very well documented and few hundred lines long, but I will if requested.
Thank you!
UPDATE: new test, by using Ander Biguri's advice:
UPDATE 2: THE CODE
function [weights, alphas, iters] = solveSVM(data, labels, C, e)
% FUNCTION [weights, alphas, iters] = solveSVM(data, labels, C, e)
%
% AUTHOR: jjepsuomi
%
% VERSION: 1.0
%
% DESCRIPTION:
% - This function will attempt to solve the optimal weights for a Support
% Vector Machines (SVM) model using active set method with gradient
% projection.
%
% INPUTS:
% "data" a n-by-m data matrix. The number of rows 'n' corresponds to the
% number of data points and the number of columns 'm' corresponds to the
% number of variables.
% "labels" a 1-by-n row vector of data labels from the set {-1,1}.
% "C" Box costraint upper limit. This will constrain the values of 'alphas'
% to the range 0 <= alphas <= C. If hard-margin SVM model is required set
% C=Inf.
% "e" a real value corresponding to the convergence criterion, that is if
% solution Xi and Xi-1 are within distance 'e' from each other stop the
% learning process, i.e. IF |F(Xi)-F(Xi-1)| < e ==> stop learning process.
%
% OUTPUTS:
% "weights" a vector corresponding to the optimal decision line parameters.
% "alphas" a vector of alpha-values corresponding to the optimal solution
% of the dual optimization problem of SVM.
% "iters" number of iterations until learning stopped.
%
% EXAMPLE USAGE 1:
%
% 'Hard-margin SVM':
%
% data = [0 0;2 2;2 0;3 0];
% labels = [-1 -1 1 1];
% [weights, alphas, iters] = solveSVM(data, labels, Inf, 10^-100)
%
% EXAMPLE USAGE 2:
%
% 'Soft-margin SVM':
%
% data = [0 0;2 2;2 0;3 0];
% labels = [-1 -1 1 1];
% [weights, alphas, iters] = solveSVM(data, labels, 0.8, 10^-100)
% STEP 1: INITIALIZATION OF THE PROBLEM
format long
% Calculate linear kernel matrix
L = kron(labels', labels);
K = data*data';
% Hessian matrix
Qd = L.*K;
% The minimization function
L = #(a) (1/2)*a'*Qd*a - ones(1, length(a))*a;
% Gradient of the minimizable function
gL = #(a) a'*Qd - ones(1, length(a));
% STEP 2: THE LEARNING PROCESS, ACTIVE SET WITH GRADIENT PROJECTION
% Initial feasible solution (required by gradient projection)
x = zeros(length(labels), 1);
iters = 1;
optfound = 0;
while optfound == 0 % criterion met
% Negative of the gradient at initial solution
g = -gL(x);
% Set the active set and projection matrix
Aq = labels; % In plane y^Tx = 0
P = eye(length(x))-Aq'*inv(Aq*Aq')*Aq; % In plane projection
% Values smaller than 'eps' are changed into 0
P(find(abs(P-0) < eps)) = 0;
d = P*g'; % Projection onto plane
if ~isempty(find(x==0 | x==C)) % Constraints active?
acinds = find(x==0 | x==C);
for i = 1:length(acinds)
if (x(acinds(i)) == 0 && d(acinds(i)) < 0) || x(acinds(i)) == C && d(acinds(i)) > 0
% Make the constraint vector
constr = zeros(1,length(x));
constr(acinds(i)) = 1;
Aq = [Aq; constr];
end
end
% Update the projection matrix
P = eye(length(x))-Aq'*inv(Aq*Aq')*Aq; % In plane / box projection
% Values smaller than 'eps' are changed into 0
P(find(abs(P-0) < eps)) = 0;
d = P*g'; % Projection onto plane / border
end
%%%% DISPLAY INFORMATION, THIS PART IS NOT NECESSAY, ONLY FOR DEBUGGING
if Aq*x ~= 0
disp('ACTIVE SET CONSTRAINTS Aq :')
Aq
disp('CURRENT SOLUTION x :')
x
disp('MULTIPLICATION OF Aq and x')
Aq*x
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Values smaller than 'eps' are changed into 0
d(find(abs(d-0) < eps)) = 0;
if ~isempty(find(d~=0)) && rank(P) < length(x) % Line search for optimal lambda
lopt = ((g*d)/(d'*Qd*d));
lmax = inf;
for i = 1:length(x)
if d(i) < 0 && -x(i) ~= 0 && -x(i)/d(i) <= lmax
lmax = -x(i)/d(i);
elseif d(i) > 0 && (C-x(i))/d(i) <= lmax
lmax = (C-x(i))/d(i);
end
end
lambda = max(0, min([lopt, lmax]));
if abs(lambda) < eps
lambda = 0;
end
xo = x;
x = x + lambda*d;
iters = iters + 1;
end
% Check whether search direction is 0-vector or 'e'-criterion met.
if isempty(find(d~=0)) || abs(L(x)-L(xo)) < e
optfound = 1;
end
end
%%% STEP 3: GET THE WEIGHTS
alphas = x;
w = zeros(1, length(data(1,:)));
for i = 1:size(data,1)
w = w + labels(i)*alphas(i)*data(i,:);
end
svinds = find(alphas>0);
svind = svinds(1);
b = 1/labels(svind) - w*data(svind, :)';
%%% STEP 4: OPTIMALITY CHECK, KKT conditions. See KKT-conditions for reference.
weights = [b; w'];
datadim = length(data(1,:));
Q = [zeros(1,datadim+1); zeros(datadim, 1), eye(datadim)];
A = [ones(size(data,1), 1), data];
for i = 1:length(labels)
A(i,:) = A(i,:)*labels(i);
end
LagDuG = Q*weights - A'*alphas;
Ac = A*weights - ones(length(labels),1);
alpA = alphas.*Ac;
LagDuG(any(abs(LagDuG-0) < 10^-14)) = 0;
if ~any(alphas < 0) && all(LagDuG == zeros(datadim+1,1)) && all(abs(Ac) >= 0) && all(abs(alpA) < 10^-6)
disp('Optimal found, Karush-Kuhn-Tucker conditions satisfied.')
else
disp('Optimal not found, Karush-Kuhn-Tucker conditions not satisfied.')
end
% VISUALIZATION FOR 2D-CASE
if size(data, 2) == 2
pinds = find(labels > 0);
ninds = find(labels < 0);
plot(data(pinds, 1), data(pinds, 2), 'o', 'MarkerFaceColor', 'red', 'MarkerEdgeColor', 'black')
hold on
plot(data(ninds, 1), data(ninds, 2), 'o', 'MarkerFaceColor', 'blue', 'MarkerEdgeColor', 'black')
Xb = min(data(:,1))-1;
Xe = max(data(:,1))+1;
Yb = -(b+w(1)*Xb)/w(2);
Ye = -(b+w(1)*Xe)/w(2);
lineh = plot([Xb Xe], [Yb Ye], 'LineWidth', 2);
supvh = plot(data(find(alphas~=0), 1), data(find(alphas~=0), 2), 'g.');
legend([lineh, supvh], 'Decision boundary', 'Support vectors');
hold off
end
NOTE:
If you run the EXAMPLE 1, you should get an output starting with the following:
As you can see, the multiplication between Aq and x don't produce value 0, even though they should. This is not a bad thing in this particular example, but if I have more data points with lots of decimals in them this inaccuracy becomes bigger and bigger problem, because the calculations are not exact. This is bad for example when I'm searching for a new direction vector when I'm moving towards the optimal solution in gradient projection method. The search direction isn't exactly the correct direction, but close to it. This is why I want the exactly correct values...is this possible?
I wonder if the decimals in the data points have something to do with the accuracy of my results. See the picture below:
So the question is: Is this caused by the data or is there something wrong in the optimization procedure...
Do you use format function inside your script? It looks like you used somewhere format rat.
You can always use matlab eps function, that returns precision that is used inside matlab. The absolute value of -1/18014398509481984 is smaller that this, according to my Matlab R2014B:
format long
a = abs(-1/18014398509481984)
b = eps
a < b
This basically means that the result is zero (but matlab stopped calculations because according to eps value, the result was just fine).
Otherwise you can just use format long inside your script before the calculation.
Edit
I see inv function inside your code, try replacing it with \ operator (mldivide). The results from it will be more accurate as it uses Gaussian elimination, without forming the inverse.
The inv documentation states:
In practice, it is seldom necessary to form the explicit inverse of a
matrix. A frequent misuse of inv arises when solving the system of
linear equations Ax = b. One way to solve this is with x = inv(A)*b. A
better way, from both an execution time and numerical accuracy
standpoint, is to use the matrix division operator x = A\b. This
produces the solution using Gaussian elimination, without forming the
inverse.
With the provided code, this is how I tested:
I added a break-point on the following code:
if Aq*x ~= 0
disp('ACTIVE SET CONSTRAINTS Aq :')
Aq
disp('CURRENT SOLUTION x :')
x
disp('MULTIPLICATION OF Aq and x')
Aq*x
end
When the if branch was taken, I typed at console:
K>> format rat; disp(x);
12/65
28/65
32/65
8/65
K>> disp(x == [12/65; 28/65; 32/65; 8/65]);
0
1
0
0
K>> format('long'); disp(max(abs(x - [12/65; 28/65; 32/65; 8/65])));
1.387778780781446e-17
K>> disp(eps(8/65));
1.387778780781446e-17
This suggests that this is a displaying problem: the format rat deliberately uses small integers for expressing the value, on the expense of precision. Apparently, the true value of x(4) is the next one to 8/65 than can be possibly put in double format.
So, this begs the question: are you sure that numeric convergence depends on flipping the least significant bit in a double precision value?

Multiplying a vector times the inverse of a matrix in Matlab

I have a problem multiplying a vector times the inverse of a matrix in Matlab. The code I am using is the following:
% Final Time
T = 0.1;
% Number of grid cells
N=20;
%N=40;
L=20;
% Delta x
dx=1/N
% define cell centers
%x = 0+dx*0.5:dx:1-0.5*dx;
x = linspace(-L/2, L/2, N)';
%define number of time steps
NTime = 100; %NB! Stability conditions-dersom NTime var 50 ville en fått helt feil svar pga lambda>0,5
%NTime = 30;
%NTime = 10;
%NTime = 20;
%NTime = 4*21;
%NTime = 4*19;
% Time step dt
dt = T/NTime
% Define a vector that is useful for handling teh different cells
J = 1:N; % number the cells of the domain
J1 = 2:N-1; % the interior cells
J2 = 1:N-1; % numbering of the cell interfaces
%define vector for initial data
u0 = zeros(1,N);
L = x<0.5;
u0(L) = 0;
u0(~L) = 1;
plot(x,u0,'-r')
grid on
hold on
% define vector for solution
u = zeros(1,N);
u_old = zeros(1,N);
% useful quantity for the discrete scheme
r = dt/dx^2
mu = dt/dx;
% calculate the numerical solution u by going through a loop of NTime number
% of time steps
A=zeros(N,N);
alpha(1)=A(1,1);
d(1)=alpha(1);
b(1)=0;
c(1)=b(1);
gamma(1,2)=A(1,2);
% initial state
u_old = u0;
pause
for j = 2:NTime
A(j,j)=1+2*r;
A(j,j-1)=-(1/dx^2);
A(j,j+1)=-(1/dx^2);
u=u_old./A;
% plotting
plot(x,u,'-')
xlabel('X')
ylabel('P(X)')
hold on
grid on
% update "u_old" before you move forward to the next time level
u_old = u;
pause
end
hold off
The error message I get is:
Matrix dimensions must agree.
Error in Implicit_new (line 72)
u=u_old./A;
My question is therefore how it is possible to perform u=u_old*[A^(-1)] in Matlab?
David
As knedlsepp said, v./A is the elementwise division, which is not what you wanted. You can use either
v/A provided that v is a row vector and its length is equal to the number of columns in A. The result is a row vector.
A\v provided that v is a column vector and its length is equal to the number of rows in A
The results differ only in shape: v/A is the transpose of A'\v'