I am trying to generate some random standard normal variables and correlate them
In particular I want:
where is the vector I want to simulate, and the given covariance matrix.
I am doing the following in matlab:
Simulate a vector of uncorrelated Gaussian random variables,
then find a square root of , i.e. a matrix such that .
Then the target vector is given by
Here is my matlab code:
N = 500000
u_1 = normrnd(zeros(N,1),1);
u_2 = normrnd(zeros(N,1),1);
u_3 = normrnd(zeros(N,1),1);
u_4 = normrnd(zeros(N,1),1);
rv = [u_1 '; u_2'; u_3'; u_4'];
VarCov = [1 -0.87 0.0 -0.6;
-0.87 1 0.0 0.0;
0.0 0.0 1 0.0
-0.6 0.0 0.0 1];
ch = chol(VarCov);
result = ch * rv;
However the chol(VarCov) is giving an error saying that the matrix is not positive semi definite.
Related
I use fminunc to find the value of B (2x4 matrix) that minimzes the difference between the corresponding elements in two vectors as indicated in the attached code. In other words, I want to find the B that makes the elements of beta_d (1x4 vector) which is a function of B matrix, equal to the corresponding ones of a "given" beta_u (1x4 vector), i.e. beta_d(1,1) = beta_u(1,1) && beta_d(1,2) = beta_u(1,2) && beta_d(1,3) = beta_u(1,3) && beta_d(1,4) = beta_u(1,4).
However, I usually receive the following message without getting any result and the program seems to go on an infinite loop!
Local minimum found.
Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.
<stopping criteria details>
The code is as follows:
% System paramters:
N = 2;
K = 4;
C_l = 4;
H = [-0.3208 -0.9784; -1.5994 -1.4689; -1.5197 -0.4568; -0.0993 -0.7667]; % 4*2 matrix
A = [-1 1; 0 1]; % 2x2 matrix
C = [-0.20 0.4 0.6 -0.2; -0.2 0.4 0.6 -0.2; 0.4 0.2 -0.2 0.4; 0.4 0.2 -0.2 0.4]; % 4x4 matrix
P = [250000 0 0 0; 0 250000 0 0; 0 0 250000 0; 0 0 0 250000]; % 4x4 matrix
beta_u = [50.2207 50.2207 20.3433 20.3433]; % 1x4 vector
beta_d = zeros(1,4); % intial value
B = zeros(2,4); % intial value
% store inputs to a struct for shorter syntax
s = struct();
[s.H,s.A,s.C,s.P,s.C_l,s.N,s.K] = deal(H,A,C,P,C_l,N,K);
%fminunc optimization
while (sum(abs(beta_u-beta_d))>=0.1)
initial_guess = randn(2,4);
OLS = #(B_d,input_vars)sum((beta_u-myfun(B_d,input_vars)).^2); % ordinary least squares cost function
opts = optimoptions(#fminunc,'MaxIterations',10000,'MaxFunctionEvaluations',50000,'CheckGradients',true);
B = fminunc(OLS, initial_guess, opts,s);
input_vars = s;
[beta_d, D_d] = myfun(B,input_vars);
end
% calculate beta_d from B and the other inputs
function [beta_d, D_d] = myfun(B,input_vars)
% load parameters
s = input_vars;[H,A,C,P,C_l,N,K]=deal(s.H,s.A,s.C,s.P,s.C_l,s.N,s.K);
for j = 1:1:N
d(j) = (B(j,:)*P*B(j,:)')/((2^(2*C_l))-(norm(A(:,j))^2));
end
D_d = diag(d);
for i = 1:1:K
V_d(i) = C(i,:)*P*B'*H(i,:)'*inv(1+H(i,:)*(A'*D_d*A+B*P*B')*H(i,:)');
sigma_d(i) = norm((V_d(i)*H(i,:)*B-C(i,:))*(P^(1/2)))^2+(V_d(i)^2)*(1+H(i,:)*A'*D_d*A*H(i,:)');
beta_d(i) = ((P(i,i))/sigma_d(:,i));
end
end
I am novice to deep learning so I begin with the simplest test case: XOR learning.
In the new edition of Digital Image Processing by G & W the authors give an example of XOR learning by a deep net with 3 layers: input, hidden and output (each layer has 2 neurons.), and a sigmoid as the network activation function.
For network initailization they say: "We used alpha = 1.0, an inital set of Gaussian random weights of zero mean and standard deviation of 0.02" (alpha is the gradient descent learning rate).
Training was made with 4 labeled examples:
X = [1 -1 -1 1;1 -1 1 -1];%MATLAB syntax
R = [1 1 0 0;0 0 1 1];%Labels
I have written the following MATLAB code to implement the network learing process:
function output = neuralNet4e(input,specs)
NumPat = size(input.X,2);%Number of patterns
NumLayers = length(specs.W);
for kEpoch = 1:specs.NumEpochs
% forward pass
A = cell(NumLayers,1);%Output of each neuron in each layer
derZ = cell(NumLayers,1);%Activation function derivative on each neuron dot product
A{1} = input.X;
for kLayer = 2:NumLayers
B = repmat(specs.b{kLayer},1,NumPat);
Z = specs.W{kLayer} * A{kLayer - 1} + B;
derZ{kLayer} = specs.activationFuncDerive(Z);
A{kLayer} = specs.activationFunc(Z);
end
% backprop
D = cell(NumLayers,1);
D{NumLayers} = (A{NumLayers} - input.R).* derZ{NumLayers};
for kLayer = (NumLayers-1):-1:2
D{kLayer} = (specs.W{kLayer + 1}' * D{kLayer + 1}).*derZ{kLayer};
end
%Update weights and biases
for kLayer = 2:NumLayers
specs.W{kLayer} = specs.W{kLayer} - specs.alpha * D{kLayer} * A{kLayer - 1}' ;
specs.b{kLayer} = specs.b{kLayer} - specs.alpha * sum(D{kLayer},2);
end
end
output.A = A;
end
Now, when I am using their setup (i.e., weights initalizaion with std = 0.02)
clearvars
s = 0.02;
input.X = [1 -1 -1 1;1 -1 1 -1];
input.R = [1 1 0 0;0 0 1 1];
specs.W = {[];s * randn(2,2);s * randn(2,2)};
specs.b = {[];s * randn(2,1);s * randn(2,1)};
specs.activationFunc = #(x) 1./(1 + exp(-x));
specs.activationFuncDerive = #(x) exp(-x)./(1 + exp(-x)).^2;
specs.NumEpochs = 1e4;
specs.alpha = 1;
output = neuralNet4e(input,specs);
I'm getting (after 10000 epoches) that the final output of the net is
output.A{3} = [0.5 0.5 0.5 0.5;0.5 0.5 0.5 0.5]
but when I changed s = 0.02; to s = 1; I got output.A{3} = [0.989 0.987 0.010 0.010;0.010 0.012 0.0.98 0.98] as it should.
Is it possible to get these results with `s=0.02;' and I am doing something wrong in my code? or is standard deviation of 0.02 is just a typo?
Based on your code, I don't see any errors. In my knowledge, the result that you got,
[0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5]
That is a typical result of overfitting. There are many reasons for this to happen, such as too many epochs, too large learning rate, too small sample data, and others.
On your example, s=0.02 limits the values of randomized weights and biases. Changing that to s=1 makes the randomized values unchanged/unscaled.
To make the s=0.02 one work, you can try minimizing the number of epochs or maybe lowering the alpha.
Hope this helps.
I have a sequence S :
S= 'ABCD' % which means A<B<C<D
I want to convert S into a matrix M[i,j] which have to satisfy those conditions :
M[i,j] , M[j,i] are random
M[i,i] =0.5
M[i,j] + M[j,i] = 1
M[i,j] < M[j,i] % For example: if A<B then M[A,B] < M[B,A]
For example: if we have S = 'ABCD', the M matrix will be expected as follows:
A B C D
A o.5 0.25 0.2 0.1
B 0.75 0.5 0.35 0.15
C 0.8 0.65 0.5 0.4
D 0.9 0.85 0.6 0.5
How to create that kind of above matrix from a given sequence ?
From your question it appears you want
to fill the lower part of the matrix with random entries uniformly distributed on the interval (0,0.5) (so that condition 4 in your question be satisfied);
the upper part is then computed according to condition 3; and
the diagonal is determined by condition 2.
You can do that as follows:
n = 4; %// size
M = NaN(n); %// preallocate
M(1:n+1:end) = 0.5; %// fill diagonal
ind_lower = tril(true(n), -1); %// logical index for lower part
M(ind_lower) = 0.5*rand(n*(n-1)/2, 1); %// fill lower part
M_aux = NaN(n); %// auxiliary variable to fill upper part
M_aux(ind_lower) = 1-M(ind_lower).';
M_aux = M_aux.';
M(ind_lower.') = M_aux(ind_lower.'); %// fill upper part
Example result:
M =
0.5000 0.5214 0.7573 0.5999
0.4786 0.5000 0.9291 0.7891
0.2427 0.0709 0.5000 0.5421
0.4001 0.2109 0.4579 0.5000
Here's another similar approach:
n = 4;
M = tril(rand(n)*0.5, -1);
P = triu(1-M.', 1);
M = M + P + eye(n)*0.5;
Result:
M =
0.500000 0.987433 0.711005 0.944642
0.012567 0.500000 0.782633 0.902365
0.288995 0.217367 0.500000 0.783708
0.055358 0.097635 0.216292 0.500000
I am using matlab for part of my final year project. I am solving a geometric series such as the sum of x^j, starting from j=0 up to n-1. I have the following code so far:
$Variable dictionary
%N Number of terms to sum
%alpha Sum of series
%x Vector of constants
%n Loop counter
N = input('Enter the number of terms to sum: ');
alpha = 0;
x = [0.9 0.99 0.999 0.9999 0.99999 0.999999];
for n = 0:N-1
alpha = alpha + (x.^(n));
end
format long
alpha
When I run this script it is allowing me to put in the values of x in the script as a vector but asks the user for values of n. Is there anyway I can amend my code so that I can put the n in myself? And make it more than one value of n?
Thanks
Maybe this is what you want (no loops needed):
x = [0.9 0.99 0.999 0.9999 0.99999 0.999999];
n = [1 2 5];
alphas = sum(bsxfun(#power, x(:), n(:).')); %'// one result for each value of n
Modify this part of code:
for n = 1: length(N)
alpha = alpha + (x.^(N(n)));
end
And pass the N as vector [10 100 1000]
Here a solution:
x = [0.9 0.99 0.999 0.9999 0.99999 0.999999];
nlist = [10,100,1000];
for elm = nlist
alpha = alpha + (x.^(elm));
end
I want to solve these equations using MATLAB and I am sure there is a non zero solution. The equations are:
0.7071*x + 0.7071*z = x
-0.5*x + 0.7071*y + 0.5*z = y
-0.5*x - 0.7071*y + 0.5*z = z
I wrote in MATLAB:
[x,y,z]=solve('0.7071 * x+0.7071 * z=x','-0.5 * x+0.7071 * y+0.5 * z=y','-0.5 * x-0.7071 * y+0.5 * z=z');
But the result is x = y = z = 0.
As I said I am sure that there is a solution. Can any one help?
You're looking for a non-trivial solution v to A*v=v with v=[x;y;z] and...
A =
0.70710678118655 0 0.70710678118655
-0.50000000000000 0.70710678118655 0.50000000000000
-0.50000000000000 -0.70710678118655 0.50000000000000
You can transform this into (A-I)v=0 where I is the 3x3 identity matrix. What you have to do to find a nontrivial solution is checking the null space of A-I:
>> null(A-eye(3))
ans =
0.67859834454585
-0.67859834454585
0.28108463771482
So, you have a onedimensional nullspace. Otherwise you'd see more than one column. Every linear combination of the columns is a point in this null space that A-I maps to the null vector. So, every multiple of this vector is a solution to your problem.
Actually, your matrix A is a rotation matrix of the first kind because det(A)=1 and A'*A=identity. So it has an eigenvalue of 1 with the rotation axis as corresponding eigenvector. The vector I computed above is the normalized rotation axis.
Note: For this I replaced your 0.7071 with sqrt(0.5). If rounding errors are a concern but you know in advance that there has to be a nontrivial solution the best bet is to do a singular value decomposition of A-I and pick the right most right singular vector:
>> [u,s,v] = svd(A-eye(3));
>> v(:,end)
ans =
0.67859834454585
-0.67859834454585
0.28108463771482
This way you can calculate a vector v that minimizes |A*v-v| under the constraint that |v|=1 where |.| is the Euclidean norm.
I don't think you need to use the solve function as your equations is a system of linear equations.
Recast as a matrix equation:
Ax = B
In your case:
| -0.2929 0.0 0.7071 | | x | | 0 |
| -0.5 -0.2929 0.5 | | y | = | 0 |
| -0.5 -0.7071 -0.5 | | z | | 0 |
Use the built-in functionally of MATLAB to solve it. See e.g. MATLAB: Solution of Linear Systems of Equations.
The core of MATLAB is to solve this kind of equation.
Using FreeMat (an open-source MATLAB-like environment with
a GPL license; direct download URL for Windows installer):
A = [ -0.2929 0.0 0.7071; -0.5 -0.2929 0.5; -0.5 -0.7071 -0.5 ]
B = [0.0; 0.0; 0.0]
A\B
ans =
0
0
0
So the solution is: x = 0, y = 0, z = 0
The solution can also be derived by hand. Starting from the last two equations:
-0.5*x + 0.7071*y + 0.5*z = y
-0.5*x - 0.7071*y + 0.5*z = z
0.2929*y = -0.5*x + 0.5*z
0.7071*y = -0.5*x + 0.5*z
0.2929*y = 0.7071*y
Thus y = 0.0 and:
0.7071*y = -0.5*x + 0.5*z
0 = -0.5*x + 0.5*z
0 = -0.5*x + 0.5*z
0.5*x = 0.5*z
x = z
Inserting in the first equation:
0.7071*x + 0.7071*z = x
0.7071*x + 0.7071*x = x
1.4142*x = x
Thus x = 0.0. And as x = z, then z = 0.0.
x = 0, y = 0, z = 0 is the correct solution. This problem can be easily worked by hand.
A = [ 0.7071 0 0.7071 ;
-0.5 0.7071 0.5 ;
-0.5 -0.7071 0.5 ];
B = [1 ; 1 ; 1];
AA = A-diag(B)
-0.2929 0 0.7071
-0.5 -0.2929 0.5
-0.5 -0.7071 -0.5
BB = B-B
0
0
0
AA\BB
0
0
0
Friends use MATLAB command rref(A) for Ax=B.... It will give a upper triangular Matrix. Then u can calculate non-trivial solutions easily... But keep in mind that if rref(A)=I (as in your case) then non-trivial solutions do not exist.
BEST OF LUCK