I am running into issues when attempting to shift a Hessian to be positive definite for an optimization problem in Matlab. An example of my problem is:
H=[1 2 2;
2 3 2;
1 3 1];
[V,D]=eig(H);
While H*V-V*D, as it should, essentially equals zero:
V*D*V' does not provide the original H matrix
I actually tried to run your code:
H = [
1 2 2;
2 3 2;
1 3 1
];
[V,D] = eig(H);
test = norm(H*V-V*D,inf) / norm(V*D,inf);
and I'm seeing no obvious problems with the example you posted. The eigenvalues and vectors that I get satisfy their defining equation with high accuracy:
test = 7.57596318689868e-16
On the top of that, as #kpg987 pointed out, you have to use the inverse of V, not the transposed version of V. And if you perform the following test:
test = V * D * inv(V);
You will obtain your original hessian (or something very very close to it).
Related
a = [1 2 3
2 4 6
3 6 9];
b = pinv(a);
[U,S,V] = svd(a);
T = S;
T(find(S~=0)) = 1./S(find(S~=0));
svda = V * T' * U';
I find that the pinv method in Matlab uses the SVD decomposition to calculate pseudo-inverse, so I tried to solve the matrix a.
And as shown above, theoretically the b should be equal with svda, but the Matlab result said they are totally different. Why?
b is
0.00510204081632653 0.0102040816326531 0.0153061224489796
0.0102040816326531 0.0204081632653061 0.0306122448979592
0.0153061224489796 0.0306122448979592 0.0459183673469388
svda is
-2.25000000000000 -5.69876639328585e+15 3.79917759552390e+15
-2.14021397132170e+15 1.33712246709292e+16 -8.20074512351222e+15
1.42680931421447e+15 -7.01456098285751e+15 4.20077088383351e+15
How does pinv get to its result?
REASON:
Thanks to Cris, I check my S, and it do have 2 very large number, and that is the source of this strange result.
S:
14.0000000000000 0 0
0 1.00758232556386e-15 0
0 0 5.23113446604828e-17
By pinv method and Cris method, this 2 latter numbers should set to 0 which I didnt do. So here is the reason。
pinv doesn't just invert all the non-zero values of S, it also removes all the nearly zero values first. From the documentation:
pinv(A,TOL) treats all singular values of A that are less than TOL as zero. By default, TOL = max(size(A)) * eps(norm(A)).
pinv more or less does this:
[U,S,V] = svd(a);
I = find(abs(S) > max(size(a)) * eps(norm(a)));
T = zeros(size(S));
T(I) = 1 ./ S(I);
svda = V * T.' * U';
On my machine, isequal(svda,b) is true, which is a bit of a coincidence because the operations we're doing here are not exactly the same as those done by pinv, and you could expect rounding errors to be different. You can see what pinv does exactly by typing edit pinv in MATLAB. It's a pretty short function.
Note that I used T.', not T'. The former is the transpose, the latter is the complex conjugate transpose (Hermitian transpose). We're dealing with real-valued matrices here, so it doesn't make a difference in this case, but it's important to use the right operator.
I've been set a question asking me to solve a system of linear equations. In the question it states I should set up a matrix A and column vector b to solve the equation Ax=b, where x is the column vector (w x y z).
A = [1 1 1 1; 0 1 4 -2; 2 0 -2 1; 1 -2 -1 1]
b = [28;7;22;-4]
A1 = inv(A).*b
sum(A1,2)
This is what I've done so far, however I know the answer that MATLAB gives me is incorrect, as the right solutions should be w=10.5, x=9, y=2.5, z=6.
Can someone point me in the right direction/ show me where I'm going wrong? (I'm fairly new to MATLAB so very unsure about it all).
Thanks.
A = [1 1 1 1; 0 1 4 -2; 2 0 -2 1; 1 -2 -1 1];
b = [28;7;22;-4];
A1 = A \ b;
ans = sum(A1,2);
For a reference concerning the \ operator, please read this: https://it.mathworks.com/help/matlab/ref/mldivide.html
The correct code to use your technique would be:
A1 = inv(A) * b;
but as you may notice, the Matlab code analyzer will point out that:
For solving a system of linear equations, the inverse of a
matrix is primarily of theoretical value. Never use the inverse of a
matrix to solve a linear system Ax=b with x=inv(A)*b, because it is
slow and inaccurate.
Replace inv(A)*b with A\b
Replace b*inv(A) with b/A
and that:
INV(A)*b can be slower than A\b
In an attempt to speed up for loops (or eliminate all together), I've been trying to pass matrices into functions. I have to use sine and cosine as well. However, when I attempt to find the integral of a matrix where the elements are composed of sines and cosines, it doesn't work and I can't seem to find a way to make it do so.
I have a matrix SI that is composed of sines and cosines with respect to a variable that I have defined using the Symbolic Math Toolbox. As such, it would actually be even better if I could just pass the SI matrix and receive a matrix of values that is the integral of the sine/cosine function at every location in this matrix. I would essentially get a square matrix back. I am not sure if I phrased that very well, but I have the following code below that I have started with.
I = [1 2; 3 4];
J = [5 6; 7 8];
syms o;
j = o*J;
SI = sin(I + j);
%SI(1,1) = sin(5*o + 1)
integral(#(o) o.*SI(1,1), 0,1);
Ideally, I would want to solve integral(#(o) o*SI,0,1) and get a matrix of values. What should I do here?
Given that A, B and C are all N x N matrices, for the moment, let's assume they're all 2 x 2 matrices to make the example I'm illustrating more succinct to understand. Let's also define o as a mathematical symbol based on your comments in your question above.
syms o;
A = [1 2; 3 4];
B = [5 6; 7 8];
C = [9 10; 11 12];
Let's also define your function f according to your comments:
f = o*sin(A + o*B + C)
We thus get:
f =
[ o*sin(5*o + 10), o*sin(6*o + 12)]
[ o*sin(7*o + 14), o*sin(8*o + 16)]
Remember, for each element in f, we take the corresponding elements in A, B and C and add them together. As such, for the first row and first column of each matrix, we have 1, 5 and 9. As such, A + o*B + C for the first row, first column equates to: 1 + 5*o + 9 = 5*o + 10.
Now if you want to integrate, just use the int command. This will find the exact integral, provided that the integral can be solvable in closed form. int also can handle matrices so it will integrate each element in the matrix. You can call it like so:
out = int(f,a,b);
This will integrate f for each element from the lower bound a to the upper bound b. As such, supposing our limits were from 0 to 1 as you said. Therefore:
out = int(f,0,1);
We thus get:
out =
[ sin(15)/25 - sin(10)/25 - cos(15)/5, sin(18)/36 - sin(12)/36 - cos(18)/6]
[ sin(21)/49 - sin(14)/49 - cos(21)/7, sin(24)/64 - sin(16)/64 - cos(24)/8]
Bear in mind that out is defined in the symbolic math toolbox. If you want the actual numerical values, you need to cast the answer to double. Therefore:
finalOut = double(out);
We thus get:
finalOut =
0.1997 -0.1160
0.0751 -0.0627
Obviously, this can generalize for any size M x N matrices, so long as they all share the same dimensions.
Caveat
sin, cos, tan and the other related functions have their units in radians. If you wish for the degrees equivalent, append a d at the end of the function (i.e. sind, cosd, tand, etc.)
I believe this is the answer you're after. Good luck!
I have a 3x3 matrix, A. I also compute a value, g, as the maximum eigen value of A. I am trying to change the element A(3,3) = 0 for all values from zero to one in 0.10 increments and then update g for each of the values. I'd like all of the other matrix elements to remain the same.
I thought a for loop would be the way to do this, but I do not know how to update only one element in a matrix without storing this update as one increasingly larger matrix. If I call the element at A(3,3) = p (thereby creating a new matrix Atry) I am able (below) to get all of the values from 0 to 1 that I desired. I do not know how to update Atry to get all of the values of g that I desire. The state of the code now will give me the same value of g for all iterations, as expected, as I do not know how to to update Atry with the different values of p to then compute the values for g.
Any suggestions on how to do this or suggestions for jargon or phrases for me to web search would be appreciated.
A = [1 1 1; 2 2 2; 3 3 0];
g = max(eig(A));
% This below is what I attempted to achieve my solution
clear all
p(1) = 0;
Atry = [1 1 1; 2 2 2; 3 3 p];
g(1) = max(eig(Atry));
for i=1:100;
p(i+1) = p(i)+ 0.01;
% this makes a one giant matrix, not many
%Atry(:,i+1) = Atry(:,i);
g(i+1) = max(eig(Atry));
end
This will also accomplish what you want to do:
A = #(x) [1 1 1; 2 2 2; 3 3 x];
p = 0:0.01:1;
g = arrayfun(#(x) eigs(A(x),1), p);
Breakdown:
Define A as an anonymous function. This means that the command A(x) will return your matrix A with the (3,3) element equal to x.
Define all steps you want to take in vector p
Then "loop" through all elements in p by using arrayfun instead of an actual loop.
The function looped over by arrayfun is not max(eig(A)) but eigs(A,1), i.e., the 1 largest eigenvalue. The result will be the same, but the algorithm used by eigs is more suited for your type of problem -- instead of computing all eigenvalues and then only using the maximum one, you only compute the maximum one. Needless to say, this is much faster.
First, you say 0.1 increments in the text of your question, but your code suggests you are actually interested in 0.01 increments? I'm going to operate under the assumption you mean 0.01 increments.
Now, with that out of the way, let me state what I believe you are after given my interpretation of your question. You want to iterate over the matrix A, where for each iteration you increase A(3, 3) by 0.01. Given that you want all values from 0 to 1, this implies 101 iterations. For each iteration, you want to calculate the maximum eigenvalue of A, and store all these eigenvalues in some vector (which I will call gVec). If this is correct, then I believe you just want the following:
% Specify the "Current" A
CurA = [1 1 1; 2 2 2; 3 3 0];
% Pre-allocate the values we want to iterate over for element (3, 3)
A33Vec = (0:0.01:1)';
% Pre-allocate a vector to store the maximum eigenvalues
gVec = NaN * ones(length(A33Vec), 1);
% Loop over A33Vec
for i = 1:1:length(A33Vec)
% Obtain the version of A that we want for the current i
CurA(3, 3) = A33Vec(i);
% Obtain the maximum eigen value of the current A, and store in gVec
gVec(i, 1) = max(eig(CurA));
end
EDIT: Probably best to paste this code into your matlab editor. The stack-overflow automatic text highlighting hasn't done it any favors :-)
EDIT: Go with Rody's solution (+1) - it is much better!
I'm trying to use MatLab code as a way to learn math as a programmer.
So reading I'm this post about subspaces and trying to build some simple matlab functions that do it for me.
Here is how far I got:
function performSubspaceTest(subset, numArgs)
% Just a quick and dirty function to perform subspace test on a vector(subset)
%
% INPUT
% subset is the anonymous function that defines the vector
% numArgs is the the number of argument that subset takes
% Author: Lasse Nørfeldt (Norfeldt)
% Date: 2012-05-30
% License: http://creativecommons.org/licenses/by-sa/3.0/
if numArgs == 1
subspaceTest = #(subset) single(rref(subset(rand)+subset(rand))) ...
== single(rref(rand*subset(rand)));
elseif numArgs == 2
subspaceTest = #(subset) single(rref(subset(rand,rand)+subset(rand,rand))) ...
== single(rref(rand*subset(rand,rand)));
end
% rand just gives a random number. Converting to single avoids round off
% errors.
% Know that the code can crash if numArgs isn't given or bigger than 2.
outcome = subspaceTest(subset);
if outcome == true
display(['subset IS a subspace of R^' num2str(size(outcome,2))])
else
display(['subset is NOT a subspace of R^' num2str(size(outcome,2))])
end
And these are the subset that I'm testing
%% Checking for subspaces
V = #(x) [x, 3*x]
performSubspaceTest(V, 1)
A = #(x) [x, 3*x+1]
performSubspaceTest(A, 1)
B = #(x) [x, x^2, x^3]
performSubspaceTest(B, 1)
C = #(x1, x3) [x1, 0, x3, -5*x1]
performSubspaceTest(C, 2)
running the code gives me this
V =
#(x)[x,3*x]
subset IS a subspace of R^2
A =
#(x)[x,3*x+1]
subset is NOT a subspace of R^2
B =
#(x)[x,x^2,x^3]
subset is NOT a subspace of R^3
C =
#(x1,x3)[x1,0,x3,-5*x1]
subset is NOT a subspace of R^4
The C is not working (only works if it only accepts one arg).
I know that my solution for numArgs is not optimal - but it was what I could come up with at the current moment..
Are there any way to optimize this code so C will work properly and perhaps avoid the elseif statments for more than 2 args..?
PS: I couldn't seem to find a build-in matlab function that does the hole thing for me..
Here's one approach. It tests if a given function represents a linear subspace or not. Technically it is only a probabilistic test, but the chance of it failing is vanishingly small.
First, we define a nice abstraction. This higher order function takes a function as its first argument, and applies the function to every row of the matrix x. This allows us to test many arguments to func at the same time.
function y = apply(func,x)
for k = 1:size(x,1)
y(k,:) = func(x(k,:));
end
Now we write the core function. Here func is a function of one argument (presumed to be a vector in R^m) which returns a vector in R^n. We apply func to 100 randomly selected vectors in R^m to get an output matrix. If func represents a linear subspace, then the rank of the output will be less than or equal to m.
function result = isSubspace(func,m)
inputs = rand(100,m);
outputs = apply(func,inputs);
result = rank(outputs) <= m;
Here it is in action. Note that the functions take only a single argument - where you wrote c(x1,x2)=[x1,0,x2] I write c(x) = [x(1),0,x(2)], which is slightly more verbose, but has the advantage that we don't have to mess around with if statements to decide how many arguments our function has - and this works for functions that take input in R^m for any m, not just 1 or 2.
>> v = #(x) [x,3*x]
>> isSubspace(v,1)
ans =
1
>> a = #(x) [x(1),3*x(1)+1]
>> isSubspace(a,1)
ans =
0
>> c = #(x) [x(1),0,x(2),-5*x(1)]
>> isSubspace(c,2)
ans =
1
The solution of not being optimal barely scratches the surface of the problem.
I think you're doing too much at once: rref should not be used and is complicating everything. especially for numArgs greater then 1.
Think it through: [1 0 3 -5] and [3 0 3 -5] are both members of C, but their sum [4 0 6 -10] (which belongs in C) is not linear product of the multiplication of one of the previous vectors (e.g. [2 0 6 -10] ). So all the rref in the world can't fix your problem.
So what can you do instead?
you need to check if
(randn*subset(randn,randn)+randn*subset(randn,randn)))
is a member of C, which, unless I'm mistaken is a difficult problem: Conceptually you need to iterate through every element of the vector and make sure it matches the predetermined condition. Alternatively, you can try to find a set such that C(x1,x2) gives you the right answer. In this case, you can use fminsearch to solve this problem numerically and verify the returned value is within a defined tolerance:
[s,error] = fminsearch(#(x) norm(C(x(1),x(2)) - [2 0 6 -10]),[1 1])
s =
1.999996976386119 6.000035034493023
error =
3.827680714104862e-05
Edit: you need to make sure you can use negative numbers in your multiplication, so don't use rand, but use something else. I changed it to randn.