I am trying to compute this
in MATLAB but the code requires about 8 hours to compile. In particular e, Ft=[h(t);q(t)] and Omega are 2x1 matrices (e' is 1x2), Gamma is a 2x2 matrix and n=30. Can someone help me to optimize this code?
I tried in this way:
aux=[0;0];
for k=0:29
for j=1:k-1
aux=[aux Gamma^j*Omega];
end
E(t,k+1)= e'*(sum(aux,2)+Gamma^k*[h(t);q(t)]);
end
Vix=1/30*sum(E,2);
EDIT
now I changed into this and it is faster, but I am not sure that I am applying correctly the formula in the picture...
for t=2:T
% 1. compute today's volatility
csi(t) = log(SP500(t)/SP500(t-1))-r(t)+0.5*h(t);
q(t+1) = omega+rho*q(t)+phi*((csi(t)-lambda*sqrt(h(t)))^2-h(t));
h(t+1) = q(t)+alpha*((csi(t)-lambda*sqrt(h(t)))^2-q(t))+beta*(h(t)-q(t));
for k=1:30
aux=zeros(2,k);
for j=0:k-1
aux(:,j+1)=Gamma^j*Omega;
end
E(t,k)= e'*(sum(aux,2)+Gamma^k*[h(t);q(t)]);
end
end
Vix(2:end)=1/30*sum(E(2:end,:),2);
(I don't need Vix(1))
Here are some reasons I can think of:
REPEATED COPYING(No preallocation) The main reason for the long run time is the line aux=[aux Gamma^j*Omega] line, in which an array is concatenated at every loop iteration. MATLAB's debugger should have flagged this for you in its editor and should have cited that "memory preallocation" using zeros should be implemented.
Essentially, when one concatenates arrays this way, MATLAB is internally making copies of the array at every loop iteration, thus, in addition to the math operations copying is taking place. As the array grows, the copying operations become ever more expensive. This is avoided by preallocation, which consists of predefining the size of the storage array (in this case the variable aux) so that MATLAB doesn't have to keep on allocating space on the go. Try:
aux = zeros(2, 406); %Creates a 2 by 406 array. I explain how I get 406 below:
p = 0; %A variable that indexes the columns of aux
for k=0:29
for j=1:k-1
p = p+1; %Update column counter
aux(:,p) = Gamma^j*Omega; % A 2x2 matrix multiplied by a 2x1 matrix yields a 2x1.
end
E(t,k+1)= e'*(sum(aux,2)+Gamma^k*[h(t);q(t)]);
end
Vix=1/30*sum(E,2);
Now, MATLAB simply overwrites the individual elements of aux instead of copying aux, and concatenating it with Gamma^j*Omega, and then overwriting aux. Essentially, the above makes MATLAB allocate space for aux ONCE instead of 406 times. I figured out that aux ends up being a 2 by 406 array for the n=30 case in the end by running this code:
p = 0;
for k = 0:29
for j = 1:k-1
p = p + 1;
end
end
To know the final size of aux for other values of n you should see if a formula for it is available (or derive your own).
LOOPING TRANSPOSITION OF A CONSTANT?
Next, e'. As you may know, ' is the transpose operation. From your sample code, the variable e is not edited inside the for loops, yet you have the ' operator inside the outer for loop. If you perform the transpose operation once outside the outer for loop you save yourself the expense of transposing it at every loop iteration.
RUNNING TOTAL
As a final note, I would suggest replacing sum(aux,2) with a variable that keeps a running total. This is because currently, this makes MATLAB sum over the entirety of aux at every loop iteration.
Hope this helps mate.
I have been asked to obtain the first 15 triplets according to this series and this code ought to work. However, it does only produce a table (15*3) filled with zero rather than the 15 Pythagorean triplets? Any help will be welcome.
A = zeros(15, 3);
ii = 1;
for c = 5:120
c2=c^2;
for a=1:c-1
a2=a^2;
for b=a:c-1
if c2-(a2+b^2) == 0
A(ii,1) = a;
A(ii,2) = b;
A(ii,3) = c;
ii=ii+1;
if A(15, 1) ~= 0
flag = 1;
break
end
end
end
if flag == 1
break
end
end
if flag == 1
break
end
end
T1 = array2table(A);
disp(T1)
So, the code generated a correct table on application-restart before failing on all subsequent attempts. And, now I notice that the code runs successfully only for the first time after every relaunch of the application. (Resolved, thanks Dan Pollard.)
Also, interested in knowing if there is any way to not write an upper limit (120) into the code.
I don't think your if statement is ever satisfied. For example, for c=5, you'd expect a=3, b=4 to be a triplet. But you're only letting a and b go up to floor(sqrt(c-1)), which is 2.
Do you mean to let a and b go up to floor(sqrt(c2-1))?
Edit As the question has changed.
When you run the code, Matlab creates all the variables which you assign, and stores them in the workspace. This can be useful, but here it's hurting you as you have the variable flag which is stored as 1. This means that when the code runs, it checks if flag==1 after the first run through b, which it is, so the code ends. Resolve this by placing clear; at the beginning of your script.
There isn't a practical way to remove the upper limit on c. Matlab has the built-in variable Inf but at best Matlab won't let you use it in that context. Realistically you could just replace the 120 with a really large number, but this will take more time and more memory as the number gets bigger. Computers have a finite RAM to store matlab arrays in though, and there are infinitely many pythagorean triples, so doing the calculation without an upper limit will fail in some way.
Code
clear;clc
T=800;
Pc=48.45;
Tc=375;
w=0.153;
R=82.06;
a=((0.45724)*(R^2)*(Tc^2))/Pc;
b=((0.07780)*R*Tc)/Pc;
B=(0.37464+(1.54226*w)-(0.26992*(w^2)));
Tr=T/Tc;
s=(1+(B*(1-sqrt(Tr))))^2;
for Vm=90:5:1000
P=((R*T)/(Vm-b))-((a*s)/((Vm)^2+(2*b*Vm)-b^2));
end
plot(Vm, P)
Problem
Every time I run this code, it comes out with a completely empty plot with just numbers on both axes as the image shown below. I've checked my code a few times, but I still can't find the problem, especially since the code runs with no errors. The result I am supposed to be getting on this plot is the behavior of P as the value of Vm increases.
The result of the code
Additional information about the source of the question
Here's the original question if you're interested (Exercise 1).
The original question (Exercise 1)
Try displaying your variables. You'll see Vm is not an array, rather it's a single-valued scalar. When you loop over Vm it takes one value at a time; it doesn't build an array.
MATLAB can do calculations on multiple values at once, so if you define Vm to be an array and drop the loop I'm guessing it'll work...
Try something like this (replace the for-loop with these lines):
Vm = 90:5:1000
P=((R*T)./(Vm-b))-((a*s)./((Vm).^2+(2*b.*Vm)-b^2));
P will then be an array. Notice we use .* rather than * when multiplying by the array Vm since we want to do element-wise multiplication, rather than matrix multiplication. Similarly we use ./ rather than / and .^ rather than ^.
EDIT: If you need to use a for-loop then you could define both P and Vm as arrays, and then work on each element separately within a loop:
Vm = 90:5:1000;
P = NaN(size(Vm));
for i=1:numel(Vm)
P(i)=((R*T)./(Vm(i)-b))-((a*s)./((Vm(i)).^2+(2*b.*Vm(i))-b^2));
end
Since the above is working on scalar values, it doesn't matter if you use .* or *...
I have some code which uses sparse indexing (and there's no way that I can get around that). I run this in a function, and use it for two problems, where the sizes of all the variables involved do not change. However, for one problem, the sparse indexing part takes 5 seconds, and for the other, takes 25 seconds.
I checked the size of every variable involved, and they are the same for both problems. I also checked that xv is a full matrix for both problem types.
So, anyone else ever run into something weird like this? Any ideas as to why this would happen? Mainly I am trying to make the code more efficient, and while 5 seconds is ok for my particular application, 25 seconds (especially when I can't explain it) is very bad.
Edit: Here is a link to a photo that profiles this weird behavior. The runtime values were recorded on the third run to ensure that the size of X is also not changing. And I did check that xv is a dense (not sparse) matrix both times.
https://www.dropbox.com/s/i41j6afanzbjdyg/weird_bcd_thing.png?dl=0
Thanks so much for any help!
Code below (runs in a for loop). If I use ptype = 1, then it's 5 seconds, ptype = 3 is 25 seconds.
clvec = cliques{k};
xcurr = full(X(clvec));
xv = reshape(xcurr - Z(offset_index(k) + 1 : offset_index(k) + ncl^2),ncl,ncl);
%these two functions both take a dense symmetric matrix and return a dense symmetric matrix, and in both cases the size is the same for a given k.
if ptype == 1
xv = proj_PSD(xv,0,0);
elseif ptype == 3
xv = proj_Schoenberg(xv,0);
end
Xd = vec(xv) - xcurr;
%THIS IS THE WEIRD LINE
tic
X(clvec) = xv;
toc;
In the 'WEIRD LINE' : X(clvec) = xv;
You are using a random access to a sparse matrix.
This access in a sparse matrix is not constant and depends on its data. The time is may depend on the matrix values and the indices you are trying to access.
This is not the case in regular matrix, where you usually get a stable access time, and faster.
In order to assure a stable constant access try to change the implementation based on your specific matrix usage, try to avoid values assign by random access.
See next code for as a reference:
X = sparse(randi(100,50,1),randi(100,50,1),randn(1),100,100);
for i=1:10000
rand_inds{i} = randperm(10000,100);
end
for i=1:100
ti = tic;
X(rand_inds{i}) = 3;
to_X(i) = toc(ti);
end
Xf = full(X);
for i=1:100
ti = tic;
Xf(rand_inds{i}) = 3;
to_Xf(i) = toc(ti);
end
figure;plot(to_X);hold on;plot(to_Xf,'r');
I solved my problem! I'm posting the answer because I think it's interesting.
One thing I didn't mention in the question is that the loop goes from k = 1 to k = L, and for ptype = 3, we add one more step, and that's assigning all the diagonal indices to 0:
X(diag_index) = 0
where diag_index is computed ahead of time.
The problem is, instead of just assigning the values to 0, MATLAB will automatically discard these indices, and the next loop, when accessing diagonal indices, it has to re-allocate for X. So, I changed that line to
X(diag_index) = eps;
and now they both run equally fast! (It's not the best solution, since that's going to be a source of error later, but there's no more mystery!)
The answer is never what you think it would be...
I must to use angle = atan2(norm(cross(a,b)),dot(a,b)), for calculating the angle between two vectors a,b and these are double type and norm is undefined for this type. How do I resolve this problem? I need to calculate the angle between two vectors this way.
In your comments, you have shown us how you are actually writing out the angle calculation and it is not the same as how you have put it in your post.
atan2(norm(cross(I(i,j,:),I_avg)),dot(I(i,j,:),I_avg));
I is an image you are loading in. I'm assuming it's colour because of the way you are subsetting I. Because I is a 3D matrix, doing I(i,j,:) will give you a 1 x 1 x 3 vector when in fact this has to be a 1D vector. norm does not recognize this structure which is why you're getting this error. Therefore, you need to use squeeze to remove the singleton dimensions so that this will become a 3 x 1 vector, rather than a 1 x 1 x 3 vector. As such, you need to rewrite your code so that you're doing this instead. Bear in mind that in your comments, angle is always overwritten inside the for loop, so you probably want to save the results of each pixel. With this, you probably want to create a 2D array of angles that will store these results. In other words:
I=imread('thesis.jpg');
I = double(I);
angles = zeros(m,n);
I_avg = squeeze(I_avg); %// Just in case
for i=1:m
for j=1:n
pixels = squeeze(I(i,j,:)); %// Add this statement and squeeze
angles(i,j) = atan2(norm(pixels,I_avg)),dot(pixels,I_avg)); %// Change
end
end
Minor note
MATLAB has a built-in function called angle that determines the angle from the origin to a complex number in the complex plane. It is not recommended you call your variable angle as this will unintentionally shadow over the angle function, and any other code that you create from this point onwards may rely on that actual angle function, and you will get unintended results.
Another minor note
Using i and j as loop variables is not recommended. These letters are reserved for the complex number, and this can produce unintentional results. Take a look at this question and post by Shai here - Using i and j as variables in Matlab. As such, it is suggested you use other variable names instead.
As #rayryeng has successfully answered this question, I would like to turn my post into a more general one by sharing my experience in debugging in Matlab. I hope anyone who somehow managed to find this post get more or less thinking about the habits a good programmer should have.
The question goes like: "How would I do if I get errors?"
Here's an excellent article by Eric in which he lists the rule-of-thumbs when you encounter a bug and wish to get rid of it. It's originally been cited by Stackoverflow, and that's the reason I read it.
If you still get no clue / idea how you can play with your code, see how this person does:
Pin-point the buggy line
(The number should start with 0) Make sure before running a script, you clear out any previously stored variables, including the notorious i and j's (you should never see them in any workspace). If any one is needed for the buggy code to run, save('buggy.mat','importantvar') before clear and load('buggy.mat') after clear.
By doing so, you can isolate your buggy code from anything else, which could have bad influences. For example, in a previously called script, there is a line
double = [2,4,6]; % you should never name a variable `double`
and in the next script, you have
>> e = str2num('uint8(200)')
e =
200
>> double(e)
Index exceeds matrix dimensions.
>>
>> f = single(2.36)
f =
2.3600
>> double(f)
Subscript indices must either be real positive integers or
logicals.
>>
The reason is double is no longer an inbuild function, but a user-defined variable. Too bad to pick up a name carelessly!
....anyway, let's clear the workspace and get rid of double.
>> clear
Read the error message, thoroughly.
Now let's begin with OP's problem. The original code (trimmed) goes like this -
img = imread('peppers.png');
a = img(300,200,:);
b = img(200,300,:);
d = norm(cross(a,b));
.... hence the error
Undefined function 'norm' for input arguments of type 'uint8'.
Error in untitled (line 6)
d = norm(cross(a,b));
Most beginners are only interested in the first line of the error message, which by it alone usually doesn't provide any useful help, or only in the red color, which leads to the famous question "my code does not work!"
But think twice. You still have another 2 lines unread! Error in untitled (line 6) says I'm running a script named untitled and the (first) error lies in line 6, and the code in that line is d = norm(cross(a,b));.
Now, at least you know a little more about your code - "My code d = norm(cross(a,b)); doesn't work!"
Although most likely we may also vote this kind of question to get closed, it's still much much better than a simply "It does not work!".
Now we can pin-point the buggy line
try
% this line will raise an error
d = norm(cross(a,b));
catch err
disp(err.message)
end
Look into the functions
First, make sure the inner function cross works as expected -
>> cross(a,b)
ans(:,:,1) =
0
ans(:,:,2) =
255
ans(:,:,3) =
0
>>
Good. So now we can even narrow down the error to the outer norm.
One more thing to mention. You can always find Mathworks' documentation for any in-build function, by typing "matlab function", such as "matlab norm" in Google (or any other search engine) and clicking on the first result. If you prefer, you can also type in Matlab command window doc _function_ such as doc norm and read the doc in Matlab. It's of course a pleasure of us on Stackoverflow to give you the reference by doing the same thing, but it takes a longer time because a human is, in this aspect, always slower than a search engine.
The error reads Undefined function 'norm' for input arguments of type 'uint8'.. So the input for norm should not be uint8, unsigned 8-bit integer. But what should it be?
% why `norm` "does not work"?
% this line runs perfectly well
norm(cross([1,2,3], [4,5,6]))
% so what is working?
class([1,2,3]) % so `norm` works for `double`
One thing we can do now is convert a and b to double precision. Let's try it now.
% try fixing 'uint8' error
a2 = double(a);
b2 = double(b);
whos a b % now they are double, which `norm` should work for
try
% this line will raise an error
d = norm(cross(a2,b2));
catch err
disp(err.message)
end
Now the error becomes Input must be 2-D.. What's wrong with the input?
% what is "must be 2-D" error?
size(a2) % a2 is 3-D
disp(b2) % b2 is also 3-D
This gives output in command window
ans =
1 1 3
(:,:,1) =
255
(:,:,2) =
150
(:,:,3) =
0
In OP's problem, he/she is trying to calculate something about color difference (to the best of my knowledge) which involves the angle between two color vectors in RGB space. So the vectors are needed. With imread, each pixel of the image is stored as 3 elements in the matrix, first 2 dimension being its physical position, the 3 dimension being RGB channel components. Hence pixel(200,300) with color rgb[255,150,0] is stored by us in variable b wihch is a 3-D vector.
By understanding what we need and what Matlab can do, we can combine these two points into one. We need the norm of the cross product of a and b, while the useful information (the 3 component values) is stored in the 3rd dimension. Matlab can calculate the norm of the cross product of a vector with all its information in the 1st dimension. (Here, "dimension" refers to that of the Matlab variable; a vector with 3 elements in its 1st dimension is physically a 3-D vector).
After thinking twice, we are now able to debug our code - just put all 3 elements into the 1st dimension.
% so we want the 3 elements in the 3rd dimension become in the 1st dim
a3 = squeeze(a2);
b3 = reshape(b2,numel(b2),[]);
try
d = norm(cross(a3,b3));
catch err
disp(err.message)
end
d
Bonus: If by default Matlab treats a 3-D vector as a "1-D array", then most probably the cross function has not been working correctly. Let's make a check -
>> clear
>> a = [1,2,3]
a =
1 2 3
>> b=[4,5,6]
b =
4 5 6
>> cross(a,b)
ans =
-3 6 -3
>>
The result should be the same as the one we can get by calculating by hand.
Now if we put the components into the 3rd dimension of the variable -
>> clear
>> a(1,1,:)=[1,2,3]
a(:,:,1) =
1
a(:,:,2) =
2
a(:,:,3) =
3
>> b(1,1,:)=[4,5,6]
b(:,:,1) =
4
b(:,:,2) =
5
b(:,:,3) =
6
>> cross(a,b)
ans(:,:,1) =
-3
ans(:,:,2) =
6
ans(:,:,3) =
-3
>>
.... seems OK. cross also puts the result in the 3rd dimension. In fact, Mathworks' documentation says
If A and B are vectors, then they must have a length of 3.
If A and B are matrices or multidimensional arrays, then they must
have the same size. In this case, the cross function treats A and B as
collections of three-element vectors. The function calculates the
cross product of corresponding vectors along the first array dimension
whose size equals 3.
At last, one thing is always correct to anyone who wants to do something with programming - be cautious and prudent when writing your code.