I have some code which uses sparse indexing (and there's no way that I can get around that). I run this in a function, and use it for two problems, where the sizes of all the variables involved do not change. However, for one problem, the sparse indexing part takes 5 seconds, and for the other, takes 25 seconds.
I checked the size of every variable involved, and they are the same for both problems. I also checked that xv is a full matrix for both problem types.
So, anyone else ever run into something weird like this? Any ideas as to why this would happen? Mainly I am trying to make the code more efficient, and while 5 seconds is ok for my particular application, 25 seconds (especially when I can't explain it) is very bad.
Edit: Here is a link to a photo that profiles this weird behavior. The runtime values were recorded on the third run to ensure that the size of X is also not changing. And I did check that xv is a dense (not sparse) matrix both times.
https://www.dropbox.com/s/i41j6afanzbjdyg/weird_bcd_thing.png?dl=0
Thanks so much for any help!
Code below (runs in a for loop). If I use ptype = 1, then it's 5 seconds, ptype = 3 is 25 seconds.
clvec = cliques{k};
xcurr = full(X(clvec));
xv = reshape(xcurr - Z(offset_index(k) + 1 : offset_index(k) + ncl^2),ncl,ncl);
%these two functions both take a dense symmetric matrix and return a dense symmetric matrix, and in both cases the size is the same for a given k.
if ptype == 1
xv = proj_PSD(xv,0,0);
elseif ptype == 3
xv = proj_Schoenberg(xv,0);
end
Xd = vec(xv) - xcurr;
%THIS IS THE WEIRD LINE
tic
X(clvec) = xv;
toc;
In the 'WEIRD LINE' : X(clvec) = xv;
You are using a random access to a sparse matrix.
This access in a sparse matrix is not constant and depends on its data. The time is may depend on the matrix values and the indices you are trying to access.
This is not the case in regular matrix, where you usually get a stable access time, and faster.
In order to assure a stable constant access try to change the implementation based on your specific matrix usage, try to avoid values assign by random access.
See next code for as a reference:
X = sparse(randi(100,50,1),randi(100,50,1),randn(1),100,100);
for i=1:10000
rand_inds{i} = randperm(10000,100);
end
for i=1:100
ti = tic;
X(rand_inds{i}) = 3;
to_X(i) = toc(ti);
end
Xf = full(X);
for i=1:100
ti = tic;
Xf(rand_inds{i}) = 3;
to_Xf(i) = toc(ti);
end
figure;plot(to_X);hold on;plot(to_Xf,'r');
I solved my problem! I'm posting the answer because I think it's interesting.
One thing I didn't mention in the question is that the loop goes from k = 1 to k = L, and for ptype = 3, we add one more step, and that's assigning all the diagonal indices to 0:
X(diag_index) = 0
where diag_index is computed ahead of time.
The problem is, instead of just assigning the values to 0, MATLAB will automatically discard these indices, and the next loop, when accessing diagonal indices, it has to re-allocate for X. So, I changed that line to
X(diag_index) = eps;
and now they both run equally fast! (It's not the best solution, since that's going to be a source of error later, but there's no more mystery!)
The answer is never what you think it would be...
I must to use angle = atan2(norm(cross(a,b)),dot(a,b)), for calculating the angle between two vectors a,b and these are double type and norm is undefined for this type. How do I resolve this problem? I need to calculate the angle between two vectors this way.
In your comments, you have shown us how you are actually writing out the angle calculation and it is not the same as how you have put it in your post.
atan2(norm(cross(I(i,j,:),I_avg)),dot(I(i,j,:),I_avg));
I is an image you are loading in. I'm assuming it's colour because of the way you are subsetting I. Because I is a 3D matrix, doing I(i,j,:) will give you a 1 x 1 x 3 vector when in fact this has to be a 1D vector. norm does not recognize this structure which is why you're getting this error. Therefore, you need to use squeeze to remove the singleton dimensions so that this will become a 3 x 1 vector, rather than a 1 x 1 x 3 vector. As such, you need to rewrite your code so that you're doing this instead. Bear in mind that in your comments, angle is always overwritten inside the for loop, so you probably want to save the results of each pixel. With this, you probably want to create a 2D array of angles that will store these results. In other words:
I=imread('thesis.jpg');
I = double(I);
angles = zeros(m,n);
I_avg = squeeze(I_avg); %// Just in case
for i=1:m
for j=1:n
pixels = squeeze(I(i,j,:)); %// Add this statement and squeeze
angles(i,j) = atan2(norm(pixels,I_avg)),dot(pixels,I_avg)); %// Change
end
end
Minor note
MATLAB has a built-in function called angle that determines the angle from the origin to a complex number in the complex plane. It is not recommended you call your variable angle as this will unintentionally shadow over the angle function, and any other code that you create from this point onwards may rely on that actual angle function, and you will get unintended results.
Another minor note
Using i and j as loop variables is not recommended. These letters are reserved for the complex number, and this can produce unintentional results. Take a look at this question and post by Shai here - Using i and j as variables in Matlab. As such, it is suggested you use other variable names instead.
As #rayryeng has successfully answered this question, I would like to turn my post into a more general one by sharing my experience in debugging in Matlab. I hope anyone who somehow managed to find this post get more or less thinking about the habits a good programmer should have.
The question goes like: "How would I do if I get errors?"
Here's an excellent article by Eric in which he lists the rule-of-thumbs when you encounter a bug and wish to get rid of it. It's originally been cited by Stackoverflow, and that's the reason I read it.
If you still get no clue / idea how you can play with your code, see how this person does:
Pin-point the buggy line
(The number should start with 0) Make sure before running a script, you clear out any previously stored variables, including the notorious i and j's (you should never see them in any workspace). If any one is needed for the buggy code to run, save('buggy.mat','importantvar') before clear and load('buggy.mat') after clear.
By doing so, you can isolate your buggy code from anything else, which could have bad influences. For example, in a previously called script, there is a line
double = [2,4,6]; % you should never name a variable `double`
and in the next script, you have
>> e = str2num('uint8(200)')
e =
200
>> double(e)
Index exceeds matrix dimensions.
>>
>> f = single(2.36)
f =
2.3600
>> double(f)
Subscript indices must either be real positive integers or
logicals.
>>
The reason is double is no longer an inbuild function, but a user-defined variable. Too bad to pick up a name carelessly!
....anyway, let's clear the workspace and get rid of double.
>> clear
Read the error message, thoroughly.
Now let's begin with OP's problem. The original code (trimmed) goes like this -
img = imread('peppers.png');
a = img(300,200,:);
b = img(200,300,:);
d = norm(cross(a,b));
.... hence the error
Undefined function 'norm' for input arguments of type 'uint8'.
Error in untitled (line 6)
d = norm(cross(a,b));
Most beginners are only interested in the first line of the error message, which by it alone usually doesn't provide any useful help, or only in the red color, which leads to the famous question "my code does not work!"
But think twice. You still have another 2 lines unread! Error in untitled (line 6) says I'm running a script named untitled and the (first) error lies in line 6, and the code in that line is d = norm(cross(a,b));.
Now, at least you know a little more about your code - "My code d = norm(cross(a,b)); doesn't work!"
Although most likely we may also vote this kind of question to get closed, it's still much much better than a simply "It does not work!".
Now we can pin-point the buggy line
try
% this line will raise an error
d = norm(cross(a,b));
catch err
disp(err.message)
end
Look into the functions
First, make sure the inner function cross works as expected -
>> cross(a,b)
ans(:,:,1) =
0
ans(:,:,2) =
255
ans(:,:,3) =
0
>>
Good. So now we can even narrow down the error to the outer norm.
One more thing to mention. You can always find Mathworks' documentation for any in-build function, by typing "matlab function", such as "matlab norm" in Google (or any other search engine) and clicking on the first result. If you prefer, you can also type in Matlab command window doc _function_ such as doc norm and read the doc in Matlab. It's of course a pleasure of us on Stackoverflow to give you the reference by doing the same thing, but it takes a longer time because a human is, in this aspect, always slower than a search engine.
The error reads Undefined function 'norm' for input arguments of type 'uint8'.. So the input for norm should not be uint8, unsigned 8-bit integer. But what should it be?
% why `norm` "does not work"?
% this line runs perfectly well
norm(cross([1,2,3], [4,5,6]))
% so what is working?
class([1,2,3]) % so `norm` works for `double`
One thing we can do now is convert a and b to double precision. Let's try it now.
% try fixing 'uint8' error
a2 = double(a);
b2 = double(b);
whos a b % now they are double, which `norm` should work for
try
% this line will raise an error
d = norm(cross(a2,b2));
catch err
disp(err.message)
end
Now the error becomes Input must be 2-D.. What's wrong with the input?
% what is "must be 2-D" error?
size(a2) % a2 is 3-D
disp(b2) % b2 is also 3-D
This gives output in command window
ans =
1 1 3
(:,:,1) =
255
(:,:,2) =
150
(:,:,3) =
0
In OP's problem, he/she is trying to calculate something about color difference (to the best of my knowledge) which involves the angle between two color vectors in RGB space. So the vectors are needed. With imread, each pixel of the image is stored as 3 elements in the matrix, first 2 dimension being its physical position, the 3 dimension being RGB channel components. Hence pixel(200,300) with color rgb[255,150,0] is stored by us in variable b wihch is a 3-D vector.
By understanding what we need and what Matlab can do, we can combine these two points into one. We need the norm of the cross product of a and b, while the useful information (the 3 component values) is stored in the 3rd dimension. Matlab can calculate the norm of the cross product of a vector with all its information in the 1st dimension. (Here, "dimension" refers to that of the Matlab variable; a vector with 3 elements in its 1st dimension is physically a 3-D vector).
After thinking twice, we are now able to debug our code - just put all 3 elements into the 1st dimension.
% so we want the 3 elements in the 3rd dimension become in the 1st dim
a3 = squeeze(a2);
b3 = reshape(b2,numel(b2),[]);
try
d = norm(cross(a3,b3));
catch err
disp(err.message)
end
d
Bonus: If by default Matlab treats a 3-D vector as a "1-D array", then most probably the cross function has not been working correctly. Let's make a check -
>> clear
>> a = [1,2,3]
a =
1 2 3
>> b=[4,5,6]
b =
4 5 6
>> cross(a,b)
ans =
-3 6 -3
>>
The result should be the same as the one we can get by calculating by hand.
Now if we put the components into the 3rd dimension of the variable -
>> clear
>> a(1,1,:)=[1,2,3]
a(:,:,1) =
1
a(:,:,2) =
2
a(:,:,3) =
3
>> b(1,1,:)=[4,5,6]
b(:,:,1) =
4
b(:,:,2) =
5
b(:,:,3) =
6
>> cross(a,b)
ans(:,:,1) =
-3
ans(:,:,2) =
6
ans(:,:,3) =
-3
>>
.... seems OK. cross also puts the result in the 3rd dimension. In fact, Mathworks' documentation says
If A and B are vectors, then they must have a length of 3.
If A and B are matrices or multidimensional arrays, then they must
have the same size. In this case, the cross function treats A and B as
collections of three-element vectors. The function calculates the
cross product of corresponding vectors along the first array dimension
whose size equals 3.
At last, one thing is always correct to anyone who wants to do something with programming - be cautious and prudent when writing your code.
this is likely a few lines of code but I cannot figure it out...
I need to perform a convolution operation of experimental data with an analytical function which is singular at the beginning of the integration range. So if I use "conv" it won't work...
I have two experimental vectors, phi(time) and time
then want to calculate T(t)
T = \int_{0}^{t}\phi \left ( t-\tau \right )\frac{\textup{d}\tau}{\tau ^{1/2}}
So the the singularity at tau = 0 makes "conv" not work. I've been fiddling with anonymous functions and using quadgk to handle the singularity and the like, but can't make anything work. I would have bet anything that this is a solved problem but can't find anything like it. Any help is very much appreciated.
EDIT: solved it this way:
function T = TCalc(time, power)
warning off MATLAB:quadgk:NonFiniteValue %suppress warning at the zero point
T=zeros(size(time));
for par = 1:numel(time)
T(par) = s1(time(par));
end
function y = r1(t)
y = interp1q(time,power,t');%notice that t is transposed! to make interp1q work.
y = y';
end
function y = s1(tfin)
y = quadgk(#(t) (r1(tfin-t))./t.^(0.5),0,tfin,'RelTol',1.e-2);
end
end
I guess, the inelegant trick here is to make Matlab think the experimental data is an analytical function by passing through interp1, and using quadgk to tolerate a singularity at the limit of integration. About one second for 1000-point vectors. Ignoring or setting the tau = 0 point to a small value doesn't work, it still perceives a singularity and the result is wrong.
I am trying to use solve() to solve a system of equations of the following form
eq1=a1x+a2y;
eq2=b1x+b2y;
where a1 = .05 for values of x<5, .1 for values of 5
Is there a way to solve for this using solve? As in sol = solve(eq1,eq2);
I'm not sure what you're trying to do here. Can you please post a real example (with numbers) and what you would like the output to be?
I think you're trying to solve linear simultaeneous equations. Assuming that is what you are trying to do:
I would suggest multiplying all of your equations by 20, so that your minimum quanta size of 0.05 becomes 1.00. Your problem then becomes the solution of linear equations for integer values.
Note that if the system is fully constrained (that is, if there are n independent constraints on the n equations you want to solve) then there will only be one solution and it may not necessarily be an integer solution. For example the system:
1 = 2a + 4b
3 = a + b
has the solution a = 5.5, b = -2.5. No other solution is possible.
For under-constrained systems, i.e.
0 = 3x + y
x > 0
Then there will be an infinite number of solutions, some of which may have both x and y being integer values. (Or there may be no integer solutions at all.)
Okay let me give you a quick rundown.
if you want to solve an equation or a system of equations and conditions then you need to define them as such, so let me explain.
so by example
clear all; %just to be safe
syms x y b
a=0.5;
somevalue=1;
someothervalue=3;
eq1= a*x+a*y == somevalue; %this is your first equation
eq2= b*x+b*y == someothervalue; %this is your 2nd equation
cond1= x<5; %this is a condition which matlab sees as an "equation"
eqs=[eq1,eq2,cond1]; %these are the equations and conditions you want to solve for, use this for solve
eqs=[eq1,eq2]; %use this for vpasolve and set your condition in range
vars=[x,y,b]; %these are the variable you want to solve for
range = [-Inf 5; NaN NaN; NaN NaN]; %NaN means you set no range
%you can use solve or vpasolve, second one being numeric, which is the one you'll probably want
n=5;
sol=zeros(n,numel(vars));
for i = 1:n
temp1 = vpasolve(eqs, vars, range, 'random', true);
temp = vpasolve(eqs, vars, 'random', true);
sol(i,1) = temp.x;
sol(i,2) = temp.y;
sol(i,3) = temp.b;
end
sol
Now when I run this myself I can't get the range to properly work for some reason, still trying to figure that out. When you don't set a range it works just fine, if you can use the solve function then there also isn't a problem.
In theory the range function should work fine like this so it might be a bug on my end.
If you use solve you have some nice options where you can use assume to set extra conditions that are a bit more advanced, like only checking for real solutions or only integers, etc.
Keeping simple, take a matrix of ones i.e.
U_iso = ones(72,37)
and some parameters
ThDeg = 0:5:180;
dtheta = 5*pi/180;
dphi = 5*pi/180;
Th = ThDeg*pi/180;
Now the code is
omega_iso = 0;
for i = 1:72
for j=1:37
omega_iso = omega_iso + U_iso(i,j)*sin(Th(j))*dphi*dtheta;
end
end
and
D_iso = (4 * pi)/omega_iso
This code is fine. It take a matrix with dimension 72*37. The loop is an approximation of the integral which is further divided by 4pi to get ONE value of directivity of antenna.
Now this code gives one value which will be around 1.002.
My problem is I dont need 1 value. I need a 72*37 matrix as my answer where the above integral approximation is implemented on each cell of the 72 * 37 matrix. and thus the Directviity 'D' also results in a matrix of same size with each cell giving the same value.
So all we have to do is instead of getting 1 value, we need value at each cell.
Can anyone please help.
You talk about creating a result that is a function essentially of the elements of U. However, in no place is that code dependent on the elements of U. Look carefully at what you have written. While you do use the variable U_iso, never is any element of U employed anywhere in that code as you have written it.
So while you talk about defining this for a matrix U, that definition is meaningless. So far, it appears that a call to repmat at the very end would create a matrix of the desired size, and clearly that is not what you are looking for.
Perhaps you tried to make the problem simple for ease of explanation. But what you did was to over-simplify, not leaving us with something that even made any sense. Please explain your problem more clearly and show code that is consistent with your explanation, for a better answer than I can provide so far.
(Note: One option MIGHT be to use arrayfun. Or the answer to this question might be more trivial, using simple vectorized operations. I cannot know at this point.)
EDIT:
Your question is still unanswerable. This loop creates a single scalar result, essentially summing over the entire array. You don't say what you mean for the integral to be computed for each element of U_iso, since you are already summing over the entire array. Please learn to be accurate in your questions, otherwise we are just guessing as to what you mean.
My best guess at the moment is that you might wish to compute a cumulative integral, in two dimensions. cumtrapz can help you there, IF that is your goal. But I'm not sure it is your goal, since your explanation is so incomplete.
You say that you wish to get the same value in each cell of the result. If that is what you wish, then a call to repmat at the end will do what you wish.