pre allocation of struct in matlab is a problem.
Please see following code in matlab profiler
time calls line
2 65 sizeofTLS= 10000;
< 0.01 2 66 LaserS(sizeofTLS).POI(n)={0};
0.03 2 67 LaserS(sizeofTLS).dis(n)={0};
0.04 2 68 LaserS(sizeofTLS).plane(n)={0};
69
70
< 0.01 2 71 for it=1:sizeofTLS
16.74 2823212 72 LaserS(it).POI(1:n)={0};
16.91 2823212 73 LaserS(it).dis(1:n)={0};
16.88 2823212 74 LaserS(it).plane(1:n)={0};
1.04 2823212 75 end
How can I improve it(72,73,74)?
The best way to preallocate structs is with the following syntax:
myEmptyCell = num2cell( zeros(1,n) );
b = repmat( struct('POI', myEmptyCell ,...
'dis', myEmptyCell,...
'plane', myEmptyCell ) , sizeofTLS, 1 );
This is ~10x faster than not preallocating on my laptop.
Related
How to improve the distance calculation on the 2 separated datasets?
This is the code:
X = [ 3.6 79
1.8 54
3.333 74
2.283 62
4.533 85
2.883 55
4.7 88
3.6 85
1.95 51
4.35 85
1.833 54
3.917 84
4.2 78
1.75 47
4.7 83
2.167 52
1.75 62
4.8 84
1.6 52
4.25 79
1.8 51
1.75 47
3.45 78
3.067 69
4.533 74
3.6 83
1.967 55
4.083 76
3.85 78
4.433 79
4.3 73
4.467 77
3.367 66
4.033 80
3.833 74
2.017 52
1.867 48
4.833 80
1.833 59
4.783 90 ]
clc;
close all;
figure;
h(1) = plot(X(:,1),X(:,2),'bx');
hold on;
X1 = X(1:3,:);
X2 = X(4:40,:);
h(2) = plot(X1(1:3,1), X1(1:3,2),'rs','MarkerSize',10);
k=5;
[D2 ind] = sort(squeeze(sqrt(sum(bsxfun(#minus,X2,permute(X1,[3 2 1])).^2,2))))
ind_closest = ind(1:k,:)
x_closest = X(ind_closest,:)
for j = 1:length(x_closest);
h(3) =plot(x_closest(j,1),x_closest(j,2),'ko','MarkerSize',10);
end
The output is shown as in the picture below:
The problem is, the code does not pick the closest data points of red squared data points. I also tried to use pdist2 function from statistical toolbox,the result yields similar with the bsxfun function that i applied in my code.
I'm not sure which part in the code need to improve so that i can pick the data points that closest to the target.
Really appreciate if anyone can help me to improve my code
If the closest point means closest to X, line 19 & line 20 should be replaced as
[D2 ind] = sort(squeeze(sqrt(sum(bsxfun(#minus,X,permute(X1,[3 2 1])).^2,2))))
ind_closest = ind(2:k+1,:)
If the closest point means closest to X2, then try this:
x_closest = X2(ind_closest,:)
In the meanwhile, I modified your code a little bit, since your h(3) could be optimized.
clc; clear; close all;
%load fisheriris
%X=meas(:,3:4);
load X
X=unique(X,'rows');
figure;
h(1) = plot(X(:,1),X(:,2),'bx');
hold on;
X1 = X([5 15 30],:);
h(2) = plot(X1(:,1), X1(:,2),'rs','MarkerSize',10);
[D2,ind] = sort(squeeze(sqrt(sum(bsxfun(#minus,X,permute(X1,[3 2 1])).^2,2))));
k=3;
ind_closest = unique(ind(2:k+1,:));
x_closest = X(ind_closest,:);
h(3) =plot(x_closest(:,1),x_closest(:,2),'ko','MarkerSize',10);
axis equal
It seems to be working fine.
I am trying to do classification using neural network and I have written the following code. Is this the code required to perform the training and classification?
%n1 to s5(n1=147,n2=205,n3=166,n4=204,n5=167,b1=156,b2=172,b3=153,b4=151,b5=160,r1=133,r2=135,r3=190,r4=143,ru1=133,ru2=153,ru3=154,ru4=137,s1=132,165,130,136,148)
%code:
T = [n1,n2,n3,n4,n5,b1,b2,b3,b4,b5,r1,r2,r3,r4,r5,ru1,ru2,ru3,ru4,s1,s2,s3,s4,s5];
x = [0 0 0 0 0 1 1 1 1 1 2 2 2 2 2 3 3 3 3 4 4 4 4 4];
net1 = newff(minmax(T),[30 20 1],{'logsig','logsig','purelin'},'trainrp');
net1.trainParam.show = 1000;
net1.trainParam.lr = 0.04;
net1.trainParam.epochs = 7000;
net1.trainParam.goal = 1e-5;
[net1] = train(net1,T,x);
save net1 net1
Additionally, if I have more samples with more features then how should I represent it in T and X? How do I write T and x? For example:
sample 1 ..... 123 0.56 78 127 .......0
sample 2 .......127 0.89 56 132 ........0
sample3...... 134 0.72 65 140...1
sample4 156 0.55 69 145 .....1
sample 5 112 0.10 12 120 .......2
sample 6 123 0.15 24 99 .......2
sample 7 95 0.32 98 198 ....3
sample 8 90 0.45 90 200...... 3
Yes your solution appears correct. I should note, newff is obsolete as of R2010b NNET 7.0 and it's last used in R2010a NNET 6.0.4. The recommended function is now is feedforwardnet. The feedforwardnet implementation should be as follows
net = feedforwardnet([30 20]);
net.layers{1:2}.transferFcn = 'logsig'
net.trainParam.show = 1000;
net.trainParam.lr = 0.04;
net.trainParam.epochs = 7000;
net.trainParam.goal = 1e-5;
[net1] = train(net1,T,x);
save net1 net1
I'm not quite sure what you're trying to do with minmax but that should be the basic structure of the feedforward NN.
To structure T and x, say you have the following data:
[123 0.56 78 127] belongs to class 1
[127 0.89 56 132] belongs to class 1
[134 0.72 65 140] belongs to class 2
[156 0.55 69 145] belongs to class 2
[112 0.10 12 120] belongs to class 3
[123 0.15 24 99] belongs to class 3
You can set T and x as follows:
T = [ 123 0.56 78 127; 127 0.89 56 132; 134 0.72 65 140; 156 0.55 69 145; 112 0.10 12 120; 123 0.15 24 99];
x = [1; 1; 2; 2; 3; 3];
I have a p-by-p-by-n tensor. I want to extract diagonal element for each p-by-p slice. Are there anyone know how to do this without looping?
Thank you.
Behold the ever mighty and ever powerful bsxfun for vectorizing MATLAB problems to do this task very efficiently using MATLAB's linear indexing -
diags = A(bsxfun(#plus,[1:p+1:p*p]',[0:n-1]*p*p))
Sample run with 4 x 4 x 3 sized input array -
A(:,:,1) =
0.7094 0.6551 0.9597 0.7513
0.7547 0.1626 0.3404 0.2551
0.2760 0.1190 0.5853 0.5060
0.6797 0.4984 0.2238 0.6991
A(:,:,2) =
0.8909 0.1493 0.8143 0.1966
0.9593 0.2575 0.2435 0.2511
0.5472 0.8407 0.9293 0.6160
0.1386 0.2543 0.3500 0.4733
A(:,:,3) =
0.3517 0.9172 0.3804 0.5308
0.8308 0.2858 0.5678 0.7792
0.5853 0.7572 0.0759 0.9340
0.5497 0.7537 0.0540 0.1299
diags =
0.7094 0.8909 0.3517
0.1626 0.2575 0.2858
0.5853 0.9293 0.0759
0.6991 0.4733 0.1299
Benchmarking
Here's few runtime tests comparing this bsxfun based approach against repmat + eye based approach for big datasizes -
***** Datasize: 500 x 500 x 500 *****
----------------------- With BSXFUN
Elapsed time is 0.008383 seconds.
----------------------- With REPMAT + EYE
Elapsed time is 0.163341 seconds.
***** Datasize: 800 x 800 x 500 *****
----------------------- With BSXFUN
Elapsed time is 0.012977 seconds.
----------------------- With REPMAT + EYE
Elapsed time is 0.402111 seconds.
***** Datasize: 1000 x 1000 x 500 *****
----------------------- With BSXFUN
Elapsed time is 0.017058 seconds.
----------------------- With REPMAT + EYE
Elapsed time is 0.690199 seconds.
One suggestion I have is to create a p x p logical identity matrix, replicate this n times in the third dimension, and then use this matrix to access your tensor. Something like this, supposing that your tensor was stored in A:
ind = repmat(logical(eye(p)), [1 1 n]);
out = A(ind);
Example use:
>> p = 5; n = 3;
>> A = reshape(1:75, p, p, n)
A(:,:,1) =
1 6 11 16 21
2 7 12 17 22
3 8 13 18 23
4 9 14 19 24
5 10 15 20 25
A(:,:,2) =
26 31 36 41 46
27 32 37 42 47
28 33 38 43 48
29 34 39 44 49
30 35 40 45 50
A(:,:,3) =
51 56 61 66 71
52 57 62 67 72
53 58 63 68 73
54 59 64 69 74
55 60 65 70 75
>> ind = repmat(logical(eye(p)), [1 1 n]);
>> out = A(ind)
out =
1
7
13
19
25
26
32
38
44
50
51
57
63
69
75
You'll notice that we grab the diagonals of the first slice, followed by the diagonals of the second slice, etc. up until the last slice. These values are all concatenated into a single vector.
Just reading Divakar's answer and trying to understand why he again is roughly 10 times faster than my idea I put together code mixing both, and ended up with code which is faster:
A=reshape(A,[],n);
diags2 = A(1:p+1:p*p,:);
For a 500x500x500 tensor I get 0.008s for Divakar's solution and 0.005s for my solution using Matlab 2013a. Probably plain indexing is the only way to beat bsxfun.
Say I have the following columns vector Z
1 53 55 57 60 64 68 70 71 72 74 76 77 78 79 80 255
I want to use it to create a matrix such that each row would contain all the number between (and including) 2 adjacent elements in Z
So the output matrix should be something like this:
1 2 3 .... 53
53 54 55
55 56 57
57 58 60
....
80 81 ... 255
I've been searching for something similar but couldn't find it.
Thanks
See if this works for you -
lens = diff(Z)+1;
mask1 = bsxfun(#le,[1:max(lens)]',lens); %//'
array1 = zeros(size(mask1));
array1(mask1) = sort([1:255 Z(2:end-1)]);
out = array1.'; %//'# out is the desired output
Try this to break the monotony of bsxfun :) :
d = diff(Z);
N = max(d)+1;
R = zeros(length(Z)-1,N);
for i = 1:length(Z)-1
R(i,1:1+d(i)) = Z(i):Z(i+1);
end
EDIT:
I know that the general consensus is that one always should try to avoid loops in Matlab, but is this valid for this example? I know that this is a broad question, so lets focus on this particular problem and compare bsxfun to JIT loop. Comparing the two proposed solutions:
the code used for testing:
Z = [1 53 55 57 60 64 68 70 71 72 74 76 77 78 79 80 255];
%[1 3 4, 6];
nn = round(logspace(1,4,10));
tm1_nn = zeros(length(nn),1);
tm2_nn = zeros(length(nn),1);
for o = 1:length(nn)
tm1 = zeros(nn(o),1);
tm2 = zeros(nn(o),1);
% approach1
for k = 1:nn(o)+1
tic
d = diff(Z);
N = max(d)+1;
R = zeros(length(Z)-1,N);
for i = 1:length(Z)-1
R(i,1:1+d(i)) = Z(i):Z(i+1);
end
tm1(k) = toc;
end
%approach 2
for k = 1:nn(o)+1
tic
lens = diff(Z)+1;
mask1 = bsxfun(#le,[1:max(lens)]',lens); %//'
array1 = zeros(size(mask1));
array1(mask1) = sort([1:255 Z(2:end-1)]);
out = array1.';
tm2(k) = toc;
end
tm1_nn(o) = mean(tm1);%sum(tm1);%mean(tm1);%
tm2_nn(o) = mean(tm2);%sum(tm2);%mean(tm2);%
end
semilogx(nn,tm1_nn, '-ro', nn,tm2_nn, '-bo')
legend('JIT loop', 'bsxfun')
xlabel('log_1_0(Number of runs)')
%ylabel('Sum execution time')
ylabel('Mean execution time')
grid on
I encountered other tasks previously where the loop was faster. (or I mess up the comparison?)
Assume we have the following data:
H_T = [36 66 21 65 52 67 73; 31 23 19 33 36 39 42]
P = [40 38 39 40 35 32 37]
Using MATLAB 7.0, I want to create three new matrices that have the following properties:
The matrix H (the first part in matrix H_T) will be divided to 3 intervals:
Matrix 1: the 1st interval contains the H values between 20 to 40
Matrix 2: the 2nd interval contains the H values between 40 to 60
Matrix 3: the 3rd interval contains the H values between 60 to 80
The important thing is that the corresponding T and P will also be included in their new matrices meaning that H will control the new matrices depending on the specifications defined above.
So, the resultant matrices will be:
H_T_1 = [36 21; 31 19]
P_1 = [40 39]
H_T_2 = [52; 36]
P_2 = [35]
H_T_3 = [66 65 67 73; 23 33 39 42]
P_3 = [38 40 32 37]
Actually, this is a simple example and it is easy by looking to create the new matrices depending on the specifications, BUT in my values I have thousands of numbers which makes it very difficult to do that.
Here's a quick solution
[~,bins] = histc(H_T(1,:), [20 40 60 80]);
outHT = cell(3,1);
outP = cell(3,1);
for i=1:3
idx = (bins == i);
outHT{i} = H_T(:,idx);
outP{i} = P(idx);
end
then you access the matrices as:
>> outHT{3}
ans =
66 65 67 73
23 33 39 42
>> outP{3}
ans =
38 40 32 37