I'm got a script that produce a column of 36 values.
I would save these 36 values in rows in Excel.
At the moment, I have to run the script each time so I can change the xlrange value, eg: A1 to A1000.
I tried looping the script, and tried to write the values into a new column of a new variable, say mm.
For i=1:1000
Scriptnamehere
mm(i,:)=m or mm(:,i)
Write in excel script here
End
It failed to recognize i for mm.
You have to preallocate the Matrix mm:
N = 1000; % number of iterations
num_rows = 36; % number of values in every iteration
mm = zeros(num_rows, N); % preallocation
for k = 1:N % don't use i as index variable
% call script with k, receive m
mm(:, k) = m;
end
Maybe use simple assignemt mm = m (I suppose m is the value You got from the script), in You case You tried to assign 36 values to for example mm(1), which would not work. On the other hand I would not recomend to use i as variable for the looping, because it is already predefined by Matlab as imaginary number
For i=1:1000
Scriptnamehere
mm = m
Write in excel script here
End
Related
I have a data, which may be simulated in the following way:
N = 10^6;%10^8;
K = 10^4;%10^6;
subs = randi([1 K],N,1);
M = [randn(N,5) subs];
M(M<-1.2) = nan;
In other words, it is a matrix, where the last row is subscripts.
Now I want to calculate nanmean() for each subscript. Also I want to save number of rows for each subscript. I have a 'dummy' code for this:
uniqueSubs = unique(M(:,6));
avM = nan(numel(uniqueSubs),6);
for iSub = 1:numel(uniqueSubs)
tmpM = M(M(:,6)==uniqueSubs(iSub),1:5);
avM(iSub,:) = [nanmean(tmpM,1) size(tmpM,1)];
end
The problem is, that it is too slow. I want it to work for N = 10^8 and K = 10^6 (see commented part in the definition of these variables.
How can I find the mean of the data in a faster way?
This sounds like a perfect job for findgroups and splitapply.
% Find groups in the final column
G = findgroups(M(:,6));
% function to apply per group
fcn = #(group) [mean(group, 1, 'omitnan'), size(group, 1)];
% Use splitapply to apply fcn to each group in M(:,1:5)
result = splitapply(fcn, M(:, 1:5), G);
% Check
assert(isequaln(result, avM));
M = sortrows(M,6); % sort the data per subscript
IDX = diff(M(:,6)); % find where the subscript changes
tmp = find(IDX);
tmp = [0 ;tmp;size(M,1)]; % add start and end of data
for iSub= 2:numel(tmp)
% Calculate the mean over just a single subscript, store in iSub-1
avM2(iSub-1,:) = [nanmean(M(tmp(iSub-1)+1:tmp(iSub),1:5),1) tmp(iSub)-tmp(iSub-1)];tmp(iSub-1)];
end
This is some 60 times faster than your original code on my computer. The speed-up mainly comes from presorting the data and then finding all locations where the subscript changes. That way you do not have to traverse the full array each time to find the correct subscripts, but rather you only check what's necessary each iteration. You thus calculate the mean over ~100 rows, instead of first having to check in 1,000,000 rows whether each row is needed that iteration or not.
Thus: in the original you check numel(uniqueSubs), 10,000 in this case, whether all N, 1,000,000 here, numbers belong to a certain category, which results in 10^12 checks. The proposed code sorts the rows (sorting is NlogN, thus 6,000,000 here), and then loop once over the full array without additional checks.
For completion, here is the original code, along with my version, and it shows the two are the same:
N = 10^6;%10^8;
K = 10^4;%10^6;
subs = randi([1 K],N,1);
M = [randn(N,5) subs];
M(M<-1.2) = nan;
uniqueSubs = unique(M(:,6));
%% zlon's original code
avM = nan(numel(uniqueSubs),7); % add the subscript for comparison later
tic
uniqueSubs = unique(M(:,6));
for iSub = 1:numel(uniqueSubs)
tmpM = M(M(:,6)==uniqueSubs(iSub),1:5);
avM(iSub,:) = [nanmean(tmpM,1) size(tmpM,1) uniqueSubs(iSub)];
end
toc
%%%%% End of zlon's code
avM = sortrows(avM,7); % Sort for comparison
%% Start of Adriaan's code
avM2 = nan(numel(uniqueSubs),6);
tic
M = sortrows(M,6);
IDX = diff(M(:,6));
tmp = find(IDX);
tmp = [0 ;tmp;size(M,1)];
for iSub = 2:numel(tmp)
avM2(iSub-1,:) = [nanmean(M(tmp(iSub-1)+1:tmp(iSub),1:5),1) tmp(iSub)-tmp(iSub-1)];
end
toc %tic/toc should not be used for accurate timing, this is just for order of magnitude
%%%% End of Adriaan's code
all(avM(:,1:6) == avM2) % Do the comparison
% End of script
% Output
Elapsed time is 58.561347 seconds.
Elapsed time is 0.843124 seconds. % ~70 times faster
ans =
1×6 logical array
1 1 1 1 1 1 % i.e. the matrices are equal to one another
I have the following data:
no_gridpoints = 640 % amount of columns in considered
surfaceelevation % a 1x640 array with surface elevation
Terskol1752, Terskol1753, ... Terskol2017 % 365x1 arrays with daily mean temperatures for 1 year of which the fifth colomn contains the temperature data
I want to create temp_glacier files with the corresponding year in the file name. This with a loop over all the years (1752-2017) by using the sprintf command in the loop:
for k = 1752:2017
for m = 1:no_gridpoints
sprintf('temp_glacier%d(m)',k) = sprintf('Terskol%d(:,5)',k) + surfaceelevation
end
end
However, I always get the error 'Subscripted assignment dimension mismatch.'. Can anyone tell me what I am doing wrong?
Thanks
As stated in my comment: it looks like you're mistaking sprintf for eval. The expression within sprintf is not evaluated, so your assignment is stating "make this string = this other string added to an array" - it makes no sense.
To correct your code as-is, you could do the following
for k = 1752:2017
for m = 1:no_gridpoints
eval(sprintf('temp_glacier%d(m) = Terskol%d(:,5) + surfaceelevation', k, k))
end
end
This is a bad idea
It would be far better practise for you to store your yearly data in a single cell array (or because it's numerical and the same size, just a standard matrix) rather than 266 individually named variables. I say better practise because if you to mean to use eval, you should know it should be avoided!
This method would look like the following:
Terskol = [ ... ] % your data here, in a 266*365 matrix where each row is a year
for k = (1752:2017) - 1751 % We actually want to loop through rows 1 to 266
for m = 1:no_gridpoints
% Your arrays were 1D, so you were originally getting a scalar temp val
% We can do that here like so...
temp_glacier(m) = Terskol(k, 5) + surfaceelevation;
% Now do something with temp_glacier, or there was no point in this loop!
% ...
end
end
Vectorising the inner loop:
for k = (1752:2017) - 1751 % We actually want to loop through rows 1 to 266
temp_glacier = repmat( Terskol(k, 5) + surfaceelevation, 1, no_gridpoints );
% Do something with temp_glacier...
end
l_0=1.5;
l_1=1.6;
Lambda_min=2*(1+1)*l_0;
Lambda_max=2*(1+1)*l_1;
n_0=linspace(2,2.11,10);
n_1=linspace(2.30,2.50,10);
for i=1:10
for j=1:10
for k=1:10
l(i) = Lambda_min * ( Lambda_max/Lambda_min)^(i/10)
sum=sum(l)
d_0(:,j)= l(i)/((n_0(i)/n_1(i)+1))
d_1(:,k)= (n_0(i)/n_1(i))*d_0(:,j)
end
end
end
First of all; I want to find values of l(i) which is a vector, then take the sum of that vector. second, for d_0(:,j) I want to create a matrix so I can plot it later, that takes different values from l(i),n_0,n_1 each time. If I take the values for n_0 and n_1 and put in the for loop I will get index error because it should be logic or integer number.
My matrix is overwritten and do not know how to avoid it. Note, I want in d_0 and d_1 n_0 and n_1 to take values from linspace. for example in the first iteration n_0= 2 n_1= 2.30 then second iteration take the next value in linspace.
I tried to see the value of n_0(i) and does it give me 10 iterations. It gives me more that that overwritten.
Try:
l_0=1.5;
l_1=1.6;
Lambda_min = 4*l_0;
Lambda_max = 4*l_1;
n_0 = linspace(2,2.11,10) % don't add semicolon so you can check this is giving 10 values
n_1 = linspace(2.30,2.50,10) %
for i=1:10
l(i) = Lambda_min * ( Lambda_max/Lambda_min)^(i/10) % should give you 10 values
end
d_0= l./((n_0./n_1+1)); % This will only give you a vector, not a matrix.
d_1= (n_0./n_1).*d_0;
Lsum = sum(l); % should give you one value
I am trying to concatenate several structs. What I take from each struct depends on a function that requires a for loop. Here is my simplified array:
t = 1;
for t = 1:5 %this isn't the for loop I am asking about
a(t).data = t^2; %it just creates a simple struct with 5 data entries
end
Here I am doing concatenation manually:
A = [a(1:2).data a(1:3).data a(1:4).data a(1:5).data] %concatenation function
As you can see, the range (1:2), (1:3), (1:4), and (1:5) can be looped, which I attempt to do like this:
t = 2;
A = [for t = 2:5
a(1:t).data
end]
This results in an error "Illegal use of reserved keyword "for"."
How can I do a for loop within the concatenate function? Can I do loops within other functions in Matlab? Is there another way to do it, other than copy/pasting the line and changing 1 number manually?
You were close to getting it right! This will do what you want.
A = []; %% note: no need to initialize t, the for-loop takes care of that
for t = 2:5
A = [A a(1:t).data]
end
This seems strange though...you are concatenating the same elements over and over...in this example, you get the result:
A =
1 4 1 4 9 1 4 9 16 1 4 9 16 25
If what you really need is just the .data elements concatenated into a single array, then that is very simple:
A = [a.data]
A couple of notes about this: why are the brackets necessary? Because the expressions
a.data, a(1:t).data
don't return all the numbers in a single array, like many functions do. They return a separate answer for each element of the structure array. You can test this like so:
>> [b,c,d,e,f] = a.data
b =
1
c =
4
d =
9
e =
16
f =
25
Five different answers there. But MATLAB gives you a cheat -- the square brackets! Put an expression like a.data inside square brackets, and all of a sudden those separate answers are compressed into a single array. It's magic!
Another note: for very large arrays, the for-loop version here will be very slow. It would be better to allocate the memory for A ahead of time. In the for-loop here, MATLAB is dynamically resizing the array each time through, and that can be very slow if your for-loop has 1 million iterations. If it's less than 1000 or so, you won't notice it at all.
Finally, the reason that HBHB could not run your struct creating code at the top is that it doesn't work unless a is already defined in your workspace. If you initialize a like this:
%% t = 1; %% by the way, you don't need this, the t value is overwritten by the loop below
a = []; %% always initialize!
for t = 1:5 %this isn't the for loop I am asking about
a(t).data = t^2; %it just creates a simple struct with 5 data entries
end
then it runs for anyone the first time.
As an appendix to gariepy's answer:
The matrix concatenation
A = [A k];
as a way of appending to it is actually pretty slow. You end up reassigning N elements every time you concatenate to an N size vector. If all you're doing is adding elements to the end of it, it is better to use the following syntax
A(end+1) = k;
In MATLAB this is optimized such that on average you only need to reassign about 80% of the elements in a matrix. This might not seam much, but for 10k elements this adds up to ~ an order of magnitude of difference in time (at least for me).
Bare in mind that this works only in MATLAB 2012b and higher as described in this thead: Octave/Matlab: Adding new elements to a vector
This is the code I used. tic/toc syntax is not the most accurate method for profiling in MATLAB, but it illustrates the point.
close all; clear all; clc;
t_cnc = []; t_app = [];
N = 1000;
for n = 1:N;
% Concatenate
tic;
A = [];
for k = 1:n;
A = [A k];
end
t_cnc(end+1) = toc;
% Append
tic;
A = [];
for k = 1:n;
A(end+1) = k;
end
t_app(end+1) = toc;
end
t_cnc = t_cnc*1000; t_app = t_app*1000; % Convert to ms
% Fit a straight line on a log scale
P1 = polyfit(log(1:N),log(t_cnc),1); P_cnc = #(x) exp(P1(2)).*x.^P1(1);
P2 = polyfit(log(1:N),log(t_app),1); P_app = #(x) exp(P2(2)).*x.^P2(1);
% Plot and save
loglog(1:N,t_cnc,'.',1:N,P_cnc(1:N),'k--',...
1:N,t_app,'.',1:N,P_app(1:N),'k--');
grid on;
xlabel('log(N)');
ylabel('log(Elapsed time / ms)');
title('Concatenate vs. Append in MATLAB 2014b');
legend('A = [A k]',['O(N^{',num2str(P1(1)),'})'],...
'A(end+1) = k',['O(N^{',num2str(P2(1)),'})'],...
'Location','northwest');
saveas(gcf,'Cnc_vs_App_test.png');
I have got a text file which contains 14000 rows and 7 columns. I have to take each of this columns. Then I have to find the sum of 1st 40 elements, then the next 40 elements(41-80) then the next 40 (81-120) and so on. I have written a MATLAB code for this. It is:
clc;clear all;close all;
fid = fopen('sks.txt');
datacell = textscan(fid,'%f%f%f%*[^\n]',...
'delimiter','\t');
fclose(fid);
A = datacell{1};
B=datacell{2};
l=size(A);
k=40;
sum=0;
for x=1:k
sum=sum+A(x);
end;
sum
for y=1:((l/k)-1)
sum1=0;
for i=((y*k)+1):((y+1)*k)
sum1=sum1+A(i);
end;
end;
I am getting all the sets of sums correcly with this code but I want all the answers as a single matrix. That is presently all the answers are obtained seperately. There are 350 seperate answers. I want all 350 in a single matrix. I want something like
Sum=23
34
87
.......
and so on.
Not
sum=23
sum=34
sum=87
I am not good at coding or MATLAB. So if there is a simpler way other than this you are more than welcome to give me that. Or please help me modify this code.:-)
You can assign those values in such a way (matrix will expand with every iteration):
sum = [] % init output matrix
for y=1:((l/k)-1)
sum1=0;
for i=((y*k)+1):((y+1)*k)
sum1=sum1+A(i);
end;
sum = [sum; sum1] % it will automatically expend matrix
end;
or as You know the final output of the Sum matrix (length(1:((l/k)-1)))
sum = zeros(1,length(1:((l/k)-1))) % init output matrix
ii = 1; % matrix index
for y=1:((l/k)-1)
sum1=0;
for i=((y*k)+1):((y+1)*k)
sum1=sum1+A(i);
end;
sum(ii) = sum1 % assign value to matrix
ii = ii + 1; % increment matrix index
end;
Not the whole answer - but will make the code much cleaner and faster:
You might run into some problems as sum is actually the name of a built-in function - which accidentally does exactly what you do in loops (just quicker).
Use stuff like:
mySum = sum(A)
and
mySum1 = mySum1 + sum( ((y*k)+1):((y+1)*k) )
Thanks for the answers guys. I got a simpler way. So I thought I will share it here. There is a command called reshape in matlab so I used it like this
A = datacell{1};
A1=squeeze( sum( reshape(A, 40, [], 1) ) );
A1 will give the sum of each set of 40 elements.