Binary file - Matrix Manipulation - matlab
I have several binary files with a known structure (8,12000,real*4). 8 is the number of variables, and 12000 represent the time steps. From these binaries I need to get a final 16x9 matrix defined as followed:
the first column contains a filename identification.
on the diagonal are located the extreme values (maxima and minima) of the corresponding variable.
the simultaneous values of the other variables shall be given in the rows.
At the moment I'm using this code
for i = 1:num_files
for j = 1:num_ext
fid = fopen([fullfile(BEGINPATH{1},FILELIST{i}) '.$' VAREXTENSION{j}],'r');
A = fread(fid,[DLC_dimens{i}{3}(1) DLC_dimens{i}{3}(2)],'real*4');
for k = 1:size(A,1)
[max_val(k,i) max_idx(k,i)] = max(A(k,:));
[min_val(k,i) min_idx(k,i)] = min(A(k,:));
end
fclose(fid);
end
end
% Pre-allocate ULS matrices
uloads = cell(2*size(A,1),size(A,1)+1);
uloads_temp = cell(2*size(A,1),size(A,1)+1);
% Generate ULS matrix for the first file
for i = 1:size(uloads,1)
uloads{i,end} = DLC_dimens{1}(1);
end
fid = fopen([fullfile(BEGINPATH{1},FILELIST{1}) '.$' VAREXTENSION{1}],'r');
A = fread(fid,[DLC_dimens{i}{3}(1) DLC_dimens{i}{3}(2)],'real*4');
for j = 1:size(uloads,2)-1
uloads{1,j} = A(j,max_idx(1,1))*DLC_dimens{1}{4};
uloads{2,j} = A(j,min_idx(1,1))*DLC_dimens{1}{4};
uloads{3,j} = A(j,max_idx(2,1))*DLC_dimens{1}{4};
uloads{4,j} = A(j,min_idx(2,1))*DLC_dimens{1}{4};
uloads{5,j} = A(j,max_idx(3,1))*DLC_dimens{1}{4};
uloads{6,j} = A(j,min_idx(3,1))*DLC_dimens{1}{4};
uloads{7,j} = A(j,max_idx(4,1))*DLC_dimens{1}{4};
uloads{8,j} = A(j,min_idx(4,1))*DLC_dimens{1}{4};
uloads{9,j} = A(j,max_idx(5,1))*DLC_dimens{1}{4};
uloads{10,j} = A(j,min_idx(5,1))*DLC_dimens{1}{4};
uloads{11,j} = A(j,max_idx(6,1))*DLC_dimens{1}{4};
uloads{12,j} = A(j,min_idx(6,1))*DLC_dimens{1}{4};
uloads{13,j} = A(j,max_idx(7,1))*DLC_dimens{1}{4};
uloads{14,j} = A(j,min_idx(7,1))*DLC_dimens{1}{4};
uloads{15,j} = A(j,max_idx(8,1))*DLC_dimens{1}{4};
uloads{16,j} = A(j,min_idx(8,1))*DLC_dimens{1}{4};
end
fclose(fid);
% ULS temporary matrix generation
uls = uloads;
for i = 2:num_files
fid = fopen([fullfile(BEGINPATH{1},FILELIST{i}) '.$' VAREXTENSION{1}],'r');
A = fread(fid,[8 12000],'float32');
for j = 1:size(uloads,1)
uloads_temp{j,9} = DLC_dimens{i}(1);
end
for k = 1:size(uloads,2)-1
uloads_temp{1,k} = A(k,max_idx(1,i))*DLC_dimens{i}{4};
uloads_temp{2,k} = A(k,min_idx(1,i))*DLC_dimens{i}{4};
uloads_temp{3,k} = A(k,max_idx(2,i))*DLC_dimens{i}{4};
uloads_temp{4,k} = A(k,min_idx(2,i))*DLC_dimens{i}{4};
uloads_temp{5,k} = A(k,max_idx(3,i))*DLC_dimens{i}{4};
uloads_temp{6,k} = A(k,min_idx(3,i))*DLC_dimens{i}{4};
uloads_temp{7,k} = A(k,max_idx(4,i))*DLC_dimens{i}{4};
uloads_temp{8,k} = A(k,min_idx(4,i))*DLC_dimens{i}{4};
uloads_temp{9,k} = A(k,max_idx(5,i))*DLC_dimens{i}{4};
uloads_temp{10,k} = A(k,min_idx(5,i))*DLC_dimens{i}{4};
uloads_temp{11,k} = A(k,max_idx(6,i))*DLC_dimens{i}{4};
uloads_temp{12,k} = A(k,min_idx(6,i))*DLC_dimens{i}{4};
uloads_temp{13,k} = A(k,max_idx(7,i))*DLC_dimens{i}{4};
uloads_temp{14,k} = A(k,min_idx(7,i))*DLC_dimens{i}{4};
uloads_temp{15,k} = A(k,max_idx(8,i))*DLC_dimens{i}{4};
uloads_temp{16,k} = A(k,min_idx(8,i))*DLC_dimens{i}{4};
end
if uloads_temp{1,1}(:) > uls{1,1}(:)
uls(1,:) = uloads_temp(1,:);
end
if uloads_temp{2,1}(:) < uls{2,1}(:)
uls(2,:) = uloads_temp(2,:);
end
if uloads_temp{3,2}(:) > uls{3,2}(:)
uls(3,:) = uloads_temp(3,:);
end
if uloads_temp{4,2}(:) < uls{4,2}(:)
uls(4,:) = uloads_temp(4,:);
end
if uloads_temp{5,3}(:) > uls{5,3}(:)
uls(5,:) = uloads_temp(5,:);
end
if uloads_temp{6,3}(:) < uls{6,3}(:)
uls(6,:) = uloads_temp(6,:);
end
if uloads_temp{7,4}(:) > uls{7,4}(:)
uls(7,:) = uloads_temp(7,:);
end
if uloads_temp{8,4}(:) < uls{8,4}(:)
uls(8,:) = uloads_temp(8,:);
end
if uloads_temp{9,5}(:) > uls{9,5}(:)
uls(9,:) = uloads_temp(9,:);
end
if uloads_temp{10,5}(:) < uls{10,5}(:)
uls(10,:) = uloads_temp(10,:);
end
if uloads_temp{11,6}(:) > uls{11,6}(:)
uls(11,:) = uloads_temp(11,:);
end
if uloads_temp{12,6}(:) < uls{12,6}(:)
uls(12,:) = uloads_temp(12,:);
end
if uloads_temp{13,7}(:) > uls{13,7}(:)
uls(13,:) = uloads_temp(3,:);
end
if uloads_temp{14,7}(:) < uls{14,7}(:)
uls(14,:) = uloads_temp(14,:);
end
if uloads_temp{15,8}(:) > uls{15,8}(:)
uls(15,:) = uloads_temp(15,:);
end
if uloads_temp{16,8}(:) < uls{16,8}(:)
uls(16,:) = uloads_temp(16,:);
end
fclose(fid);
end
Now comes the question: I was thinking of a procedure where
I generate a temporary uloads_temp matrix just with the first file;
Calculate the uloads matrix for the i-th file (i = 2:num_files)
Compare the terms on the diagonal between i-th uloads matrix and temporary uloads_temp:
a) if the elements of the i-th ulaods are major (minor) than the respective uloads_temp values
b) update uloads_temp rows the condition a) occurs.
I hope I explained everything properly. Could you please give me a hint on how to perform the described loops?
I thank you all in advance.
WKR,
Francesco
P.S. : everything could be reproduced by means fo matrices filled of random numbers; I just copied and pasted my code with reference to a file list.
Related
How to loop through the columns of a Matlab table by using the headings?
I am trying to make a for-loop for the Matlab code below. I have named each column with JAN90, FEB90, etc. all the way up to AUG19, which can be found in a matrix named "data". At this point I need to change the month and year manually to obtain the result I want. Is there a way to iterate over the columns by the column name? Would it be easier to name the columns Var1, Var2 etc.? clear; clc; data = readtable('Data.xlsx','ReadVariableNames',false); data(1,:) = []; data.Var2 = str2double(data.Var2); data.Var3 = str2double(data.Var3); data.Var4 = str2double(data.Var4); data.Var5 = str2double(data.Var5); data.Var6 = str2double(data.Var6); data.Var7 = str2double(data.Var7); data.Var8 = str2double(data.Var8); data.Var9 = str2double(data.Var9); data.Var10 = str2double(data.Var10); data.Var11 = str2double(data.Var11); data.Var12 = str2double(data.Var12); data.Var13 = str2double(data.Var13); data(:,1) = []; data = table2array(data); data = array2table(data.'); data = table2cell(data) data = cell2table(data, 'VariableNames',{'JAN90','FEB90','MAR90','APR90','MAY90','JUN90','JUL90','AUG90'... ,'SEP90','OCT90','NOV90','DEC90','JAN91','FEB91','MAR91','APR91','MAY91','JUN91','JUL91','AUG91'... ,'SEP91','OCT91','NOV91','DEC91','JAN92','FEB92','MAR92','APR92','MAY92','JUN92','JUL92','AUG92'... ,'SEP92','OCT92','NOV92','DEC92','JAN93','FEB93','MAR93','APR93','MAY93','JUN93','JUL93','AUG93'... ,'SEP93','OCT93','NOV93','DEC93','JAN94','FEB94','MAR94','APR94','MAY94','JUN94','JUL94','AUG94'... ,'SEP94','OCT94','NOV94','DEC94','JAN95','FEB95','MAR95','APR95','MAY95','JUN95','JUL95','AUG95'... ,'SEP95','OCT95','NOV95','DEC95','JAN96','FEB96','MAR96','APR96','MAY96','JUN96','JUL96','AUG96'... ,'SEP96','OCT96','NOV96','DEC96','JAN97','FEB97','MAR97','APR97','MAY97','JUN97','JUL97','AUG97'... ,'SEP97','OCT97','NOV97','DEC97','JAN98','FEB98','MAR98','APR98','MAY98','JUN98','JUL98','AUG98'... ,'SEP98','OCT98','NOV98','DEC98','JAN99','FEB99','MAR99','APR99','MAY99','JUN99','JUL99','AUG99'... ,'SEP99','OCT99','NOV99','DEC99','JAN00','FEB00','MAR00','APR00','MAY00','JUN00','JUL00','AUG00'... ,'SEP00','OCT00','NOV00','DEC00','JAN01','FEB01','MAR01','APR01','MAY01','JUN01','JUL01','AUG01'... ,'SEP01','OCT01','NOV01','DEC01','JAN02','FEB02','MAR02','APR02','MAY02','JUN02','JUL02','AUG02'... ,'SEP02','OCT02','NOV02','DEC02','JAN03','FEB03','MAR03','APR03','MAY03','JUN03','JUL03','AUG03'... ,'SEP03','OCT03','NOV03','DEC03','JAN04','FEB04','MAR04','APR04','MAY04','JUN04','JUL04','AUG04'... ,'SEP04','OCT04','NOV04','DEC04','JAN05','FEB05','MAR05','APR05','MAY05','JUN05','JUL05','AUG05'... ,'SEP05','OCT05','NOV05','DEC05','JAN06','FEB06','MAR06','APR06','MAY06','JUN06','JUL06','AUG06'... ,'SEP06','OCT06','NOV06','DEC06','JAN07','FEB07','MAR07','APR07','MAY07','JUN07','JUL07','AUG07'... ,'SEP07','OCT07','NOV07','DEC07','JAN08','FEB08','MAR08','APR08','MAY08','JUN08','JUL08','AUG08'... ,'SEP08','OCT08','NOV08','DEC08','JAN09','FEB09','MAR09','APR09','MAY09','JUN09','JUL09','AUG09'... ,'SEP09','OCT09','NOV09','DEC09','JAN10','FEB10','MAR10','APR10','MAY10','JUN10','JUL10','AUG10'... ,'SEP10','OCT10','NOV10','DEC10','JAN11','FEB11','MAR11','APR11','MAY11','JUN11','JUL11','AUG11'... ,'SEP11','OCT11','NOV11','DEC11','JAN12','FEB12','MAR12','APR12','MAY12','JUN12','JUL12','AUG12'... ,'SEP12','OCT12','NOV12','DEC12','JAN13','FEB13','MAR13','APR13','MAY13','JUN13','JUL13','AUG13'... ,'SEP13','OCT13','NOV13','DEC13','JAN14','FEB14','MAR14','APR14','MAY14','JUN14','JUL14','AUG14'... ,'SEP14','OCT14','NOV14','DEC14','JAN15','FEB15','MAR15','APR15','MAY15','JUN15','JUL15','AUG15'... ,'SEP15','OCT15','NOV15','DEC15','JAN16','FEB16','MAR16','APR16','MAY16','JUN16','JUL16','AUG16'... ,'SEP16','OCT16','NOV16','DEC16','JAN17','FEB17','MAR17','APR17','MAY17','JUN17','JUL17','AUG17'... ,'SEP17','OCT17','NOV17','DEC17','JAN18','FEB18','MAR18','APR18','MAY18','JUN18','JUL18','AUG18'... ,'SEP18','OCT18','NOV18','DEC18','JAN19','FEB19','MAR19','APR19','MAY19','JUN19','JUL19','AUG19'}); m = [1 2 3 6 12 24 36 60 84 120 240 360]'; for i=1:100 t = i; data.X_1 = (1-exp(-m./t))./(m./t); data.X_2 = ((1-exp(-m./t))./(m./t))-exp(-m./t); model_1 = fitlm(data, 'FEB95 ~ X_1 + X_2'); RSS(100,:) = zeros ; res = model_1.Residuals.Raw; res(any(isnan(res), 2), :) = []; RSS(i) = sum(res.^2); end RSS(:,2) = [1:1:100]; min = min(RSS(:,1)); t = find(RSS(:,1) == min) data.X_1 = (1-exp(-m./t))./(m./t); data.X_2 = ((1-exp(-m./t))./(m./t))-exp(-m./t); model_1 = fitlm(data, 'FEB95 ~ X_1 + X_2') res = model_1.Residuals.Raw; res(any(isnan(res), 2), :) = []; RSS = sum(res.^2) intercept = model_1.Coefficients.Estimate(1,1); beta_1 = model_1.Coefficients.Estimate(2,1); beta_2 = model_1.Coefficients.Estimate(3,1); Yhat = intercept + beta_1.*data.X_1 + beta_2.*data.X_2; plot(m, Yhat) hold on scatter(m, data.FEB95) I.e "FEB95" should be dynamic? Any suggestions?
Her is how I would approach your problem. First realize that VarNames=data.Properties.VariableNames will get a list of all column names in the table. You could then loop over this list. For example for v=VarNames current_column = v{1}; % .... % Define the model spec for the current column model_spec = [current_column ' ~ X_1 + X_2']; % and create the model model_1 = fitlm(data, model_spec); % ... Continue computation ... and collect results in a table or array end
Calculating values in a for loop
I have a for loop that is storing values. For some reason, it is calculating values for all the number up to and including those in the for loop, instead of just the ones in the array. t = 3600:50:172800;x = 0.1;y = 0; ro = 0.1; T = zeros(1,length(t)); for Cm = 1E6:1E6:4E6 for i = 1:length(t) T = T_ILS(x,y,ro,Cm,t); Tall(Cm,:) = [T]; end end The error I get is "Requested 2000000x3385 array exceeds maximum array size preferences". I would like the for loop to calculate just the Cm values and not every number in between.
You are not computing every value in between, but your indices are messed up. Cm takes values in the millions, but you use it to index Tall(Cm,:). You probably want t = 3600:50:172800;x = 0.1;y = 0; ro = 0.1; T = zeros(1,length(t)); Cm = 1E6:1E6:4E6; for j = 1:length(Cm) for i = 1:length(t) T(i) = T_ILS(x,y,ro,Cm(j),t(i)); end Tall(j,:) = [T]; end Notice that the main function call assigns to T(i) and uses t(i) in the function arguments to justify the existence of the for loop.
In the inner loop, T is the output of the function T_ILS, using the same arguments each time. for i = 1:length(t) T = T_ILS(x,y,ro,Cm,t); Tall(Cm,:) = [T]; end I don't know what this function computes, but you probably wanted to do this instead for i = 1:length(t) T = T_ILS(x,y,ro,Cm,t(i)); Tall(Cm,:) = [T]; end to account for each value of the vector t, or even better: for t = 3600:50:172800 T = T_ILS(x,y,ro,Cm,t); Tall(Cm,:) = [T]; end EDIT: Also, to make this answer complete, I'd like to merge #MadPhysicist's answer with mine. The result would be Cm = 1E6:1E6:4E6; x = 0.1;y = 0; ro = 0.1; T = zeros(1,length(t)); for i = 1:length(Cm) for t = 3600:50:172800; T = T_ILS(x,y,ro,Cm(i),t); Tall(i,:) = [T]; end end
getting nan values after computing mean
the program bellow runs just fine but at some point after computing the mean for every column of the matrix and subtracting them from rows. I do get some NAN values (last 600 rows of the matrix patch) i can't figure out where the problem is. Thanks for hint. nPatchSize = 9; %% fprintf('Loading data ...') if(~exist('imagedata.mat','file')) filePattern = 'C:\File\my_file'; category_list = dir(filePattern); numbClasses = 0; numImagesPerScene = 100; numPatchesPerImage = 20; patch = nan(15*numImagesPerScene*numPatchesPerImage, nPatchSize*nPatchSize); counter = 1; for tempCounter = (1:length(category_list)) if(category_list(tempCounter).isdir) switch(category_list(tempCounter).name) case {'.','..'} otherwise numbClasses = numbClasses + 1; imageList = dir(strcat(filePattern,'\', category_list(tempCounter).name)); % for tempCounter1 = (1:length(imageList)); for tempCounter1 = (1:numImagesPerScene); if(~imageList(tempCounter1).isdir) imageName = strcat(filePattern,'\', category_list(tempCounter).name, '\', imageList(tempCounter1).name); myImage = imread(imageName); if(size(myImage,3)==3) myImage = rgb2gray(myImage); end myImage = double(myImage)/255; %myImage = imresize (myImage, S); for temp = 1:numPatchesPerImage randStaCol = randi([1 size(myImage,1)-nPatchSize+1]);% extracting random patches from image (colum) randStaRow = randi([1 size(myImage,2)-nPatchSize+1]);% extracting random patches from image (row) tmp = myImage(randStaCol:randStaCol+nPatchSize-1, randStaRow:randStaRow+nPatchSize-1); patch(counter, :) = tmp(:)'; counter = counter + 1; end %bmTrainXbin = [bmTrainXbin; myImage(:)']; % label = [label;numbClasses]; end end end end end save('imagedata.mat','patch') else load('imagedata.mat') end fprintf('done\n') %% patch = bsxfun(#minus, patch, mean(patch,1)); At the end when checking patch which is my final matrix it gives me only NAN values.
Repeating the same statement using loop
This is the same statement for 3 times. how to do this with loop. Here, the matrix G1 dimension is 3*10. so in each part i am taking different column. G2 = f_range_m(1:8:length(timeline_ms),:); %%%% measured range G1 = OrgiRange(1:8:end,:); M1 = G1(:,1); dist11 = abs(bsxfun( #minus,G2,M1)); [~,col1] = min(dist11,[],2); Result_1 = diag(G2(:,col1)); M2 = G1(:, 2); dist22 = abs(bsxfun(#minus,G2, M2)); [~,col2] = min(dist22,[],2); Result_2 = diag(G2(:,col2)); M3 = G1(:, 3); dist33 = abs(bsxfun(#minus,G2, M3)); [~,col3] = min(dist33,[],2); Result_3 = diag(G2(:,col3)); I am trying this way. However not getting desired output. Result dimension should be 13*3. obj = 3; for ix = 1:obj M = G1(:,ix); dist = abs(bsxfun(#minus,G2,M)); [~,col] = min (dist,[],2); Result = diag(G2(:,col)); end
With your updated code you nearly solved it. Everything which remains is writing to the columns of Result instead of overwriting it in each iteration. obj = 3; Result=nan(size(G1,1),obj); for ix = 1:obj M = G1(:,ix); dist = abs(bsxfun(#minus,G2,M)); [~,col] = min (dist,[],2); Result(:,ix) = diag(G2(:,col)); end
Subscripted assignment dimension mismatch in matlab
I executed this code using Feature Matrix 517*11 and Label Matrix 517*1. But once the dimensions of matrices change the code cant be run. How can I fix this? The error is: Subscripted assignment dimension mismatch. in this line : edges(k,j) = quantlevels(a); Here is my code: function [features,weights] = MI(features,labels,Q) if nargin <3 Q = 12; end edges = zeros(size(features,2),Q+1); for k = 1:size(features,2) minval = min(features(:,k)); maxval = max(features(:,k)); if minval==maxval continue; end quantlevels = minval:(maxval-minval)/500:maxval; N = histc(features(:,k),quantlevels); totsamples = size(features,1); N_cum = cumsum(N); edges(k,1) = -Inf; stepsize = totsamples/Q; for j = 1:Q-1 a = find(N_cum > j.*stepsize,1); edges(k,j) = quantlevels(a); end edges(k,j+2) = Inf; end S = zeros(size(features)); for k = 1:size(S,2) S(:,k) = quantize(features(:,k),edges(k,:))+1; end I = zeros(size(features,2),1); for k = 1:size(features,2) I(k) = computeMI(S(:,k),labels,0); end [weights,features] = sort(I,'descend'); %% EOF function [I,M,SP] = computeMI(seq1,seq2,lag) if nargin <3 lag = 0; end if(length(seq1) ~= length(seq2)) error('Input sequences are of different length'); end lambda1 = max(seq1); symbol_count1 = zeros(lambda1,1); for k = 1:lambda1 symbol_count1(k) = sum(seq1 == k); end symbol_prob1 = symbol_count1./sum(symbol_count1)+0.000001; lambda2 = max(seq2); symbol_count2 = zeros(lambda2,1); for k = 1:lambda2 symbol_count2(k) = sum(seq2 == k); end symbol_prob2 = symbol_count2./sum(symbol_count2)+0.000001; M = zeros(lambda1,lambda2); if(lag > 0) for k = 1:length(seq1)-lag loc1 = seq1(k); loc2 = seq2(k+lag); M(loc1,loc2) = M(loc1,loc2)+1; end else for k = abs(lag)+1:length(seq1) loc1 = seq1(k); loc2 = seq2(k+lag); M(loc1,loc2) = M(loc1,loc2)+1; end end SP = symbol_prob1*symbol_prob2'; M = M./sum(M(:))+0.000001; I = sum(sum(M.*log2(M./SP))); function y = quantize(x, q) x = x(:); nx = length(x); nq = length(q); y = sum(repmat(x,1,nq)>repmat(q,nx,1),2);
I've run the function several times without getting any error. I've used as input for "seq1" and "seq2" arrays such as 1:10 and 11:20 Possible error might rise in the loops for k = 1:lambda1 symbol_count1(k) = sum(seq1 == k); end if "seq1" and "seq2" are defined as matrices since sum will return an array while symbol_count1(k) is expected to be single value. Another possible error might rise if seq1 and seq2 are not of type integer since they are used as indexes in M(loc1,loc2) = M(loc1,loc2)+1; Hope this helps.