My code is below. In the code, I am evaluating only the data in the 'fb2010' file. I want to add other files" 'fb2020', 'fb2030', and 'fb2040' and evaluate their data by the same code. My question is how to apply a for loop and include the other data files. I tried, but I got confused by the for loop.
load('fb2010'); % loading the data
x = fb2010(3:1:1502,:);
% y_filt = filter(b,a,x); % filtering the received signal
y_filt= filter(b,a,x,[],2);
%%%%%%% fourier transform
nfft = length(y_filt);
res = fft(y_filt,nfft,2)/nfft;
res2 = res(:,1:nfft/2+1); %%%% taking single sided spectrum
res3 = fft(res2,[],2);
for i = 3:1:1500 %%%% dividing each row by first row.
resd(i,:) = res3(i,:)./res3(1,:);
end
I'm assuming that your files are MAT-files, not ASCII. You can do this by having load return a struct and using dynamic field referencing:
n = 4;
for i = 1:n
vname = ['fb20' int2str(i) '0']; % Create file/variable name based on index
s = load(vname); % Load data as struct (overwriting previous s)
x = s.(vname)(3:1:1502,:); % Access struct with dynamic field reference
% Rest of your code
...
end
If you're using a plain ASCII file, load won't produce a struct. However, such files are much simpler (see documentation for load/save). The following code would probably work:
n = 4;
for i = 1:n
vname = ['fb20' int2str(i) '0']; % Create file/variable name based on index
s = load(vname); % Load data as matrix (overwriting previous s)
x = s(3:1:1502,:); % Directly index matrix
% Rest of your code
...
end
It would be a good idea to add the file extension to your load command to make your code more readable.
Related
This question already has answers here:
matlab iterative filenames for saving
(4 answers)
Closed 2 years ago.
I have a large data set (vector) I want to split up in to n smaller sets to look at later with other scripts. I.e.if n = 10 I want to turn one 1x80000000 double in to ten 1x8000000 doubles. My thoughts are turn the original in to a n by m matrix then save each row of the matrix in to it's own vector, as follows.
%data-n-splitter
n = 10 %number of sections
L = length(data);
Ls = L/n;
Ls = floor(Ls);
Counter = 1;
%converting vector to matrix
datamatrix = zeros(n,Ls);
for k = 1:n
datamatrix(k,:) = data(Counter:Counter+ Ls - 1);
Counter = Counter + Ls;
end
How do I make matlab loop this part of the code n times:
%save each row of matrix as seperate vector
P1 = datamatrix(1,:);
P2 = datamatrix(2,:);
P3 = datamatrix(3,:);
P4 = datamatrix(4,:);
P5 = datamatrix(5,:);
P6 = datamatrix(6,:);
P7 = datamatrix(7,:);
P8 = datamatrix(8,:);
P9 = datamatrix(9,:);
P10 = datamatrix(10,:);
Example answer that I'm hoping for:
for k = 1:n
P('n') = datamatrix(n,:);
end
I've seen some articles about using cell arrays but the scripts I'm passing the variables to aren't set up for this so I'd rather not go down that route if possible.
There are several options:
use a struct, which comes closest to what you are hoping for,
use a cell, more convenient looping but no access over meaningful names,
use a higher-dimension matrix (in your case it is only 2D, but the same applies for 3D or higher). This is the most memory-efficient option.
To round this off, you could also use a table, which is a hybrid of a struct and a cell as you can use both notations to access it. There is no other benefit.
Now, how to do this? The simplest (and best) solution first: create a 2D matrix with reshape
Ary = 1:10; % I shrank your 1x80000000 array to 1x10 but you'll get the idea
%% create new structure
Mat = reshape(Ary,5,2);
%% access new structure (looping over columns)
for i = 1:size(Ary,2)
% access columns through slicing
ary_sct = Mat(:,i);
% do something
end
Pro: memory efficient (requires the same amount of memory as the initial array); easy looping
Con: only works if you can slice the initial array evenly
Next: create a cell
Ary = 1:10;
n = 2; % number of sections
L = floor(length(Ary)/n);
% allocate memory
C = cell(1,n);
%% create new structure
for i = 1:n
% access the content of a cell with {}
C{i} = Ary((i-1)*L+1:i*L);
end
%% access new structure (looping over entries)
for i = 1:length(C)
% access the content of a cell with {}
ary_sct = C{i};
% do something
end
Pro: You can store anything in a cell. Every data type and -- what is often more important -- of any dimension
Con: The accessing the content (through {}) or accessing the element (through ()) is a bit annoying if your are a beginner; each element require a memory overhead of about 60 bytes as those are pointers, which need to store the information where and on what they are pointing.
Next: use a struct
Ary = 1:10;
n = 2; % number of sections
L = floor(length(Ary)/n);
% create empty struct
S = struct();
%% create new structure
for i = 1:n
% create fieldname (must start with a character!)
fld = num2str(i,'F%d');
% write to field (note the brackets)
S.(fld) = Ary((i-1)*L+1:i*L);
end
%% access new structure (looping over fieldnames)
% get all field names
FlNms = fieldnames(S);
for i = 1:length(FldNames)
% access field names (this is a cell!)
fld = FldNms{i};
% access struct
ary_sct = S.(fld);
% do something
end
Pro: Field names are convenient to keep the overview of your data
Con: accessing field names in a loop is a bit tedious; each element require a memory overhead of about 60 bytes as those are pointers, which need to store the information where and on what they are pointing.
I have a function that is part of an object that I have name TestData. Here are the properties and its constructor:
classdef TestData
properties
metaData = []; % stores meta data in Nx2 array
data = []; % stores data in PxQ array
colLabels = []; % labels columns
colUnits = []; % provides units
temp = []; % temporary to bypass setters while structuring data
end
methods
%% Constructor
function this = TestData(varargin) % Will take in a variable number of inputs
if (nargin == 0)
return;
end
if (nargin == 1 && ischar(varargin{1})) % Case of the input being a file (.txt, .s*p, etc..)
this = readDataFromFile(this, varargin{1});
return;
elseif (nargin == 4) % Case of the input being raw data. Note that the order matters.
this.metaData = varargin{1};
this.colLabels = varargin{2};
this.colUnits = varargin{3};
this.data = varargin{4};
return
else
error('Not enough input arguments.');
end
end
Now, let us say that I have 42 text files that I want to pass into the object. I have created a function for this called extractData that sifts through the data and finds a user input x-axis and y-axis. It does so by first calling a function readDataFromFile, which sets the property arrays that are outlined in the code above. It is a fairly long function and I won't include it for the time being, but I will say that I do know it is working.
The problem: I make an object of the class called myTestData and perform this operation in the CW, myTestData=myTestData.extractData('Tip Type','Tip S/N'). This will look for the Tip Type and Tip S/N. As you can see in the code, the type and S/N are in column 3 of the matrix. If the string is found, it takes the row it was found on, accesses column 3 of that row, and places that value into the temp array. In my case, the type is passive and the S/N is AM66191. It will write this to the temp array, but then on the second iteration of the for loop it will write something else over the first row and the passive and AM66191 will be displayed on the second row. It will do this all the way until the end, where on row 42 I will see the type and S/N, but all the other rows are just garbage. I have attached an image of what the temp array looks like after the second iteration of the for loop. Here is the code for the extractData function:
function this = extractData(this, xAxis, yAxis)
s = dir('*.txt'); % Gather all text files
disp(length(s))
for i=1:length(s) % Loop through and gather data until last element of strcuct
j = 1;
fid = s(i).name; % Open file in read-only mode
this = this.readDataFromFile(fid);
disp(fid)
x = this.metaData(find(contains(this.metaData(:,1), xAxis)),3);
this.temp(i,j) = x;
disp(this.temp(i,j))
j = j+1;
y = this.metaData(find(contains(this.metaData, yAxis)),3); %#ok<*FNDSB>
this.temp(i,j) = y;
disp(this.temp(i,j))
end %for
end %extractData
This is a followup to a previous issue I was having.
I want to give an offset to a signal then add some delay in it and calculate RMSE for that but when taking difference I am having the following issue:
I would like to ask the following things:
How can I solve the above problem?
Will anybody please explain in simple words what iddata does - because I have studied different portals including MATLAB but remained unable to get a good concept.
How can I store data of type iddata in cell for subtraction in the last part of my code?
Code with Problem :
drv(1)=load('123.mat');
t = drv(1).x;
ref = drv(1).y;
angle = drv(1).z;
Fs = 1000;
t1 =t';
ref1= ref';
d_data = iddata(ref1, t1, 1/Fs);
%% Add offset:
x = 1;
afterOffset1= {};
for i = 100:10:130
T = getTrend(d_data);
% <detrend data if needed>
T.InputOffset = i;
T.OutputOffset = i;
afterOffset = retrend(d_data,T);
afterOffset1{x,1}= afterOffset;
x= x+1 ;
end
%% Add delay:
y=20;
afterDelay1= {};
for i = 1:1:4
% delaySamples = i; % Must be a non-negative value
% afterDelay = iddata([NaN(delaySamples,1); d_data.OutputData],...
% [d_data.InputData; NaN(delaySamples,1)], 1/Fs);
afterOffset1{i}.Tstart = y;
afterDelay1{i,1}= afterOffset1{i};
y= y+10;
end
%% Plot:
n = size(afterDelay1,1);
figure();
for i=1:1:n
subplot(2,2,i);
plot(d_data);
hold all
plot(afterDelay1{i});
end
sig_diff = angle(1)-afterDelay1;
square_error(i,:) = (sig_diff(i)).^2;
mse(i,:)= mean(square_error(i));
rmse(i,:) = sqrt(mse(i));
sig_diff = d_data_1 - afterDelay; % <<<<<<<<<<<<<<<<<<<<<< Problem is here
% square_error = (sig_diff).^2;
% mse= mean(square_error);
% rmse = sqrt(mse);
end
You most likely want the OutputData attribute from the iddata object which is the output or y signal of your problem:
sig_diff = angle(1)-afterDelay1.OutputData;
Also note that this will give you a column vector, but your code later on assumes it's a row vector. You may want to transpose this data after you perform the above calculation before proceeding:
sig_diff = angle(1)-afterDelay1.OutputData;
sig_diff = sig_diff.';
In general, iddata is a function that creates an object that represents input and output time or frequency domain data. Take note that when you create an iddata object, the input matrix can potentially have multiple sources and so each column dictates a source. The same can be said for the output where each column dictates an output. Therefore, it is very important that you transpose your data prior to using this function to ensure that each signal is in a separate column, or just use a single column to represent one input / output.
Inside the object has a variety of attributes, including the sampling time or sampling frequency, the valid domain and range that the function takes on and finally accessing the input and output data. OutputData is one of these fields. I'd recommend looking at the documentation that talks about all of the attributes that you can access with iddata. OutputData is clearly defined here: https://www.mathworks.com/help/ident/ref/iddata.html
Can someone help me to understand how I can save in matlab a group of .csv files, select only the columns in which I am interested and get as output a final file in which I have the average value of the y columns and standard deviation of y axes? I am not so good in matlab and so I kindly ask if someone to help me to solve this question.
Here what I tried to do till now:
clear all;
clc;
which_column = 5;
dirstats = dir('*.csv');
col3Complete=0;
col4Complete=0;
for K = 1:length(dirstats)
[num,txt,raw] = xlsread(dirstats(K).name);
col3=num(:,3);
col4=num(:,4);
col3Complete=[col3Complete;col3];
col4Complete=[col4Complete;col4];
avgVal(K)=mean(col4(:));
end
col3Complete(1)=[];
col4Complete(1)=[];
%columnavg = mean(col4Complete);
%columnstd = std(col4Complete);
% xvals = 1 : size(columnavg,1);
% plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');
B = reshape(col4Complete,[5000,K]);
m=mean(B,2);
C = reshape (col4Complete,[5000,K]);
S=std(C,0,2);
Now I know that I should compute mean and stdeviation inside for loop, using mean()function, but I am not sure how I can use it.
which_column = 5;
dirstats = dir('*.csv');
col3Complete=[]; % Initialise as empty matrix
col4Complete=[];
avgVal = zeros(length(dirstats),2); % initialise as columnvector
for K = 1:length(dirstats)
[num,txt,raw] = xlsread(dirstats(K).name);
col3=num(:,3);
col4=num(:,4);
col3Complete=[col3Complete;col3];
col4Complete=[col4Complete;col4];
avgVal(K,1)=mean(col4(:)); % 1st column contains mean
avgVal(K,2)=std(col4(:)); % 2nd column contains standard deviation
end
%columnavg = mean(col4Complete);
%columnstd = std(col4Complete);
% xvals = 1 : size(columnavg,1);
% plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');
B = reshape(col4Complete,[5000,K]);
meanVals=mean(B,2);
I didn't change much, just initialised your arrays as empty arrays so you do not have to delete the first entry later on and made avgVal a column vector with the mean in column 1 and the standard deviation in column 1. You can of course add two columns if you want to collect those statistics for your 3rd column in the csv as well.
As a side note: xlsread is rather heavy for reading files, since Excel is horribly inefficient. If you want to read a structured file such as a csv, it's faster to use importdata.
Create some random matrix to store in a file with header:
A = rand(1e3,5);
out = fopen('output.csv','w');
fprintf(out,['ColumnA', '\t', 'ColumnB', '\t', 'ColumnC', '\t', 'ColumnD', '\t', 'ColumnE','\n']);
fclose(out);
dlmwrite('output.csv', A, 'delimiter','\t','-append');
Load it using csvread:
data = csvread('output.csv',1);
data now contains your five columns, without any headers.
I'm new to Matlab.
I'm trying to apply PCA function(URL listed below)into my palm print recognition program to generate the eigenpalms. My palm print grey scale images dimension are 450*400.
Before using it, I was trying to study these codes and add some codes to save the eigenvector as .mat file. Some of the %comments added by me for my self understanding.
After a few days of studying, I still unable to get the answers.
I decided to ask for helps.I have a few questions to ask regarding this PCA.m.
PCA.m
What is the input of the "options" should be? of "PCA(data,details,options)"
(is it an integer for reduced dimension? I was trying to figure out where is the "options" value passing, but still unable to get the ans. The msgbox of "h & h2", is to check the codes run until where. I was trying to use integer of 10, but the PCA.m processed dimension are 400*400.)
The "eigvector" that I save as ".mat" file is ready to perform Euclidean distance classifier with other eigenvector? (I'm thinking that eigvector is equal to eigenpalm, like in face recognition, the eigen faces. I was trying to convert the eigenvector matrix back to image, but the image after PCA process is in Black and many dots on it)
mySVD.m
In this function, there are two values that can be changed, which are MAX_MATRIX_SIZE set by 1600 and EIGVECTOR_RATIO set by 0.1%. May I know these values will affect the results? ( I was trying to play around with the values, but I cant see the different. My palm print image dimension is set by 450*400, so the Max_matrix_size should set at 180,000?)
** I hope you guys able to understand what I'm asking, please help, Thanks guys (=
Original Version : http://www.cad.zju.edu.cn/home/dengcai/Data/code/PCA.m
mySVD: http://www.cad.zju.edu.cn/home/dengcai/Data/code/mySVD.m
% Edited Version by me
function [eigvector, eigvalue] = PCA(data,details,options)
%PCA Principal Component Analysis
%
% Usage:
% [eigvector, eigvalue] = PCA(data, options)
% [eigvector, eigvalue] = PCA(data)
%
% Input:
% data - Data matrix. Each row vector of fea is a data point.
% fea = finite element analysis ?????
% options.ReducedDim - The dimensionality of the reduced subspace. If 0,
% all the dimensions will be kept.
% Default is 0.
%
% Output:
% eigvector - Each column is an embedding function, for a new
% data point (row vector) x, y = x*eigvector
% will be the embedding result of x.
% eigvalue - The sorted eigvalue of PCA eigen-problem.
%
% Examples:
% fea = rand(7,10);
% options=[]; %store an empty matrix in options
% options.ReducedDim=4;
% [eigvector,eigvalue] = PCA(fea,4);
% Y = fea*eigvector;
%
% version 3.0 --Dec/2011
% version 2.2 --Feb/2009
% version 2.1 --June/2007
% version 2.0 --May/2007
% version 1.1 --Feb/2006
% version 1.0 --April/2004
%
% Written by Deng Cai (dengcai AT gmail.com)
%
if (~exist('options','var'))
%A = exist('name','kind')
% var = Checks only for variables.
%http://www.mathworks.com/help/matlab/matlab_prog/symbol-reference.html#bsv2dx9-1
%The tilde "~" character is used in comparing arrays for unequal values,
%finding the logical NOT of an array,
%and as a placeholder for an input or output argument you want to omit from a function call.
options = [];
end
h2 = msgbox('not yet');
ReducedDim = 0;
if isfield(options,'ReducedDim')
%tf = isfield(S, 'fieldname')
h2 = msgbox('checked');
ReducedDim = options.ReducedDim;
end
[nSmp,nFea] = size(data);
if (ReducedDim > nFea) || (ReducedDim <=0)
ReducedDim = nFea;
end
if issparse(data)
data = full(data);
end
sampleMean = mean(data,1);
data = (data - repmat(sampleMean,nSmp,1));
[eigvector, eigvalue] = mySVD(data',ReducedDim);
eigvalue = full(diag(eigvalue)).^2;
if isfield(options,'PCARatio')
sumEig = sum(eigvalue);
sumEig = sumEig*options.PCARatio;
sumNow = 0;
for idx = 1:length(eigvalue)
sumNow = sumNow + eigvalue(idx);
if sumNow >= sumEig
break;
end
end
eigvector = eigvector(:,1:idx);
end
%dt get from C# program, user ID and name
evFolder = 'ev\';
userIDName = details; %get ID and Name
userIDNameWE = strcat(userIDName,'\');%get ID and Name with extension
filePath = fullfile('C:\Users\***\Desktop\Data Collection\');
userIDNameFolder = strcat(filePath,userIDNameWE); %ID and Name folder
userIDNameEVFolder = strcat(userIDNameFolder,evFolder);%EV folder in ID and Name Folder
userIDNameEVFile = strcat(userIDNameEVFolder,userIDName); % EV file with ID and Name
if ~exist(userIDNameEVFolder, 'dir')
mkdir(userIDNameEVFolder);
end
newFile = strcat(userIDNameEVFile,'_1');
searchMat = strcat(newFile,'.mat');
if exist(searchMat, 'file')
filePattern = strcat(userIDNameEVFile,'_');
D = dir([userIDNameEVFolder, '*.mat']);
Num = length(D(not([D.isdir])))
Num=Num+1;
fileName = [filePattern,num2str(Num)];
save(fileName,'eigvector');
else
newFile = strcat(userIDNameEVFile,'_1');
save(newFile,'eigvector');
end
You pass options in a structure, for instance:
options.ReducedDim = 2;
or
options.PCARatio =0.4;
The option ReducedDim selects the number of dimensions you want to use to represent the final projection of the original matrix. For instance if you pick option.ReducedDim = 2 you use only the two eigenvectors with largest eigenvalues (the two principal components) to represent your data (in effect the PCA will return the two eigenvectors with largest eigenvalues).
PCARatio instead allows you to pick the number of dimensions as the first eigenvectors with largest eigenvalues that account for fraction PCARatio of the total sum of eigenvalues.
In mySVD.m, I would not increase the default values unless you expect more than 1600 eigenvectors to be necessary to describe your dataset. I think you can safely leave the default values.