I have some *.dat files in a folder, I would like to extract a particular column (8th column) from all of the files and put into a excel file. I have run a for loop, but it only gives me the results of final run (i.e. if there are 10 number of files, it only returns me 8th column of the 10th files).
data = cell(numel(files),1);
for i = 1:numel(files)
fid = fopen(fullfile(pathToFolder,files(i).name), 'rt');
H = textscan(fid, '%s', 4, 'Delimiter','\n');
C = textscan(fid, repmat('%f ',1,48), 'Delimiter',' ', ...
'MultipleDelimsAsOne',true, 'CollectOutput',true);
fclose(fid);
H = H = H{1}; C = C{1};
data{i} = C;
B = C(:,8);
end
Looking for your help on this.
It would be greatly appreciated.
You are overwriting B each iteration. B(:,i) will put each column 8 of C in a column of B.
Related
I am trying to import every rows that starts with '//', I have tried to extract it with the script below. can anybody check my script please?
formatSpec = '//NFE=%f //ElapsedTime=%f //SBX=%f //DE=%f //PCX=%f //SPX=%f //UNDX=%f //UM=%f //Improvements=%f //Restarts=%f //PopulationSize=%f //ArchiveSize=%f //MutationIndex=%f %*f';
N=1
k = 0;
while ~feof(fileID)
k = k+1;
C = textscan(fileID,formatSpec,N,'CommentStyle','#','Delimiter','\n');
end
It is not clear to me how you want the output to look, but here is one possibilitiy:
fid = fopen(filename, 'rt');
dataset = textscan(fid, '%s', 'delimiter', '\n', 'headerlines', 0);
fclose(fid);
result = regexp(dataset{1}, '//([A-Za-z].*)=([0-9\.].*)', 'tokens');
result = result(cellfun(#(x) ~isempty(x), result));
result contains both the type, e.g. NFE or SBX, and the number (albeit in character format).
I have a set of data in the following format, and I would like to import each block in order to analyze them with Matlab.
Emax=0.5/real
----------------------------------------------------------------------
4.9750557 14535
4.9825821 14522
4.990109 14511
4.9976354 14491
5.0051618 14481
5.0126886 14468
5.020215 14437
5.0277414 14418
5.0352678 14400
5.0427947 14372
5.0503211 14355
5.0578475 14339
5.0653744 14321
Emax=1/real
----------------------------------------------------------------------
24.965595 597544
24.973122 597543
24.980648 597543
24.988174 597542
24.995703 597542
25.003229 597542
I have modified this piece of code from MathWorks, but I think, I have problems dealing with the spaces between each column.
Each block of data consist of 3874 rows and is divided by a text (Emax=XX/real) and a line of ----, unfortunately is the only way the software export the data.
Here is one way to import the data:
% read file as a cell-array of lines
fid = fopen('file.dat', 'rt');
C = textscan(fid, '%s', 'Delimiter','');
C = C{1};
fclose(fid);
% remove separator lines
C(strncmp('---',C,3)) = [];
% location of section headers
headInd = [find(strncmp('Emax=', C, length('Emax='))) ; numel(C)+1];
% extract each section
num = numel(headInd)-1;
blocks = struct('header',cell(num,1), 'data',cell(num,1));
for i=1:num
% section header
blocks(i).header = C{headInd(i)};
% data
X = regexp(C(headInd(i)+1:headInd(i+1)-1), '\s+', 'split');
blocks(i).data = str2double(vertcat(X{:}));
end
The result is a structure array containing the data from each block:
>> blocks
blocks =
2x1 struct array with fields:
header
data
>> blocks(2)
ans =
header: 'Emax=1/real'
data: [6x2 double]
>> blocks(2).data(:,1)
ans =
24.9656
24.9731
24.9806
24.9882
24.9957
25.0032
This should work. I don't think textscan() will work with a file like this because of the breaks between blocks.
Essentially what this code does is loop through lines between blocks until it finds a line that matches the data format. The code is naive and assumes that the file will have exactly the number of blocks lines per block that you specify. If there were a fixed number of lines between blocks it would be a lot easier and you could remove the first inner loop and replace with just ~=fgets(fid) once for each line.
function block_data = readfile(in_file_name)
fid = fopen(in_file_name, 'r');
delimiter = ' ';
line_format = '%f %f';
n_cols = 2; % Number of numbers per line
block_length = 3874; % Number of lines per block
n_blocks = 2; % Total number of blocks in file
tline = fgets(fid);
line_data = cell2mat(textscan(tline,line_format,'delimiter',delimiter,'MultipleDelimsAsOne',1));
block_n = 0;
block_data = zeros(n_blocks,block_length,n_cols);
while ischar(tline) && block_n < n_blocks
block_n = block_n+1;
tline = fgets(fid);
if ischar(tline)
line_data = cell2mat(textscan(tline,line_format,'delimiter',delimiter,'MultipleDelimsAsOne',1));
end
while ischar(tline) && isempty(line_data)
tline = fgets(fid);
line_data = cell2mat(textscan(tline,line_format,'delimiter',delimiter,'MultipleDelimsAsOne',1));
end
line_n = 1;
while line_n <= block_length
block_data(block_n,line_n,:) = cell2mat(textscan(tline,line_format,'delimiter',delimiter,'MultipleDelimsAsOne',1));
tline = fgets(fid);
line_n = line_n+1;
end
end
fclose(fid)
I've have a
function [Q,A] = load_test(filename) which is loading in a text file. I would like the function to skip empty lines, but i'm not sure how to do it.
I have tried to use
~isempty(x), ~ischar(x)
but I keep getting an error message. my code so far is:
fid = fopen(filename);
data = textscan(fid, '%s','delimiter','\n');
fclose(fid);
Q = cellfun(#(x) x(1:end-2), data{1}, 'uni',0);
A = cellfun(#(x) x(end) == 'T' || x(end) == 'F' && ~isempty(x),data{1});
what do I need to do ?
Code
%%// Your code
fid = fopen(filename);
data = textscan(fid, '%s','delimiter','\n')
fclose(fid);
%%// Additional code
%%// 1. Remove empty lines
c1 = ~cellfun(#isempty,data{:})
t1 = data{:,:}(c1,:)
%%// 2. Select only the lines that have F or T as end characters
lastInLine = regexp(t1,'.$','match','lineanchors') %%// Get the end characters
%%// Get a binary array of rows that have F or T at the end
c2 = strcmp(vertcat(lastInLine{:}),'F') | strcmp(vertcat(lastInLine{:}),'T')
%%// Finally select those rows/lines
data = {t1(c2,:)}
Please note that I am not sure if you still need Q and A.
The program myfile.m reads a txt file that contains a total of 25 names and numbers like
exemple:
John doughlas 15986
Filip duch 357852
and so on.
The program converts them to
15986 Doughlas John
357852 duch Filip
This is without function, with it I get too many outputs.
Error message:
Error using disp
Too many output arguments.
Error in red4 (line 26)
array = disp(All);
Original code below:
function array = myfile(~)
if nargin == 0
dirr = '.';
end
answer = dir(dirr);
k=1;
while k <= length(answer)
if answer(k).isdir
answer(k)=[];
else
filename{k}=answer(k).name;
k=k+1;
end
end
chose=menu( 'choose file',filename);
namn = char(filename(chose));
fid = fopen(namn, 'r');
R = textscan(fid,'%s %s %s');
x=-1;
k=0;
while x <= 24
x = k + 1;
All = [R{3}{x},' ',R{1}{x},' ',R{2}{x}];
disp(All)
k = k + 1;
end
fclose(fid);
Now I have got many good answers from people and sites like functions but I cant get the results like the above with function.
I have tried combining them and got some results:
y = 15986 & [a,z,b] = myfile
y = 25 & myfile = x
y = numbers name1,2,3,4 and so one & myfile = fprintf(All)
y = & I used results().namn,
numbers name 1 & results().id, results().lastname
y =
numbers name 2 and so on.
The result I want is:
y = myfile
y =
15986 Doughlas John
357852 duch Filip
update: Change it like Eitan T said but did't get the result like above.
Got the result:
'John doughlas 15986'
'Filip duch 357852'
function C = myfile()
if nargin == 0
dirr = '.';
end
answer = dir(dirr);
k=1;
while k <= length(answer)
if answer(k).isdir
answer(k)=[];
else
filname{k}=answer(k).name;
k=k+1;
end
end
chose=menu( 'choose',filname);
name = char(filname(chose));
fid = fopen(name, 'r');
C = textscan(fid, '%s', 'delimiter', '');
C = regexprep(C{1}, '(\w+) (\w+) (\w+)', '$3 $2 $1');
fclose(fid);
Why use loops? Read the lines at once with textscan and use regexprep to manipulate the words:
fid = fopen(filename, 'r');
C = textscan(fid, '%s', 'delimiter', '');
C = regexprep(C{1}, '(\w+) (\w+) (\w+)', '$3 $2 $1')
fclose(fid);
The result is a cell array C, each cell storing a line. For your example, you'll get a 2×1 cell array:
C =
'15986 doughlas John'
'357852 duch Filip'
I'm not sure what you want to do with it, but if you provide more details I can improve my answer further.
Hope this helps!
I have a comma separated file with 182 rows and 501 columns, of which 500 columns are of type number (features) while the last column are strings (labels).
Example: 182x501 dimension
1,3,4,6,.........7, ABC
4,5,6,4,.........9, XYZ
3,4,5,3,.........2, ABC
How can I load this file so it will have a data set with a matrix, B, containing the number as my features, and a vector, C, containing the strings as my labels?
d = dataset(B, C);
Build a format specifier for textscan based on the number and types of columns, and have it read the file for you.
nNumberCols = 500;
format = [repmat('%f,', [1 nNumberCols]) '%s'];
fid = fopen(file);
x = textscan(fid, format);
fclose(fid);
B = cat(2, x{1:nNumberCols});
C = x{end};
You could use the textscan function. For example:
fid = fopen('test.dat');
% Read numbers and string into a cell array
data = textscan(fid, '%s %s');
% Then extract the numbers and strings into their own cell arrays
nums = data{1};
str = data{2};
% Convert string of numbers to numbers
for i = 1:length(str)
nums{i} = str2num(nums{i}); %#ok<ST2NM>
end
% Finally, convert cell array of numbers to a matrix
nums = cell2mat(nums);
fclose(fid);
Note that I have made a number of assumptions here, based on the file format you have specified. For example, I assume that there are no spaces after the commas following a number, but that there is a space immediately preceding the string at the end of each line.
To can make the above code more flexible by using a more considered format specifier (the second argument to textscan). See the section Basic Conversion Specifiers in the textscan documentation.
For example, if you have the following data in a file named data.txt:
1,3,4,6,7, ABC
4,5,6,4,9, XYZ
3,4,5,3,2, ABC
you can read it into a matrix B and a cell array C using the code
N = 5; % Number of numeric data to read
fid = fopen('data.txt');
B = []; C = {};
while ~feof(fid) % repeat until end of file is reached
b = fscanf(fid, '%f,', N); % read N numeric data separated by a comma
c = fscanf(fid, '%s', 1); % read a string
B = [B, b];
C = [C, c];
end
C
B
fclose(fid);
to give
C =
'ABC' 'XYZ' 'ABC'
B =
1 4 3
3 5 4
4 6 5
6 4 3
7 9 2