I want to read in a text file that contains some strings but mostly numbers. I want to be able to ignore the strings and only look at the numbers. I want to plot those values on a 3D plane. The data looks like this:
Tech4:<152.266724,173.189377,27.995975>
<117.880638,156.116531,27.999983>
<129.849899,59.195660,27.999983>
<249.321121,60.605404,27.999983>
<224.120361,139.072739,28.000668>
<171.188950,143.490921,56.933430>
<171.188950,143.490921,83.548088>
<171.188950,143.490921,27.999985>
I believe to read in a file is just:
File = textread('testFile.txt');
How can I only look at those values and then plot it.
Thanks!
fid = fopen([pathname,filename]);
tline = fgetl(fid);
CX = [];
CY = [];
CZ = [];
while ischar(tline)
% skip < and >
tline = substr(tline, 1, length(tline)-2)
% extract numbers
temp = textscan(tline,'%n%n%n', 'delimiter',',');
CX(end+1,:) = [temp(1)];
CY(end+1,:) = [temp(2)];
CZ(end+1,:) = [temp(3)];
tline = fgetl(fid);
end
fclose(fid);
and then plot it using
plot3(CX, CY, CZ)
function call.
Add the check for "Tech4:" at the beginning however...
I think you can also directly use textscan in a one-liner:
fid = fopen('testFile.txt');
data = textscan(fid,'%*s%f,%f,%f');
fclose(fid);
this loads the values from all rows with the specified format into the variable data.
no matlab around to test it out though.
fscanf is an option to, the same kind of parameters as textscan.
EDIT: typo, you want to detect floats (%f) of course, and not integers (%d)
EDIT2: got matlab and tested it out, this works here for your sample input ^^
fid = fopen('testFile.txt');
data = textscan(fid,'%*s%f%f%f','Delimiter',',<>')
fclose(fid);
Related
I have a data file that contains parameter names and values with an equal sign in between them. It's like this:
A = 1234
B = 1353.335
C =
D = 1
There is always one space before and after the equal sign. The problem is some variables don't have values assigned to them like "C" above and I need to weed them out.
I want to read the data file (text) into a cell and just remove the lines with those invalid statements or just create a new data file without them.
Whichever is easier, but I will eventually read the file into a cell with textscan command.
The values (numbers) will be treated as double precision.
Please, help.
Thank you,
Eric
Try this:
fid = fopen('file.txt'); %// open file
x = textscan(fid, '%s', 'delimiter', '\n'); %// or '\r'. Read each line into a cell
fclose(fid); %// close file
x = x{1}; %// each cell of x contains a line of the file
ind = ~cellfun(#isempty, regexp(x, '=\s[\d\.]+$')); %// desired lines: space, numbers, end
x = x(ind); %// keep only those lines
If you just want to get the variables, and reject lines that do not have any character, this might work (the data.txt is just a txt generated by the example of data you have given):
fid = fopen('data.txt');
tline = fgets(fid);
while ischar(tline)
tmp = cell2mat(regexp(tline,'\=(.*)','match'));
b=str2double(tmp(2:end));
if ~isnan(b)
disp(b)
end
tline = fgets(fid);
end
fclose(fid);
I am reading the txt file line by line, and using general expressions to get rid of useless chars, and then converting to double the value read.
I have boatloads of tab delimited textfiles that contain numerical data in 1000x2 format.
They're named file00001.txt - file10000.txt
I would like to write a script to load each of these files and make a variable containing ONLY the 400th row of the 2nd column of each of these files.
After that I'm going to try and plot a graph with the data I collected - but that's not important here.
I would be very grateful for your help.
Edit -
My most recent endeavour is:
numfiles = 10;
mydata = cell(1, numfiles);
for k = 1:numfiles
myfilename = sprintf('DM0000%d.txt', k);
mydata{k} = importdata(myfilename);
end
I'm running into a few problems -
1) if numfiles is >9, the 10th file data entry in the mydata variable comes up as []. This may have something to do with the naming method of my files? They're named in this fashion:
DM00000 ...DM00009, DM00010, DM00011, etc.
2) Also this is pretty slow to load, someone said using fopen, if so where should I put it in and how?
I'm guessing it'd be somewhere along the lines of fopen('filename', 'r')?
Based on your edit, this is what I'd recommend:
numfiles = 10;
row = 400;
column = 2;
data = zeros(1, numfiles);
for k = 1:numfiles
filename = sprintf('DM%05d.txt', k);
fid = fopen(filename,'r');
tempdata = textscan(fid, '%f%f');
fclose(fid);
data(k) = tempdata{column}(row);
end
I've updated the formatspec in sprintf to create the filenames correctly (you were missing the padding with zeros). I'm using textscan to import the data as doubles (change the %f to something else if required - check out the formatspec documentation). I also changed data to be a matrix rather than a cell array. You mentioned that you'd want to plot the data, so it'll be easier if it's a matrix and I couldn't see any need to use a cell array here.
I need to read the following csv file in MATLAB:
2009-04-29 01:01:42.000;16271.1;16271.1
2009-04-29 02:01:42.000;2.5;16273.6
2009-04-29 03:01:42.000;2.599609;16276.2
2009-04-29 04:01:42.000;2.5;16278.7
...
I'd like to have three columns:
timestamp;value1;value2
I tried the approaches described here:
Reading date and time from CSV file in MATLAB
modified as:
filename = 'prova.csv';
fid = fopen(filename, 'rt');
a = textscan(fid, '%s %f %f', ...
'Delimiter',';', 'CollectOutput',1);
fclose(fid);
But it returs a 1x2 cell, whose first element is a{1}='ÿþ2', the other are empty.
I had also tried to adapt to my case the answers to these questions:
importing data with time in MATLAB
Read data files with specific format in matlab and convert date to matal serial time
but I didn't succeed.
How can I import that csv file?
EDIT After the answer of #macduff i try to copy-paste in a new file the data reported above and use:
a = textscan(fid, '%s %f %f','Delimiter',';');
and it works.
Unfortunately that didn't solve the problem because I have to process csv files generated automatically, which seems to be the cause of the strange MATLAB behavior.
What about trying:
a = textscan(fid, '%s %f %f','Delimiter',';');
For me I get:
a =
{4x1 cell} [4x1 double] [4x1 double]
So each element of a corresponds to a column in your csv file. Is this what you need?
Thanks!
Seems you're going about it the right way. The example you provide poses no problems here, I get the output you desire. What's in the 1x2 cell?
If I were you I'd try again with a smaller subset of the file, say 10 lines, and see if the output changes. If yes, then try 100 lines, etc., until you find where the 4x1 cell + 4x2 array breaks down into the 1x2 cell. It might be that there's an empty line or a single empty field or whatever, which forces textscan to collect data in an additional level of cells.
Note that 'CollectOutput',1 will collect the last two columns into a single array, so you'll end up with 1 cell array of 4x1 containing strings, and 1 array of 4x2 containing doubles. Is that indeed what you want? Otherwise, see #macduff's post.
I've had to parse large files like this, and I found I didn't like textscan for this job. I just use a basic while loop to parse the file, and I use datevec to extract the timestamp components into a 6-element time vector.
%% Optional: initialize for speed if you have large files
n = 1000 %% <# of rows in file - if known>
timestamp = zeros(n,6);
value1 = zeros(n,1);
value2 = zeros(n,1);
fid = fopen(fname, 'rt');
if fid < 0
error('Error opening file %s\n', fname); % exit point
end
cntr = 0
while true
tline = fgetl(fid); %% get one line
if ~ischar(tline), break; end; % break out of loop at end of file
cntr = cntr + 1;
splitLine = strsplit(tline, ';'); %% split the line on ; delimiters
timestamp(cntr,:) = datevec(splitLine{1}, 'yyyy-mm-dd HH:MM:SS.FFF'); %% using datevec to parse time gives you a standard timestamp vector
value1(cntr) = splitLine{2};
value2(cntr) = splitLine{3};
end
%% Concatenate at the end if you like
result = [timestamp value1 value2];
Can someone tell me why I am receiving this error --
??? The following error occurred converting from cell to
double:
Error using ==> double
Conversion to double from cell is not possible.
Error in ==> test at 18
CX(end+1,:) = temp(1);
Here is the code:
file = fopen('C:\Program Files (x86)\Notepad++\testFile.txt'); % open text file
tline = fgetl(file); % read line by line and remove new line characters
%declare empty arrays
CX = [];
CY = [];
CZ = [];
while ischar(tline) % true if tline is a character array
temp = textscan(fid,'%*s%f%f%f','Delimiter',',<>'); % loads the values from all rows with the specified format into the variable data
CX(end+1,:) = temp(1);
CY(end+1,:) = temp(2);
CZ(end+1,:) = temp(3);
tline = fgetl(file);
end
fclose(file); % close the file
plot3(CX, CY, CZ) % plot the data and label the axises
xlabel('x')
ylabel('y')
zlabel('z')
grid on
axis square
Quick guess: does using curly braces help?
CX(end+1,:) = temp{1}
Use cell2mat to convert from a cell array (what textscan returns) to a numeric array, which you can use (like append to, in your case) with other numeric arrays.
I would also recommend using vertcat rather than the approach you've taken to concatenating:
CX = vertcat(CX, cell2mat(temp(1)));
Or, you could read all 3 values in to a row and concatentate into a N-by-3 matrix instead... lots of options.
In Matlab, after creating a certain number of lines and printing them to a file, I have the need to delete a line and rewrite the rest of the data to that same file. When I do so, the new data overwrites the previous data, but since the data is shorter than the original, there are still remnants of the original data. Does anyone have any idea what the best/most efficient way to delete that extra data is?
Here is a simplified example of what I'm trying to do:
fid = fopen('file.txt','w');
for i=1:10
fprintf(fid,'%i\r\t',i);
end
frewind(fid);
for i=3:5
fprintf(fid,'%i\r\t',i);
end
fprintf(fid,'EOF');
fclose(fid);
I've looked all over, but I can't seem to find the solution to my question. Any suggestions?
Without using any temp files, you can do the following:
fid = fopen('file.txt', 'wt');
for i=1:10
fprintf(fid, '%i\n', i);
end
frewind(fid);
for i=3:5
fprintf(fid, '%i\n', i);
end
pos = ftell(fid); % get current position in file
fclose(fid);
% read from begining to pos
fid = fopen('file.txt', 'r');
data = fread(fid, pos);
fclose(fid);
% overwite file with data read
fid = fopen('file.txt', 'w');
fwrite(fid, data);
fclose(fid);
Printing "EOF" won't work - nice try!
There are Unix system calls truncate and ftruncate that will do that, given either a file descriptor (truncate) or handle (ftruncate) in the first argument and a desired length in the second.
I'd try and see if Matlab supports ftruncate. Failing that... if worst came to worst you could copy-write the file to a new file, stopping and closing the new file when you hit what you consider the end of data.
To follow up on Carl Smotricz's suggestion of using two files, you can use MATLAB's DELETE and MOVEFILE commands to avoid system calls:
fid = fopen('file.txt','wt');
for i=1:10
fprintf(fid,'\t%i\r',i);
end
fclose(fid);
fid = fopen('file.txt','rt');
fidNew = fopen('fileNew.txt', 'wt');
for i = 1:2
s = fgetl(fid);
fprintf(fidNew, '%s\r', s);
end
for i=4:10
fprintf(fidNew, '\t%i\r', i);
end
fclose(fid);
fclose(fidNew);
delete('file.txt');
movefile('fileNew.txt', 'file.txt')