I had a text-based file with .ptx suffix. It contains the point cloud information please see the following example
100
50
0.352 -5.207 -0.823 0.238 61 61 61
0.345 -5.202 -0.824 0.234 60 60 60
...
Question:
How can I load the file from the third row (ignore the first two rows) and save is as a matrix.
I would recommend using textscan.
Something like:
in = textscan('sample.ptx','%f %f %f %f %f %f %f','HeaderLines',2)
You can specify a number of header lines to skip using 'HeaderLines'. The %f refers to the format of the input data. Hope that helps.
Here is a full example of how to apply textscan and transform the result in to a matrix:
fid = fopen('textscantest.txt','r');
assert(fid~=1); % verify file is opened
C = textscan(fid,'%f %f %f %f %f %f %f','HeaderLines',2);
fclose(fid);
M = [C{:}]
Note that since you want it all in the same matrix, you need the same data type and all %f is required for each column.
Related
I am pretty new to matlab. I've been reading the documentation but can figure it out why matlab does not correctly read the string from file. What I am trying to do is to read a mixed data type from file. Some sample data is:
t a e incl lasc aper meanan truean rupnode rdnnode name
0.000000 1.2712052487 0.8899021688 22.2458 265.2511471042 322.1539251184 -13.6281352271 -130.986 0.155342 0.889756 phaet_000018
0.000000 1.2712052478 0.8899021575 22.2458 265.2511428392 322.1539270642 -13.6281369694 -130.986 0.155342 0.889756 phaet_000044
0.000000 1.2712052496 0.8899021868 22.2458 265.2511587897 322.1539149438 -13.6281365049 -130.986 0.155342 0.889755 phaet_000006
The first line is header. So here is what I've done so far:
fid = fopen('data.dat');
header = fgetl(fid); # I read the header
Now I read the data:
data = fscanf(fid,'%f %f %f %f %f %f %f %f %f %f %s',[11 inf]);
data1 = data';
fclose(fid);
I can now access the first element as:
data1(1,1)
However, when I do:
data(1,11)
instead of phaet_000018 I am getting a number (112). Any idea what I am doing wrong?
There are a few issues with your code.
First, your sizeA input to fscanf is backwards. sizeA with a vector input is defined as:
Read at most m*n numeric values or character fields. n can be Inf, but m cannot. The output, A, is m-by-n, filled in column order.
So you've asked fscanf to give you 11 rows and whatever number of columns. You can't have an Inf row specification so you'll want to remove the third input entirely and reshape your data afterwards.
For example:
fid = fopen('data.dat');
header = fgetl(fid);
data = fscanf(fid,'%f %f %f %f %f %f %f %f %f %f %s');
fclose(fid);
% We just happen to know this explicitly, not knowledge to generally assume
ncols = 22;
% Reshape and transpose
data = reshape(data, ncols, []).';
Gives us a 3 x 22 data array, which is kinda sorta what we want.
So where are the extra columns coming from? For %s fields, fscanf reads the string until it encounters whitespace. Because the output of fscanf is a numeric array it must convert this string into a numeric value, so it converts each character to its numeric equivalent (double(letter)) and outputs that into the matrix.
Using our above data matrix as an example, we have:
>> char(data(1, 11:end))
ans =
phaet_000018
With this in mind, your initial code only happens to work because all of your strings are the same length. If we change the length of one or more of the strings, this data import will fail:
Error using reshape
Product of known dimensions, 22, not divisible into total number of elements, 65.
Error in testcode (line 11)
data = reshape(data, ncols, []).';
So what can we do instead? If you need this string from your data I would recommend trying textscan:
fid = fopen('data.dat');
header = fgetl(fid);
data = textscan(fid, '%f %f %f %f %f %f %f %f %f %f %s');
fclose(fid);
This will read your data into a 1x11 cell array, where each column corresponds to a column in your data:
>> data{1} % t
ans =
0
0
0
To collect your numeric data you can iterate through the cell array, or you can use the 'CollectOutput' flag in textscan:
fid = fopen('data.dat');
header = fgetl(fid);
data = textscan(fid, '%f %f %f %f %f %f %f %f %f %f %s', 'CollectOutput', true);
fclose(fid);
Which will output a 1x2 cell array, where data{1} is your numeric array and data{2} is a cell array containing your strings:
>> data{1} % Numeric data
ans =
0 1.2712 0.8899 22.2458 265.2511 322.1539 -13.6281 -130.9860 0.1553 0.8898
0 1.2712 0.8899 22.2458 265.2511 322.1539 -13.6281 -130.9860 0.1553 0.8898
0 1.2712 0.8899 22.2458 265.2512 322.1539 -13.6281 -130.9860 0.1553 0.8898
>> data{2} % Strings
ans =
3×1 cell array
'phaet_000018'
'phaet_000044'
'phaet_000006'
I have a cell variable (Size:2639516x12, Bytes:3863876744, Class:cell) and I want to make a selection, considering the first row. So for instance if I have
A:
1997 FD 89
1997 GD 65
1999 FDK 87
2010 UY 123
I would like to get
B:
1997 FD 89
1997 GD 65
To get to cell A I use the following code:
% Transfer csv file to matlab
Data_file = fopen('Data.csv');
Data = textscan(Data_file,'%q %q %q %q %f %f %f %f %s %f %f %f %s %f %s %f %s %f %f %f %s','delimiter',',','headerlines', 1);
fclose(Data_file);
%Convert numbers into strings
F_5=Data{:,5};
F_6=num2cell(Data{:,6});
F_7=num2cell(Data{:,7});
%Get the first 4 numbers within variable F_5
F_5A=max(0,fix(log10(F_5)+1)-4);
F_5B=fix(F_9./10.^F_5A);
%Convert number into string
F_5C = num2cell(F_5B);
%Create new cell A w/ variables I want
A=[F_5C Data{:,1} Data{:,2} Data{:,3} Data{:,4} F_6 F_7];
Using logical indexing
B = A(cell2mat(A(:,1))==1997,:);
Thanks to excaza for mentioning that values may not be rounded
If the year values are not properly rounded (i.e some cells have value as 1996.999999 or 1997.0001) then use
e = 0.001 %\\some small value
B = A(abs(cell2mat(A(:,1))-1997)<e,:);
You can use the following code to extract specific rows based on the first column of a.
b=a(~cellfun('isempty',(cellfun(#(x) find(x==1997),a(:,1),'UniformOutput',false))),:);
Here is how it works:
a =
[1997] 'FD' [ 89]
[1997] 'GD' [ 65]
[1999] 'FDK' [ 87]
[2010] 'UY' [123]
b=a(~cellfun('isempty',(cellfun(#(x) find(x==1997),a(:,1),'UniformOutput',false))),:);
b =
[1997] 'FD' [89]
[1997] 'GD' [65]
I am reading a tab-delimited file. Five representative lines of this file are:
Date Time Property Path 1 Path 2 Path 3 Path 4 Path 5 Path 6 Path 7 Path 8
Lev 1 Lev 1 Lev 1 Lev 1 Lev 1 Lev 1 Lev 1 Lev 1
1/1 00:00:00 F1 (sm³/s) -1.3405E-003 -1.1170E-002 -1.0123E-004 9.7769E-003 -8.4673E-004 1.1710E-003 2.6890E-004 2.2413E-003
1/1 01:00:00 F1 (sm³/s) 1.9988E-004 1.6655E-003 2.2252E-004 1.6883E-003 1.8612E-003 2.0221E-004 2.0795E-004 1.7333E-003
1/1 02:00:00 F1 (sm³/s) -4.0722E-004 -3.3931E-003 -4.4324E-004 -2.1177E-003 -3.7075E-003 -2.5364E-004 -3.7330E-004 -3.1115E-003
When I use the following format string I get the expected results:
test = '1/1 00:00:00 F1 (sm³/s) -1.3405E-003 -1.1170E-002 -1.0123E-004 9.7769E-003 -8.4673E-004 1.1710E-003 2.6890E-004 2.2413E-003';
textscan(test, '%*s %*s %*s %*s %f %f %f %f %f %f %f %f')
Gives me:
ans =
[-0.0013] [-0.0112] [-1.0123e-04] [0.0098] [-8.4673e-04] [0.0012] [2.6890e-04] [0.0022]
Which is what I want, but when I attempt:
C = textscan(fid,...
'%*s %*s %*s %*s %f %f %f %f %f %f %f %f',...
'CollectOutput', false,...
'Headerlines', 2);
I get a 1x8 cell of empty cells.
What is the error in the format string translation?
I don't think there's anything wrong with your format string specifically.
Try pulling in the lines individually with fgetl or similar and just check that there's nothing you weren't expecting in the file. For example - your code seems to work for me but I can replicate your error by putting an additional blank line at the start of the file, which causes textscan to try and read the second header line as a line of data (and fail inelegantly). That particular error can be removed by increasing the value of HeaderLines.
fid = fopen('test.txt');
fgetl(fid) % repeat until you see your first line of data
Now, I try to use you code and it's work!
file=('d.txt');
fid=fopen(file);
C = textscan(fid,...
'%*s %*s %*s %*s %f %f %f %f %f %f %f %f',...
'CollectOutput', false,...
'Headerlines', 2);
Output:
celldisp(C)
C{1} =
-0.0013405
0.00019988
-0.00040722
C{2} =
-0.01117
0.0016655
-0.0033931
C{3} =
-0.00010123
0.00022252
-0.00044324
C{4} =
0.0097769
0.0016883
-0.0021177
C{5} =
-0.00084673
0.0018612
-0.0037075
C{6} =
0.001171
0.00020221
-0.00025364
C{7} =
0.0002689
0.00020795
-0.0003733
C{8} =
0.0022413
0.0017333
-0.0031115
I came across a problem where my textscan would only grab empty cell arrays, google search led me here. I solved it by using fgetl(fid) a couple of times and then frewind(fid), (fid being your variable for fopen) something about reading the lines made it easier to bring in the values.
I want to read the training.csv file with Octave for the Kaggle competition.
The file contains 16 fields. First 15 are the coordinates of keypoints. The 16th is is the image, which is 9216 numbers (0 to 255) separated by space.
Tried, but with no luck the followings:
- data = csvread('training.csv');
- data = dlmread('training.csv', ',');
- [l1,l2,l3,l4,l5,l6,l7,l8,l9,l10,l11,l12,l13,l14,l15, image] =
textread("training.csv", "%f %f %f %f %f %f %f %f %f %f %f %f %f %f
%f %s", "delimiter", ",", "endofline", "\n", "headerlines", 1);
Note:
The file also contains missing data.
The first 3 lines of the csv file are here: pastebin.com/pwBQgcfa
Thanks,
This post contributed greatly to figure this out.
The key is to:
Remove the header row
Replace the ",,"(double commas) with "0"
Replace ","(single comma) with " " (space)
The code to read the file
fn = 'training_space.txt';
M = dlmread(fn);
I am trying to interpret data from an eye tracking device. The files exported from the eye tracker are in ASCII format.
Recording files that contain data from a single eye only look like this (only four rows shown):
6372825 645.3 275.4 1362.0 ...
6372826 644.6 274.0 1364.0 ...
6372827 644.2 273.2 1365.0 ...
6372828 642.5 272.7 1367.0 ...
Note that the dots at the end of each row above are a part of the output file, i.e. I haven't added them for the purposes of this question. I normally detect these dots and later throw them away.
The format of the above columns is [timestamp, X, Y, pupilSize, {...}]
A recording from both eyes looks like this (only four rows shown):
505076 416.8 755.4 1148.0 23.6 751.1 1239.0 .....
505077 417.0 758.4 1143.0 23.7 753.1 1244.0 .....
505078 416.7 761.4 1146.0 24.6 752.1 1249.0 .....
505079 416.1 764.8 1150.0 27.3 750.2 1250.0 .....
In this case, the data format is [timestamp, X(left), Y(left), pupilSize(left), X(right), Y(right), pupilSize(right), {.....}]
In both cases, I'd like to extract the numbers from the text and assign them to an array. Here's how I do this for recordings from a single eye:
eyeData = textscan(fid,'%d %f %f %f %s');
I can do the same for binocular recordings, using the following code:
eyeData = textscan(fid,'%d %f %f %f %f %f %f %s');
The trouble is, I'd like to be able to automatically detect whether the data I'm dealing with are monocular or binocular. In other words, I need a way of determining whether the ASCII file has five columns or eight. Note that the last column in both cases just consists of a series of dots. Whilst I typically just throw this away, it may be useful in determining the number of eyes in the recording (since monocular recordings end each row with ... and binocular with .....)
Any ideas as to how I might work out how many columns are in each ASCII file are welcome!
You can read the first data line, check the number of columns and then revert the file position indicator For example:
pos = ftell(fid);
cols = numel(regexp(fgetl(fid), '\s*([^\s]*)\s*'));
fseek(fid, pos, 'bof');
This can be followed by:
if (cols == 5)
eyeData = textscan(fid, '%d %f %f %f %s');
else
eyeData = textscan(fid, '%d %f %f %f %f %f %f %s');
end
By the way, note that you can tell textscan to discard the dots by using %*s instead of the last %s in the pattern string.
You can count the columns in a file with a shell command, which you can call from MATLAB using
s = system(shell_command);
To produce a 'shell_command' that fits your needs check out the following link
unix - count of columns in file