I am trying to read a csv file use textscan. The fields are seperated with ',' . I used the following code, but it only read in one line of data into the matrix W.
I also tried dlmread(), it got the number of fields wrong.
The file is contructed under linux, matlab is under linux.
file_id = fopen('H:\data\overlapmatrices\cos.mat.10');
W = textscan(file_id, '%f', 'delimiter', ',' , 'EndOfLine', '\r\n');
fclose(file_id);
clear file_id;
you might wanna try csvread, it should do the trick.
or you could alway do something dirty like
fid = fopen( filename );
tline = fgetl(fid);
while ischar(tline) %or some other check
%sscanf(tline...
tline = fgetl(fid);
end
The problem could be in how the end of line is represented in the file (see also this article on Wikipedia). While \r\n (the combination of a carriage return and a newline character) is common on Windows, \n (just the newline character) is the standard on Linux and other Unix systems.
But as ben is saying, csvread might be an easier way how to read the file.
Related
Neither of the following line write "a\b" into file
fid = fopen('myfile.txt','w'); fprintf(fid, 'a\b'); fclose(fid);
fid = fopen('myfile.txt','wb'); fprintf(fid, 'a\b'); fclose(fid);
Probably, Matlab does backslash escaping during save to file.
How to disable this "feature"?
The string should remain intact, i.e. fprintf(fid, 'a\\b') is not a solution, length('a\b')==3 should be true.
You could use conversion characters in fprintf, i.e. %s in this case
fid = fopen('myfile.txt','w'); fprintf(fid, '%s', 'a\b'); fclose(fid);
Your condition length('a\b')==3 isn't violated like this, if I am not mistaken
Another alternative is to use fwrite:
fwrite(fid,'a\b','uchar');
If you use 'char' mode, the string will be encoded differently depending on how the file was opened (e.g. UTF encoding).
I have a huge .txt file and parts of which I want to parse (using text scan), say I have 10000 line data and a part which starts at line 300, the part also has a header of 10 lines say,how can I skip the first 300 lines(not using header function of text scan of course as I then wont be able to get my actual 10 line header) or is there a way in which I can jump to line 300 and start text scan from there as if 301 line was the first line.
So, assuming your data is generated by the following (since you have not mentioned how it's formatted in your question):
fid = fopen('datafile.txt', 'w');
for i=1:300
fprintf(fid, 'ignore%d\n', i);
end
for i=301:310
fprintf(fid, 'header%d\n', i);
end
for i=311:10000
fprintf(fid, '%d\n', i);
end
fclose(fid);
You can read the data by first using fgetl to advance to the 300th line, then using textscan twice to get the header info and the data. Basically, the thing to remember is that textscan works starting from the place where fid is pointing. So, if it's pointing to the 301st line, it'll start scanning from there. So, here's the code to read the file above, starting from line 301:
fid = fopen('datafile.txt', 'r');
for i=1:300
fgetl(fid);
end
scannedHeader = textscan(fid, '%s', 10);
scannedData = textscan(fid, '%d');
fclose(fid);
NB: if the data is always the same format, you can use ftell to know where to skip to exactly then use fseek to go to that offset.
I want to load a csv file in a matrix using matlab.
I used the following code:
formatSpec = ['%*f', repmat('%f',1,20)];
fid = fopen(filename);
X = textscan(fid, formatSpec, 'Delimiter', ',', 'CollectOutput', 1);
fclose(fid);
X = X{1};
The csv file has 1000 rows and 21 columns.
However, the matrix X generated has 2000 columns and 20 columns.
I tried using different delimiters like '\t' or '\n', but it doesn't change.
When I displayed X, I noticed that it displayed the correct csv file but with extra rows of zeros every 2 rows.
I also tried adding the 'HeaderLines' parameters:
`X = textscan(fid, formatSpec1, 'Delimiter', '\n', 'CollectOutput', 1, 'HeaderLines', 1);`
but this time, the result is an empty matrix.
Am I missing something?
EDIT: #horchler
I could read with no problem the 'test.csv' file.
There is no extra comma at the end of each row. I generated my csv file with a python script: I read the rows of another csv file, modified these (selecting some of them and doing arithmetic operations on them) and wrote the new rows on another csv file. In order to do this, I converted each element of the first csv file into floats...
New Edit:
Reading the textscan documentation more carefully, I think the problem is that my input file is neither a textfile nor a str, but a file containing floats
EDIT: three lines from the file
0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,0,0,2
1,-0.3834323,-1.92452324171,-1.2453254094,0.43455627857,-0.24571121,0.4340657,1,1,0,0,0,0.3517396202,1,0,0,0.3558122164,0.2936975319,0.4105696144,0,1,0
-0.78676,-1.09767,0.765554578,0.76579043,0.76,1,0,0,323124.235998,1,0,0,0,1,0,0,1,0,0,0,2
How about using regex ?
X=[];
fid = fopen(filename);
while 1
fl = fgetl(fid);
if ~ischar(fl), break, end
r =regexp(fl,'([-]*\d+[.]*\d*)','match');
r=r(1:21); % because your line 2nd is somehow having 22 elements,
% all lines must have same # elements or an error will be thrown
% Error: CAT arguments dimensions are not consistent.
X=[X;r];
end
fclose(fid);
Using csvread to read a csv file seems a good option. However, I also tend to read csv files with textscan as files are sometimes badly written. Having more options to read them is therefore necessary.
I face a reading problem like yours when I think the file is written a certain way but it is actually written another way. To debug it I use fgetl and print, for each line read, both the output of fgetl and its double version (see the example below). Examining the double version, you may find which character causes a problem.
In your case, I would first look at multiple occurrences of delimiters (',' and '\t') and , in 'textscan', I would activate the option 'MultipleDelimsAsOne' (while turning off 'CollectOutput').
fid = fopen(filename);
tline = fgetl(fid);
while ischar(tline)
disp(tline);
double(tline)
pause;
tline = fgetl(fid);
end
fclose(fid);
I have an array which looks like this:
cloud =
7.5059 51.4406
7.5057 51.4445
7.5048 51.4484
7.5034 51.4522
7.5014 51.4558
7.4989 51.4593
7.4958 51.4627
7.4923 51.4658
7.4884 51.4686
.
.
all i want is to write this array to a text file as it is, in the same format. I tried both fprintf and dlmwritebut i'm able to produce the exact same format. I know its an easy one, but I'm only asking after trying a lot.
Have you looked into string formatting?
fid = fopen( 'myFile.txt', 'w' );
for ii=1:size(cloud,1)
fprintf( fid, '%.5g\t%.5g\r\n', cloud(ii,1), cloud(ii,2) );
end
fclose( fid ); % do not forget to close the file :-)
Have you considered saveing into ascii file?
save( 'myFile.txt', 'cloud', '-ascii', '-tabs' );
EDIT:
End-of-line issue: for text file there are several way of marking the end of line: On windows it is usually required to print \r\n, for Mac and Linux sometimes it is enough to use \r and sometimes \n (I'm not 100% sure). So, you might need to experiment with it a bit to find out what works best for your machine. (Thanks #Rody for correcting me here)
Accuracy: the number in the formatting string %.5g determines the accuracy of the printed number. Again, you can play with it till you are satisfied with the results.
WINDOWS
Here's one way:
fid = fopen('cloud.txt', 'w');
fprintf(fid, '%.4f\t%.4f\r\n', cloud.');
fclose(fid)
Here's the more readable way:
dlmwrite('cloud.txt', cloud, 'delimiter', '\t', 'precision', 4, 'newline', 'pc')
LINUX
Here's one way:
fid = fopen('cloud.txt', 'w');
fprintf(fid, '%.4f\t%.4f\n', cloud.');
fclose(fid)
Here's the more readable way:
dlmwrite('cloud.txt', cloud, 'delimiter', '\t', 'precision', 4);
I have a big text file containing data that needs to be extracted and inserted into a new text file. I possibly need to store this data in an cell/matrix array ?
But for now, the question is that I am trying to test a smaller dataset, to check if the code below works.
I have a code in which it opens a text file, scans through it and replicates the data and saves it in another text file called, "output.txt".
Problem : It doesn't seem to save the file properly. It just shows an empty array in the text file, such as this " [] ". The original text file just contains string of characters.
%opens the text file and checks it line by line.
fid1 = fopen('sample.txt');
tline = fgetl(fid1);
while ischar(tline)
disp(tline);
tline = fgetl(fid1);
end
fclose(fid1);
% save the sample.txt file to a new text fie
fid = fopen('output.txt', 'w');
fprintf(fid, '%s %s\n', fid1);
fclose(fid);
% view the contents of the file
type exp.txt
Where do i go from here ?
It's not a good practice to read an input file by loading all of its contents to memory at once. This way the file size you're able to read is limited by the amount of memory on the machine (or by the amount of memory the OS is willing to allocate to a single process).
Instead, use fopen and its related function in order to read the file line-by-line or char-by- char.
For example,
fid1 = fopen('sample.txt', 'r');
fid = fopen('output.txt', 'w');
tline = fgetl(fid1);
while ischar(tline)
fprintf(fid, '%s\n', tline);
tline = fgetl(fid1);
end
fclose(fid1);
fclose(fid);
type output.txt
Of course, if you know in advance that the input file is never going to be large, you can read it all at once using by textread or some equivalent function.
Try using textread, it reads data from a text file and stores it as a matrix or a Cell array. At the end of the day, I assume you would want the data to be stored in a variable to manipulate it as required. Once you are done manipulating, open a file using fopen and use fprintf to write data in the format you want.