MATLAB : read data file in real time - matlab

Question

I have an application, A, that writes to a File.
I want to use MATLAB to read N lines in realtime from this file.
My question is related to this stack post: How to plot real time data from text file in MATLAB
The author of one of the answers, mentions the following approach:
You can't plot using hard real time conditions, thus it can always happen that matlab misses a 10ms timeslot. You have to use option 2 to get all data.
To get started: Write a function which only reads the new data which was written since last call. To achieve this, do not close the file handle. It stores the position.
As such, here is my code:
myfile_fid=fopen(filePath, 'rt')
waitForFileToHaveData(filePath, 10);
for readingIdx = 1:10
fgetl(myfile_fid)
end
My waitForFileToHaveData function, is defined as follows:
function waitForFileToHaveData(filePath, desired_length)
if (getNumLinesOfFile(filePath) < desired_length)
disp('###### not enough data in the file');
pause(0.02);
waitForFileToHaveData(filePath, desired_length);
end
end
function num = getNumLinesOfFile(file_to_read)
[status, result] = system( ['wc -l ', file_to_read] );
if(status~=1)
scanCell = textscan(result,'%u %s');
num = scanCell{1} - 2;
else
num = 0;
end
end
Result:
When I get into the for loop, myfile_fid evaluates to 3, while fgetl(myfile_fid) evaluates to -1. If I print out the results of getNumLinesOfFile(filePath), I see 20. The odd part is that if I wait, say for the file to have 40 lines, and execute the code above, I do not get the error. I tried to look at the documentation to see why fgetl returns back -1, but I cannot seem to find it in 2018b MATLAB documentation. There is mention that the myfile_fid can return a -1, but that is only if the file cannot be opened. However, at runtime, myfile_id evaluates to 3.
Using MATLAB, is it possible to read N number of lines since last read in a file that is being written to by another application?

Answer 1

fgetl returns -1 when fileID reaches the end-of-file marker, See Matlab fgetl documentation. This means that if the first result from fgetl is -1 then the file is empty.
I'm not sure why you are getting -1 if getNumLinesOfFile returns 20, check the code carefully if you are reading the same file. Maybe the file has changed?
I wrote here MATLAB code that checks if 10 new lines were added and and then gets them with fgetl:
myfile_fid = fopen(filePath, 'rt');
newLines = 10;
linesRead = 0;
while(waitForFileToHaveData(filePath, linesRead + newLines))
linesRead = linesRead + newLines;
for readingIdx = 1:newLines
line = fgetl(myfile_fid)
end
end
fclose(myfile_fid);
I updated the waitForFileToHaveData function to return 1:
function ready = waitForFileToHaveData(filePath, desired_length)
while (getNumLinesOfFile(filePath) < desired_length)
disp('###### not enough data in the file');
pause(0.02);
end
ready = 1;
end
Note:
If the file had exactly 10 lines with no end-line marker at line 10, and you read them, then another 10 lines were added, fileID now points to the end-line of line 10 and the first line fgetl will return is the end-line at line 10, but since fgetl removes the end-line, it returns an empty array.
Side note:
the function waitForFileToHaveData uses recursion which is inefficient. You can easily use a while loop.