Why does readmatrix in Matlab skip the first n lines? - matlab

Question

In my simulation I am writing data to file using writematrix, then later reading it back using readmatrix. I am appending to a single file at each time step, each line is the same length or longer than the previous line.
For some reason when using readmatrix on the output file, the first n lines are skipped entirely, as in not read at all. For example, my file looks like this:
...
11.8,1,2,3,4,5,6,7,8,9,10,2
11.9,1,2,3,4,5,6,7,8,9,10,2
...
12.3,1,2,3,4,5,6,7,8,9,10,2
12.4,7,8,9,10,7,8,9,10,1,2,1,1,2,3,4,5,6,3,4,5,6,1
12.5,7,8,9,10,7,8,9,10,1,2,1,1,2,3,4,5,6,3,4,5,6,1
...
30.5,7,8,9,10,7,8,9,10,1,2,2,1,2,3,4,5,6,3,4,5,6,2
30.6,7,8,9,10,7,8,9,10,1,2,2,1,2,3,4,5,6,3,4,5,6,2
30.7,17,18,19,20,1,2,7,8,9,10,1,1,2,3,4,5,6,3,4,5,6,2,11,12,13,14,15,16,7,8,9,10,1
30.8,17,18,19,20,1,2,7,8,9,10,1,1,2,3,4,5,6,3,4,5,6,2,11,12,13,14,15,16,7,8,9,10,1
...
(the first column is a time stamp, so the first ellipsis represents t=0 to t=11.7. At t=30.7 there is another step jump in the number of entries), and when I read using the command
data = readmatrix('/path/to/file/data.csv');
the matrix data looks like
12.4 7 8 9 10 7 8 9 10 1 2 1 1 2 3 4 5 6 3 4 5 6 1
12.5 7 8 9 10 7 8 9 10 1 2 1 1 2 3 4 5 6 3 4 5 6 1
12.6 7 8 9 10 7 8 9 10 1 2 1 1 2 3 4 5 6 3 4 5 6 1
...
30.5 7 8 9 10 7 8 9 10 1 2 2 1 2 3 4 5 6 3 4 5 6 2
30.6 7 8 9 10 7 8 9 10 1 2 2 1 2 3 4 5 6 3 4 5 6 2
30.7 17 18 19 20 1 2 7 8 9 10 1 1 2 3 4 5 6 3 4 5 6 2 11 12 13 14 15 16 7 8 9 10 1
30.8 17 18 19 20 1 2 7 8 9 10 1 1 2 3 4 5 6 3 4 5 6 2 11 12 13 14 15 16 7 8 9 10 1
...
That is to say, all the entries before t=12.4 (i.e. the first step jump in line length) are skipped.
In the file, if I delete everything before the first step jump (i.e everything before t=12.4), then I get the same matrix data, so we can conclude the subsequent step jumps cause no issue. If I delete everything from the second step jump (i.e. everything after t=30.6) then it still skips all the entries before t=12.4. If I have no step jumps (i.e. only t=0 to t=12.3) then it happily reads in the first lines.
I've tried reading the same file using csvread and it returns all of the data from the beginning of the file (albeit padded with zeros instead of nans), so I'm confident the issue isn't with the file.
Why is this happening?
A minimum working example is the first code block without the ellipses.
For reference, the first lines have 12 csvs, and each step jump increase that by 11
Edit:
Output from detectImportOptions
ans =
DelimitedTextImportOptions with properties:
Format Properties:
Delimiter: {','}
Whitespace: '\b\t '
LineEnding: {'\n' '\r' '\r\n'}
CommentStyle: {}
ConsecutiveDelimitersRule: 'split'
LeadingDelimitersRule: 'keep'
EmptyLineRule: 'skip'
Encoding: 'UTF-8'
Replacement Properties:
MissingRule: 'fill'
ImportErrorRule: 'fill'
ExtraColumnsRule: 'addvars'
Variable Import Properties: Set types by name using setvartype
VariableNames: {'Var1', 'Var2', 'Var3' ... and 20 more}
VariableTypes: {'double', 'double', 'double' ... and 20 more}
SelectedVariableNames: {'Var1', 'Var2', 'Var3' ... and 20 more}
VariableOptions: Show all 23 VariableOptions
Access VariableOptions sub-properties using setvaropts/getvaropts
PreserveVariableNames: false
Location Properties:
DataLines: [4 Inf]
VariableNamesLine: 0
RowNamesColumn: 0
VariableUnitsLine: 0
VariableDescriptionsLine: 0
To display a preview of the table, use preview

Answer 1

Matlab's readmatrix is trying to be smart and locate a 2-D matrix within the data model of the CSV file you're passing it. It looks like it's passing over the first few lines which don't have explicit trailing empty "cells".
You can control this by setting the import options. Run opts = detectImportOptions(...); on your file and have a look at the DataLines property. If it doesn't start at 1, set it to [1 Inf] to force readmatrix to read in all the lines. And then call readmatrix, explicitly passing in that options structure.
To do this compactly (and probably more efficiently), call readmatrix with an explicit option right off the bat like this:
readmatrix(path2mat,delimitedTextImportOptions('DataLines',[0,Inf]))