Output from large .csv files generated by Matlab - matlab

Question

I am downloading some data in static and time series format using Matlab codes (and toolboxes). The codes give out the results by default in a .csv file for static and time series separately. Although the static data turns out fine, the time series data is huge and the .csv file doesn't load all the data. I tried changing the output file extension to .dta and then also .mat in order to view the output in Stata or Matlab. Also tried writing a little loop where the data loaded into large .csv file can be split into two worksheets within the same file. But none of it has worked. Although I am used to some basic coding in Matlab, I am new to dealing with such large datasets. Any help on this would be very much appreciated.
Thank you- Veronica

Answer 1

How about using these two bits of VBScript to split your CSV file into 2 pieces. There will be 599,999 lines in the first output file (part1.csv) and the rest in the second file (part2.csv).
Save this as part1.vbs
Set fso = CreateObject ("Scripting.FileSystemObject")
Set stdout = fso.GetStandardStream (1)
LineNum=1
Do While Not WScript.StdIn.AtEndOfStream
REM Read in next line of input
Line = WScript.StdIn.ReadLine()
If LineNum<600000 Then
stdout.WriteLine(Line)
End If
LineNum=LineNum+1
Loop
Save this as part2.vbs
Set fso = CreateObject ("Scripting.FileSystemObject")
Set stdout = fso.GetStandardStream (1)
LineNum=1
Do While Not WScript.StdIn.AtEndOfStream
REM Read in next line of input
Line = WScript.StdIn.ReadLine()
If LineNum>=600000 Then
stdout.WriteLine(Line)
End If
LineNum=LineNum+1
Loop
Then you can do this at the Command Prompt to split your file in two:
cscript /nologo part1.vbs YourFile.CSV > part1.csv
cscript /nologo part2.vbs YourFile.CSV > part2.csv