Swift : Streaming / Writing out a CSV file - swift

Question

I am using my phone to record some sensor data and store it on device via SQLite via SharkORM(DBAccess).
I now want to write that data out to a CSV file however, I am now up to 1.6 million records.
Currently, I am looping through 1000 records, adding them to a string and at the end writing them out. However, there must be a better way to do it?
func writeRawFile()
{
let fileName = "raw"
let DocumentDirURL = try! FileManager.default.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: true)
let fileURL = DocumentDirURL.appendingPathComponent(fileName).appendingPathExtension("csv")
var data = "time,lat,lon,speed,x_acc,y_acc,z_acc,gyro_x,gyro_y,gyro_z,compass,orientation\r\n"
let count = RawReading.query().count()
var counter = 0;
let df = DateFormatter()
df.dateFormat = "y-MM-dd H:m:ss.SSSS"
for i in stride(from:0, to: count, by: 1000)
{
autoreleasepool {
for result in RawReading.query().offset(Int32(i)).limit(1000).fetch()
{
if let raw : RawReading = result as? RawReading
{
if (Double(raw.speed!) > 3.0) //1 Meter per Second = 2.236936 Miles per Hour
{
//print(df.string(from: raw.date!))
data += "\(df.string(from: raw.date!)),\(raw.lat!),\(raw.lon!),\(raw.speed!),\(raw.x!),\(raw.y!),\(raw.z!),\(raw.xx!),\(raw.yy!),\(raw.zz!),\(raw.compass!),\(raw.orientation!)" + "\r\n"
counter += 1
}
}
}
print ("Written \(i) of \(count)")
}
}
print("Count \(count) \(counter)")
//write the file, return true if it works, false otherwise.
do{
try data.write(to: fileURL, atomically: true, encoding: String.Encoding.utf8 )
} catch{
print("error")
}
}

Answer 1

Open a FileHandle for writing, then build and write each line separately, so that you don't have to keep the entire file contents
in memory:
do {
let file = try FileHandle(forWritingTo: fileURL)
defer { file.closeFile() }
for <... your loop ...> {
let line = ... // build one CSV line
file.write(line.data(using: .utf8)!)
}
} catch let error {
// ...
}
You can also write to a temporary file first and then rename it to the
actual file, in order to avoid a damaged file if anything
went wrong.