FileAppend data loss in a loop - autohotkey

Question

In a loop, I use FileAppend to add text to a file. In every iteration of the loop, the data is accumulated in a variable and FileAppend is called every n iterations of the loop to save the data. Very occasionally, after a FileAppend call, the data accumulated in the variable during the next iteration is lost. As it is very intermittent, I could not reproduce this behavior. It seems like if, in some situation, the script would need a delay after FileAppend. Is this a known issue? I've search AHK forums and this site without report of such issue.
Here is a piece of code where this happens:
Loop, %intMax% ; for each record in the collection
{
if !Mod(A_Index, intProgressIterations)
; update the progress bar and save the data every intProgressIterations
; (when intProgressIterations / A_Index = 0)
{
ProgressUpdate(A_index, intMax, strProgressText)
; update progress bar only every %intProgressIterations% iterations
FileAppend, %strData%, %strFilePath%
strData := ""
; save the data accumulated in strData and empty it
}
strData := strData . BuildData(objCollection[A_Index])
; build the data for this record and add it to strData
}
More precisely, it is the content of one (or more) iteration of the line strData := strData . BuildData(objCollection[A_Index]) that is lost.

Answer 1

Could be any number of things. The file could be locked, there could be an error in your BuildData function that causes it not to produce data.
I'd recommend checking the last modified date before and after you append the data.
If it's the same, you can either try again and/or notify the user.
As to your question about the delay, it shouldn't need it, the script does not continue to the next line of code until it finishes writing to the file.

Answer 2

Following findings about file locking, this code with error management will be safer:
Loop, %intMax% ; for each record in the collection
{
if !Mod(A_Index, intProgressIterations)
; update the progress bar and save the data every intProgressIterations
; (when intProgressIterations / A_Index = 0)
{
ProgressUpdate(A_index, intMax, strProgressText)
; update progress bar only every %intProgressIterations% iterations
Loop
{
FileAppend, %strData%, %strFilePath%
; save the data accumulated in strData and empty it after the loop
if ErrorLevel
Sleep, 20
}
until !ErrorLevel or (A_Index > 50) ; after 1 second (20ms x 50), we have a problem
if (ErrorLevel)
strError := strError . "Error writing line " . A_Index . " Error: #" . A_LastError . "`n"
strData := ""
}
strData := strData . BuildData(objCollection[A_Index])
; build the data for this record and add it to strData
}

Answer 3

As an added point to this issue (for future searches)... I found that trying to FileAppend within a loop when the output file is on Dropbox, will also cause data loss.
Dropbox will detect updates to the file during the early cycles of the loop, and then lock it while it sends the updated (updating) file to the Dropbox servers. I've tracked data loss in a text file as short as 150 lines of text.
The solution I used was to create a temp output file outside of the Dropbox-monitored main folder, and write to it instead. When the loop is complete, copy the temp file back to Dropbox, and then delete the temp file.
Here's an partial code example of how I did the switch with a completed temp file, after the loop had finished:
; Copy the Daily Scrub temp file to var holding the filepath to todo.txt, then delete the Daily Scrub temp file
FileCopy, C:\temp-dailyscrub.txt, %todoFilePathParse%, 1
FileDelete, C:\temp-dailyscrub.txt