FSCTL_MOVE_FILE on Windows XP, System volume, FAT32 - windows-xp

Question

I am having a problem with defragmenting files on windows xp, fat32 system volume. I am not writing a defragmenter but instead a part of the solution requires a certain set of files to be laid out continuously on the disk. To ensure this i am using FSCTL_MOVE_FILE ioctl to move file extents into a single free space extent of sufficient size on the volume. The process goes as follows:
1) Create a file:
return m_file.Create(path,
GENERIC_READ | GENERIC_WRITE,
0, NULL,
CREATE_ALWAYS,
FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH |
FILE_ATTRIBUTE_SYSTEM | FILE_ATTRIBUTE_HIDDEN,
NULL);
2) Fill file with zeroes.
3) Check that file is fragmented, if it is, acquire volume bitmap with FSCTL_GET_VOLUME_BITMAP, find free cluster chain of sufficient size.
4) Use FSCTL_MOVE_FILE to defragment the file into found extent as such:
MOVE_FILE_DATA input;
input.FileHandle = fileHandle;
input.StartingVcn.QuadPart = 0;
input.StartingLcn.QuadPart = freeExtent.lcn;
input.ClusterCount = totalFileClusters;
DWORD bytesReturned = 0; // unused
::DeviceIoControl(
volumeHandle,
FSCTL_MOVE_FILE,
&input,
sizeof(input),
NULL,
0,
&bytesReturned,
NULL);
That last call works fine on NTFS system and regular volumes. Non-system volumes on XP also present no problem. However on FAT32 system volumes on XP i almost always get INVALID_ARGUMENT (87) error. Files are quite large, about 700MB. The volume has about 10GB free space. After failed fsctl it can be seen that part of the file was actually moved before the error occured. I tried several attempts, but so far all 50 of them failed. I am aware that moving a large file this way can fail due to a previously free cluster further down the road becoming occupied by something else on the volume, especially if the volume has a lot of activity (like the system volumes usually have). But i have no idea how to mitigate this given i have no kernel presence. What i am doing wrong and/or how can i do better?

Answer 1

The short answer here is no, you cannot get there from user mode. You'll always be racing with the rest of the applications/operating system processes that are manipulating the filesystem.
If you really want to go down this path, you're going to need to write a fairly complex kernel driver (File System Minifilter) to assist with synchronizing this operation. This can get tricky, particularly for moves on volumes that host page/swap files.
Good luck!

Answer 2

Two problems. First, on a live system you are always racing against other activity (as the previous answerer suggested). As an example, once you have retrieved the volume bitmap, it may already be out of date due to operations in other processes that you don't see.
Second, there is probably a limitation in FAT32 for how much you can move at a time. Perhaps you should consider moving files in chunks of 256Kb (aka the cache manager mapping view size).
If you encounter an error, such as hitting space the was formerly free, you have much less work to reason about.
Finally, if you have a 700Mb file, if it exists in 700 1Mb chunks, I doubt that you'd see any significant performance improvements over a single 700Mb chunk.