Using GetPrivateProfileStringW with UTF-8 encoded files? - unicode

Question

Up until now my INI text files have been Unicode encoded and I have been doing things like this to read it:
::GetPrivateProfileStringW(
strUpdateSection,
strTalkKey,
_T(""),
strTalk.GetBuffer( _MAX_PATH ),
_MAX_PATH,
m_strPathINI );
strTalk.ReleaseBuffer();
The file gets downloaded from the internet first and then accessed. But I was finding that for Arabic the text file was getting corrupted. Unless I change the encoding to UTF-8. When I do that, it downloads right. But then it doesn't read the data right from the INI.
So does a INI file have to be Unicode encoded or should it also be OK to use UTF-8 with the function calls?
I think it is about time I convert this part of my program to UTF-8 encoded XML files instead! Way to go.
But wanted to ask the question first.
I should clarify that the text file was initial saved as UTF-8 using NotePad.
Update
This is what it looks like when I read the file:
This is how I download the file:
BOOL CUpdatePublicTalksDlg::DownloadINI()
{
CInternetSession iSession;
CHttpFile *pWebFile = NULL;
CWaitCursor wait;
CFile fileLocal;
TCHAR szError[_MAX_PATH];
int iRead = 0, iBytesRead = 0;
char szBuffer[4096];
BOOL bOK = FALSE;
DWORD dwStatusCode;
CString strError;
// ask user to go online
if( InternetGoOnline( (LPTSTR)(LPCTSTR)m_strURLPathINI, GetSafeHwnd(), 0 ) )
{
TRY
{
// our session should already be open
// try to open up internet session to my URL
// AJT V10.4.0 Use flag INTERNET_FLAG_RELOAD
pWebFile = (CHttpFile*)iSession.OpenURL( m_strURLPathINI, 1,
INTERNET_FLAG_TRANSFER_BINARY |
INTERNET_FLAG_DONT_CACHE | INTERNET_FLAG_RELOAD);
if(pWebFile != NULL)
{
if( pWebFile->QueryInfoStatusCode( dwStatusCode ) )
{
// 20x codes mean success
if( (dwStatusCode / 100) == 2 )
{
// Our downloaded file is only temporary
m_strPathINI = theApp.GetTemporaryFilename();
if( fileLocal.Open( m_strPathINI,
CFile::modeCreate|CFile::modeWrite|CFile::typeBinary ) )
{
iRead = pWebFile->Read( szBuffer, 4096 );
while( iRead > 0 )
{
iBytesRead += iRead;
fileLocal.Write( szBuffer, iRead );
iRead = pWebFile->Read( szBuffer, 4096 );
}
fileLocal.Close();
bOK = TRUE;
}
}
else
{
// There was a problem!
strError.Format( IDS_TPL_INVALID_URL, dwStatusCode );
AfxMessageBox( strError,
MB_OK|MB_ICONERROR );
}
}
}
else
{
AfxMessageBox( ID_STR_UPDATE_CHECK_ERR, MB_OK|MB_ICONERROR );
}
}
CATCH( CException, e )
{
e->GetErrorMessage( szError, _MAX_PATH );
AfxMessageBox( szError, MB_OK|MB_ICONERROR );
}
END_CATCH
// Tidy up
if( pWebFile != NULL )
{
pWebFile->Close();
delete pWebFile;
}
iSession.Close();
}
return bOK;
}
This is how I read the file:
int CUpdatePublicTalksDlg::ReadTalkUpdateINI()
{
int iLastUpdate = 0, iUpdate;
int iNumTalks, iTalk;
NEW_TALK_S sTalk;
CString strUpdateSection, strTalkKey, strTalk;
// How many possible updates are there?
m_iNumUpdates = ::GetPrivateProfileIntW(
_T("TalkUpdates"),
_T("NumberUpdates"),
0,
m_strPathINI );
// What what the last talk update count?
iLastUpdate = theApp.GetLastTalkUpdateCount();
// Loop available updates
for( iUpdate = iLastUpdate + 1; iUpdate <= m_iNumUpdates; iUpdate++ )
{
// Build section key
strUpdateSection.Format( _T("Update%d"), iUpdate );
// How many talks are there?
iNumTalks = ::GetPrivateProfileIntW(
strUpdateSection,
_T("NumberTalks"),
0,
m_strPathINI );
// Loop talks
for( iTalk = 1; iTalk <= iNumTalks; iTalk++ )
{
// Build key
strTalkKey.Format( _T("Talk%d"), iTalk );
// Get talk information
::GetPrivateProfileStringW(
strUpdateSection,
strTalkKey,
_T(""),
strTalk.GetBuffer( _MAX_PATH ),
_MAX_PATH,
m_strPathINI );
strTalk.ReleaseBuffer();
// Decode talk information
DecodeNewTalk( strTalk, sTalk );
// Does it already exists in the database?
// AJT v11.2.0 Bug fix - we want *all* new talks to show
//if( !IsExistingTalk( sTalk.iTalkNumber ) )
//{
//if(!LocateExistingTheme(sTalk, false))
AddNewTalkToListBox( sTalk );
//}
}
}
// Return the actual amount of updates possible
return m_iNumUpdates - iLastUpdate;
}
This is the file being downloaded:
http://publictalksoftware.co.uk/TalkUpdate_Arabic2.ini
Update
It seems that the file is corrupt at the point of being downloaded:
Please see updated Watch:
Making progress, I now confirm the data OK at this point: