SDL Tridion - Published files are not UTF-8 Encoded - encoding

I've set the publication target within SDL Tridion to push files (aspx) as UTF-8.
The pages are published but i'm still seeing some weird encoding issues when rendered in a browser.
I can edit the files on the server using notepad and save them as UTF-8, which fixes the issue. So I'm a little stuck as to why this is happening, and annoyingly i've seen this before but cannot remember how it was resolved.
I have tried some other options in the publishing target for example 'System Default' but i'm still seeing the same issue.
Thanks
John

Did you try these 7 steps to solve encoding issues from Elena?

In IIS / .NET I never had problems after setting the Publication Target to UTF-8 and setting this in web.config:
<globalization fileEncoding="UTF-8" requestEncoding="UTF-8"
responseEncoding="UTF-8"/>
If you're using a Windows-Service based deployer you may want to set the JVM encoding to UTF-8, as explained here.

If this is a system you've inherited from someone else, check to make sure the code page isn't explicitly set in the templating. This would override the default set on the publication target.

Related

The greyed-out file might have an impact on the program

For a while, I have noticed that my config.js file is grey, compared to most other files in the editor. I also have an issue where, other than text/html and image/x-icon, the MIME type of static assets cannot be established. I am wondering whether the two issues are related.
Aside from setting up the server, the config file serves a base Url along with NodeJS global variables to be interpolated into placeholders in the HTML document. While I don't see any reason for there to be an associated problem, I am perplexed why the file is grey and not the same colour as the other .js files.
I have tried to find information on the problem, but I cannot describe the problem well enough to find answers to the issue browser searching. I have tried to find a VSCode forum, but there isn't one where I could make enquiry, even if I were able to articulate my problem.
If anyone has answer to why the file is greyed-out I would appreciate it.

Wrong encoding in Joomla 3

I am using joomla 3 which changes character ' into â€.
Secondly it also converting any space into Â.
I tried using
and
on myhead file but still the problem persist.
My database is with collation of utf8 too.
I am using no editor on my joomla administrator.
Also on my windows operating system it is working fine but when i push files to linux server, it shows these weird sign.
I tried google search alot but in vain.
Any help will be appreciated.
One possible reason is that the files you are pushing to the Linux server have encoding other than UTF-8, they might be Windows-1252.
Here's one suggestion to test if this might be the case.
Create a new text document in Notepad, then Save As, and under Encoding select UTF-8
Upload this test file and check the results. If this still doesn't help, you might want to double check preferences on your SFTP application, just in case it is overriding file encoding.
Good luck!
I had same problem, I deactivate Google ModPagespeed on my Cpanel and my problem has been solved.
Remove "ModPagespeed On" from .htaccess

How to setup Visual Studio Code to detect and set the correct encoding on file open

I recently started to use Visual Studio Code on Server Systems where I did not have Studio IDE installed. I like it very much but I'm running into a problem.
When I open a file (used Notepad++ before) the editor detects the encoding and sets it for me. I have many files on windows servers that are still with windows-1252 but vscode just uses UTF-8 by default.
I know I can reopen with encoding Western (Windows 1252) but I often forget it and I have sometimes destroyed some content while saving it.
So I did not find any parameter yet, is there a way to make vscode detect the encoding and set it automatically when I open a file?
To allow Visual Studio Code to automatically detect the encoding of a file, you can set "files.autoGuessEncoding":true (in the settings.json configuration file).
https://github.com/Microsoft/vscode/pull/21416
This obviously requires an updated verison of the application compared to when the question was originally asked.
Go to File-> Preferences -> User Settings
Add (or update) the entry "files.encoding": "windows1252" to the right editor window and save
Now VSCode opens all text files using windows-1252 when there is no proper encoding information set.
EDIT:
In 2017's June release the files.autoGuessEncoding setting was introduced. When enabled it will guess the file's encoding as good as possible. Its default value is false .
Add guide by image :
File >> Preferences >> Settings
Enter autoGuessEncoding and make sure checkbox is checked
beware, auto guessing in vscode still does not work as expected, the guessing, is VERY inaccurate, and does still open as guessed encoding, even when the guess library returns also the confidence score being low - they use jschardet (https://www.npmjs.com/package/jschardet)
if the score of guess is not close to 100%, it simply should rather open in "files.encoding" instead of guessed encoding, but that does not happen, vscode authors should make better use of the confidence score jschardet returns
i open either utf-8 which guesses ok, and second encoding i use is windows-1250, which in 99% cases detects wrong as some other windows-... encoding or even iso-8859-... and such... cry daily at the computer having to experience this
tuning up the confidence checking and fallback to default encoding would do, it needs someone skilled to check their source and offer them a fix
From Peminator's answer:
Beware: auto guessing in VSCode still does not work as expected, the guessing, is VERY inaccurate,
This should slightly improve with VSCode 1.67 (Apr. 2022) with (insider released):
Allow to set files.encoding as language specific setting for files on startup
we now detect changes to the editor language and there is almost always a transition from plaintext in the beginning until languages are resolved to a target language
if we detect that the new language has a configured files.encoding override
and that encoding is different from the current encoding
and the editor is not dirty or in save conflict resolution
:
we reopen the file with the configured encoding
Unfortunately I cannot tell apart this from happening vs. the user changing the language from the editor status bar.
So if the user changes language mode to bat from to ps1 and ps1 has a configured encoding, the encoding will change.

TortoiseHg commit complains about non-ASCII characters in the message

The subject tells it all. In 2015, one of the modern VCS doesn't want to accept multilingual commit messages. Though my main development language is English, I need to be able to comment on non-English stuff happening in the software. (I wonder why a program would care to use anything but UTF-8 nowadays, but it's another story.) It says:
I've found several posts on the web, discussing similar issues, but somehow none of them covers exactly this, nor do I understand how to solve the problem. I have already tried setting the system variable
,
but TortoiseHg doesn't seem to notice the difference. On the other hand, I am not sure if I set it correctly.
You set HGENCODING as a system variable in Windows. That generally requires a restart to inherit the new environment in all processes. You may get away with closing and restarting TortoiseHg from the Windows Desktop as Explorer should get a notification that the variable changed and update its own environment.
I could reproduce the issue on my system, but once the variable is set correctly TortoiseHG committed just fine. Using HGENCODING=UTF-8 is a valid spelling for that encoding.

Configure mime-types in Subclipse?

To have browsable javadocs in a project hosted at Google-code, I got from here the recipe: set the correct mime-type property (text/html) when Subversion commits HTML pages.
Does anyone knows how to configure that in Subclipse?
This related question gives the pointer to do it in Subversive (there is a Preference tab for that). I couldn't find anything similar for Subclipse.
Someone comments about doing a Team->SetProperty for the whole javadocs dir (recursively), but that is not extension specific (I'd want text/css for stylesheets, for example); and further, I guess, if when new files are created I must remember to do it again...
Is there an alternative?
Update (2001-06-11): In case somebody is interested: I couldn't find a way to accomplish this, so I ended switching to Subversive.
To configure the MIME types information, edit the following file:
Windows 7/Vista: C:\Users[YOUR_USER]\AppData\Roaming\Subversion\config
In the config file, locate the property enable-auto-props and uncomment it. It should be set to yes. Make sure there is no space at the beginning of the line:
enable-auto-props = yes
The last section will be [auto-props] and by default everything will be commented out. Add your required file type with the appropriate config.
It's possible to do part of the work in the shell, by setting mime-type via script.