Encoding problem with my joomla site Topic is solved

[adrian_vg]

Hi all,

Just recently Google Chrome seems to have started showing Swedish national characters with yen-signs and whatnot instead of the correct Å, Ä and Ö.

My research shows that this is probably due to to that Joomla is set to use UTF-8 (the default I believe) while Chrome is set to use Latin1 (I think it's also called ISO-8859-1).

Now, further research suggests that UTF-8 can be set or changed in php.ini, httpd.conf but not in Chrome, as recent versions of Chrome reportedly does the change automatically and the user can't change that.
FWIW, when tested with Firefox, same thing happens there - weird characters when writing in Swedish are seen in the front-end.

An example can be found on https://www.srbu.se/motorcycles/158-ser ... lning-nbsp.

AFAICT, my Joomla runs the language defaults with english and swedish languages.
Httpd.conf is set to use UTF-8, as php.ini is.
Mysql seems to be using both latin1 and UTF-8 for some reason. Is this the default setting, and could this matter?

Code: Select all

Database Collation				latin1_swedish_ci
Database Connection Collation	utf8_general_ci

I've no idea where to start fixing this, there are so many variables...
Suggestions and hints are greatly appreciated!
Thanks.

[infograf768]

Column collation should be utf8mb4_unicode_ci as well as database connection collation.
You can use this tool
https://github.com/PhocaCz/PhocaDownloa ... ation2.zip

make sure you use the correct database

Screen Shot 2018-07-16 at 16.40.27.png

[Per Yngve Berg]

Do you intend to have a Multi-language site?

The Swedish text with the error appears on English pages. The tags says 'en-gb' and Joomla is in English.
You have to read up on how to configure Multi-language in Joomla with the language Switcher module.

[infograf768]

@Per Yngve Berg

I do not think the user intends to make a multingual site.
I believe his database (and specific server settings) are wrong and as his site comes from a dump (see his former posts in this forum), it may be that the wrong encoding is already in the db.
Therefore he would have to first correct his server and db settings and then re-save everywhere the wrong encodings are displayed (in some articles apparently).

[adrian_vg]

@Per Yngve Berg

I do not think the user intends to make a multingual site.
I believe his database (and specific server settings) are wrong and as his site comes from a dump (see his former posts in this forum), it may be that the wrong encoding is already in the db.
Therefore he would have to first correct his server and db settings and then re-save everywhere the wrong encodings are displayed (in some articles apparently).

You're right and may be on to something.
I'll check out the db-collation and see if that helps.

Thanks for the starting point!

Edit: Some minor parts of the site is in Swedish, but the majority is in english. That won't change significantly in the future!

[adrian_vg]

Hmm, the Phoca tool gives me an error when I try the conversion.
I did however see that Akeeba Admin tools also offer a collation change-tool. I used the custom setting with the string "utf8mb4_unicode_ci". Nothing changed as far as I can tell.

Can this be done on the command line?
For reference, what would I type then?
I have some basic knowledge with CLI mysql, but changing collations I've never done before.

The mysql used is v5.1.73, the latest available for CentOS 6.
I've a hunch this might be a problem maybe.

Would this guide be usable for my situation?
https://mathiasbynens.be/notes/mysql-utf8mb4

[infograf768]

Just look at your db to see if the collation has changed. Post a screenshot.
It should look like this.

Screen Shot 2018-07-19 at 15.36.59.png

If it does, then edit again the articles concerned and re-save.

[adrian_vg]

Just look at your db to see if the collation has changed. Post a screenshot.
It should look like this.
Screen Shot 2018-07-19 at 15.36.59.png
If it does, then edit again the articles concerned and re-save.

That's a nice overview!

How did you do that?

[infograf768]

PhpMyadmin

[adrian_vg]

Ah. I can't install that on my server. CentOS 6 is to old...

I have webmin available though. Would that work?
Can't seem to find any similar view like yours.

[adrian_vg]

Got phpMyAdmin to work after upgrading mysql to v5.5 and some other *-fu.
Now to check the collation tables.
Will report back soonish.

[adrian_vg]

Okay, it seems I have a mix of utf8_unicode_ci, utf8_general_ci and latin1_swedish_ci if I understand this correctly. Please see attached screenshoot.

Screenshot from 2018-07-20 12-44-16.png

Is there a way I can do the conversion from within phpMyAdmin?
The Phoca tool didn't seem to work for me.

[adrian_vg]

Wait a sec...
Dammit'!! Seems like I got hit by a changing char set on CLI and GUI respectively.

Edit: Wazza! All tables etc set to utf8mb4_unicode_ci!
Now to check the swedish characters in Joomla.

[adrian_vg]

Well, Joomla System Information now tells me this:

Code: Select all

Database Collation	utf8mb4_unicode_ci
Database Connection Collation	utf8mb4_unicode_ci

But I still see the yen-signs.
I'm starting to feel I should just translate the swedish pages to english and be done with it.

[adrian_vg]

I also notice that I have several instances of joomla tables in the one joomla database. Undoubtedly from the previous server moves, reinstalls etc.

Maybe I should remove the unused ones and see if it gets better...

[Webdongle]

Alternatively

• Install Akeeba and make a backup
• Create a new database, database user and password
• Create a subdomain (use a folder level with the main site not under it if possible)
• Use the backup to install your site in the subdomain
• Compare the collation of both database
• Work on the subdomain version and back it up when you have fixed your problem

Note
Working on a copy in the subdomain will prevent you breaking the live site.
Some shared Hosting create the subdomain folder under the main site and that can cause confusion if you are not aware of it's location.

[adrian_vg]

Interesting idea. I'll try that.

Am a bit unsure about the third paragraph, how do you mean subdomain not under the main site?
English is not my native language, sorry.

This site is not hosted though, it's my own private server at home.

[Webdongle]

Create the folder for the subdomain where you see the folder for the domain

path/to/domain-root (contains configuration.php robots.txt /administrator etc for domain)
path/to/subdomain-root (contains configuration.php robots.txt /administrator etc for subdomain)

= the folder for you site is level with with the one for your subdomain. In other words entering the domain folder you don't see the subdomain folder

folder structure 01.JPG

Not enter the domain folder (where you configuration.php robots.txt /administrator )

path/to/domainroot/subdomain-root (contains configuration.php robots.txt /administrator etc for domain)
= you have to enter the domain root folder to see the folder for the subdomain.

folder structure 02.JPG

[adrian_vg]

I'm sorry, the extremely hot weather currently in Sweden makes me thick...

However, using phpMyAdmin and Akeeba Admin Tools, I've managed to now set this:

Code: Select all

Database Collation	utf8mb4_unicode_ci
Database Connection Collation	utf8mb4_unicode_ci

Creating a new test article with swedish characters still gives me those weird characters.

Edit: A clarification; The previous change to the charset, a bit up, broke my database when using the Phoca Collation Tool and by extension my Joomla.
The site is restored and now back again and with the unicode setting in this post.

[adrian_vg]

Made it work.
I changed the encodings back to utf8, the default Joomla setting and then using eg "ö" for "ö" in the code.

As previously stated the Swedish pages are few.
The solution is not very clean, but works.

[Webdongle]

Have a look at the charmap for JCE editor easier than searching then typing it out

jce charmap 01.JPG

[infograf768]

using eg "ö" for "ö" in the code.

Definitely, this means that there is something wrong in your server. UTF8 characters do not need to be changed to htmlentities for Joomla to work fine.

[adrian_vg]

Have a look at the charmap for JCE editor easier than searching then typing it outjce charmap 01.JPG

Thanks!
That might be a time saver for now.
The search and replace I did yesterday wasn't very time consuming, just tedious.

[adrian_vg]

using eg "ö" for "ö" in the code.

Definitely, this means that there is something wrong in your server. UTF8 characters do not need to be changed to htmlentities for Joomla to work fine.

I agree. Problem is just what, as this happened quite suddenly a little while ago.

Anyway, now that the site looks at least superficially good for my slightly OCD-ish taste, I'll start troubleshooting with a copy instead of this live server.

Thanks all for hints and suggestions so far!