Encoding problem with my joomla site Topic is solved

General questions regarding the use of languages in Joomla! 3.x.

Moderators: infograf768, General Support Moderators

Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
Post Reply
User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Encoding problem with my joomla site

Post by adrian_vg » Mon Jul 16, 2018 9:49 am

Hi all,

Just recently Google Chrome seems to have started showing Swedish national characters with yen-signs and whatnot instead of the correct Å, Ä and Ö.

My research shows that this is probably due to to that Joomla is set to use UTF-8 (the default I believe) while Chrome is set to use Latin1 (I think it's also called ISO-8859-1).

Now, further research suggests that UTF-8 can be set or changed in php.ini, httpd.conf but not in Chrome, as recent versions of Chrome reportedly does the change automatically and the user can't change that.
FWIW, when tested with Firefox, same thing happens there - weird characters when writing in Swedish are seen in the front-end.

An example can be found on https://www.srbu.se/motorcycles/158-ser ... lning-nbsp.

AFAICT, my Joomla runs the language defaults with english and swedish languages.
Httpd.conf is set to use UTF-8, as php.ini is.
Mysql seems to be using both latin1 and UTF-8 for some reason. Is this the default setting, and could this matter?

Code: Select all

Database Collation				latin1_swedish_ci
Database Connection Collation	utf8_general_ci
I've no idea where to start fixing this, there are so many variables...
Suggestions and hints are greatly appreciated!
Thanks.
--
//AvG

User avatar
infograf768
Joomla! Master
Joomla! Master
Posts: 18629
Joined: Fri Aug 12, 2005 3:47 pm
Location: **Translation Matters**

Re: Encoding problem with my joomla site

Post by infograf768 » Mon Jul 16, 2018 2:41 pm

Column collation should be utf8mb4_unicode_ci as well as database connection collation.
You can use this tool
https://github.com/PhocaCz/PhocaDownloa ... ation2.zip

make sure you use the correct database
Screen Shot 2018-07-16 at 16.40.27.png
You do not have the required permissions to view the files attached to this post.
Jean-Marie Simonet / infograf · http://www.info-graf.fr
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group

User avatar
Per Yngve Berg
Joomla! Master
Joomla! Master
Posts: 24628
Joined: Mon Oct 27, 2008 9:27 pm
Location: Akershus, Norway

Re: Encoding problem with my joomla site

Post by Per Yngve Berg » Mon Jul 16, 2018 7:33 pm

Do you intend to have a Multi-language site?

The Swedish text with the error appears on English pages. The tags says 'en-gb' and Joomla is in English.
You have to read up on how to configure Multi-language in Joomla with the language Switcher module.

User avatar
infograf768
Joomla! Master
Joomla! Master
Posts: 18629
Joined: Fri Aug 12, 2005 3:47 pm
Location: **Translation Matters**

Re: Encoding problem with my joomla site

Post by infograf768 » Tue Jul 17, 2018 10:15 am

@Per Yngve Berg

I do not think the user intends to make a multingual site.
I believe his database (and specific server settings) are wrong and as his site comes from a dump (see his former posts in this forum), it may be that the wrong encoding is already in the db.
Therefore he would have to first correct his server and db settings and then re-save everywhere the wrong encodings are displayed (in some articles apparently).
Jean-Marie Simonet / infograf · http://www.info-graf.fr
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Thu Jul 19, 2018 12:49 pm

infograf768 wrote:@Per Yngve Berg

I do not think the user intends to make a multingual site.
I believe his database (and specific server settings) are wrong and as his site comes from a dump (see his former posts in this forum), it may be that the wrong encoding is already in the db.
Therefore he would have to first correct his server and db settings and then re-save everywhere the wrong encodings are displayed (in some articles apparently).
You're right and may be on to something.
I'll check out the db-collation and see if that helps.

Thanks for the starting point!

Edit: Some minor parts of the site is in Swedish, but the majority is in english. That won't change significantly in the future!
--
//AvG

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Thu Jul 19, 2018 1:19 pm

Hmm, the Phoca tool gives me an error when I try the conversion.
I did however see that Akeeba Admin tools also offer a collation change-tool. I used the custom setting with the string "utf8mb4_unicode_ci". Nothing changed as far as I can tell.

Can this be done on the command line?
For reference, what would I type then?
I have some basic knowledge with CLI mysql, but changing collations I've never done before.

The mysql used is v5.1.73, the latest available for CentOS 6.
I've a hunch this might be a problem maybe.

Would this guide be usable for my situation?
https://mathiasbynens.be/notes/mysql-utf8mb4
--
//AvG

User avatar
infograf768
Joomla! Master
Joomla! Master
Posts: 18629
Joined: Fri Aug 12, 2005 3:47 pm
Location: **Translation Matters**

Re: Encoding problem with my joomla site

Post by infograf768 » Thu Jul 19, 2018 1:38 pm

Just look at your db to see if the collation has changed. Post a screenshot.
It should look like this.
Screen Shot 2018-07-19 at 15.36.59.png
If it does, then edit again the articles concerned and re-save.
You do not have the required permissions to view the files attached to this post.
Jean-Marie Simonet / infograf · http://www.info-graf.fr
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Thu Jul 19, 2018 2:19 pm

infograf768 wrote:Just look at your db to see if the collation has changed. Post a screenshot.
It should look like this.
Screen Shot 2018-07-19 at 15.36.59.png
If it does, then edit again the articles concerned and re-save.
That's a nice overview!

How did you do that?
--
//AvG

User avatar
infograf768
Joomla! Master
Joomla! Master
Posts: 18629
Joined: Fri Aug 12, 2005 3:47 pm
Location: **Translation Matters**

Re: Encoding problem with my joomla site

Post by infograf768 » Thu Jul 19, 2018 2:53 pm

PhpMyadmin
Jean-Marie Simonet / infograf · http://www.info-graf.fr
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Thu Jul 19, 2018 3:20 pm

Ah. I can't install that on my server. CentOS 6 is to old...

I have webmin available though. Would that work?
Can't seem to find any similar view like yours.
--
//AvG

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Fri Jul 20, 2018 10:42 am

Got phpMyAdmin to work after upgrading mysql to v5.5 and some other *-fu.
Now to check the collation tables.
Will report back soonish.
--
//AvG

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Fri Jul 20, 2018 10:46 am

Okay, it seems I have a mix of utf8_unicode_ci, utf8_general_ci and latin1_swedish_ci if I understand this correctly. Please see attached screenshoot.
Screenshot from 2018-07-20 12-44-16.png
Is there a way I can do the conversion from within phpMyAdmin?
The Phoca tool didn't seem to work for me.
You do not have the required permissions to view the files attached to this post.
--
//AvG

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Fri Jul 20, 2018 10:52 am

Wait a sec...
Dammit'!! Seems like I got hit by a changing char set on CLI and GUI respectively.

Edit: Wazza! All tables etc set to utf8mb4_unicode_ci!
Now to check the swedish characters in Joomla.
--
//AvG

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Fri Jul 20, 2018 11:55 am

Well, Joomla System Information now tells me this:

Code: Select all

Database Collation	utf8mb4_unicode_ci
Database Connection Collation	utf8mb4_unicode_ci
But I still see the yen-signs.
I'm starting to feel I should just translate the swedish pages to english and be done with it. :'(
--
//AvG

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Fri Jul 20, 2018 12:13 pm

I also notice that I have several instances of joomla tables in the one joomla database. Undoubtedly from the previous server moves, reinstalls etc.

Maybe I should remove the unused ones and see if it gets better...
--
//AvG

User avatar
Webdongle
Joomla! Master
Joomla! Master
Posts: 34773
Joined: Sat Apr 05, 2008 9:58 pm

Re: Encoding problem with my joomla site

Post by Webdongle » Fri Jul 20, 2018 12:52 pm

Alternatively
  • Install Akeeba and make a backup
  • Create a new database, database user and password
  • Create a subdomain (use a folder level with the main site not under it if possible)
  • Use the backup to install your site in the subdomain
  • Compare the collation of both database
  • Work on the subdomain version and back it up when you have fixed your problem
Note
Working on a copy in the subdomain will prevent you breaking the live site.
Some shared Hosting create the subdomain folder under the main site and that can cause confusion if you are not aware of it's location.
http://www.weblinksonline.co.uk/
https://www.weblinksonline.co.uk/updating-joomla.html
"The definition of insanity is doing the same thing over and over again, but expecting different results": Albert Einstein

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Fri Jul 20, 2018 2:30 pm

Interesting idea. I'll try that.

Am a bit unsure about the third paragraph, how do you mean subdomain not under the main site?
English is not my native language, sorry.

This site is not hosted though, it's my own private server at home.
--
//AvG

User avatar
Webdongle
Joomla! Master
Joomla! Master
Posts: 34773
Joined: Sat Apr 05, 2008 9:58 pm

Re: Encoding problem with my joomla site

Post by Webdongle » Fri Jul 20, 2018 3:10 pm

Create the folder for the subdomain where you see the folder for the domain

path/to/domain-root (contains configuration.php robots.txt /administrator etc for domain)
path/to/subdomain-root (contains configuration.php robots.txt /administrator etc for subdomain)

= the folder for you site is level with with the one for your subdomain. In other words entering the domain folder you don't see the subdomain folder
folder structure 01.JPG


Not enter the domain folder (where you configuration.php robots.txt /administrator )

path/to/domainroot/subdomain-root (contains configuration.php robots.txt /administrator etc for domain)
= you have to enter the domain root folder to see the folder for the subdomain.
folder structure 02.JPG
You do not have the required permissions to view the files attached to this post.
http://www.weblinksonline.co.uk/
https://www.weblinksonline.co.uk/updating-joomla.html
"The definition of insanity is doing the same thing over and over again, but expecting different results": Albert Einstein

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Fri Jul 20, 2018 3:48 pm

I'm sorry, the extremely hot weather currently in Sweden makes me thick...

However, using phpMyAdmin and Akeeba Admin Tools, I've managed to now set this:

Code: Select all

Database Collation	utf8mb4_unicode_ci
Database Connection Collation	utf8mb4_unicode_ci
Creating a new test article with swedish characters still gives me those weird characters.

Edit: A clarification; The previous change to the charset, a bit up, broke my database when using the Phoca Collation Tool and by extension my Joomla.
The site is restored and now back again and with the unicode setting in this post.
--
//AvG

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Mon Jul 23, 2018 7:49 pm

Made it work.
I changed the encodings back to utf8, the default Joomla setting and then using eg "&ouml;" for "ö" in the code.

As previously stated the Swedish pages are few.
The solution is not very clean, but works.
--
//AvG

User avatar
Webdongle
Joomla! Master
Joomla! Master
Posts: 34773
Joined: Sat Apr 05, 2008 9:58 pm

Re: Encoding problem with my joomla site

Post by Webdongle » Mon Jul 23, 2018 9:01 pm

Have a look at the charmap for JCE editor easier than searching then typing it out
jce charmap 01.JPG
You do not have the required permissions to view the files attached to this post.
http://www.weblinksonline.co.uk/
https://www.weblinksonline.co.uk/updating-joomla.html
"The definition of insanity is doing the same thing over and over again, but expecting different results": Albert Einstein

User avatar
infograf768
Joomla! Master
Joomla! Master
Posts: 18629
Joined: Fri Aug 12, 2005 3:47 pm
Location: **Translation Matters**

Re: Encoding problem with my joomla site

Post by infograf768 » Tue Jul 24, 2018 5:48 am

using eg "&ouml;" for "ö" in the code.
Definitely, this means that there is something wrong in your server. UTF8 characters do not need to be changed to htmlentities for Joomla to work fine.
Jean-Marie Simonet / infograf · http://www.info-graf.fr
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Tue Jul 24, 2018 6:33 am

Webdongle wrote:
Mon Jul 23, 2018 9:01 pm
Have a look at the charmap for JCE editor easier than searching then typing it outjce charmap 01.JPG
Thanks!
That might be a time saver for now.
The search and replace I did yesterday wasn't very time consuming, just tedious.
--
//AvG

User avatar
adrian_vg
Joomla! Apprentice
Joomla! Apprentice
Posts: 44
Joined: Wed Apr 23, 2014 3:06 pm
Location: Sweden

Re: Encoding problem with my joomla site

Post by adrian_vg » Tue Jul 24, 2018 6:37 am

infograf768 wrote:
Tue Jul 24, 2018 5:48 am
using eg "&ouml;" for "ö" in the code.
Definitely, this means that there is something wrong in your server. UTF8 characters do not need to be changed to htmlentities for Joomla to work fine.
I agree. Problem is just what, as this happened quite suddenly a little while ago.

Anyway, now that the site looks at least superficially good for my slightly OCD-ish taste, I'll start troubleshooting with a copy instead of this live server.

Thanks all for hints and suggestions so far!
--
//AvG


Post Reply

Return to “Language - Joomla! 3.x”