Text truncated beginning at French accent

General questions regarding the use of languages in Joomla! 1.5.

Moderator: General Support Moderators

Locked
smallbiz
Joomla! Apprentice
Joomla! Apprentice
Posts: 32
Joined: Wed Feb 20, 2008 3:47 pm

Text truncated beginning at French accent

Post by smallbiz » Thu May 28, 2009 6:07 pm

I'm running J 1.5.10, under Apache, with PHP 5.2.9 and mySQL 5.0.77
MySQL charset: UTF-8 Unicode (utf8)
MySQL connection collation: UTF_general_ci

I create a new Joomla installation using Fantastico, and without extending Joomla at all, I update an article title with a French accent -- for example, é

As soon as I save/apply the change, the character is deleted. If I insert the character at the beginning of any string (article title or content), all text from the accented character forwards are deleted (truncated).

If I use phpMyAdmin to enter the é into the database directly, it is saved successfully.

In phpMyAdmin, all tables have the collation of latin1_swedish_ci

This seems to be limited to Joomla installations on the server -- for example, when I add French characters to text in an installation of LimeSurvey on the same server, using the same mySQL instance, the accented characters are preserved.

Can anyone point me towards a solution?

User avatar
Per Yngve Berg
Joomla! Master
Joomla! Master
Posts: 25756
Joined: Mon Oct 27, 2008 9:27 pm
Location: Akershus, Norway

Re: Text truncated beginning at French accent

Post by Per Yngve Berg » Thu May 28, 2009 6:40 pm

Use collation of utf8_general_ci instead of latin1_swedish_ci.

smallbiz
Joomla! Apprentice
Joomla! Apprentice
Posts: 32
Joined: Wed Feb 20, 2008 3:47 pm

Re: Text truncated beginning at French accent

Post by smallbiz » Thu May 28, 2009 6:50 pm

Well, I ran the util tool_phoca_changing_collation to set the collation to utf8_general_ci, and when that completes, I found the problem persists.

Interestingly, now when I enter the é character into an article title, and click save or apply, I'm booted out to the admin login page.

Interestingly, the LimeSurvey on the same mySQL box has collation = latin1_swedish_ci and stores French characters okay.

Any other ideas???

smallbiz
Joomla! Apprentice
Joomla! Apprentice
Posts: 32
Joined: Wed Feb 20, 2008 3:47 pm

Re: Text truncated beginning at French accent

Post by smallbiz » Fri May 29, 2009 12:03 am

I found the solution - 1 line in php.ini on my VPS:

change commented out line:

;default_charset = "iso-8859-1"

to

-default_charset = "utf-8"

Thanks to this useful blog http://dreweyscorner.[URL banned].com/2008/ ... pache.html

Which I'm including full text of here in the event the blog is deleted at some future date:

Before we begin, you need r00t access to key Apache, PHP and MySQL configuration files.

Let’s start with the apache config. The location paths may differ based on your server setup. Look around and be sure that you are editing real config files and not the templates.

Apache Config - /etc/httpd/conf/httpd.conf

AddDefaultCharset UTF-8

PHP Config – /etc/php.ini

default_charset = "utf-8"

MySQL Config - /etc/my.cnf

[client]
default-character-set=utf8

[mysqld]
character-set-server=utf8
default-character-set=utf8
default-collation=utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-client = utf8

Restart the above services once these updates have been applied.

Confirm if UTF-8 is Enabled:

# mysql –uroot –hlocalhost –p
# show variables like 'c%'

The above output should be:

+--------------------------+-------------------------------------------------------------------+
| Variable_name | Value |
+--------------------------+-------------------------------------------------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | utf8 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8 |
| character_set_system | utf8 |
| collation_connection | utf8_general_ci |
| collation_database | utf8_unicode_ci |
| collation_server | utf8_unicode_ci |
| completion_type | 0 |
| concurrent_insert | 1 |
| connect_timeout | 5 |
+--------------------------+-------------------------------------------------------------------+

It is possible that after these updates, PHP will continue to decode a UTF-8 character set in the form of question marks. Eg: ????

The solution is to call mysql_query() immediately after mysql_connect() has attempted a connection to the database.

$db_ = @mysql_connect (HOST, USER, PASSWORD, TRUE) or die("Could not connect");
mysql_query('SET NAMES utf8');
mysql_select_db(DB, $db_);

(Note the mysql_query('SET NAMES utf8'); above)

User avatar
infograf768
Joomla! Master
Joomla! Master
Posts: 18830
Joined: Fri Aug 12, 2005 3:47 pm
Location: **Translation Matters**

Re: Text truncated beginning at French accent

Post by infograf768 » Fri May 29, 2009 5:59 am

smallbiz wrote:I'm running J 1.5.10, under Apache, with PHP 5.2.9 and mySQL 5.0.77
MySQL charset: UTF-8 Unicode (utf8)
MySQL connection collation: UTF_general_ci

I create a new Joomla installation using Fantastico, and without extending Joomla at all, I update an article title with a French accent -- for example, é

As soon as I save/apply the change, the character is deleted. If I insert the character at the beginning of any string (article title or content), all text from the accented character forwards are deleted (truncated).

If I use phpMyAdmin to enter the é into the database directly, it is saved successfully.

In phpMyAdmin, all tables have the collation of latin1_swedish_ci

This seems to be limited to Joomla installations on the server -- for example, when I add French characters to text in an installation of LimeSurvey on the same server, using the same mySQL instance, the accented characters are preserved.

Can anyone point me towards a solution?
The usual problem here is not the php.ini, but the collation of the database BEFORE installation.
If it is not collated utf8 (it may be latin1 as default when the database is created on a share hosting environment), then all will go berserk as not only the db has to be collated utf8, but also the tables themselves, and the text fields inside the tables.
Jean-Marie Simonet / infograf · http://www.info-graf.fr
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group

User avatar
PhilTaylor-Prazgod
Joomla! Ace
Joomla! Ace
Posts: 1193
Joined: Sat Aug 20, 2005 12:32 pm
Location: Jersey, Channel Islands
Contact:

Re: Text truncated beginning at French accent

Post by PhilTaylor-Prazgod » Sun May 12, 2013 9:17 pm

This issue is now resolved on your server.

I tried many things to debug this, including downgrading your PHP version to 5.3, (I'm current lying rebuilding apache back to 5.4.x now…) - the whole process took a couple of hours debugging.

The root fix was to edit your PHP Config file php.ini (You can use PHP Configuration Editor in WHM instead of the command line to do this, I used SSH command line)

I have reset ALL the mbstring configurations back to their default, distributed settings. These being the default values listed on this page:

http://php.net/manual/en/mbstring.configuration.php

As soon as I did that, with no restarting of mysql or PHP/apache the problem was resolved and I could now save a string with accented chars in the centre of a paragraph with no truncating of data.

Kindest regards
Phil.
Phil Taylor - Full Time Joomla/PHP Security Expert
Blue Flame Digital Solutions Limited.
-- https://myJoomla.com/ Multi Award Winning Joomla Security & Auditing Service
-- https://www.phil-taylor.com/

User avatar
infograf768
Joomla! Master
Joomla! Master
Posts: 18830
Joined: Fri Aug 12, 2005 3:47 pm
Location: **Translation Matters**

Re: Text truncated beginning at French accent

Post by infograf768 » Mon May 13, 2013 7:24 am

Thanks Phil for updating this, never too late. :)
Jean-Marie Simonet / infograf · http://www.info-graf.fr
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group

User avatar
PhilTaylor-Prazgod
Joomla! Ace
Joomla! Ace
Posts: 1193
Joined: Sat Aug 20, 2005 12:32 pm
Location: Jersey, Channel Islands
Contact:

Re: Text truncated beginning at French accent

Post by PhilTaylor-Prazgod » Mon May 13, 2013 8:05 am

Sorry never looked at the date :-)
Some dude had this issue today and sent me this link so I assumed it was him. The same issue exactly, and the solution I posted worked on his server and a test server I set up for this.
Phil Taylor - Full Time Joomla/PHP Security Expert
Blue Flame Digital Solutions Limited.
-- https://myJoomla.com/ Multi Award Winning Joomla Security & Auditing Service
-- https://www.phil-taylor.com/


Locked

Return to “Language - Joomla! 1.5”