php vaporizing special characters?

rsphaeroides · Post by **rsphaeroides** » Thu Oct 20, 2005 7:33 pm

I originally posted this in templates because I though it was a css problem, but it appears now that it's actually a more general problem with some of the php components of Joomla

I've attached a couple of screen shots showing what happens when a component other than com_content is occupying the openpane

The first is with a content item, and the extended ascii character e accent grave (0233) shows up fine. The second is with a gallery component open (no it's not gallery it does the same with frontpage and other built ins) and the ascii character is unrecognized.

The css info looks the same when I look at it with firefox dev tools. The problem seems to be the html that is being generated by the php

here is the source from the pathway for each image

Code: Select all

<span class="pathway"><span class="pathway"><a href="index.php" class="pathway">Home</a> <img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" /> <a href="index.php?option=com_content&task=view&id=149&Itemid=96" class="pathway">Hotels</a> <img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" /> <a href="index.php?option=com_content&task=category&sectionid=5&id=13&Itemid=86" class="pathway"> San José</a> <img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" />   Hotel Don Carlos </span>

<span class="pathway"><span class="pathway"><a href="index.php" class="pathway">Home</a> <img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" /> <a href="index.php?option=com_content&task=view&id=149&Itemid=96" class="pathway">Hotels</a> <img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" /> <a href="index.php?option=com_content&task=category&sectionid=5&id=13&Itemid=86" class="pathway"> San Jos�</a> <img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" />   Hotel Don Carlos <img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" /> <a href="/travel/index.php?option=com_gallery2&Itemid=87&g2_view=core.ShowItem&g2_itemId=7" class="pathway">Albums</a><img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" />  <a href="/travel/index.php?option=com_gallery2&Itemid=87&g2_view=core.ShowItem&g2_itemId=315" class="pathway">></a><img src="http://costa-rica-guide.com/travel/templates/costa_rica_travel/images/arrow.png" border="0" alt="arrow" />  Hotel Don Carlos</span>

Replacing the ascii character from the Arial font with the html & # 233; helps in some cases but not all and sometimes shows up as & amp #233;

Where the heck is this coming from and what can I do about it? Any suggestions would be most appreciated.

zigzag · Post by **zigzag** » Thu Oct 20, 2005 10:16 pm

Hi,
It's probably this
http://forum.joomla.org/index.php/topic,10327.0.html

rsphaeroides · Post by **rsphaeroides** » Thu Oct 20, 2005 10:24 pm

Thanks for the suggestion @zigzag, but I've always used

If I view source that's what's at the top of each of those pages and one displays the characters correctly and the other doesn't.

My problem appears to be of the sort Ricardo was having which was never resolved in those other threads.

Vizibelli wrote: I´m with same problem here. All á, é, í, ó, ú became a "?"

Changing the english.php file solved part of the problem.
I tryed to edit some contents before seing this thread and after changing encode at english.php new problems arrived with these content.

Same content at front page is viewed with errors, but when i click on "read more", the content shows without errors.

Think we need a 1.0.2.1 version quickly.

Ricardo

It appears that the php in com_content is handling this differently than the other components (it's outputting the correct characters) and what I'd really like to know is how to bring com_frontpage etc. up to speed...

DeanMarshall · Post by **DeanMarshall** » Thu Oct 20, 2005 11:16 pm

Hi rsphaeroides,

I think you misunderstood one of the earlier posts.

In this message:
http://forum.joomla.org/index.php/topic ... l#msg70332

Tonie wrote: In the /languages/english.php from 1.0.2, there is a value:
Code: Select all
DEFINE('_ISO','charset=utf-8');
This used to be:
Code: Select all
DEFINE('_ISO','charset=iso-8859-1');
in older versions.

I haven't played with this setting, so I don't guarantee that you are going to have the results you wish.

You need to switch to utf-8 in your index.php file and all should be well.
This is part of Joomla's move towards greater support for extended character sets.

Dean

zigzag · Post by **zigzag** » Fri Oct 21, 2005 12:28 am

Also if anyones running the SMF Forum Bridged and wrapped into Joomla you need to change the charset in smf as well or these characters will start appearing in posts (it took me several days to work that out

)

http://forum.joomla.org/index.php/topic,13314.0.html

rsphaeroides · Post by **rsphaeroides** » Fri Oct 21, 2005 12:29 am

Dean,

You're exactly right about the missunderstanding

Unfortunately all switching to UTF-8 does is makes all of them wrong all of the time.
Even more unfortunately this is probably because I'm running Mambo 4.5.2.3

Any suggestions to fix it short of starting from scratch with Joomla? I've made so many hacks, additions and changes that I don't think the migration will be very smooth and I was hoping to hold out for the significant upgrades with Joomla 2.1 or so before moving over.

People must have used accents previously there are all of those International users...

Also any insight into why charset=iso-8859-1 works fine with com_content but not the other components would be great.

DeanMarshall · Post by **DeanMarshall** » Fri Oct 21, 2005 12:43 am

Hi,

Some browsers are terrible at accepting the change. You may at some point in the past 'told' your browser to show websites using the iso western charset. This overrides the html page's instruction. In your browser change this in your 'view' menu. Then close your browser. Then re-open it again and check that the setting has been maintained. If not, you may want to try again. Some browsers - IE - can require a reboot to accept such changes. Don't ask - it's a Microsoft thing.

You can set your browser to auto, which should use the encoding from the page, or you could force utf-8.

If that still doesn't work, check your .css file, I believe - based on a comment by KenMcD earlier that the .css file can override all of this. Lastly, on my localhost installation there was a setting in Apache which caused problems. I had to change this to 'force' utf-8 output.

Please also note: the utf-8 setting may be needed in two places in your index.php file, both in the .
echo '';
" />

Hope that helps.

Dean.

edited: minor typo.

rsphaeroides · Post by **rsphaeroides** » Fri Oct 21, 2005 1:16 am

Thanks Dean!
Haven't tried it yet, but that gives me a load of things to chase down! I've been after this for weeks and mostly stumbling aimlessly. Now I've got some targets!

BTW it seems consitent across several browsers although IE makes a heck of a mess...

rsphaeroides · Post by **rsphaeroides** » Fri Oct 21, 2005 5:24 pm

Wow, I'm really making a mess of this.

I've got utf-8 showing up both places in the head, and when I look at the View>Character Encoding in either Firefox or IE it says UTF-8 whenever I'm on a page in my site, but I'm still getting no special characters.

How would I determine the Apache default and force UTF-8? Is there something I need to look at in the database?

It doesn't seem to be an issue with the characters displaying wrong, it's more like they are all (or some subset of them that is chosen by indeterminate means) being stripped out as the page is being generated.

DeanMarshall · Post by **DeanMarshall** » Fri Oct 21, 2005 8:23 pm

Okay,

I can't remember the exact setting in apache - but if you search the httpd.conf file for iso you won't waste too much time.
The final possibility that you have just reminded me of is indeed the MySQL database tables which have a characterset too. I think if you ran the update sql files that the necessary change is made, but I am out on a limb on this one (guess work).

Use php MyAdmin (or similar) to see what your database settigns are.

Dean.

DeanMarshall · Post by **DeanMarshall** » Sat Oct 22, 2005 1:46 am

Ray,

I just re-read that other thread referenced earlier and now see that I mis-understood it, and that reverting to iso-8859-1 by editing the language file 'may' work for you.

Before you go hacking about with the database - have you tried changing templates, or changing back to using the 'charset=iso-8859-1' setting. Others are reporting success with transitioning back, to 8859-1. I can't claim to fully understand this issue - I was under the impression that 8859-1 was a subset of utf-8, in other words I thought utf-8 preserved and then extends the character set to support multibyte characters. Obviously I must be wrong. Check around before you do anything too drastic - and make sure you have both database and file backups before you proceed.

If this transition is taking too much time - you might consider reverting back to $ambo.

I know I had this problem and I know I solved it for me (on a test installation), but I wouldn't like to see you wreck your site on my guesswork.

Dean.

rsphaeroides · Post by **rsphaeroides** » Sat Oct 22, 2005 5:16 am

Hey Dean,
I've got backups, and I'm still very much in development as well so I'm not too worried about ruining anything.

The database may be part of this issue, but I haven't figured out how it's affecting things. I can set to UTF, Latin, Iso and some things work while others don't.

Thanks again for your suggestions. I'll keep digging
Ray

rsphaeroides · Post by **rsphaeroides** » Wed Oct 26, 2005 2:27 pm

I think I'm getting there.

Changing to UTF-8 in the language file, database, index.php, and half a dozen other places I tracked down character set settings seems to be the ultimate solution, but there is a NASTY artifact. Any content created, menus made, or titles set with accents etc. in another character set besides UTF-8 will be a mess when you switch.

If I simply retype the missing characters they show up fine. I've been editing them as I see them, but can only get away with this because I'm still developing. If I had my content in the site already I'd think long and hard before considering changing character sets. It might result in extraordinary manual editing requirements.

Thanks for the help

guilliam · Post by **guilliam** » Tue Feb 28, 2006 11:08 pm

hi ray and dean,

just to pitch in something,.. is this or is this not similar to this thread? --> http://forum.joomla.org/index.php/topic,33247.0.html

its particularly about umlauts in which i honestly say not of my TURF in mysql(charsets). @predator and @de made a little explanation on that thread as well which for me makes sense(noob on charsets

)

lemme hear your thoughts.

- g

The Joomla! Forum™

php vaporizing special characters?

php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?

Re: php vaporizing special characters?