Joomla! Discussion Forums

Does anyone know if there is a difference between a magic_quotes_gpc escaped value vs. a completely raw input that is escaped through mysql_real_escape_string or mysql_escape_string?

I thought I had read that mysql_real_escape_string actually escapes from more potentially harmful mysql characters than mysql_escape_string.  And that leads me to beg the question of whether either of those php functions will escape from more harmful characters than a magic_quotes_gpc escaped value.  Anyone know?

As an end user who has been updating componets even from big players this week that didn't have any Valid_MOS statement --

This sounds like a logical change to both joomla core and recommend to 3PDs.

@Tyler - you will have to wait until RobS and Beat come back to play. I am still trying to figure out how I spent nearly 25 years in IT and don't understand more than 10% of what people are saying! This is only made more discouraging when RobS says that the cases were articulate and well presented, and I know in order to reach 10%, I had to cheat and count the discussions on wine and whiskey.

+++

I am hearing we agree that Rliskey should go with post #49. Change it to "1" and modify the description as per that post so it's EZ.

+++
It is so important to take this highly technical, fun and geeky debate and reduce it to something that normal people can read and understand without feeling overwhelmed or stupid so that it is clear to them the choices they have, and the steps needed to secure their websites.

In two months, I cannot believe the progress that you smart guys have made. And, I hope you recognize how valuable that is to our community.

Beat -- I just now saw your seven step PHP.INI decision making guide to safety. Perfect. You are right about continuing to rewrite and simplify and SHORTEN the message. That is a perfect example of "falling off a log" easy - can't miss.

Basically, addslashes() just adds slashes before quotes, double quotes, and null charachters.  mysql_escape_string() is similiar except it is more mysql specific and it escapes a few other things like newlines, carriage returns, back slashes, and the sub characther.  mysql_real_escape string() is better because it is charset sensitive so it will respect the character encoding, which is important, because when you get to charachter sets like chinese simplified and you need multiple bytes of data to store the value of one character (normal charsets only use 1 byte per charachter), you can break out of things like addslashes() and mysql_escape_string() by using certain charachters in chinese simplified (and others similarly large charsets).  Because Joomla supports internationalization, it is important that we use mysql_real_escape_string() because the UTF-8 charachter sets allow for chinese simplified multibyte characthers that can cause problems for addslashes() and mysql_escape_string().

I hope that helps

Great info Rob, thx for that.  I'm curious though where magic_quotes_gpc = 1 stands with regard to escaping.  Does it escape the same characters as addslashes(), or is it more comprehensive like mysql_real_escape string() ?  I'm trying to figure this out because I've had no luck with net search on what characters magic_quotes_gpc = 1 will escape, and that information will be my basis for how I code my custom escaping function:


The thing is, I've seen some custom escaping functions that simply return the magic_quotes_gpc'd $value if magic_quotes_gpc is on.  Now, if in fact there is a difference between a magic_quotes_gpc $value, versus a $value that is escaped with mysql_real_escape_string(), then I'd prefer to use the strongest escaping methodology in my custom escaping function.  I just need to know how strong magic_quotes_gpc = 1 is, wrt the other native php escaping functions (ie. addslashes(),  mysql_escape_string(),  & mysql_real_escape_string() ).

As far as I know magic_quotes_gpc works exactly like addslashes().

Hi Rob,

Got your points accross and agree fully with them: that's why it's important to discuss them for 1.5, which is another class of beast than 1.0.

I like your global approach very much.

1.5 is playing in another league when it comes to its design, structure, and intended developpers. 1.5 requires good understanding of object-oriented programming among other things, which is less the case for 1.0. So, the level of successful "non-Joomla 1.0-legacy-mode" 3PD developpers will be very different, as well as their number. Also the new J! 1.5 framework, when used, allows to safely handle data in its original state. So yes, agreeing with you. For pure 1.5 J! applications, magic_quotes_gpc 0 is better and should not be less safe against developpers' bugs than magic_quotes_gpc 1 as long as only J! application framework is used.

As you agree that for 1.0, that magic_quotes_gpc 1 is better and safer, I think we are in line, and gave a simple clear answer to the community.

As a matter of fact, in my larger PHP applications, as developer, I hate having escaped non-raw variables, and prefer escaping according to the database used (e.g. mysql is ' -> \' and postgres ' -> '' (2 times "'") ) at database storage time than carrying over the escapings and changing them at database layer if required (e.g. \' -> '' for postgress).

But for small plugins and extensions, which are often written by less defensive or experienced developers, it's safer to give them safer escaped variables. And one of the big success factors of Joomla! 1.0, compared to more complex systems, has been that it's easy to change code or add code without deep programming knowledge. The result is there: 700+ extensions, but not all 100% cleanly written and secure.

Only question yet open for a consensus is that for a probably longer time than we expect, people will keep 1.5 legacy-compatibility setting ON to run existing 1.0 Joomla 3pd extension. So what should be the recommended setting be for Joomla 1.5.0 ? In light of legacy 3pd extensions, magic_quotes_gpc 1 is better security-wise, while magic_quotes_gpc 0 is cleaner for 1.5 core and pure 1.5 extensions.

I'm "a little" security biased, as you know , so it's a very good discussion. Sorry for being sometimes a little technical (tried to put those in separate paragraphs). No pun intended at all and didn't feel any from anybody either

IMO - if you start out v 1.5 with compromise, you will never acheive the level of excellence you are aiming for. Go for it and trust your developers to dig in and get better. With all the organization, documenting and training you are already doing, it can happen. But, if you let it slide from the start, it will be impossible to ever get it back. For what that's worth!

Hi,

@rliskey: There is still one occurence of "magic_gpc_quotes" that needs changing to "magic_quotes_gpc" in your (most excellent, by the way) Security Checklist.  It's in the quote under "Here are example directives for the above suggestions:".  Sorry to be a pain but anyone who does a cut-and-paste will find that the line concerned will simply be ignored by PHP and it's the kind of mistake that can be hard to track down.

Regards,
Chris.

Very true.

As said, Joomla! 1.5 is another beast, and for 1.5, I'm basically agreeing with Rob's position of principe.

In addition, as RG_EMULATION is always OFF (0) in Joomla 1.5, the risk of unescaped strings is lower as variables can't be used directly.

For 1.0.x, in the light of badly configured servers and of some 3pd extensions vulnerabilities, adding multiple lines of defense and warnings was not a luxury.

Yes, 1.5 is a new story.

But am I wrong in assuming that you can always restore unescaped status of a magic_quote_gpc escaped string using stripslashes ? even with UTF-8 ? As a matter of fact I use a private function getUnescaped() in all my larger applications which does exactly that, and seems to work fine in Chinese and other UTF-8 languages...

Right now (i just svn updated my 1.5 test-installation, the Joomla! 1.5 installer recommends magic_gpc_quote ON, so the 1.5 core devs seem to be agreeing ?

Also I didn't see a large PHP application around yet recommending magic_quotes to be OFF (e.g. just installed SMF 1.1 RC3 with UTF-8 support, and it doesn't care either). But your experiences may be different.

It would be good is to reach a consensus in core team on this, if this recommendation needs really to be changed for 1.5.

Here are two code snippets from 1.5 SVN revision 4839:
[code=\libraries\joomla\environment\request.php, line 135]
// Handle magic quotes compatability
if (get_magic_quotes_gpc() && ($result != $default))
{
if (!is_array($result) && is_string($result)) {
$result = stripslashes($result);
}
}
[/code]
and
[code=\libraries\joomla\environment\request.php, line 212]
// Handle magic quotes compatability
if (get_magic_quotes_gpc()) {
$result = JRequest::_stripSlashesRecursive( $result );
}
[/code]

This means that all extensions which use the JRequest class will get values without quotes, no matter what the magic_quotes_gpc setting is.



I've asked pretty much the same question a few weeks ago on the dev mailing list, and the answer from two core devs was to remove the recommended setting. So the installer message is still there as a "leftover" from old 1.0.x days, and most probably you'll notice that the recommended setting during the installation will go away soon
I'll bring this up on the dev mailing list again though, especially after 1.0.11 now clearly directs you to set magic_quotes to "on". Personally, I'm in favour of removing it for 1.5. We'll see what comes out of this...

In addition to Rob's post, here is an article by Chris Shiflett that explains some differences between addslashes and mysql_real_escape_string (and why addshlashes is insecure for certain multibyte charsets) for the interested reader

http://shiflett.org/archive/184

Thanks Enno for this indeed interesting condensed and hands-on reading.

Happy to see that 1.5 really took it the professional way. 

Could you please confirm or correct me, to make sure I undertood it right looking at the 1.5 code:

1.5 is in fact insensible to magic_quotes_gpc setting and this Multibyte-escaping pitfall, as stripslashes does first exactly the reverse of magic_quotes_gpc if magic_quotes_gpc is set to 1, then normal raw data handling is done in all cases in the components/modules code, and then only at the SQL-query the proper (e.g. mysql_real_escape_string for mysql databases) escaping is performed, taking also in account the multibyte settings and database type.

Right ?

In that case, magic_quotes_gpc setting is don't care and 0 can be slightly recommended for performance reasons, but is less of a security concern than in Joomla 1.0 series extensions...

And that's why in Joomla 1.5 the magic_quotes_gpc settings recommendation can be completely removed...

Right ?

Right !

Thanks so much Enno for Shiflett's blog article.  I read a lot of valuable info linked from there and learned some very important stuff about security.

I guess a good thing (wrt escaping MySQL strings) is that if one is using UTF-8, then they don't need to be concerned w/the difference between addslashes() vs mysql_real_escape_string()

Just curious Enno, do you know if magic_quotes_gpc = 1 accomplishes the same thing as addslashes(), or is it more encompassing like mysql_real_escape_string() ?

Yes, it does the same thing: http://de.php.net/manual/en/function.addslashes.php

Here is some informations I got from 1and1 support.
It is possible to update php.ini setting from a custom file.

1. create a new php.ini file with tis simple line:

2. upload this file in your root and /administrator directories
3. edit all these files:

and add this line at the top of each file :

This overwrite the settings from the common php.ini file (works for me)

But it seems this line does not support relative path... any idea?

Does it require apache reboot before it takes effect?  I know this is usually the case w/the master php.ini file.

Not at all. Modificatons are updated immediatly.

Warning:

This will only change php settings and protect the files where this line is included.

The PHP register_globals setting is precisely most useful for those files where the other standard files, like global.php, are not included and bugs/forgettings/missing security checks are missing.

The warning from backend configuration will not show up (as it's from a normal entry point), so you will not be warned of this missing barrier of protection for other entry points.

Sorry, this is not a good implementation for changing default PHP settings, security-wise.

That's why I posted here, to be sure this is a good solution. But it's not!

Rechecked source and corrected the very important paragraph 3 (SQL injections) of developper documentation for securing extensions in wiki accordingly (mosGetParam escapes and JRequest:getVar() does not.

Here the link for review and fixes/complements as needed:
http://dev.joomla.org/component/option, ... injections

The Security Checklist is very well done. But who reads it? I think every one who has a problem with his site (something like my site got hacked).

Maybe it would be better to serve the security checklist with the installation. So that every one who installs joomla is aware of the risks and the actions to take against security risks.

Or if register globals is on the site is offline until you explicitly say it should ignore register globals on or you change register globals to off. Such things that help the guys who never read the manual or the forum. It helps them to get a better and more secure joomla. And it helps the community to not always have the same postings "my site got hacked". And also those little script kiddies got no more food for there bot nets.

Actually, this is already the case with Joomla! 1.0.11

By default, for compatibility reasons at install and upgrade times, RG_EMULATION is ON by default. This triggers the security warning at installation and in backend homepage. Each of these warnings has a direct link to exactly this Security Checklist... 

Ah

well done.

As i installed joomla there was not such a thing. And i just upgraded to 1.0.11, so there was no need to take care of the list

Maybe it would make sense to show a link to the security list in all cases, including when the server settings are "correct". Even those who got those three setting "right," may have other security questions.

BTW: I'm gradually moving actual advise out of that list and into the FAQ forum where it belongs. That way the list can once again be just a high-level overview of the issues, with pointers to the best background information.

If something is added to the security checklist, please add a note what has been changed.

If you don't read every thread, you only see that there has been a change, but don't see what has changed.

Point taken. I try to minimize posts, but will do this from now on.

Because the Security Checklist has become so long, we are gradually moving the detailed information to the Joomla! FAQs. This will allow the checklist to again be just a quick checklist, with many links to more complete background information.

Security FAQ: http://forum.joomla.org/index.php/board,322.0.html

3rd Party/Non Joomla! Security FAQ: http://forum.joomla.org/index.php/board,346.0.html

thanks for your hard work organising this.