Joomla! Discussion Forums

Hello,

To avoid anyone telling me that Community Builder questions should be asked on Joomlapolis, my question is more related to J! 1.5.14 than to CB functions.

Current Platform : Joomla! 1.5.14 / CB 1.2.1 and coming up Kunena 1.5.

Aim : Link articles authors to their CB profile pages.

Possible solution : Use Zak Author 2 CB Plugin or CB Authorbot ver. 1.0.2

Problem : Upper mentionned plugins do not work anymore since I upgraded J! from 1.5.9 to 1.5.14. Actually it is known that from 1.5.10 it doesn't work. Symptom ? Instead of displaying a link to the author, the HTML link code is fully displayed and, thus, not effective.
Plugin doesn't work ? Solution should be to contact the author, but he's not responding and no plugin alternative is existing.

Now that I have some knowledge in PHP/xHTML, I'd like to correct this myself but need some help regarding J!. Focusing on Zak's one, for which code is easy.

Why doesn't it work ? It seems that HTML encapsulation used in J! 1.5.14 is thus replacing a simple HTML link by a full-text code.

Finding the bug :
FYI : plugin code and info at the end of this post.

In order to summarize, on line 60 of the code :

Gives us in HTML source :

This is displayed as :

Instead of :
(Smith being a clickable link)

As ones can see, PHP created a string with special characters which are translated by J! into generic web code making the link not effective.

Question is rather simple (easy after that long introduction ) :
How can I bypass this problem ? How would the plugin's author trick J! to have the link displayed eventually ?
(Being not fan of hardcoding or Core hack solutions...)

I had some ideas like replacing " ' " (apostrophes) by " " " (brackets) in original code, though I'm not sure this would be sufficient.

Every help will be greatly appreciated. If any fix comes up, I'd publish it onto relevant forums in Joomlapolis to help other J! members.
Thanks !


---------

Plugin structure :

• en-GB.plg_content_zakauthor.ini
• zakauthor.php
• zakauthor.xml

Nothing more

Here is the plugin code from zakauthor.php :

Have you solved the problem?
I'm interesting for it also from 1.5.10.

At Forum of Joomlapolis i have found:


Can it help you?
Thanks.

I've looked into the Joomla! code:
Yes, now Joomla! "escapes" all special characters. See the code from default Joomla! template (file 'components/com_content/views/article/tmpl/default.php')

To make your plugin(s) work again you need to remove call to the '$this->escape' function:



Edited: Typo in the code was corrected, thank you.

Thank you Yvolk.

I guess then that for the moment, no other choice than Core hacking...

Edit : This solution does not work either... for each of the upper mentionned plugins, even with removing all escape functions...

Isn't it any way to modify the plugin, returning right string to the escape function so that link works eventually ?

Hi Krycek,
In fact my proposal is NOT "Core hacking", because it suggests changing page TEMPLATE.
Page templates are, by Joomla! design, customizable. So:
1. Instead of changing file from the core you better create custom copy of it and use Joomla!'s "Template override" feature to substitute core template page with yours...
2. In a case you're already using some not default Joomla! template, Joomla! already doesn't use that "components/com_content/views/article/tmpl/default.php" file at all, so you don't see any difference after the proposed "hack"
E.g. if you selected "JA_Purity" template, you have to change (custom copy of) the file: "templates/ja_purity/html/com_content/article/default.php".

There is a related discussion with some valid points going on here:

http://www.joomlapolis.com/component/op ... itstart,0/

The code above has a little mistake. It must be, probably:



This solution works well!

I knew there was a mistake in the code and corrected it. I also knew about template overrides, but mine is custom (I made it from scratch) and am not using any override at all.

Anyhow, I don't understand why the plugin cannot be modified to return appropriate string to the escape function... Template override for that simple feature is quite out of all proportion, innit ?

Edit : Actually, I just found out that it is working for articles but not for home page.
So same trick, override or core hack page in 'components/com_content/views/frontpage/tmpl/default_item.php' with :

instead of :

The real issue here is that plugins should have a possibility to change the author text with html without needing to change any core or 3pd joomla template.

That was fine until Joomla 1.5.13, and a security "fix" disabled that possibility.

Question to Joomla dev team:

Would it be possible to do the database-content cleanup BEFORE calling the plugins triggers instead of after calling the plugin triggers, to avoid htmlspecialcharing the html code coming from joomla plugins ?

Anyone plan to do something about it?
or I just have to kill my plugin gAuthor, that uses the trick!
also CBAuthorBot, BK Author Link and Agora AuthorBot.

I made a post to joomla's public devs-list/group here to bring this to their attention:

http://groups.google.com/group/joomla-d ... a?hl=en-GB

Agora Authorbot seems to be working fin with Joomla 1.5.14. Sorry not sure about the rest

Sorry but though the JED page description says: "Now all authors names "Written By: XXXXX" on your site will be linked back to their Agora Profile.", the plugin does something else and add peace of HTML to the top of content text. It seems the new version does not use the trick!

Hazzaa - thanks for saying that, I've been working with Agora Authorbot and couldn't see any problems. I'll look at another one, now.

Author Link for Joomla! is changing the array element created_by_alias to contain both the Name and a link. Nothing is escaping the text values in the Plugin.

Here's one example from that plugin:


I assume when the core escapes the created by or created by alias field prior to presentation (as should be done), the link is corrupted by the escaping? Is that the issue?

To me, this is a good example of why the core must be the one to escape the output. Shiflett says if PHP devs focus on 2 things: filter input and escape output, the top 5 PHP security problems are gone.

In Joomla!, the core escapes output when that output is presented in the layouts. There's no place safer to do so. One might argue that doing so within the View *after* the Plugins fire to might provide better protection when non-core template overrides are used (if the developer doesn't do their job right), but never before IMO.

Beat - you made the point in the Dev CMS thread that core should escape the output before the Plugins fire to prevent double-escaping. But, that's sort of impossible since we don't know what all the other Plugins might do. It's not a feasible solution, IMO.

My take is that core is doing it right. I might recommend core moving it to inside the view after the Plugins fire. Now, some of our practices as third party developers need to also find best practices. We shouldn't change an array element to include a link in a name field since core will escape that output prior to presentation. Instead, we should insert our modifications to the document by working with the article->text field to append in or replace what is needed.

Just my opinion - good questions.
Amy

The question is, why not core treat $article->created_by_alias like $article->text. this future adds flexibility to Joomla! articles and usually is not just a field for name. I suggest core use some less harsh mechanism on escaping this kind of data.
Nobody have any problem with security and stuff but escaping everything, sometime causes lost of abilities and there is noway around for third party to practice this kind ability!

Hi Amy,

Thank you for dropping in with a proposed solution and interesting arguments.

Just to clarify, I don't wish to reinvent the wheel or start any great debate about security in web-applications (and how to avoid all these emergency security-releases), if that would be the case, I would be starting a new topic, and my post would be loooong.

Please accept my comments to yours:

- it's not because a single third-party plugin doesn't do its own security job (escaping) that it's joomla's task (additionally, previously names were already escaped in the database, therefore, the database content was already trusted, so it's even not an error of that plugin !). Following that route, joomla could escape the whole component output to do the job of poor 3p devs ?

- altering content isn't an option for adding properly a link to the author's name, or to add some fancy markup to allow cool effects. Why ? because you will miss the automatic and very varied format that templates do apply to that output, and not follow the templates best joomla practices.

- you wrote: "you made the point in the Dev CMS thread that core should escape the output before the Plugins fire to prevent double-escaping" : please read my post again, I didn't make that point. Nor did I want to make any "point". I just did a heads-up about a regression in the Joomla API affecting most author information enhancing content plugins.

I agree with you that best place to properly html-sanitize output is ... at output, but taking in account the nature of the information to display:
1) You assuming unclean text, and all previous joomla 1.0.x and 1.5.x releases assumed clean html. Thus it broke backwards-compatibility. It's as simple as that. Ian's suggestion on dev list to assume (instead of the 2 previous unclean text or clean html) unclean html is fine for me too.

2) Plus that joomla-internal "fix" for a poor input escaping of previous releases on name, won't fix any 3pd templates that do override that output based on joomla's previous example. Creating a false feeling of safety. Correct and only safe security fix would be to sanitize the database during the joomla upgrade (I also proposed that).

That's my only message, together with a simple proposal for a solution to fix that. Not a "point", just facts.

I am happy with other solutions allowing as previously to:

a) add a link to author name, and maybe a photo, buttons or any cool enhancements
b) add nice markup, e.g. title information, or photo on hover

exactly there where the name is displayed by standard joomla templates, according to standard parameters of joomla content.

But apart of database sanitization at upgrade, or exceptionally moving a little up the chain the "late" *database input* escaping done at *output* as a bug-fix, I don't see another backwards compatible solution to this. But I'm open to better solution, and will stand corrected if I missed one allowing a) and b) above

Ian's proposal on the dev list could work too (without params, just default white/black list).

Just looking for a practical solution to this regression here. Nothing more.



Indeed, good point. It's really "unclean text" vs "clean or unclean html". Ian's proposal on the devs list (link in my post above) to treat it as "unclean html" goes into that direction too.

Respectfully, Best regards,

Title and Name fields are not expected to contain HTML. By adding HTML to the Name field, you are wanting Joomla! to now treat it like the article text. Inside of com_content, there is different input filtering based on the data type and configuration option. There needs to be a consistent treatment of these data elements from input to output.

It's not an air tight security measure. You could create a Module to replace the Article Title area and not escape the output. You could create a Template Override for this purpose. (That's probably the better location but I understand it's not a good way to share code with others to use.) You could do a text search on the Article and replace it there. That's probably the best option, right?

But, I have a hard time joining in and saying the core shouldn't escape the output. I think we should follow those industry security practices and figure out how to provide flexibility to developers.

I hear you and I see your point. I just have to think that following Shiflet's advice is important.

I'll think on this a bit, too, and I don't mean to discourage you.

Thanks.
Amy

lol! We'd need a beer for that. It's gotten much better, though, since the security burn from a few years ago, hasn't it?


I think a goal might be that Joomla! core continues to improve it's ability to ensure a secure deployment, even in regard to third party extensions as much as possible, without (and this is what I understand your point to be) diminishing what we can do as third party extension developers.

I do a lot of this type of work in Template Overrides - or right inside of the Component output now. Things like Gravatars or Post Date Icons. I realize that is difficult though to share as an installable extensions. And, I realize right now, since the escaping is done in the layout, it merely works around core security. I agree with your point about layout overrides and think there is good reason to consider moving the escaping into the View, after the Plugins run.

Why after the plugins and not before? Because that gives the core the best chance to ensure a safe environment. I do not accept that we can't figure out clever ways to get things done, but, yes, we have to be more clever. A cost worthwhile to avoid a security burn like we saw a few years back in my opinion. (Maybe overprotective, yes.)

I did not mean "point" in a negative way, but rather something worth considering. I apologize if I misunderstood you, though.


That would be true *if* we forced people to always use the white list option, or required that they never allow any HTML in their text. The truth is many people do not want input sanitized to that point. So, it is very important to also escape the output. Maybe we should allow people to turn escaping off, just like we allow them to varying the input filtering. I'm in favor of offering people choice and am not a big believer in protecting people from themselves. But, if we don't offer a choice and only do it one way, then it should be the most secure way possible, in my thinking.

I agree with you on this which is why I think it could make sense in 1.6 to move this inside of the view, after the plugin events fire, and before the layout processes.

Regarding the Upgrade Process, since you've raised that point a few times. I suppose that's a good idea to run data through a filter on upgrade, much like the filtering now available on input. But, the same point holds true, unless we force everyone to use a specific white list or restrict any HTML on input, the possibility of "unclean data" exists in the database, always. That is the basis for Shiflett saying "Filter input. Escape Output." That's the four words he suggests are most important for PHP security. Not just "Filter input."

I am happy to read your points and learn from you, always.

Unless there are questions, I've made my points, for what it's worth, I understand the frustration. I would be against restricting choice to only a specified white list, although I think that's the very best default to offer. I would also be against removing escaping of output. I am in favor of clever solutions for us to increase Joomla!'s security and continue to produce cool extensions. I think we always do that, even when it's challenging to figure out how, at first.

With respect.
Amy

Amy,

While the database source of title and author isn't expected to be html, the output after events triggering was expected to be html.

That's how it worked in joomla 1.5.12 and all joomla and mambos before.

Changing that output assumption broke cool extensions (not speaking of ours here, others' are cooler than ours)

Fully agreeing to sanitize output, but as html not as text. That's what Ian proposed on the dev list. I'm fine with that.

You can also sanitize the output from database to be clean non-html text too, if you can't trust your database (seems to be the case here). That was one of my 2 proposals (apart of sanitizing the database at upgrade).

OR, alternatively, propose another method for content plugins to output html there where they could in the earlier API. Fine with that too. But putting that structural information into the content isn't a good solution as it bypasses templates and standard joomla settings for authors display.

Respectfully,

EDIT: added relevant quote above to clarify context of my answer, as phpBB didn't catch us posting same time...

Just a reminder: There IS a way to live in peace with Joomla! Core and have HTML markup in any place of the page, including parts, where Joomla! never allowed HTML.
The way is using BBCodes instead of HTML markup in the content

Please see the topic about yvBBCode extension.

By "like", didn't mean "exact" and also there is a difference between input and output. I think the misunderstanding is about how these plugin works (well, worked!) the input and processing the output is core job and it have to be clean as much as you want, but when core done with the data and want to pour it to template, the plugin simply convert the name admin to a html clean phrase <a href="link location">Administrator</a> and it doesn't suppose to save on database or coming back to core cycle. Just like templates.

As I see, there is no way to replace this trick. this is not within the content text to simply search and replace it and the template overriding does not have this kind of functionality. Also creating modules doesn't solve the problem. it just another trouble for template adjusting.

And nobody said anything about not escaping the output, but in a proper way to save data.


By cleaver way you mean simply not trust any plugin! so we just shut down and move to templates, because it's to close to core security.
It's not matter of security to just check every data before and after process in a third party code and undone the work, it's being -as you said- overprotective!
We do not talk about a manipulation a copy of Joomla! we talk about supporting the third party to create a public solution for every copy!


It's not "in the content".

I did some more reading to try to understand why Shiflett says escaping output is a security issue. In his book, he talks about examining echo and print statements and using htmlentities on database output prior to display. On his Web site, he lists two "Fliter Input. Escape Output." as his top two security prevention measures.

When inputted data is used, again, in another query, then escaping that data is a good security precaution against possible SQL injection. This would have been a better link for me to have shared. http://shiflett.org/articles/sql-injection And, that is not the case here. We are not talking about using data for a query - just printing it on the page.

Shiflett also shared a resource from the NYPHP Group. The section that is specific to escaping output starts with the title "Retrieving Data For Display in a Browser." The reason data is escaped when displayed is to handle any special characters in the content that could break the output. For example, "<rant>My rant goes here.</rant>".

Nearly everything in the layouts has been escaped, including the class parameter entered into the Administrator. The author name really shouldn't have any of those characters in it because of the input filtering on title and name fields. But, it's escaped. However, there's no escaping of the text field in the layouts, which is most likely to have that type of content.

So, I'm going to recant my previous statements and say, I am confused and don't know. I don't think this is increasing security from what I can see, so, I was definitely wrong on that.

I'll try to play around with it a bit, too. Thanks.
Amy

OK - please see this PHP Security Briefing - in particular the section on Cross Site Scripting (39, 40, 41).

The way the output is escaped in the layouts right now is consistent with good security measures. It could be done different ways. In order to make changes, though, someone should audit the individual data elements as those are captured on the frontend, filtered, used in queries, and output to the page, or another delivery method.

The datatype does matter in that one can increase security on those data elements not expected to have HTML. Such is the case with the Author Name or the Author Alias.

I see a good point raised here. Security could be handled in a different way so as to open up the application a bit more for Plugins to use those fields. Whether we like it or not, someone has to do that work and we are 100% volunteer based.

It appears to me that the Joomla! core provides a very secure (admittedly on the side of overly cautious) approach to managing the Author and Author Alias field. Those addition precautions were, indeed, added since 1.5 in an attempt to harden the application. That is good stuff - unless you happened to rely on the area that was hardened.

I'd suggest getting more involved. Either in helping to plan out 1.6's approach to input and output data handling, or in inventorying how it happens now (front to back) and proposing a patch, or getting involved with the Bug Squad - the group who volunteers lots of time to helping improve Joomla! in many ways.

Good points raised here. Thanks - it was educational.
Amy

The BBCode scripts are actually causing issues with Forum parser BBCode scripts. Both yvBBCode and RokCandy totally destroy our component as it forces the BBCode elements to html. Putting HTML editors to forums is opening a security hole in itself.
These two plugins break Agora, Kunena, ccBoard. I am not sure about the others but I think these should be avoided for the time being if at all possible.

This statement should be more general:
----
Any new extension that you install in Joomla! may cause compatibility issues with extensions that were installed earlier. It is quite a challenge to figure out what is the real cause of the problem: the "new" extension or some "old" .
So stay with what you already have as long as possible.

+ most bbcode parsers had or have vulnerabilities in converting maliciously crafted BBcode to HTML. See latest few security releases of Kunena.

Implementing a secure bbcode parser is very far from trivial.

Before applying general all-pages bbcode-to-html conversions, I would do a very serious review of their code.

Specially if those things break your page, there is a serious chance of them being vulnerable.

It is not a clean solution to this trivial Joomla 1.5.14 API bug with a trivial and secure fix as suggested by Ian on the dev list, with which I'm fine.

Do you mean Kunena has BBCode parser?
If Yes this is very good...
BTW, as I remember Information about Kunena (and the Kunena code...) is somewhat closed for the public.
Agree, thanks for the tip. I will check for the security holes found by community in HTML_BBCodeParser and for their fixes...


Of course, BBCodes won't replace "normal" HTML formatting.

BTW, if Joomla! will implement that: "Fully agreeing to sanitize output, but as html not as text"
it may sanitize BBCode output also...

Wow, this is an idea how to "sanitize" BBCode extension in general:
Just "sanitize" its output with built-in Joomla! filter, but as html not as text.

I don't understand why we talk about BB code here! This have nothing to do with user entry. It's just about allowing the plug-ins to convert a simple text to a clean html link and print it on the screen!

Let me explain to you:
The author of the first message has a problem: "Upper mentionned plugins do not work anymore since I upgraded J! from 1.5.9 to 1.5.14"

As always, there are several solutions to the problem, and I proposed two of them:
1. The fix in the template file (and there is a source code above in the topic for the fix)
2. Using BBCodes (without the example code).
To make it clear how the second proposal may me implemented in THIS case, let me show you the source code for it here, using 'zakauthor.php' plugin, mentioned above, as example.

The solution to the problem is this:
2.1. Change the code of the 'zakauthor.php' file,
Change this line:

to look something like this:


2.2. Install yvBBCode plugin and enable BBCodes replacement for the whole page. You _will_ have "clean html link"
- This is not 100% replacement (no CSS for the <a>, target="_blank"), but it creates a link.

Note: I agree with other folks here that changing template (in Joomla! core, escaping not to the text, but to HTML only...) is better solution.