[32]Save Triggers for Core Components

Locked
pasamio
Joomla! Ace
Joomla! Ace
Posts: 1318
Joined: Thu Aug 18, 2005 9:27 am
Location: San Jose, CA, USA
Contact:

[32]Save Triggers for Core Components

Post by pasamio » Mon Feb 18, 2008 2:27 am

Note: Whilst articles and com_content is heavily referenced, this can easily apply to any core component such as weblinks, menus, and more.

Introduction
One of the often requested features of Joomla! has been the implementation of content versioning (http://extensions.joomla.org/component/ ... Itemid,35/) to enable content items to be saved and stored. The need is obvious for a lot of users in government because of legistlative (legal) requirements to have older versions of the website available on request for the courts. As websites become corporate documents, more and more the requirement for record keeping has increased.

Whilst content versioning is important, an entire site must actually be version, not just the content. Important information is stored in other locations that generate content so it is imperative that more than just content is versioning, in fact the ability to version anything that goes in an out of a database table is important.

Additionally there is no ability to do "workflow" tasks for content item. For example there is no way to take a document from draft, to proofed to released.

Present Issue
There is no easy way to handle tasks like workflow and versioning as there is no easy way to intercept a save request for a component. What I propose is adding triggers to enable these requests to be monitored and even halted so that there is minimal impact on existing database structures.

Alternatives
The base problem that this is resolving is version control. The issue is that version control is a complex issue. In some cases merely keeping a copy of the old content is enough and the new vesion can go live immediately. In other cases it is required that all content be checked because even having that content out there for five minutes could cause legal risk (e.g. a user looks at a site that says that they "can do X" instead of "can not do X" and this becomes the basis of a legal defense), it only takes one person to read the information for it to become a liability.

Joomla!, the project, cannot hope to meet all requirements that might be required of the system. This is why this proposal is more interesting than hard coding a version control implementation into com_content: it allows flexibility. It allows the third party community to develop their own better implementations of version control in addition to the Core supported version control and to enable the end user to choose which one they want (or even both).

Proposal
Adding two triggers, onBeforeSave and onAfterSave, would allow developers to write their own interceptors in a manner that would be more flexible than having Joomla! write it by hand. Further more it can also form the basis of providing a complex ACL infrastructure, workflow and versioning.

Whilst the examples suggest the usage of the content component, it could be expanded to be inclusive of other components.

onBeforeSave
The onBeforeSave trigger is designed to allow the system to handle updates of content before it goes to the database. onBeforeSave would be tied to a response that would determine if the content should actually be saved. If a false has been returned by any plugin responding to an onBeforeSave event then the processing should be aborted and the content not saved. The reason why we have this behaviour is to allow for extended workflow for content. For example users may be able to update a content item but it might require another user to approve it before that new version is released to the website. Without the ability to reject a content update (the plugin rejecting should probably store it somewhere else, though it may also be an ACL plugin that rejects based on other criteria). This trigger should be passed an object (or objects) containing the new content and the old content.

The response from the content save update is an object that has three states: a default accept state, a do not save and a reject. The default accept state is used for items like simple record keeping or anything else that might go through (normal state at present). A do not save state is when an object has been saved elsewhere and the user should be directed to the normal list page with the appropriate message. The final one is a reject which means that the edit may not have been saved anywhere and has been rejected entirely and the user should be redirected back to their content item (with all of its content in tact) and a message noted as to why it was rejected.

onAfterSave
The onAfterSave occurs after the content has been saved/stored to the database. This trigger is envisaged to be used as the start of the version control system as it has past an onBeforeSave events first. This trigger should be passed an object (or objects) containing the old content and the new content.

Unlike onBeforeSave, the return value of these plugins does not matter and should not impact on the behaviour of the system from where they are called.

Implementation
Whilst in the examples the trigger call was "onBeforeSave" and "onAfterSave", this might also be renamed to "onBeforeContentSave" or "onBeforeWeblinkSave" to enable specific callbacks. Additionally enough information might be put into the generic "onBeforeSave" call to enable tests to be developed from there.

At the base level, it requires adding simple logic to handle the onBeforeSave trigger's need to reject a content save, the addition of a new plugin category ("save", since it might apply to many extensions) and a call for onAfterSave. All of this is simple code within the Joomla! Framework.

Impacts
Users
The user is presented with a slightly different interface where their content might not be saved and directly put on the web. In an environment where their content might have to be vetted before it is put live this is actually a desirable feature. The content will remain somewhere, perhaps in a third party content versioning database. It is just a matter of altering the thinking of users. This really only occurs if the user has save plugins installed.

Third Party Extensions
This change does not impact on third party extensions, however it does enabled third party developers.

Performance
There will be an impact on performance as the number of plugins increases, however this is mitigated by the fact that it will only occur when a user attempts to save an item. The true performance impact will be determined by the plugins that are triggered and what they cause the system to do (e.g. database queries).

Upgrade impacts (from 1.5.x)
The aim of this proposal is to provide as little impact to the existing database tables and to put the version control information in an external system.

For example parts of the system could be implemented by setting a new content state (e.g. -3) and creating a second table to store different versions. The published version would always have the same article id, however different values for each version could be stored seperately.

So, Article ID 4 would always have the same ID, however different versions would have a different published status (e.g. -3) and could be referenced using a secondary table (e.g. linking primary article ID, version ID and actual article ID). This would allow reuse of the core table without impacting data too much or altering the table structure. This does not proclude adding a version ID field and making this and the content ID the primary key for the table but it does give us alternative methods of thinking about things.
Sam Moffatt
Updater, Installer and Authentication Systems
JoomlaCode Backend Systems
Pie.

User avatar
instance
Joomla! Explorer
Joomla! Explorer
Posts: 302
Joined: Mon Nov 13, 2006 11:31 am
Location: Toronto, Canada

Powerful New Content Events

Post by instance » Sun Mar 02, 2008 1:39 am

What
Addition of some events related to content: onBeforeContentLoad, onAfterContentLoad, onBeforeContentSave, and onAfterContentSave.

Why
These events will facilitate a number of useful enhancements to content handling. My personal short list includes decoupling content storage from the main database, allowing new content structures, and enabling access control.

Details
Each of these events will take a JTableContent object as the first argument, and a parameter array as the second. If the article is new, the save events will include a boolean parameter, ['create']=true. The ID of a new article will be undefined at onBeforeContentSave, and will contain the new articleID at onAfterContentSave.

All events will return an integer result. The least significant bit will indicate the success or failure of the event. If ($result & 1 == 0) then the event has failed.

The "before" events will return a "continue" flag in the next most significant bit. If ($result & 2 == 0), then no call to the local database will be made to load or save the article. However in this case, the "after" event will still fire.

Change Management
The change requires overriding the JTable::load() and JTable::store() methods in JTableContent. Logic for determining if an article is new will be moved from ContentModelArticle into JTableContent::store().

Impact on Architecture
None. This doesn't even affect the API.

Implementation
The implementation is quite simple. Overriding load() and save() to inject the event triggers is all that is required.
I have surveyed the code for all uses of JTableContent. Only the use in ContentModelArticle needs to be changed.
The most significant impact is testing, since this represents the heart of the system.
==> Please do not PM me for support issues. <==
Alan Langford -- Joomla Security Strike Team, Extension Developer, Hosting Guy
Biz: http://www.abivia.net
Blog: http://www.ambitonline.com/nextrelease

pasamio
Joomla! Ace
Joomla! Ace
Posts: 1318
Joined: Thu Aug 18, 2005 9:27 am
Location: San Jose, CA, USA
Contact:

Re: Powerful New Content Events

Post by pasamio » Sun Mar 02, 2008 4:31 am

The load triggers are an interesting concept, allowing custom load ACL, which would be cool. The save triggers are something I've considered already here, albeit I want to move to more than just com_content (I feel you could do the same with the load triggers as well): http://forum.joomla.org/viewtopic.php?f=500&t=266296

The main limitation I see with your model is that you don't provide a method to easily handle message passing. If you are simply returning a value how does the user know why they can't do something (be it load or save). Perhaps their work hasn't been saved because its actually in a different table for 'drafts' because of work flow reasons. The users work has actually been saved, just not to com_content's table. Sure we could use mainframe to handle messages but we're jumping to the top layer without providing the UI the ability to intelligently handle it (e.g. what happens if you hit it via XML-RPC?).

The other question is do you put this in the actually JTableContent (which enforces it on everyone) or do you put it in the actual component. If you put it in the component you can avoid the triggers in other things like workflow tools. For example say I want to have an approval process, something that jos_content doesn't have the ability to handle (e.g new versions have to be approved before they are put on the website which isn't uncommon) so I put it into a new table. In my own component when I want to move it into the new table, how do I handle this? I end up having to avoid trapping my own event (which isn't too hard, via defines), but does it then get trapped by other events? That pathway leads to an ugly mess where due to the combination of plugins you have installed you never get anything saved because they all prevent you do something.

Just some thoughts, my own opinion (obviously) is to dump it into the component itself and make it obvious that when its edited from the component that this applies but not all of the time. Otherwise people will just reinvent the wheel and write their on JTableContent (or worse) and in that case, what is the point because they're subverting the system anyway.

Sam
Sam Moffatt
Updater, Installer and Authentication Systems
JoomlaCode Backend Systems
Pie.

User avatar
instance
Joomla! Explorer
Joomla! Explorer
Posts: 302
Joined: Mon Nov 13, 2006 11:31 am
Location: Toronto, Canada

Re: Powerful New Content Events

Post by instance » Sun Mar 02, 2008 5:17 am

pasamio wrote: The other question is do you put this in the actually JTableContent (which enforces it on everyone) or do you put it in the actual component. If you put it in the component you can avoid the triggers in other things like workflow tools. For example say I want to have an approval process, something that jos_content doesn't have the ability to handle (e.g new versions have to be approved before they are put on the website which isn't uncommon) so I put it into a new table. In my own component when I want to move it into the new table, how do I handle this? I end up having to avoid trapping my own event (which isn't too hard, via defines), but does it then get trapped by other events? That pathway leads to an ugly mess where due to the combination of plugins you have installed you never get anything saved because they all prevent you do something.
Unfortunately I think the correct answer is "both of the above", there are good reasons to trigger in com_content, and there are good reasons to trigger at the data store level (for example let's say we want to compress content before storing it in a remote DB). The biggest challenge there will probably be one of nomenclature; do we create onBeforeContentSave, onBeforeContentStore, ...Fetch, and ...Load? Yuck. Hard to make that non-confusing.
pasamio wrote: The main limitation I see with your model is that you don't provide a method to easily handle message passing. If you are simply returning a value how does the user know why they can't do something (be it load or save). Perhaps their work hasn't been saved because its actually in a different table for 'drafts' because of work flow reasons. The users work has actually been saved, just not to com_content's table. Sure we could use mainframe to handle messages but we're jumping to the top layer without providing the UI the ability to intelligently handle it (e.g. what happens if you hit it via XML-RPC?).
This highlights something that I don't think we'll solve in 1.x. This is really a topic for another paper, but we need to do some work on sequencing and exceptions every time more than one observer attaches to an event. Sooner or later we'll have to implement exception, rollback, and dependency logic ("my component needs to run with priority n; before components x, y, and z; and before a, b, and c", etc.).

Back in the 1.5 / 1.6 world, there are lots of powerful things we can do and the problems identified so far are reasonably easy to solve if we don't try to handle conflict resolution. For example, we can allocate a bit in the result integer for "yes, this has been saved" and pass it along the observer chain. As long as one of the handlers has saved the content, the end result can be "success with warnings". We can also easily hand back a richer result object that contains the flags and supporting messaging information.
==> Please do not PM me for support issues. <==
Alan Langford -- Joomla Security Strike Team, Extension Developer, Hosting Guy
Biz: http://www.abivia.net
Blog: http://www.ambitonline.com/nextrelease

pasamio
Joomla! Ace
Joomla! Ace
Posts: 1318
Joined: Thu Aug 18, 2005 9:27 am
Location: San Jose, CA, USA
Contact:

Re: Powerful New Content Events

Post by pasamio » Sun Mar 02, 2008 5:49 am

instance wrote:
pasamio wrote: The other question is do you put this in the actually JTableContent (which enforces it on everyone) or do you put it in the actual component. If you put it in the component you can avoid the triggers in other things like workflow tools. For example say I want to have an approval process, something that jos_content doesn't have the ability to handle (e.g new versions have to be approved before they are put on the website which isn't uncommon) so I put it into a new table. In my own component when I want to move it into the new table, how do I handle this? I end up having to avoid trapping my own event (which isn't too hard, via defines), but does it then get trapped by other events? That pathway leads to an ugly mess where due to the combination of plugins you have installed you never get anything saved because they all prevent you do something.
Unfortunately I think the correct answer is "both of the above", there are good reasons to trigger in com_content, and there are good reasons to trigger at the data store level (for example let's say we want to compress content before storing it in a remote DB). The biggest challenge there will probably be one of nomenclature; do we create onBeforeContentSave, onBeforeContentStore, ...Fetch, and ...Load? Yuck. Hard to make that non-confusing.
Yes, its one of the hard ones and I agree the correct answer is both because we really want both but it is a real mess to have both. Hard to work out where we put the mess as well.
instance wrote:Back in the 1.5 / 1.6 world, there are lots of powerful things we can do and the problems identified so far are reasonably easy to solve if we don't try to handle conflict resolution. For example, we can allocate a bit in the result integer for "yes, this has been saved" and pass it along the observer chain. As long as one of the handlers has saved the content, the end result can be "success with warnings". We can also easily hand back a richer result object that contains the flags and supporting messaging information.
I'd personally prefer some extra information because it also makes life easier debugging things and communicating with the user what has really happened.
Sam Moffatt
Updater, Installer and Authentication Systems
JoomlaCode Backend Systems
Pie.

User avatar
Jick
Joomla! Explorer
Joomla! Explorer
Posts: 355
Joined: Fri Aug 12, 2005 10:06 am
Contact:

Re: Powerful New Content Events

Post by Jick » Sun Mar 02, 2008 7:13 am

I once needed a plugin that created a thumb of the last image in a content item on article save in the administrator and than remove te <img from the article. This required a system plugin with some checks for com_content and task=save before execution.

It would be great to have some plugin triggers in the admin.

Arno

User avatar
Beat
Joomla! Guru
Joomla! Guru
Posts: 840
Joined: Thu Aug 18, 2005 8:53 am
Location: Switzerland
Contact:

Re: Save Triggers for Core Components

Post by Beat » Mon Mar 03, 2008 9:50 am

Excellent suggestion, independently of core-version tracking etc.

If we could add onBefore and onAfter events to all save actions which potentially affect the site it could solve many extensions-problems in an elegant way:

- content versioning
- workflows / approvals
- authorization
- "watermarking"
- and a lot more

Triggering events are not impacting performance when no plugins are using the event (compared to all the other abstraction layers and many files inclusions, which are not necessarily in RAM cache, used in Joomla 1.5) and greatly improving clean extensibility of the core.

In addition, it's not a huge task with large benefits.
Beat 8)
www.joomlapolis.com <= Community Builder + CBSubs Joomla membership payment system - team
hosting.joomlapolis.com <= Joomla! Hosting, by the CB Team

User avatar
ircmaxell
Joomla! Ace
Joomla! Ace
Posts: 1926
Joined: Thu Nov 10, 2005 3:10 am
Location: New Jersey, USA
Contact:

Re: Save Triggers for Core Components

Post by ircmaxell » Mon Mar 03, 2008 12:53 pm

Beat wrote:Triggering events are not impacting performance when no plugins are using the event (compared to all the other abstraction layers and many files inclusions, which are not necessarily in RAM cache, used in Joomla 1.5) and greatly improving clean extensibility of the core.
This is not true. Any event trigger in 1.5 has an impact in performance. This is because of the way events are fired. All plugins are grouped into one dispatcher. When you fire an event, the dispatcher needs to loop through every plugin, and check if it has that method. Since a lot of sites use a fairly large number of plugins (> 25), this loop is not insignificant. Adding arbitrary triggers is not good for performance.
Anthony Ferrara - Core Team - Development Coordinator - Bug Squad - JSST

http://moovum.com/ - The Bird is in the air! Get Mollom Anti-Spam on your Joomla! website with Moovur...
http://www.joomlaperformance.com For All Your Joomla Performance Needs

User avatar
Beat
Joomla! Guru
Joomla! Guru
Posts: 840
Joined: Thu Aug 18, 2005 8:53 am
Location: Switzerland
Contact:

Re: Save Triggers for Core Components

Post by Beat » Mon Mar 03, 2008 3:07 pm

ircmaxell wrote:
Beat wrote:Triggering events are not impacting performance when no plugins are using the event (compared to all the other abstraction layers and many files inclusions, which are not necessarily in RAM cache, used in Joomla 1.5) and greatly improving clean extensibility of the core.
This is not true. Any event trigger in 1.5 has an impact in performance. This is because of the way events are fired. All plugins are grouped into one dispatcher. When you fire an event, the dispatcher needs to loop through every plugin, and check if it has that method. Since a lot of sites use a fairly large number of plugins (> 25), this loop is not insignificant. Adding arbitrary triggers is not good for performance.
Thanks for this precision. Didn't realize that things went that way in Joomla 1.5. Joomla 1.0 triggers were quite efficient in firing, and my (wrong) assumption was that it stayed efficient in 1.5. It seems that I was wrong, sorry about that.

Still believing that for all "Save" actions, triggers are very much ok.
Beat 8)
www.joomlapolis.com <= Community Builder + CBSubs Joomla membership payment system - team
hosting.joomlapolis.com <= Joomla! Hosting, by the CB Team

User avatar
instance
Joomla! Explorer
Joomla! Explorer
Posts: 302
Joined: Mon Nov 13, 2006 11:31 am
Location: Toronto, Canada

Re: Powerful New Content Events

Post by instance » Sat Mar 08, 2008 6:07 am

Louis has (correctly) observed that triggering an even on every load would have a large, negative effect on performance. This idea won't fly wit the current event architecture.
==> Please do not PM me for support issues. <==
Alan Langford -- Joomla Security Strike Team, Extension Developer, Hosting Guy
Biz: http://www.abivia.net
Blog: http://www.ambitonline.com/nextrelease

pasamio
Joomla! Ace
Joomla! Ace
Posts: 1318
Joined: Thu Aug 18, 2005 9:27 am
Location: San Jose, CA, USA
Contact:

Re: Powerful New Content Events

Post by pasamio » Sat Mar 08, 2008 1:48 pm

However whilst the load triggers won't work properly, the save triggers should be fine as they only occur when we actually do a content save, which is not as often as compared to a load. Save operations have other things running as well, so the slow down impact isn't as bad putting it into the load.
Sam Moffatt
Updater, Installer and Authentication Systems
JoomlaCode Backend Systems
Pie.

yudiset
Joomla! Fledgling
Joomla! Fledgling
Posts: 4
Joined: Mon Jul 24, 2006 9:33 am

Re: [32]Save Triggers for Core Components

Post by yudiset » Mon Jul 21, 2008 3:45 am

Hi all,

I'm a newbie to Joomla.
I'm using it for my work and I find it a very nice piece of software :-)

I'm wondering about this proposal, how's the progress?
In case you need some tester ... I'm on ! :-)

Thank you.

pasamio
Joomla! Ace
Joomla! Ace
Posts: 1318
Joined: Thu Aug 18, 2005 9:27 am
Location: San Jose, CA, USA
Contact:

Re: [32]Save Triggers for Core Components

Post by pasamio » Mon Jul 21, 2008 3:51 am

Parts of the save triggers are already implemented in 1.5.4 for the com_content component, but not other components. You will have to wait until 1.6 to test this feature. I'm not sure what the current status of save triggers are in the development cycle however, its on the list.
Sam Moffatt
Updater, Installer and Authentication Systems
JoomlaCode Backend Systems
Pie.


Locked

Return to “Accepted - Archived”