Extending metadata of Articles (Poor implementation of content metadata saving)

[yvolk]

Goal: Extend content with additional metadata ("meta tags")
The same wish was already articulated in forum, e.g.:

I'm currently evaluating Joomla 1.5 for a future project and I hope somebody can help me with custom metatags - the project requires the content creation date and last modified date to be in the metadata amongst other things.

I have (either rightly or wrongly) found that:
1) editing the metadata parameters group in the administrator/components/com_content/models/article.xml file will give new metadata fields when editing an article which will then appear in the head section of the document if they're not empty.
2) editing the libraries/joomla/document/html/renderer/head.php file can be useful for adding custom metatags which are either constant or created programatically.

I have searched the forum/googled and found that there is a getModifiedDate function, but when I try and use it in the head.php file it always returns an empty string - unfortunately there doesn't appear to be a similar function for the created date.

I'm hoping that I can use the data stored about the article in the database table rather than recreate it by using the xml file to define the same data again.

and also:

First I must say a great solution to implement a separated metadata field in the article table! In this way the META data can be extend anyway and is automatically presented within the view.html.php within the article view. Just great.

Looking at the Article META data structure then I see that the structure is read from the article.xml from the models directory under com_content. This limits the extension of the META data structure to the articles. Sometimes you want to extend the META structure for specific purposes. Anyway it should be nice if this META data structure could be extended easily, for example with a separate XML just for the META data where third party XML meta structure could be added. This could be a "general" META XML structure used for all Joomla items (menu elements, articles, FAQ, etc..). 

Looking at the MENU elements there is no metadata field in the menu table. Make this work the same as within the article structure (by using a generic XML meta file for all core Joomla elements).

What I'm also missing is a separate field in the Joomla items (= articles, menu, FAQ, etc..) where I can put a HTML title in. Note that this is different element then the page title which should appear in the HTML body section (with any tag the designer wants to assign to).

Also notice that when that the META fields are present when they are empty. Please don't display the META field if they don't have any content.

Problem: Current implementation of Content Editors ("Article Manager"...) in Joomla! core erases any metatags, that are unknown to the Editors, so any additional metatags are not preserved after editing of the Article.
And so metadata field is useless for extentions.

1. As you may see in the saveContent function in file "www/administrator/components/com_content/controller.php", Joomla! just receives request with data from Client, packs received "meta[tags]" into metadata
field and stores that field: it doesn't care about anything, that may be stored in
metadata field earlier, acting as if in knows everything 
2. To the addition of above, it appears, that Editor receives only metatags, that are mentioned in
"/www/administrator/components/com_content/models/article.xml" file, i.e. only two meta tags for "metadata" field of content table: "robots" and "author".
I understand, that I may manually add new metatags to that "article.xml" file, but this would be hacking Joomla! core and not extension 
Proposal: As long as there is no way for Joomla extensions to extend some "list of known meta tags" of the Article (e.g. "/www/administrator/components/com_content/models/article.xml"), I propose to change Joomla! core code so, that during saving of metadata (both from backend and from frontend) all "meta tags" with non empty values will be preserved, even those, that are unknown to particular content Editor.

[Hackwar]

Hi yvolk,
I have to admit, that I don't completely understand what you are trying to achieve. Are you trying to add something like

Code: Select all

somemetatag=somedata

into the meta field in the article editing screen or are you trying to add new fields via the article.xml? In both cases I have good and bad news for you. The bad news is, that the current code and the current database structure is not able to do what you want. The good news is, that you would very well be able to achieve at least what caveywavey wants with a content plugin. That plugin could get the date of the article, category or section and write that as new meta tag in the head area.

This is not a bug and we will not be able to implement this in 1.5. I'm moving this to the normal Joomla 1.5 forum in the development area.

[yvolk]

Hi yvolk,
I have to admit, that I don't completely understand what you are trying to achieve. Are you trying to add something like

Code: Select all

somemetatag=somedata

into the meta field in the article editing screen or are you trying to add new fields via the article.xml?

Yes, I want to store "somemetatag=somedata" in "metadata" field of the "content" table.

Ok, let's be more specific.
1. I've developed content plugin, yvComment, that have an option to use content table to store comments.
2. Now I want to add one additional field: "created_by_link", - to store link to the Website of the guest, who posted this comment. Of cause, you saw similar option in many commenting systems.
3. I guess, that metadata field of content table is the right place to store this additional information (as another one meta tag), and Joomla! database structure IS able to store as much metatags (tags and their values), as I may need (this is TEXT field, so it has plenty of room ).
I implemented this saving additional meta tag in "metadata" field of Content table easily, and it works ok as long as I decide to edit this comment with "General purpose Article editor": in this case my custom metadata is erased.
As I see the code of, e.g. Article Manager in the backend, it may be easily changed to preserve any additional metadata...
4. Current behaviour of Joomla! core, in fact, locks "metadata" field from any Joomla! extension, so I called this "poor implementation" and I'm sure my proposed changes will help make at least one small step towards content (Article) extensibility. You may read about extending com_content here, and in the messages, referenced from it.

Did I expressed myself clearly?

[pixelsoul]

Joomla! database structure IS able to store as much metatags (tags and their values), as I may need (this is TEXT field, so it has plenty of room ).

Umm why would you ever want to store comments + tags in the article field ?

[yvolk]

Joomla! database structure IS able to store as much metatags (tags and their values), as I may need (this is TEXT field, so it has plenty of room ).

Umm why would you ever want to store comments + tags in the article field ?

That is a subject of another thread. Please start reading from: Reply #10 to 'yvComment - MVC Plugin-Component for Joomla! 1.5'. We may discuss this subject there...
In this thread I think, that it is obvious, that metadata field may be used to store more, than two "meta tags" (author and robots). Great value of "metadata" approach from global point of view is exactly its extensibility. "Content" (Article) is central part of Joomla site, so handling and extending its metadata is also very important.
And this is why I ask Joomla! Dev team to unlock "metadata" field for Joomla! extensions 

[masterchief]

The "intention" of the unified metadata field was to provide a basis for custom/configurable user metadata.  Basically the idea would be to allow you to drop in an xml file that defines whatever standard or home-grown scheme you want (Dublin core, etc) and store it in a params (that is, ini) format.  We just ran out of time to implement it in 1.5.

[dchapiesky]

XML... excellent.  This is exactly what I've been wanting to do.  One problem though... 

When you do impliment this...  you will need to publish an xml schema for using this metadata field... something like:


 
     
 
 
     
 


So that when a new extension which makes use of the field is added, it does not stomp on another extension which also uses the field.

This could be done in a core API class pretty easily....  getUnifiedMetaData(ExtensionName) - putUnifiedMetaData(ExtensionName)

Daniel

[masterchief]

Actually I was thinking something more along the lines of:



 



and each component would support variants for their own content types.  That's the thought anyway.

[yvolk]

The "intention" of the unified metadata field was to provide a basis for custom/configurable user metadata.  Basically the idea would be to allow you to drop in an xml file that defines whatever standard or home-grown scheme you want (Dublin core, etc) and store it in a params (that is, ini) format.  We just ran out of time to implement it in 1.5.

1. Now we know, that it is right move to use "metadata" field of "content" table to store some additional meta tags (in ini format). (BTW, this field was added by masterchief on 03-Jun-2006 - more, than a year ago...)
2. The whole framework to extend "metadata" field (through XML files) is not implemented yet and will not be done at least in the forthcoming release of Joomla!
3. So, we have at least three simple ways on this subject:
3.1 Don't use "metadata" field at all and wait for the great future, when full support of extending Article metadata will be implemented by Dev team.
3.2 Each administrator on each site: Manually hack (edit) that administrator/components/com_content/models/article.xml file (and, maybe, something else) to include new custom meta tags.
3.3 Ask Joomla! team to make small fix in the way metadata field is stored now: so custom metatags wouldn't be erased by Joomla! Content Editors (Article Manager...). And add custom meta tags to the metadata field without any framework, XML support, etc...
That "small fix" may be implemented in such a way, that it would prevent Joomla! extensions to erase metadata, unknown to them - just the same way as for Joomla! Content Editors (e.g. injecting this fix into, e.g. JTableContent class...)

I propose the third way, that gives us opportunity to extend Content (Articles) NOW!

What do you think, Community?

[ianmac]

The "intention" of the unified metadata field was to provide a basis for custom/configurable user metadata.  Basically the idea would be to allow you to drop in an xml file that defines whatever standard or home-grown scheme you want (Dublin core, etc) and store it in a params (that is, ini) format.  We just ran out of time to implement it in 1.5.

1. Now we know, that it is right move to use "metadata" field of "content" table to store some additional meta tags (in ini format). (BTW, this field was added by masterchief on 03-Jun-2006 - more, than a year ago...)
2. The whole framework to extend "metadata" field (through XML files) is not implemented yet and will not be done at least in the forthcoming release of Joomla!
3. So, we have at least three simple ways on this subject:
3.1 Don't use "metadata" field at all and wait for the great future, when full support of extending Article metadata will be implemented by Dev team.
3.2 Each administrator on each site: Manually hack (edit) that administrator/components/com_content/models/article.xml file (and, maybe, something else) to include new custom meta tags.
3.3 Ask Joomla! team to make small fix in the way metadata field is stored now: so custom metatags wouldn't be erased by Joomla! Content Editors (Article Manager...). And add custom meta tags to the metadata field without any framework, XML support, etc...
That "small fix" may be implemented in such a way, that it would prevent Joomla! extensions to erase metadata, unknown to them - just the same way as for Joomla! Content Editors (e.g. injecting this fix into, e.g. JTableContent class...)

I propose the third way, that gives us opportunity to extend Content (Articles) NOW!

What do you think, Community?

I STRONGLY STRONGLY disagree.  I know it may seem like a very small thing, but it really is too late in the game to do that.  We are at RC2, hoping to move to RC3 at some point.  It may seem like a no brainer change, but changes like this can break other things.  Additionally, there are many little improvements like this that could be made, and if all of them were implemented, we wouldn't get a release until 2020.  I think it is important that we cut off feature requests and just get the sucker out the door.

Ian

[yvolk]

I STRONGLY STRONGLY disagree.  I know it may seem like a very small thing, but it really is too late in the game to do that...

Thank you, Ian, for your reply... but let's see, what we are really facing in this particular case:
1. There is a new feature in Joomla! ("the unified metadata field"), that was implemented only partially: the field was added to the database and one component: com_content, - was equipped with its XML file, that contains info about two meta tags, with which Content Editors should deal...
2. But even this partial implementation is buggy: it works not how it was intended: when com_content stores its metatags it overwrites all metatags, instead of preserving everything, that doesn't belong to it (i.e. it has to preserve all metatags, except its two)!
3. So I'm not talking about "adding new functionality", which is not implemented yet, but about fixing the erroneous behaviour, that existent implementation already has.
4. Today it is safe to change this behaviour, because no extension relies upon metadata field (there was no any release of Joomla!, that had it), but if Dev team will delay this fix till next major release, this may cause some incompatibility with existent components (because some of them may depend (rely upon) this erroneous behaviour).

In this situation, fixing this bug is just like fixing any other bug: it should be done, even though there is always a chance that one fix may introduce new bugs  - life is life!

[ianmac]

I guess what I'm getting at is that there has to be a point where you cut something off and say, these are all the features we're going to have, and no more, otherwise we never get to release.

there was no any release of Joomla!, that had it

Yes, there have been two releases so far - RC1 and RC2.  We tread carefully before we change behaviour here.

Ian

[masterchief]

I can confirm this won't be shipping in 1.5.  As Ian says we are just too late in the game, and if we add one small, simple thing for one person then we open the flood gates for a multitude and we never actually *finish* a release.  Adding a panel to edit metadata is one thing - but it dominoes down the chain as it requires implementation in components and template outputs, requires documentation, debugging, attending to forum questions about it when it doesn't work, or attending complaints from people that didn't like the way we implemented it ... and the list goes on.

I've given you my current thought processes simply to be a guide on how we expect to use that feature when it's implemented.  That's the best I can do at the moment.  What I suggest you do is work on the whole "patch" that would be required, test it thoroughly, and then submit it for us to consider dropping directly into 1.6 having already been road tested to a degree.

[yvolk]

I see, that we're talking about different things:

I guess what I'm getting at is that there has to be a point where you cut something off and say, these are all the features
we're going to have, and no more, otherwise we never get to release.

I'm proposing fix in existing code, no new features . This existing part of code, that I propose to change, is small and it was done by someone in a hurry (it even has a bug from cut and paste from similar portion of code. See below for details).

Adding a panel to edit metadata is one thing - but it dominoes down the chain...

Of cause, I don't propose, that Dev team add anything. So I even don't want to discuss that "panel to edit metadata" etc.
What I'm talking about, is part of code inside saveContent function in the file "www/administrator/components/com_content/controller.php". Here they are, 15 lines of code:

Code: Select all

    $metadata = JRequest::getVar( 'meta', null, 'post', 'array');
    if (is_array($params))
    {
      $txt = array();
      foreach ($metadata as $k => $v) {
        if ($k == 'description') {
          $row->metadesc = $v;
        } elseif ($k == 'keywords') {
          $row->metakey = $v;
        } else {
          $txt[] = "$k=$v";
        }
      }
      $row->metadata = implode("\n", $txt);
    }

The fix is, basically, to start not from empty "$txt = array()", but from array, retrieved from database (in order to preserve existing metatags).
That's ALL!
I'm sure it is not late to make such fix now?

What I suggest you do is work on the whole "patch" that would be required, test it thoroughly, and then submit it for us to consider dropping directly into 1.6 having already been road tested to a degree.

I may promise to make fix in the scope of the above cited code. Two weekdays from the time you agree

[dchapiesky]

masterchief:

What I suggest you do is work on the whole "patch" that would be required, test it thoroughly, and then submit it for us to consider dropping directly into 1.6 having already been road tested to a degree.

yvolk:

I may promise to make fix in the scope of the above cited code. Two weekdays from the time you agree

This kind of interaction of the J! Community and the J! Developers is exactly what should happen.  I agree with masterchief that if you think you can fix something, go ahead and do it!  That yvolk requests an agreement before hand is also great because it keeps everyone on the same page.

I have noted on some threads involving LDAP, this kind of interation is already taking place.

I am conflicted though about ianmac's comments about:

I guess what I'm getting at is that there has to be a point where you cut something off and say, these are all the features we're going to have, and no more, otherwise we never get to release.

The conflict: on one hand I completely understand his reasoning - we definitely want to get to a stable release; on the other hand, this kind of comment can discourage capable developers from contributing to J! development.

I'm not saying go hack the core and send in your changes whenever you feel like it... but as masterchief did, encouraging people to contribute in a coordinated fashion is a valuable tool which the core team should also encourage with one voice.

Daniel

[ianmac]

The conflict: on one hand I completely understand his reasoning - we definitely want to get to a stable release; on the other hand, this kind of comment can discourage capable developers from contributing to J! development.

I didn't in any way intend to discourage capable developers...  I think the change is a great idea and is a great feature to add in a future release.  I just don't think that now is the time.  Members of the dev group are already getting discouraged because as they are trying to debug and fix things, other features get added and it pushes them backwards.

I wasn't responding as a dev group member - this should be obvious because I am not one.  I was replying to the question:

I propose the third way, that gives us opportunity to extend Content (Articles) NOW!

What do you think, Community?

I was replying especially to the 'NOW' part.

I think it is great when developers offer suggestions and code and tips - I do myself, as an outsider of the dev group.  But I don't think it is helpful to try and rally the community against the devs to try and change a decision that has been made.  I don't want to test the dev's patience and upset them - I recognize the great things they've done and am grateful for them.

So, all, please continue to submit patches and suggest new features, but remember to be patient and know that not everything will get jammed into the current release.

Let's sail this ship into port!

I do apologize if I offended anybody...  I was asked, as a community member to express my opinion and I did so.  Please don't see it as anything more than that.

Ian

[dchapiesky]

ianmac:  please accept my appologies - I believe that I may have used terms which may have been too strong. 

My idealistic goal is to see 1.5 finished and out the door.  But I also want to see the appropriate hooks for future development in 1.5, otherwise we just be looking at another overhaul for 1.6-2.0

It was is my mind that people should chuck as much contributions as possible and the Dev team could then cherry pick the best. 

Daniel

[ianmac]

It was is my mind that people should chuck as much contributions as possible and the Dev team could then cherry pick the best. 

Sure, except I think the time for that for 1.5 is past...  that's all.

Ian

[yvolk]

I propose the third way, that gives us opportunity to extend Content (Articles) NOW!

What do you think, Community?

I was replying especially to the 'NOW' part.

We don't understand each other just because I'm trying to find words to persuade Dev team to let me change exact lines of code for exact purpose, and you're fighting with "such late proposals", not paying attention at what exactly I propose in this one.
I already submitted two large patches to the Joomla! (one of them - JTransliteration class - in April!, see  Transliteration implemented, v.003 - there was the same talk about "submit your patch" and after I did it - "too late"  . And that contribution is not still in work...)
So you, pals, may understand, why I'm so persistent now  ...

Ok, now we're waiting for the Dev team?!

[masterchief]

Ok, let me be clear.

Please don't stop tickering with the patch.  You need to think about how the metadata is defined (that is, where the xml is stored), how people can configure it for their own purposes, that any configuration files will survive an update (in other words, won't get overwritten when an official Joomla! patch is issued), and then you need to find where is the best place to implement the metadata in the frontend - to be honest I'd have to look hard myself but it will be somewhere in JDocumentHTML I think.

Then, wait for us to finish 1.5 and see what gets messed up because something changed slightly in the API from when you started.  Then see if it meets the needs of people who are generally asking for this type of feature - trying to find the middle road between people who don't care and the purist zealots at the other extreme who will disagree with your implementation no matter what you do.

After that, it should be ready to submit for consideration in 1.6

So to be perfectly clear, we are generally only dealing with bugs and other small things that will generally help Q&T do their job.  If you want to get an idea of what we consider "in" and "out" at the moment, follow the Dev Google Group - you should get a handle on what we are actively attacking, and what we are holding back on.

This is a good suggestion, and I would have liked it to be included but we have to consider that Q&T have a very difficult job keeping up with the necessary changes that are occurring.  This may be one "quick win" but if we allow yours, we have to allow others, and there are lots of them (you should see my list).

[yvolk]

Please don't stop tickering with the patch...

Thank you, masterchief. I understand your grand style as: "Please don't let this patch to open the path of using that metadata field of the Article in a way, that may be incompatible with Joomla! v.1.6 (or later...)".
I don't think this is a good approach of "step by step implementation" ... and this unfixed bug won't prevent usage of the field.

So, I'll write something like this in the documentation of my component:

If you enabled guests to add comments, and you want to allow guest to add link to his (or her) Web page to the comment, you have to set "Allow guests to link to a site" option also. This link is stored in the "metadata" field of the comment.
Please note, that due to the known bug in Joomla! core (see Topic: Extending metadata of Articles (Poor
implementation of content metadata saving)) if you edit existing Comment from
the backend using Joomla! "Article Manager", this link would be
erased. So don't use "Article Manager" to edit comments, made by guests, in
conjunction with this option.

[jpatti]

I'm not sure those in this thread understand that this is not a feature request, but a needed bug fix. Let me try to explain what happens as I use Yuri's extension for comments.

The extension writes a bit to the meta field, as extensions are supposed to be able to do. Specifically, it writes the user's URL so their name in the comment can be linked back to their site. Forget how this is implemented and XML and all that, all it does is write to the field.

On my site, users post comments using a guest account I set up in Joomla. The posted comments are not approved until I review them, so they are stored in Joomla as articles that are unpublished. Because my site doesn't have a front-end (I have no reason to have user accounts), I have been approving comments from the back-end using article manager. I go in and mark them all as published in the back-end. However, because of a bug, this deletes the data Yuri's extension had written to the database.

Apparently, this does not occur when the article is edited from the front-end, only the back-end article manager decides to delete data it doesn't own in this manner. The front-end and back-end article managers do not work the same way; so this is definitely a bug. Even if you completely disagree with Yuri, then you should be fixing the front-end article manager to *also* overwrite meta data! One way or another, this is a bug.

Apparently for me, my only work-around to avoid this bug is to publish a login box on my site for no one but me to use and thus be able to mark these articles as published from the front-end. I expect this to be confusing for users as I don't have user accounts on my web site. Guess I'll post an unlinked page to hide it on, but this really isn't a very elegant solution.

This thread frustrated me. Yuri has posted the code to fix this at http://joomlacode.org/gf/project/joomla ... 709several

It's a bug and you've got the code to fix it, so what the heck is the problem?

[myradesign]

Is this bug fixed in the latest release?

[ooffick]

Mod note: Please open your own topic, if you have a specific question about something, This topic is almost 2 years old.

[yvolk]

Mod note: Please open your own topic, if you have a specific question about something, This topic is almost 2 years old.

Hi ooffick,
the most important thing is exactly that this bug is two years old and is not yet fixed

@myradesign: Yes, I've just looked into the code - no good news...

[ooffick]

the most important thing is exactly that this bug is two years old and is not yet fixed

As far as I see the, it is not a bug, but a new feature request. But that was already discussed above. Did you look into Joomla 1.6 yet?

Olaf

[myradesign]

Understood, Olaf. Thanks for checking yvolk.

Myra

[davidjay]

Was this ever resolved?

I an hoping to add a custom field to the article editor. I want to provide a place for my users to place a link to an image that Facebook will reference when a visitor to the site shares the page.

Anyone have any info on how I could accomplish this?

[andypooz]

To bring this back to the original topic, can i ask someone for a good example of how I can 'use' this metadata? I have set up additional metadata fields in articles in 1.6, and I want to use them to further filter the results of a latest news module. What is the syntax for getting, say, the value of metadata key 'example'? I have heard people referring to 'exploding' these values- any clues?

@davidjay

The file you want to edit is edit_metadata.php in administrator/components/com_content/views/article/tmpl
This was in 1.6, so I'm not sure about 1.5 if that's what you're using

[davidjay]

I'm running 1.5 but I'll look into it. Thanks.