Joomla! Discussion Forums

Hi folks,

having played around quite a bit with 1.5RC2 I finally figured out, that Joomla! seems to have been design for creating dublicate content to a large extent. If you want to use all those funky article linking features like show intro text, show read more, title linkable, section linkable, category linkable, you creaze a lot of duplicate content. At this point it doesn't matter if you're using parameter URL's or SEF-URL's. The SEF option just rewrites URL parameters into folders and/or files, that's it.

Starting with that incredible "Home" link in each and every main menu for your front page (/home resp. /index.php), which should simply link to the websites root "/" instead of anywhere else, I am wondering, if I have completely misunderstood the whole site structure theory of Joomla! at all? I did some research and figured out, that this really seems to be a major issue. There're quite a lot of webmasters out there asking the same questions. However, the core team as well as most of the old hands in this forum seem not a bit to care about the issue. There's hardly any usable information to be found of how to avoid all that duplicate content. C'mon guys, are you really accepting all that DC for your websites? I can't believe that. How can I create unique article URL's? How can I create unique listing (section, category) links without featuring the same content multiple times?

Thank you very much for any tricks, comments and hints. And yes, I know that 1.5 is still in RC mode and should not be used for production sites.... However, the plague seems to be sticking in the core already. This should've nothing to do with bug and issue fixing until the stable version.

Bunjip.

Download RC 3. I follow this particular concept very closely and have been for some time. A few weeks ago, Johan completely got rid of the last bit of system generated duplicate content.

Joomla! v 1.5 RC3 does *not* duplicate content for intro text, read more and title linkable, (which, by the way is three ways of saying the same thing ).

A blog and an article is a common representation of a summary and detail look at a blog. This is no difference between this and any other blogging software, beit WordPress, Drupal, etc. There is a summary page, and a permanent URL. This is not duplicate content.

In the 1.0.x series, there was some system duplicated content at the article level for Frontpage and the article, but that is gone in v 1.5.

Searches, frontpage, Most popular, any module for that matter, category, section blogs, section lists, can all be used to create a unique and permanent URL without the system duplicating these values. Spend some time with RC 3 and celebrate our good fortune at having developers who really get it and really care.



Yes, I think you have completely misunderstood. The home links work perfectly fine in RC3.



If you cannot find information on this topic, then honestly, you have not looked very well at all. I cannot think of any other topic that has had more airplay. Perhaps for your first post, you might have *asked* instead of accused people of "not caring." That is simply not nice.

Moving forward...

To avoid duplication content in Joomla! v 1.5, do this:
1. Do not use the PDF, email to friend or print icons in the article.
2. Do not link an article to a menu item more than one time, if you are using a section, category or article link.
3. If you must link to an article from a second menu item, use the "URL" menu type and an existing URL.

The end! That's all you have to do to avoid creating duplicate content in Joomla! v 1.5. 



See above for avoidance of DC.



No real plague. Just mistaken thinking. RC 3 is in good shape. Hope that helps!
Amy

Hi Amy,

thanks a lot for your quick and comprehensive reply. First of all I feel, I need to apologize: I did not want to accuse anybody, especially nobody of the hard-working core developer team, at all. If my words appear a bit harsh, this might be caused by the fact, that I'm not a native speaker  . Sometimes I look for a decent expression to keep my posting short but full of content. In deed, with my previous posting I wanted to figure out, if really nobody else cares about that DC stuff, that - you need to confess - used to be a severe issue up until RC2, didn't it? Haven't upgraded to RC3 of course - my fault, sorry for that. (b.t.w. The official German Joomla portal http://www.joomla.de has still not announced RC3, that's why I wasn't aware..)

So, instantly I upgraded to RC3 and checked. Home link works, great! Using "link titles" for front page and menu page (category links menu) article lists (blog style) ends up with two different URL's for one and the same article, built as follows:

/index.php?option=com_content&view=article&id=48:article-alias&catid=33:category-alias
/index.php?view=article&id=48%3Aarticle-alias&option=com_content&Itemid=54

That's still duplicate content to me? Of course, that's without SEF option (can't use it on my local system [win xp + xampp]). However, does SEF solve that issue finally? Plus: I'd suggest to offer unique URL's to all the parameter URL's purists, too.

Again, any hint is highly appreciated
Bunjip.

My apologies, then! 



Good!



Would you please provide the names of the articles? I changed my RC3 to standard URLs and have looking been looking, without success, for those values. Which articles and/or menu links are you talking about?



I use .htaccess and SEF URLs on my XP XAMPP localhost. Just rename htaccess.txt to .htaccess in a DOS prompt. Works fine.



No problem. We care enough to want to help!
Amy

Edit: confusing language.

Hi again,





I'm not sure, if I got your question right. Well, I've got a menu containing two items:

- Home (Type: Articles/Front Page)
- Section Name (Type: Articles/Section/Blog)

The front page shows the latest article, which is connected to "Section Name" -> "Category Name". In
Menu Item Manager >> menu item "Home" >> "Parameters - Component"
I set "Title Linkable: Yes". Furthermore, in
Menu Item Manager >> menu item "Section Name" >> "Parameters - Component"
I also set "Title Linkable: Yes".

Now, if I go to the front page and copy the URL behind articles title, I receive the first link. Clicking on the menu item "Section Name" the same article, of course, is listed again with linkable title - as intended. However, when I copy the URL of that title link, I receive the second URL which is obviously different to the one from the front page.





DOS prompt -> great tip, I'm going to check that soon :-)


Cheers,
Bunjip.

Bunjip -

Sorry for the lack of clarify. I know how to activate Title links. What I am unable to do is find this duplicate content you are describing.

Maybe I misunderstood. It appeared to me that you installed RC3, did not activate SEF URLs and immediately identified system generated duplicate content. You provided the URLs for this duplicate content beginning with index.php--> and so forth. I configured my localhost to match yours by removing the SEF URL options, and searched for those URLS. But, I cannot find them.

My question for you is this:

• Did you find duplicate content when reviewing the RC3 default install? And, if so, what menu link did you click and what is the name of the article for those two URLs you posted?
• If you created this article and menu item yourself, please describe the exact steps I must take to reproduce your examples.

Thanks!
Amy

Hi Amy,



Actually, I did not use a standard installation. I used a self developed template, created own articles, menus and so on. Anyway, now I did install an RC3 leaving everything absolutely standard. I left each and every setting at their default values and installed the example page. Here is, how you can find the way, DC is produced:

1) Call front page of example site
2) Find the "We are Volunteers" article in the middle of the front page
3) Copy the URL behind "Read more >>" to clipboard and paste it to any text file
4) Surf to menu item "The News" in "Main Menu" (http://localhost/joomla/index.php?optio ... &Itemid=50)
5) Grab URL of the same article's "Read more" link and compare it to the 1st one.

They look like as shown here:

http://localhost/joomla/index.php?view= ... &Itemid=50
http://localhost/joomla/index.php?optio ... &Itemid=50

Hope, I can help to identify the problems.

I stand corrected. There are four steps! You have to use SEF URLs to get unique, single, permanent, drop dead sexy URLs and avoid duplicate content.

When you turn on SEF URLs, you will find this URL for frontpage "read more..." and the blog links for "The News" menu item.

You do not have to use the .htaccess or suffix options, but why would you not? 

With .htaccess you can get rid of the /index.php/:


And with the suffix option, you get index.html (or .pdf, etc.)

Please do not bother coming back with "but I want no duplicate content *and* ugly URLs." If so, too bad. You have to pick! hehe!

Take care,
Amy

Amy, it's time to say 'thank you very much' to you, once again. By the way, I've never ever honestly thought about using parametric URL's for the live site. I'd use rewritten URL's in the end with no doubt, sorry for letting this unclear until now. Now I've learned, that

• SEF option does not only rewrite but also normalize URL's (didn't know that, honestly. Plus, that's brandnew in RC3, right?)
• Files can be named without suffix in DOS mode.

Thanx a lot, I appreciate your patience with me very much  :D
Greets,
Bunjip

p.s. When using an .htaccess file in my webroot, I get "ERROR 500, Server error! The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there was an error in a CGI script." from the Apache in my WinXP xampp environment. But I guess, that's to be asked elsewhere, isn't it  ????
=> solved, needed to uncomment "LoadModule rewrite_module modules/mod_rewrite.so" in httpd.conf

Thanks to you, I learned that, as well. CLEVER!  Very clever!

Hope to see you in the forums, again.
Amy

This is a great thread (bookmarked it!).

As someone who does SEO for a living, and looking to learn Joomla!, this is the kind of thing I'm researching right now. I was looking at some extensions to help with SEO and Joomla!, but it looks like with 1.5 these are unnecessary. True?

Thanks,
Dan

Dan -

Certainly not for URLs. There's lots to SEO, of course, that doesn't just relate to SEF URLs or duplicate content. For example, I imagine Microformats will start to become very important in the next year. Charl van Niekerk's Google Summer of Code project was focused on the Semantic Web and Microformat integration into Joomla!.

I'd be curious what you end up deciding are important extensions needed to give J! sites a boost. If you blog on it or develop an article, please let me know!

Amy

Microformats are already integrated?

I wouldn't be surprised if microformats become a standard practice in SEO (specifically when working with a company that does business on a local level). When creating content with Joomla! we can edit html (right?) so these could be added anyway. I take it an integration makes it easier to do.

I've just started looking at Joomla! this past week, any statements of "what it needs" at this point would be premature. But if I decide to use it I'll definitely try to help make it better.

Thanks for your reply.
Dan

That's what it's all about! Thanks, Dan. Look forward to seeing your contributions!

Amy

FYI - Charl plans to release his work after J v 1.5 is finalized, although we are welcome to take it and "hack" it in now.

Oh, I want to do as little "hacking" as possible! For coding, I'm good with html and css but that's it!

Amy,

unfortunately, I need to resume this topic once again. Having played around a bit with the default installation, I found DC URL's again in the content section. I'm still talking about the 'test data' default standard installation of RC3 with SEF / mod_rewrite.

You can find two different URL's for one and the same article ('What is the FTP layer for?') here:

1. Front page >> 'What is the FTP layer for?' (link titles = 'yes')
2. Main Menu >> FAQ >> General >> 'What is the FTP layer for?'

The URL's:

1.http://localhost/test/joomlastandard/ca ... -layer-for
2. http://localhost/test/joomlastandard/fa ... -layer-for

Did I just get confused by thinking too much about the duplicate content thing or what is going wrong here?

Again, any help is appreciated.

I don't think you are going to get around that issue.  Anything that you publish to the front page and have accessible through another menu item is inherently going to give you 2 different links. It is the same article, but you are accessing it through two different paths

Bunjip  - Good question.



@ohc has the exact correct answer.



Which hits right on my point #2 on how to avoid duplicate content.



That is not *system* generated duplicate content, it is *user* generated duplicate content. There are two menu items FAQ and
Category Blog that both lead to the same article. I would rather not have example articles create "user generated duplicate content" because it confuses the issue.  I wonder if they might be able to change that.

Thanks Bunjip and ohc! That is a perfect example of what *not* to do if you want to avoid duplicate content.

Amy

ohc,
thanks for your comment.



I've already felt that I was not really realizing what that point #2 actually means for ones website structure.



At this point I disagree. But let me say this first: I am not discussing that matter because I like to split hairs or so. I'm just looking for best practice solutions that might help to improve the whole project a little bit at the end. Now, in my opinion what ohc says is system generated duplicate content in its purest form. Why? From a logical point of view, Joomla! is working with content items such as articles. And it uses structure items such as categories. Each item has an ID as a unique identifier in the database. If an item needs to be referenced, that ID is the only key needed. Wherever I link to such an item (e.g. an articel), I use that ID only (or, of course an alias like ). There's absolutely no need to include 'position' or 'view' information in a rewritten URL (like 'categoryblog').

With the section and category ideology it's hard enough to set up a decent site structure, as J! does not support multi level category tree structures. If one additionally has to watch out, what menu item should display which of ones content items - and what menu items should not despite being logically equal, it's getting even harder to create userfriendly site structures.

Consider, how will Joomla ever solve another major problem of the cms: cross referencing of a content item (article) in more than one structure item (category, section)? The way Joomla! handles article linking right now does not provide any basis for cross linking without DC.

I'm really curious how other experts and newbies think about that topic?

Greets.
Bunjip.

So, basically, we turn off functionality, such as e-mail, PDF, and print????

  You're not serious?  How does FUNCTIONALITY affect DC?  If it does, there are some serious architectural issues.

Always appreciate your help, Amy.  You're a star and a wonderful hep to us all.

I do have to agree with the issues Bunjip has very succinctly written about.  Our (our company) developers are rapidly determining that apart from the rest of the RC3 "issues" at the moment, they're more concerned about DC, and the existing 3 tiered structure which although not specifically an RC3 "feature", makes our BA's scream in pain.  To not be able to effectively crosslink, then not only will Joomla! sites create DC, it makes properly designed taxonomies/structures next to SEO'd useless considering current best practices.

Bunjip hits on a very very good point.  As an ex programmer the mixing of hierarchical items (category/section) with the unique ID is confusing.  Either you go hierarchical (think of old mainframe database like IMS), or you have each content item (article) and ONLY refer to that.

Now, you have the unique ID already - and it's in a RDBMS - so how about the developers review the current limitations described in this (and numerous other threads), and review these pretty major issues.  It's all fine and well and good and lovely to have roadmaps and all the touchy feely stuff, but right now there are people all over the world and even across my street trying to get sites up.  While some of the more well-versed Joomla! people may be able to get their sites going and have no issues, I would suggest that from what I've seen, they usually involve some for of modded-code or hack or workaround, which doesn't help anyone apart from their Clients.

Cheers, Cindy

#1 - PDF, email to friend, print icons - I agree it's silly that using those introduces duplicate content. But, you need to complain to Google about it. Rest assured, they punish all websites equally, whether those sites were built with straight HTML or a CMS. This is not a Joomla! issue. If you don't believe me, do a quick search on Google for Duplicate Content and PDFs. Then, you'll find complaining from all over the planet about this problem.

#2 - If you include content more than once in your menu system, you will include content more than once in your site. Try doing it on an HTML site. Put the document into two places. Folks, that is just fundamental duplicate content and it is not generated by the system, it is generated by the person. Why the worries, though? You are in complete and total control. If you don't want duplicate content, don't do it.

#3 - How does Joomla! solve the problem of using an article in multiple categories? It won't. Think of categories as a physical storage location. You can store one thing in one place at one time. It's the law of nature. Tagging, however, allows you to reference - and include - the same article in different ways. If you want to do that, you need some type of tagging extension. I know of two tagging components currently under development for v 1.5.

#4 - Yes, Joomla! has a primary key - id - for each entity in it's database, including articles. What must also be considered, however, is that the menu item creates a location on your website. And, Joomla! allows you to tell it exactly how you want that location arranged. You can specify modules and templates for that menu item/location. So, you cannot use the article's primary key, exclusively, Joomla! must understand the menu item (ItemID) in order to fulfill your requests for modules and templates.

#5 - There is *no* three level constraint in Joomla! and there never was. It is only imagined. Please see the thread entitled Can't understand joomla structure to understand more. If you have questions about that, please post in that thread.

Hope that helps, folks! This is a time to celebrate. With RC3, Jinx killed off all system generated duplicate content. Now, it is completely and totally within our control as site developers to avoid duplicate content. Yes, we have to think about it. So, certainly, the user interface and technical concepts should continue to be improved. This takes time and I am here to tell you, it's progressing.
Amy

It was never a question of not believing you, Amy.

http://www.alledia.com/blog/search-engine-optimisation-%28seo%29/get-out-of-joomla-pdf-hell/ which is followed from the link you provided seems to indicate that it's only the PDF issue, not the e-mail or print functionality.

I'll get around to modding this posting with your other points about tagging, but it's late here in Australia and I need to get some shut-eye after a week of dealing with constant RC3 issues from my developers.

Cindy

lol! Cindy - that's just a phrase. In hindsight, it was a poor choice of words since it seemed to suggest something other than what I intended.  What I meant to emphasize is that this is a condition that has nothing to do with Joomla!. (Your comments talked about architecture problems.)

The Alledia site is *excellent.* Steve *is* the Joomla! SEO expert. I recommend spending time there if you are interested in learning about the topic. He also offers an inexpensive booklet where he has consolidated his work in this area.

Email to friend would not create duplicate content. You are right. 

Print likely does. It has to do with Google following links. If it finds the same content multiple times spidering your site, then, we are into duplicate content.

I'll post back in a bit because I think there might already be a solution in v 1.5 for PDFs. But, I need to look at it a bit more.

ttyl,
Amy

Hello,
I am using RC3.

I have an article published on my frontpage. The link for the article is mysite.com/category-name/34-category-name/52-article-title.html

I click on it and view the article. The article title is clickable. Now the link is mysite.com/category-name/52-article-title.html.

Dublicate content!

What am I doing wrong?

No idea. Can you give me the URL to your site?

Thanks!
Amy

My site is under construction and not ready for public viewing. I PMed you the url



You are wrong here. In a static html site, if I want to link to a page, I will use the same URL no matter where I want to link it from. In other CMSs (well, Drupal anyway), I do not have to worry about duplicate content when linking to an article from different menus. The concept is simple: ONE article, ONE url, no matter if I link to it from the frontpage, other page, main menu or other menu.

This seems to be a Joomla problem, certainly not a person problem

jiakomo -

How motivating to read "You are wrong here." Especially when you continue with how Joomla! doesn't measure up to < insert another open source CMS name here. > Then, you want my help? 

Guess what? I use Drupal, as well. WordPress, too. You are right - this cannot happen in either of those environments.

But, to say it is a Joomla! problem means you don't understand the flexibility you are getting in exchange.

In Drupal and WordPress the article page is the same - always - same plugins for WP - same modules for Drupal. With Joomla!, you can select modules *and* templates for your menu items. So, those can vary for the same article - or different articles - it is not always the same.

As with anything else - flexibility means you have to think about it - make another choice. It's a tradeoff.

But, it certainly is not true, IMO, to say that Joomla! offering a choice means that Joomla! has a problem. You just have flexibility. You can choose to do it like Drupal and WordPress. Or, you can choose to have options.

Hope that helps. I'll try to find time to look at your URLs - although - I am not interested in this debate and certainly do not like pitting open source projects against one another. I value the efforts of all of the communities.

Amy 

Hi Amy,

this is not a matter of Joomla vs (insert cms) debate. I commented on an actual problem that I have, and I believe it is a Joomla problem not a person problem, as you implied.

I think this is not a forum to just praise Joomla, but also comment on possible problems, and eliminate them.

I must admit I didn't know this.

However, I think that Joomla's system should be flexible enough so that when I link to the same article, with the same template and the same modules, then the URL remains the same. Then, I will not have to worry about linking to my articles from different menus.

I hope that makes sense.

You do realize you can avoid the problem by not linking to an article with two menu items? That seems pretty easy to me.

You can also link to an existing URL. So, if you have already created a menu item - but you want two - use an external URL menu type and the URL you already have. Then, you will not create duplicate content. 

My point to you was hardly that I wanted you to "praise Joomla!", frankly, I'm even more insulted that you would jump to that conclusion. I only ask that you try to understand before you suggest that I am wrong and J! is "wrong" and this other CMS is right.

It's just different. That is all.  Learn to use the tool. Then, comments are seasoned with experience and more helpful towards building meaningful improvements.

All the best,
Amy