How to Optimize Joomla with more 100,000 Content Items?

Discussion regarding Joomla! 3.x Performance issues.

Moderator: General Support Moderators

Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
Post Reply
vnnguyendung
Joomla! Apprentice
Joomla! Apprentice
Posts: 26
Joined: Wed Jul 09, 2008 8:40 am

How to Optimize Joomla with more 100,000 Content Items?

Post by vnnguyendung » Tue Dec 25, 2018 8:20 am

I love Joomla, and I have many websites with Joomla. Now, I have 2 two websites with more 100,000 Content Items.

My websites has been optimized with JCH, JotCache. But I have many members log in, and add more 100 content items every day.

When the member logs in, the page has no cache, and is very slow (Extremely long TTFB).

I analyze the process, and JAccess::preloadPermissions() had more 100.000 loops and called more 100.000 JDatabaseDriverMysqli->fetchObject.
This may be the cause of slow and overloaded websites.

I have read these:
- https://github.com/joomla/joomla-cms/issues/13542
- https://github.com/joomla/joomla-cms/pull/6728
- https://github.com/joomla/joomla-cms/pull/14268

How to Optimize my website?
Should I switch to Drupal or Typo3, October for website with more 100,000 Content Items, multiple users login?

Thank you.

122.922 rows in table #__content
sql_cout_table_content.png
123.763 rows in table #__assets
sql_cout_table_asset.png
debug in single article (with url ?tmpl=component)
Time: 3048.18 ms / 3907.22 ms Memory: 74.095 MB / 85.50 MB Application: After Access::preloadPermissions (com_content)
aricle_single.png
Add $count in file Access.php
code_assess_count.png
And var_dump result: 123.594 loop
vardump_article.png
You do not have the required permissions to view the files attached to this post.
Last edited by vnnguyendung on Tue Dec 25, 2018 9:42 am, edited 5 times in total.

vnnguyendung
Joomla! Apprentice
Joomla! Apprentice
Posts: 26
Joined: Wed Jul 09, 2008 8:40 am

Re: How to Optimize Joomla with more 100,000 Content Items

Post by vnnguyendung » Tue Dec 25, 2018 8:24 am

JAccess::preloadPermissions() called 124.719 JDatabaseDriverMysqli->fetchObject.
profile_article.png
Debug in category blog
article_category.png

And var_dump result in category
vardump_category.png
You do not have the required permissions to view the files attached to this post.

User avatar
Per Yngve Berg
Joomla! Master
Joomla! Master
Posts: 25096
Joined: Mon Oct 27, 2008 9:27 pm
Location: Akershus, Norway

Re: How to Optimize Joomla with more 100,000 Content Items?

Post by Per Yngve Berg » Tue Dec 25, 2018 10:17 am

Could the new JSON data type in Mysql 8.0 be a solution?

https://dev.mysql.com/doc/refman/8.0/en/json.html

vnnguyendung
Joomla! Apprentice
Joomla! Apprentice
Posts: 26
Joined: Wed Jul 09, 2008 8:40 am

Re: How to Optimize Joomla with more 100,000 Content Items?

Post by vnnguyendung » Tue Dec 25, 2018 1:53 pm

Per Yngve Berg wrote:
Tue Dec 25, 2018 10:17 am
Could the new JSON data type in Mysql 8.0 be a solution?
I continue test
------------------------
php 7.0.15 - MariaDB 10.1.21
Time: 3048.18 ms / 3907.22 ms Memory: 74.095 MB / 85.50 MB Application: After Access::preloadPermissions (com_content)
------------------------
php 7.2.13 - MariaDB 10.1.37
Time: 1928.11 ms / 2737.76 ms Memory: 74.091 MB / 83.02 MB Application: After Access::preloadPermissions (com_content)
-----------------------

system-information.png
php72_debug_article.png
You do not have the required permissions to view the files attached to this post.

vnnguyendung
Joomla! Apprentice
Joomla! Apprentice
Posts: 26
Joined: Wed Jul 09, 2008 8:40 am

Re: How to Optimize Joomla with more 100,000 Content Items?

Post by vnnguyendung » Wed Dec 26, 2018 5:45 am

I view on github at https://github.com/joomla/joomla-cms/issues/13542, and drewgg commented that he see https://github.com/joomla/joomla-cms/pull/12809 solves the issue above, merged to Joomla 4

User avatar
AMurray
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 4159
Joined: Sat Feb 13, 2010 7:35 am
Location: Australia

Re: How to Optimize Joomla with more 100,000 Content Items?

Post by AMurray » Sat Dec 29, 2018 10:43 pm

Joomla 4 is not yet stable - it's still Alpha. I wouldn't use it on production sites.
Regards,
--------------------------------------------------------------
A Murray
Millennium Falcon - it's the ship that made the Kessel run in less than 12 parsecs! The fastest hunk of junk in the galaxy.

Forsh
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 216
Joined: Sat Aug 20, 2005 3:05 pm
Location: 2029 Larimer St, Denver, CO 80205
Contact:

Re: How to Optimize Joomla with more 100,000 Content Items?

Post by Forsh » Sun Dec 30, 2018 7:15 am

What the heck. I was having trouble with my 7k of pages … dang, if you have some tips or best practices once you get your 100,000 page site going, post an update about what you did please. I would be super-curious to know your recipe and visit your site to see it running smooth with 100,000 indexed pages. :eek:

vnnguyendung
Joomla! Apprentice
Joomla! Apprentice
Posts: 26
Joined: Wed Jul 09, 2008 8:40 am

Re: How to Optimize Joomla with more 100,000 Content Items?

Post by vnnguyendung » Mon Dec 31, 2018 11:16 am

Forsh wrote:
Sun Dec 30, 2018 7:15 am
What the heck. I was having trouble with my 7k of pages … dang, if you have some tips or best practices once you get your 100,000 page site going, post an update about what you did please. I would be super-curious to know your recipe and visit your site to see it running smooth with 100,000 indexed pages. :eek:
I cache evething on site. I use cache on my site with extension:

First, I use JCH Optimize to Combines JavaScript & CSS, minifies and compresses JavaScript. JCH Optimize cache file css and js so make website more fast.

Next, I use JotCache with config: turn on Browser Caching, Edit page Invalidate, And option turn off cache with view="form".

Next, I use cloudflare on my domain: turn on "DNS and HTTP Proxy -CDN". Cloudflare CDN help Cached Requests, Stop Threats.


Post Reply

Return to “Performance - Joomla! 3.x”