Long slow queries resulting in 503

[powerjoe]

Hi all,

I currently have the following slow queries running on my website;

Code: Select all

Executed 0 min ago for 3 sec on Database --> ********
SELECT a.id, a.title, a.alias, a.introtext, a.fulltext, a.checked_out, a.checked_out_time, a.catid, a.created, a.created_by, a.created_by_alias, CASE WHEN a.modified = '0000-00-00 00:00:00' THEN a.created ELSE a.modified END as modified, a.modified_by, uam.name as modified_by_name,CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END as publish_up,a.publish_down, a.images, a.urls, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits, a.xreference, a.featured, a.language, LENGTH(a.fulltext) AS readmore,CASE WHEN badcats.id is not null THEN 0 ELSE a.state END AS state,c.title AS category_title, c.path AS category_route, c.access AS category_access, c.alias AS category_alias,CASE WHEN a.created_by_alias > ' ' THEN a.created_by_alias ELSE ua.name END AS author,ua.email AS author_email,parent.title as parent_title, parent.id as parent_id, parent.path as parent_route, parent.alias as parent_alias,ROUND(v.rating_sum / v.rating_count, 0) AS rating, v.rating_count as rating_count,c.published, CASE WHEN badcats.id is null THEN c.published ELSE 0 END AS parents_publishedFROM sxm_content AS aLEFT JOIN sxm_categories AS c ON c.id = a.catidLEFT JOIN sxm_users AS ua ON ua.id = a.created_byLEFT JOIN sxm_users AS uam ON uam.id = a.modified_byLEFT JOIN sxm_categories as parent ON parent.id = c.parent_idLEFT JOIN sxm_content_rating AS v ON a.id = v.content_idLEFT OUTER JOIN (SELECT cat.id as id FROM sxm_categories AS cat JOIN sxm_categories AS parent ON cat.lft BETWEEN parent.lft AND parent.rgt WHERE parent.extension = 'com_content' AND parent.published != 1 GROUP BY cat.id ) AS badcats ON badcats.id = c.idWHERE a.access IN (1,1,2,7) AND c.access IN (1,1,2,7) AND CASE WHEN badcats.id is null THEN a.state ELSE 0 END = 1 AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2016-10-06 17:53:47') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2016-10-06 17:53:47')ORDER BY c.lft, CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END DESC , a.created
Executed 0 min ago for 2 sec on Database --> *********
SELECT a.id, a.title, a.alias, a.introtext, a.fulltext, a.checked_out, a.checked_out_time, a.catid, a.created, a.created_by, a.created_by_alias, CASE WHEN a.modified = '0000-00-00 00:00:00' THEN a.created ELSE a.modified END as modified, a.modified_by, uam.name as modified_by_name,CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END as publish_up,a.publish_down, a.images, a.urls, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits, a.xreference, a.featured, a.language, LENGTH(a.fulltext) AS readmore,CASE WHEN badcats.id is not null THEN 0 ELSE a.state END AS state,c.title AS category_title, c.path AS category_route, c.access AS category_access, c.alias AS category_alias,CASE WHEN a.created_by_alias > ' ' THEN a.created_by_alias ELSE ua.name END AS author,ua.email AS author_email,parent.title as parent_title, parent.id as parent_id, parent.path as parent_route, parent.alias as parent_alias,ROUND(v.rating_sum / v.rating_count, 0) AS rating, v.rating_count as rating_count,c.published, CASE WHEN badcats.id is null THEN c.published ELSE 0 END AS parents_publishedFROM sxm_content AS aLEFT JOIN sxm_categories AS c ON c.id = a.catidLEFT JOIN sxm_users AS ua ON ua.id = a.created_byLEFT JOIN sxm_users AS uam ON uam.id = a.modified_byLEFT JOIN sxm_categories as parent ON parent.id = c.parent_idLEFT JOIN sxm_content_rating AS v ON a.id = v.content_idLEFT OUTER JOIN (SELECT cat.id as id FROM sxm_categories AS cat JOIN sxm_categories AS parent ON cat.lft BETWEEN parent.lft AND parent.rgt WHERE parent.extension = 'com_content' AND parent.published != 1 GROUP BY cat.id ) AS badcats ON badcats.id = c.idWHERE a.access IN (1,1,5) AND c.access IN (1,1,5) AND CASE WHEN badcats.id is null THEN a.state ELSE 0 END = 1 AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2016-10-06 17:53:49') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2016-10-06 17:53:49')ORDER BY c.lft, CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END DESC , a.created

These create a huge load on the web-server which gives me a lot of down time. Looking at the query it seems that the DB table for the articles is causing this, correct me if I'm wrong.

Now for my website, I have the following;
- Joomla caching is off
- Cloudflare caching
- SSL Certificate WildCard
- Use Ad Agency Pro for my ad's. I do not cache these particular "zones" due to issues with this component
- Jsecure Authentication for back-end protection
- LoginOne Business to control login's per user
- Content Statistics to track user analytics
- RSForms PRO for forms
- PhocaDownload for our paid subscribers

Server specs;
- 4x 3.0 GHz CPU Cores
- 7GB Ram
- 40GB Storage
- CentOS
- cPanel

Actual website in question; https://www.thedailyherald.sx

Error log is packed with;

Code: Select all

[error] Execute of /home/******/public_html/index.php stopped because of load 34.10

The server gets overloaded pretty much every second.

I've been trying to fix this issue for quite some time now but I am out of ideas.

Does anyone else have any ideas or fixes?

Hoping for a quick response.

PJ

[powerjoe]

Alright for the people that are facing the same issue, I resolved this issue by changing the core files. I'd prefer to avoid this but I was left without an option.

In components/com_content/models/articles.php, I replaced line 431

From:

Code: Select all

$query->where('(a.publish_up = ' . $nullDate . ' OR a.publish_up <= ' . $nowDate . ')')
                ->where('(a.publish_down = ' . $nullDate . ' OR a.publish_down >= ' . $nowDate . ')');

To:

Code: Select all

$query->where('(a.publish_up >= DATE_SUB(NOW(), INTERVAL 1 YEAR))');

This might work for you or not. It worked for me and the server load is back to normal once again.

I truly hope a joomla dev member will look into this issue and see if we can get a fix in one of the upcoming updates.

PJ

[powerjoe]

*BUMP*

Almost two years later still fighting the same issue. My top 10 slow queries take a little over 32 seconds.

[csthomas]

The one reason of slow queries in that column publish_up is not indexed in the database.

I suggest to create an issue at https://issues.joomla.org/tracker/joomla-cms or directly on github https://github.com/joomla/joomla-cms/issues

You can also check my patches at https://github.com/joomla/joomla-cms/pulls/csthomas
which mostly concerns improving performance in different places.

If the patch works correctly, add a comment in the specific PR. Each patch/PR requires 2 success tests.