WARNING: Be careful with 'stats' components for Joomla!
Moderator: General Support Moderators
Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Security and Performance FAQs
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Security and Performance FAQs
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
-
- Joomla! Enthusiast
- Posts: 142
- Joined: Wed Aug 17, 2005 3:19 pm
- Contact:
WARNING: Be careful with 'stats' components for Joomla!
I thought I would write up a quick post from a web hosting providers prospective and try to give you an insight into a major problem we have been seeing with a particular type of Joomla component - stats. At the present time we host several thousands websites powered by Joomla including the official joomla.org site so I am basing this post on a fair bit of experience when it comes to Joomla-based hosting.
One type of component available for Joomla is a 'stats' component which allows users to gain statistical data about their site visitors directly through Joomla. A fair number of Joomla users would think "fantastic" and go ahead and install it. I mean being able to get all that statistical data about your visitors directly through Joomla sounds great.
However, what a lot of people don't seem to understand is that most of these stats components increase the load on the MySQL server by a dramatic amount above and beyond the core Joomla install. In some cases we have seen these stats components bring down entire managed servers hosting large Joomla sites. I should point out that while this particular issue is most noticeable on larger sites it affects all Joomla-powered sites - large or small. I should also point out that while this particular problem could apply to any Joomla component we have seen it most noticeable with the stats components.
I want to stress that this is not a problem with the Joomla core itself which we believe to be very stable. The problem is with the stats components that are being released through the official Extensions and other websites as add-ons for Joomla. In my opinion statistical data about site visitors should continue to be gathered and processed at the server level and not through a platform such as Joomla which adds excessive server loads to core services such as MySQL every time a visitor hits your website. This added load not only hits server performance but it also impacts the speed of your website.
We receive sales emails from people at least a couple of times per week who are looking to move their hosting because their current host tells them their Joomla installation is killing the server. It isn't in fact the Joomla installation that's at fault most of the time - it is a stats component which is installed under Joomla. It is often hard for a hosting provider who is not familiar with Joomla to tell that the component is to blame and not Joomla itself. This isn't their fault at all and it doesn't mean they are a bad host, it just means they aren't experts with Joomla.
My advice to anyone using Joomla would be to steer well clear of these 'stats' components to avoid running into problems with your web hosting provider's abuse / policy enforcement team. I would also advise anyone already running any of these stats components to drop them from your Joomla installation. You may not have encountered a problem yet because your site doesn't receive enough concurrent traffic at the moment but I can guarantee it will become a bigger issue as your site grows. I would also like to kindly ask that the developers of these components take this issue to heart before releasing any further components like this on unsuspecting victims.
Thanks for taking the time to read my post and I hope it may have proved useful to a few of you
One type of component available for Joomla is a 'stats' component which allows users to gain statistical data about their site visitors directly through Joomla. A fair number of Joomla users would think "fantastic" and go ahead and install it. I mean being able to get all that statistical data about your visitors directly through Joomla sounds great.
However, what a lot of people don't seem to understand is that most of these stats components increase the load on the MySQL server by a dramatic amount above and beyond the core Joomla install. In some cases we have seen these stats components bring down entire managed servers hosting large Joomla sites. I should point out that while this particular issue is most noticeable on larger sites it affects all Joomla-powered sites - large or small. I should also point out that while this particular problem could apply to any Joomla component we have seen it most noticeable with the stats components.
I want to stress that this is not a problem with the Joomla core itself which we believe to be very stable. The problem is with the stats components that are being released through the official Extensions and other websites as add-ons for Joomla. In my opinion statistical data about site visitors should continue to be gathered and processed at the server level and not through a platform such as Joomla which adds excessive server loads to core services such as MySQL every time a visitor hits your website. This added load not only hits server performance but it also impacts the speed of your website.
We receive sales emails from people at least a couple of times per week who are looking to move their hosting because their current host tells them their Joomla installation is killing the server. It isn't in fact the Joomla installation that's at fault most of the time - it is a stats component which is installed under Joomla. It is often hard for a hosting provider who is not familiar with Joomla to tell that the component is to blame and not Joomla itself. This isn't their fault at all and it doesn't mean they are a bad host, it just means they aren't experts with Joomla.
My advice to anyone using Joomla would be to steer well clear of these 'stats' components to avoid running into problems with your web hosting provider's abuse / policy enforcement team. I would also advise anyone already running any of these stats components to drop them from your Joomla installation. You may not have encountered a problem yet because your site doesn't receive enough concurrent traffic at the moment but I can guarantee it will become a bigger issue as your site grows. I would also like to kindly ask that the developers of these components take this issue to heart before releasing any further components like this on unsuspecting victims.
Thanks for taking the time to read my post and I hope it may have proved useful to a few of you
- Chris
- stingrey
- Joomla! Hero
- Posts: 2756
- Joined: Mon Aug 15, 2005 4:36 pm
- Location: Marikina, Metro Manila, Philippines
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
It should be noted that this is the reason we do not include a detailed statistics gathering fuction into the Joomla! core.
As noted by Chris, this kind of functionality it best dealt with at server level and also the functionality of server level statistics gathering applications is far more complete also most good web host provide such server level statistics gathering as part of their packages.
As noted by Chris, this kind of functionality it best dealt with at server level and also the functionality of server level statistics gathering applications is far more complete also most good web host provide such server level statistics gathering as part of their packages.
Rey Gigataras
http://www.wizmediateam.com <-- great team of talented Web Designers and Programmers!
http://about.me/reygigataras <-- About Me
Partner, Business Development & Project Manager, Event Manager, Sports Coach
http://www.wizmediateam.com <-- great team of talented Web Designers and Programmers!
http://about.me/reygigataras <-- About Me
Partner, Business Development & Project Manager, Event Manager, Sports Coach
- infograf768
- Joomla! Master
- Posts: 19133
- Joined: Fri Aug 12, 2005 3:47 pm
- Location: **Translation Matters**
Re: WARNING: Be careful with 'stats' components for Joomla!
I have asked all International Partners and Forum mods to forward this Warning to their respective communities.
Jean-Marie Simonet / infograf
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group
---------------------------------
ex-Joomla Translation Coordination Team • ex-Joomla! Production Working Group
- pe7er
- Joomla! Master
- Posts: 24974
- Joined: Thu Aug 18, 2005 8:55 pm
- Location: Nijmegen, Netherlands
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
When you state that stats component have a major negative impact on the speed of your webserver, you are referring to all kinds of Joomla stats components?Rochen wrote:[..] The problem is with the stats components that are being released through the official Extensions and other websites as add-ons for Joomla. In my opinion statistical data about site visitors should continue to be gathered and processed at the server level and not through a platform such as Joomla which adds excessive server loads to core services such as MySQL every time a visitor hits your website. This added load not only hits server performance but it also impacts the speed of your website.
[..]
My advice to anyone using Joomla would be to steer well clear of these 'stats' components to avoid running into problems with your web hosting provider's abuse / policy enforcement team. I would also advise anyone already running any of these stats components to drop them from your Joomla installation. You may not have encountered a problem yet because your site doesn't receive enough concurrent traffic at the moment but I can guarantee it will become a bigger issue as your site grows. I would also like to kindly ask that the developers of these components take this issue to heart before releasing any further components like this on unsuspecting victims.
And not just one in particular from http://extensions.joomla.org/component/ ... Itemid,35/ ?
Kind Regards,
Peter Martin, Global Moderator
Company website: https://db8.nl/en/ - Joomla specialist, Nijmegen, Netherlands
The best website: https://the-best-website.com
Peter Martin, Global Moderator
Company website: https://db8.nl/en/ - Joomla specialist, Nijmegen, Netherlands
The best website: https://the-best-website.com
- brian
- Joomla! Master
- Posts: 12785
- Joined: Fri Aug 12, 2005 7:19 am
- Location: Leeds, UK
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
The problem is that all the J stats components work by grabbing the data and then storing them in a mysql database. If your site is even moderately busy this will result on lots of mysql entries and a rapidy increasing database.
Traditionaly webstats are gathered at the server level by apache in flat files. A stats analysis program then reads those files and presents you with the results ONLY when you access it. Which for most people will only be on an occasional basis.
Brian
Traditionaly webstats are gathered at the server level by apache in flat files. A stats analysis program then reads those files and presents you with the results ONLY when you access it. Which for most people will only be on an occasional basis.
Brian
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/
- leolam
- Joomla! Master
- Posts: 20652
- Joined: Mon Aug 29, 2005 10:17 am
- Location: Netherlands/ Germany/ S'pore/Bogor/ North America
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
@ Rey;
would it be possible to place a sort of "warning" button on the extention site pointing to this excellent information provided by "Rochen" or get a pop-up with the warning on some of the components etc with similar information? This might avoid many problems and reduce issues at the forum....Should not be too much of a task...?
2ct's
Leo
would it be possible to place a sort of "warning" button on the extention site pointing to this excellent information provided by "Rochen" or get a pop-up with the warning on some of the components etc with similar information? This might avoid many problems and reduce issues at the forum....Should not be too much of a task...?
2ct's
Leo
Joomla's #1 Professional Services Provider:
#Joomla Professional Support: https://gws-desk.com -
#Joomla Specialized Hosting Solutions: https://gws-host.com -
#Joomla Professional Support: https://gws-desk.com -
#Joomla Specialized Hosting Solutions: https://gws-host.com -
- dknight
- Joomla! Guru
- Posts: 682
- Joined: Thu Aug 18, 2005 5:14 am
- Location: Kuala Lumpur, Malaysia
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Leo,
We have already updated the description to inform users about this potential performance issue:
http://extensions.joomla.org/component/ ... Itemid,35/
We have already updated the description to inform users about this potential performance issue:
http://extensions.joomla.org/component/ ... Itemid,35/
Lee - Mosets Consulting
Developer of Real Estate and Directory component for Joomla!
Website: www.mosets.com
Developer of Real Estate and Directory component for Joomla!
Website: www.mosets.com
- leolam
- Joomla! Master
- Posts: 20652
- Joined: Mon Aug 29, 2005 10:17 am
- Location: Netherlands/ Germany/ S'pore/Bogor/ North America
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Good call Lee! Tnxdknight wrote: Leo,
We have already updated the description to inform users about this potential performance issue:
http://extensions.joomla.org/component/ ... Itemid,35/
Joomla's #1 Professional Services Provider:
#Joomla Professional Support: https://gws-desk.com -
#Joomla Specialized Hosting Solutions: https://gws-host.com -
#Joomla Professional Support: https://gws-desk.com -
#Joomla Specialized Hosting Solutions: https://gws-host.com -
- keliix06
- Joomla! Ace
- Posts: 1022
- Joined: Wed Aug 17, 2005 11:46 pm
- Location: Minneapolis, MN
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
I couldn't agree with this more. We're also hosting thousands of Joomla and Mambo installs, and it is the single biggest problem we see.
Doyle Lewis
BuyHTTP Internet Services
http://www.buyhttp.com/joomla_hosting.html - No Overselling Guarantee. Your Joomla site, faster.
http://www.joomlademo.com - 30 day free trial of Joomla
BuyHTTP Internet Services
http://www.buyhttp.com/joomla_hosting.html - No Overselling Guarantee. Your Joomla site, faster.
http://www.joomlademo.com - 30 day free trial of Joomla
- focalguy
- Joomla! Guru
- Posts: 909
- Joined: Fri Aug 19, 2005 2:46 am
- Location: Washington State, USA
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Thanks for the information. My site is quite small so I haven't noticed anything yet. I guess I have been using both the stats gathered in cPanel scripts and the Joomla Components because I feel the components display the information I am most interested in in a better format. The information is there with the other scripts, but I haven't found a way to get at it as quickly or easily as the components. If this were not the case, I would drop the component immediately.
- ilox
- Joomla! Explorer
- Posts: 444
- Joined: Thu Aug 25, 2005 3:29 pm
- Location: Adelaide, South Australia
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Can I please have a little bit of clarification? In the Admin backend, Site --> Global Configuration --Statistics, there are options to collect stats.
If so then perhaps from 1.1 onwards we could have more than just a warning symbol appear next to the stats option warning people of likely consequences for activating that option? Yes, it does say, "Warning! Large amounts of data will be collected" but this doesn't really explain just what that might mean to efficiency or server work.
A newcomer (like I was) is unlikely to be aware of the hit that such an option might bring to the site and the servers. Maybe this page needs an introductory text warning placed above the options so that users are very clearly aware of the consequences?
Like.
Collecting statistics through the use of these options is likely to cause extra workload on the servers. Some service providers may take regulatory action in the event of such extra workload. It is suggested that these options only be selected if your server provider is unable to provide you with site statistics that are usually collected by the server software.
- Statistics: No Yes Enable/disable collection of site statistics
- Log Content Hits by Date: No Yes Warning! Large amounts of data will be collected
- Log Search Strings: No Yes
If so then perhaps from 1.1 onwards we could have more than just a warning symbol appear next to the stats option warning people of likely consequences for activating that option? Yes, it does say, "Warning! Large amounts of data will be collected" but this doesn't really explain just what that might mean to efficiency or server work.
A newcomer (like I was) is unlikely to be aware of the hit that such an option might bring to the site and the servers. Maybe this page needs an introductory text warning placed above the options so that users are very clearly aware of the consequences?
Like.
Collecting statistics through the use of these options is likely to cause extra workload on the servers. Some service providers may take regulatory action in the event of such extra workload. It is suggested that these options only be selected if your server provider is unable to provide you with site statistics that are usually collected by the server software.
Cheers, Ian
"Always remember. Love is the purest feeling, the wisest thought and the strongest reason. Always!"
by Sea-Life
"Always remember. Love is the purest feeling, the wisest thought and the strongest reason. Always!"
by Sea-Life
-
- Joomla! Intern
- Posts: 61
- Joined: Mon Dec 12, 2005 5:46 pm
Re: WARNING: Be careful with 'stats' components for Joomla!
Could we blend the solutions to create a better one?
I mean, the non-Joomla stats read from a flat file, which means no MySQL resources chewed up.
The Joomla stats provide better information for the Webmaster.
Can we put them together and let Mambo use flat files so we get better performance and better information?
I mean, the non-Joomla stats read from a flat file, which means no MySQL resources chewed up.
The Joomla stats provide better information for the Webmaster.
Can we put them together and let Mambo use flat files so we get better performance and better information?
- stingrey
- Joomla! Hero
- Posts: 2756
- Joined: Mon Aug 15, 2005 4:36 pm
- Location: Marikina, Metro Manila, Philippines
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
The short answer is yes this also can affect the performance of a site and is why it is `off` be default.ilox wrote: Can I please have a little bit of clarification? In the Admin backend, Site --> Global Configuration --Statistics, there are options to collect stats.
Rey Gigataras
http://www.wizmediateam.com <-- great team of talented Web Designers and Programmers!
http://about.me/reygigataras <-- About Me
Partner, Business Development & Project Manager, Event Manager, Sports Coach
http://www.wizmediateam.com <-- great team of talented Web Designers and Programmers!
http://about.me/reygigataras <-- About Me
Partner, Business Development & Project Manager, Event Manager, Sports Coach
-
- Joomla! Apprentice
- Posts: 21
- Joined: Sat Apr 15, 2006 5:47 pm
Re: WARNING: Be careful with 'stats' components for Joomla!
Since installing stat components is not advised what would be a suitable solution (or remedy) for a webmaster who needs this function? Also, I'm interested in knowing how reporting components (e.g. JamboReport) affect MYSQL and server loads.
- trompete
- Joomla! Explorer
- Posts: 273
- Joined: Thu Sep 01, 2005 11:19 pm
- Location: Minneapolis, USA
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
On behalf of BSQ Sitestats, I'd like to defend my architecture a bit. I created the component because the component I was using at the time, TFSForMambo, was generating 20 database queries per page. BSQ Sitestats does a single INSERT for a normal page load and an extra SELECT/INSERT in the rare case that a hit comes from a search engine. This performs very well. I'm currently showing 45 other queries per page on my Joomla 1.0.8 installation.
The other reason I use BSQ Sitestats instead of the native stats is that the native stats cause an update query to occur the mos_content table, invalidating Mysql's query cache for that record. The mysql query cache is wonderful, but it only works if you aren't updating records each time you view them.
I'd love to hear any feedback on performance you guys have as Joomla administrators.
I'm sorry to see that all of the stats components are grouped together as one malicious entity.
Brent
The other reason I use BSQ Sitestats instead of the native stats is that the native stats cause an update query to occur the mos_content table, invalidating Mysql's query cache for that record. The mysql query cache is wonderful, but it only works if you aren't updating records each time you view them.
I'd love to hear any feedback on performance you guys have as Joomla administrators.
I'm sorry to see that all of the stats components are grouped together as one malicious entity.
Brent
Developer, bsq_sitestats module.
www.bs-squared.com
www.bs-squared.com
- eyezberg
- Joomla! Hero
- Posts: 2859
- Joined: Thu Aug 25, 2005 5:48 pm
- Location: Geneva mostly
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
I am starting to think this might be the reason my previous host kicked me out..
Would me nice to have more detailed exaplantions and tests about Core stats and each (of the not so many) components.
I agree with trompete that it is not fair to just issue a general statement about stats collection without differentiating how these are collected and or stored etc.. According to Brians post, it looks like using flat files would solve most issues? What about that, is that correct, or are there other solutions also?
Good to warn, better to continue this discussion with the dev's and come up with (a) solution(s).
Would me nice to have more detailed exaplantions and tests about Core stats and each (of the not so many) components.
I agree with trompete that it is not fair to just issue a general statement about stats collection without differentiating how these are collected and or stored etc.. According to Brians post, it looks like using flat files would solve most issues? What about that, is that correct, or are there other solutions also?
Good to warn, better to continue this discussion with the dev's and come up with (a) solution(s).
Sometimes one pays most for the things one gets for nothing.
The important thing is not to stop questioning. Curiosity has its own reason for existing. AE
http://joomla15.[URL banned].com for J! 1.5 screenshots
http://www.eyezberg.com
The important thing is not to stop questioning. Curiosity has its own reason for existing. AE
http://joomla15.[URL banned].com for J! 1.5 screenshots
http://www.eyezberg.com
- masyomo
- Joomla! Apprentice
- Posts: 44
- Joined: Tue Nov 15, 2005 12:21 am
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
We removed BSQ stats for the same reason - our database went from 2mb to 30mb and its only a small userbase. Most of the stats are available through cpanel anyway although it was interesting to see user reports but not crucial.trompete wrote: On behalf of BSQ Sitestats, I'd like to defend my architecture a bit.
I'd love to hear any feedback on performance you guys have as Joomla administrators.
I'm sorry to see that all of the stats components are grouped together as one malicious entity.
Brent
The other component beefing up our database is remository which now stores files in mysql. On their site they say that databases will run with no problem up to 4GB so is it the load on the database thats the problem rather than the storage? Our main worry was backing up the database but that was before I'd thought we could just exclude the big tables from the backup & still have all the crucial info
- trompete
- Joomla! Explorer
- Posts: 273
- Joined: Thu Sep 01, 2005 11:19 pm
- Location: Minneapolis, USA
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
What about server load? The above statement has to do with disk space. Server load is not the same as disk space.masyomo wrote: We removed BSQ stats for the same reason - our database went from 2mb to 30mb
Disk space can be an issue, and is a valid reason not to use a stats product. But don't confuse that with peformance. For example, the "stuff" I work on at my day job generates 50 GB of recorded data a day, but it does 150k transactions per second. This is great performance, even though it eats up a lot of disk.
Developer, bsq_sitestats module.
www.bs-squared.com
www.bs-squared.com
- masyomo
- Joomla! Apprentice
- Posts: 44
- Joined: Tue Nov 15, 2005 12:21 am
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
yeah thats what I meant - I removed it because doing a backup was taking forever but hadn't thought that I could just leave out the heavy tables - and then when looking into Remository I saw the thing about the database still working fine up to 4GB and so thought maybe it wasn't such a problem afterall.
I didn't know anything about the server load so can't say unfortunately but now I'm learning!!
I didn't know anything about the server load so can't say unfortunately but now I'm learning!!
-
- Joomla! Apprentice
- Posts: 9
- Joined: Sun Feb 26, 2006 4:27 pm
Re: WARNING: Be careful with 'stats' components for Joomla!
The little statistics module that comes with the Joomla install which show count of members, server OS, number news items, etc., isn't a culprit, is it?
- Websmurf
- Joomla! Hero
- Posts: 2230
- Joined: Fri Aug 19, 2005 2:23 pm
- Location: The Netherlands
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
No that information is allready registered anywayCCI wrote: The little statistics module that comes with the Joomla install which show count of members, server OS, number news items, etc., isn't a culprit, is it?
Adam van Dongen - Developer
- Blocklist, ODT Indexer, EasyFAQ, Easy Guestbook, Easy Gallery, YaNC & Redirect -
http://www.joomla-addons.org - http://www.bandhosting.nl
- Blocklist, ODT Indexer, EasyFAQ, Easy Guestbook, Easy Gallery, YaNC & Redirect -
http://www.joomla-addons.org - http://www.bandhosting.nl
- Michiel_1981
- Joomla! Intern
- Posts: 71
- Joined: Sun Aug 28, 2005 6:38 am
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
This warning might as well go out for most sef components.
regards,
Michiel
regards,
Michiel
- notexa
- Joomla! Intern
- Posts: 51
- Joined: Fri Dec 16, 2005 12:55 am
- Location: Belgium
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Could you ellaborate on this one ?Michiel_1981 wrote: This warning might as well go out for most sef components.
How do eg Opensef, advanced Sef, ... have a negative impact on server load ?
Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.
- Websmurf
- Joomla! Hero
- Posts: 2230
- Joined: Fri Aug 19, 2005 2:23 pm
- Location: The Netherlands
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Well, mostly due to the amount of queries that it will use to build op the sef urls. Most sef components will querie the database a few times for each link on the page. These aren't very heavy queries (simple selects, so cached in Mysql 4.1 and above) but they will be there.
If you're running a high traffic site, this will play a role.
If you're running a high traffic site, this will play a role.
Adam van Dongen - Developer
- Blocklist, ODT Indexer, EasyFAQ, Easy Guestbook, Easy Gallery, YaNC & Redirect -
http://www.joomla-addons.org - http://www.bandhosting.nl
- Blocklist, ODT Indexer, EasyFAQ, Easy Guestbook, Easy Gallery, YaNC & Redirect -
http://www.joomla-addons.org - http://www.bandhosting.nl
- kenmcd
- Joomla! Champion
- Posts: 5672
- Joined: Thu Aug 18, 2005 2:09 am
- Location: California
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Performance Comparison of MySQL Queries by SEF Componentsnotexa wrote:Could you ellaborate on this one ?Michiel_1981 wrote: This warning might as well go out for most sef components.
How do eg Opensef, advanced Sef, ... have a negative impact on server load ?
http://www.open-sef.org/forum/opensef-d ... rison.html
██ LibreTraining
- Michiel_1981
- Joomla! Intern
- Posts: 71
- Joined: Sun Aug 28, 2005 6:38 am
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Hi kenmcd,
I was just searching for the that post.
If you look at the data you can concluded this, this avarage queries extra.
opensef might stand a change (rc5) because it only ads 18 queries, but all those others.
Now picture a shared host running a lot of joomla sites with those sef components. What will happen to your cache mechanisme? Does it matter, as 50+ individual select statements are slower then 1 update statement, more so if the database isn't on the same server!
BSQ sitestats has a problems, I admit, it gets fat very fast. Brent is working on this. The next release will group more urls together and has a better filter so post queries won't be counted as unique urls. The release after that the database structure will be changed and the hits be stored more efficiently.
but speed/serverload it will beat the sef components and that's were this topic was about.
regards,
Michiel
[edit]
typo
I was just searching for the that post.
If you look at the data you can concluded this, this avarage queries extra.
- opensef rc4 - 181
- opensef rc5 - 18
- sef advance - 50
- 404sef - 126
opensef might stand a change (rc5) because it only ads 18 queries, but all those others.
Now picture a shared host running a lot of joomla sites with those sef components. What will happen to your cache mechanisme? Does it matter, as 50+ individual select statements are slower then 1 update statement, more so if the database isn't on the same server!
BSQ sitestats has a problems, I admit, it gets fat very fast. Brent is working on this. The next release will group more urls together and has a better filter so post queries won't be counted as unique urls. The release after that the database structure will be changed and the hits be stored more efficiently.
but speed/serverload it will beat the sef components and that's were this topic was about.
regards,
Michiel
[edit]
typo
Last edited by Michiel_1981 on Sat May 06, 2006 12:19 am, edited 1 time in total.
- kenmcd
- Joomla! Champion
- Posts: 5672
- Joined: Thu Aug 18, 2005 2:09 am
- Location: California
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
All of the SEF queries can be cached with sufficient memory.
Inserts are another story.
I have some experience tuning high performance ad servers - lots of inserts.
But again this can be tuned for high performance.
Unfortunately most shared MySQL servers are not tuned well, and on top of that they are usually overloaded with too many users. They usually take an average machine with average memory and then don't tune it properly for what they do have. Add way too many users = sloooowwwww. The silly solution - add query limits.
Ironically the users who need a Joomla Stats component the most - those without log file access - are the ones most likely to be affected by the query load/limits.
Don'cha know it's always sumthin'
- Rosanne Rosannadana
Inserts are another story.
I have some experience tuning high performance ad servers - lots of inserts.
But again this can be tuned for high performance.
Unfortunately most shared MySQL servers are not tuned well, and on top of that they are usually overloaded with too many users. They usually take an average machine with average memory and then don't tune it properly for what they do have. Add way too many users = sloooowwwww. The silly solution - add query limits.
Ironically the users who need a Joomla Stats component the most - those without log file access - are the ones most likely to be affected by the query load/limits.
Don'cha know it's always sumthin'
- Rosanne Rosannadana
██ LibreTraining
- trompete
- Joomla! Explorer
- Posts: 273
- Joined: Thu Sep 01, 2005 11:19 pm
- Location: Minneapolis, USA
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
Agree. On the querying side, BSQ Sitestats uses the Cache lite libraries to minimize database IO by only doing it every 30 seconds or so. I've found that file IO works a lot better than database IO for a couple reasons:kenmcd wrote: Ironically the users who need a Joomla Stats component the most - those without log file access - are the ones most likely to be affected by the query load/limits.
1. Most people do database IO, so that cache fills up quickly and is invalidated in a circle of thrashing.
2. Database IO is socket-based. Yeah, it is just a handle, but TCP has its slow transmission ramp up...etc. Mysql really needs to make a working shared memory interface between the client libraries and their server for maximum throughput. At that poing, mysql would be just as good as a file system.
3. Reading of files is implicitly cached thanks to our friend, Mr. virtual memory.
4. Profit???
Developer, bsq_sitestats module.
www.bs-squared.com
www.bs-squared.com
- Michiel_1981
- Joomla! Intern
- Posts: 71
- Joined: Sun Aug 28, 2005 6:38 am
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
What exactly is your point, that they both are affected by misconfigured Shared host Setups or only the stats?kenmcd wrote: Unfortunately most shared MySQL servers are not tuned well, and on top of that they are usually overloaded with too many users. They usually take an average machine with average memory and then don't tune it properly for what they do have. Add way too many users = sloooowwwww. The silly solution - add query limits.
Ironically the users who need a Joomla Stats component the most - those without log file access - are the ones most likely to be affected by the query load/limits.
[little off topic maybe]
I'm willing to run a test on my local server to compare speed/load between these components and a clean joomla install, if you are so kind to provide some details what should be fare. How much memory should mysql get to cache per domain and how much hits/connections would be fare in this case. Just to see what the affects are.
Short, How many domains share how much of mysql cache on how many concurrent connections does it have. So we can scale down.
regards,
Michiel
- kenmcd
- Joomla! Champion
- Posts: 5672
- Joined: Thu Aug 18, 2005 2:09 am
- Location: California
- Contact:
Re: WARNING: Be careful with 'stats' components for Joomla!
HUH?Michiel_1981 wrote: What exactly is your point, that they both are affected by misconfigured Shared host Setups or only the stats?
[little off topic maybe]
I'm willing to run a test on my local server to compare speed/load between these components and a clean joomla install, if you are so kind to provide some details what should be fare. How much memory should mysql get to cache per domain and how much hits/connections would be fare in this case. Just to see what the affects are.
Short, How many domains share how much of mysql cache on how many concurrent connections does it have. So we can scale down.
regards,
Michiel
Are you upset about something?
If that is so, I do not have any idea of what or why.
What exactly is your point?
██ LibreTraining