The Joomla! Forum ™



Forum rules


Forum Rules
READ ME <-- please read before posting, this means YOU.



Post new topic Reply to topic  [ 26 posts ] 
Author Message
PostPosted: Mon Jan 30, 2012 6:50 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
This issue gets mentioned time and again, but then it seems to slip down the back of the feature request's sofa and forgotten, so I'm going to raise it again.

Could you please resolve the review scoring system used in the JED? At the moment it is, to put not too fine a point on it, unfair, meaning that my core product scores "worse" than many of my less developed competitors despite the level of work we put into it being significantly higher.


Let's take a look at how things are now.
http://extensions.joomla.org/extensions ... ns/booking

Jomres is listed on the JED under Vertical Markets - Booking : Currently it is placed 8th in the scores but this seems odd, given the amount of work we put into it, and the reviews. Let's look at some of the competition :


Roombooker. 2nd.
Added 05/04/2010
Last updated 5/01/2011.
10 reviews.
http://extensions.joomla.org/extensions ... king/12082
Forum http://www.joomplace.com/forum/joomla-c ... ooker.html

Jongman. 5th.
Added 01/10/2010
Last updated 24/12/2010.
10 reviews.
http://extensions.joomla.org/extensions ... king/14221
Forum http://www.joomlant.org/forum/6-jongman ... ystem.html

Table boss. 7th.
Added 22/09/2007
Last updated 08/01/2011 7 reviews.
http://extensions.joomla.org/extensions ... oking/3110
Forum http://tableboss.com/component/option,c ... 1/lang,en/

Jomres 8th.
Added 05/03/2006
Last updated 30/01/2012
88 reviews.
http://extensions.joomla.org/extensions ... ooking/335
Forum http://www.jomres.net/forum


Jomres aside, none of them have been updated in the last year.

I've linked to these product's forum pages. From them you can see that there's almost no activity in them, at least two of them look dead. In comparison, our forum alone has had a dozen posts this morning, and that's not including the emails we've responded to via the ticket system.

Since these extensions were last updated, Jomres has seen 43 new versions. It's probably one of the most heavily developed of the Joomla components. It has two people supporting and developing for it, full time. Indeed, if you look at the reviews you'll see that we pride ourselves on our speedy and accurate support.


Yet, somehow, Jomres is still showing 8th in the list. Why is that? I believe that it's the number of reviews we've got that are actually holding us back from getting a better score. We have more users, whereas these stagnant projects aren't gaining new users therefore their scores aren't changing. It's their inactivity that's hurting Jomres', and more importantly potential user's impressions of Jomres. In a community that agrees that software that's under development is going to be of higher quality than code that's simply been ignored for the last few years, surely this is an anathema? It's actually to my competitor's benefit that they don't update their products and don't encourage users to review them on the JED. If that's not a bit mad, I don't know what is.

Naturally, this is very frustrating to me, and I think it's jolly unfair :)

How can we improve the scoring system on the JED?

Well, I've done some back of a fag packet calculations. My numbers might be slightly off, but if they are it's not by much.

First, let's look at the existing data.

Review score values (sum and count of reviews)

Code:
Roombooker Score 48, reviews 10.
Jongman Score 44, reviews 10.
Table boss Score 35 reviews 7.
Jomres Score 366 reviews 81*


*Yes, I know JED says 88 reviews, but not all of them have ratings attached, as those reviews predate the rating functionality that was added later.

Months since last update

Code:
Roombooker 0
Jongman 13
Table boss 12
Jomres 0



JED ratings (How JED currently scores extensions)

Code:
Room booker 4.8
Jongman 4.57
Table boss 4.36
Jomres 4.23



Simple averages score

Code:
Table boss 5
Room booker  4.8
Jomres 4.51
Jongman 4.4


Clearly, averages aren't going to help,

Weighted averages (n1 * N1) + (n2 * N2) / (n1 + n2) are a minor improvement, but they don't take an extension's inactivity into account, let's see the numbers :

Simple weighted averages

Code:
Table boss (7x5)/5 = 35/7 = 5
Icebooking* (22x5)+(1x3)/(22+1) = 113/23 = 4.91
Jomres (63x5)+(9x4)+(1x1) / (63+9+1) = 352/73 = 4.82
Room booker (6x5)+(2x4)/(6+2) = 38/8 = 4.75
Jongman (4x5)+(6x4)/(4+6) = 44/10 = 4.4


* I've added Icebooking in too to ensure that these figures don't negatively impact currently well reviewed and favoured extensions. I'm not looking for an unfair advantage here. Instead, I'm hoping to generate some discussion around how the scoring system can be improved to fairly reflect an extension's ongoing value within the Joomla environment.


Now, Table boss, which has the fewest reviews but is second oldest after Jomres and is obviously ignored by it's developer, demonstrates that this formula isn't working as it's ranked the highest now. Is there anything we can do to improve this?

Yes, we can throw the number of months since the last update into the mix.

Weighted averages including last update

Code:
Jomres ( (63x5)+(9x4)+(1x1) -0)   / (63+9+1) = 352/73 = 4.82
Icebooking* ( (22x5)+(1x3) -6) /(22+1) = 107/23 = 4.65
Table boss ( (7x5) -12 ) /5 = 23/7 = 3.28
Room booker ( (6x5)+(2x4) - 0 ) /(6+2) = 26/8 = 4.75
Jongman ( (4x5)+(6x4)/(4+6) - 13 )  = 31/10 = 3.1


Now, you may disagree with this calculation as it's favourable to Jomres. Icebooking has 23 reviews, of which 22 are 5 stars, so currently it appears to be more popular than Jomres, however bear in mind that it's been around a fraction of the amount of time that Jomres has and is not updated as often, so like everything you need to take things with a pinch of salt, however this calculation DOES appear to favour more recently updated extensions than abandoned ones. If Icebooking had been updated in the last couple of months then it would rank higher than Jomres.

Sure, there might be some people who will think that massaging their changelog will suffice to keep their extension rated better, but isn't that really the point, to get people updating their products?

Remember, I've based these calculations off the information that's currently available in the JED's db. There's no point in throwing other weighting factors into the mix, such as the frequency of updates, if we haven't captured that information. The JED might have done, but there's no evidence to say that it has therefore we'll assume it's not available therefore we can't use it.



NB. Before you mention it, yes I know that all of those mentioned extensions (except Icebooking) are J1.5 only and will be unpublished on the 1st of April, that's not my point. My point is that the scoring system used actually penalises me for having more reviews.

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Last edited by Vimes on Thu Feb 02, 2012 3:41 pm, edited 1 time in total.

Top
 Profile  
 
PostPosted: Mon Jan 30, 2012 8:17 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
Of course, in my hurry to not upset the missus and get grumbled at for letting dinner get cold, I forgot to wrap up the post. Now that I'm fed and watered...

I'd appreciate it if my developer colleagues who also have extensions listed on the JED would take a look at the figures posted here and let us know what they think about these proposals.

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Tue Jan 31, 2012 9:27 am 
User avatar
Joomla! Master
Joomla! Master
Online

Joined: Fri Aug 12, 2005 7:19 am
Posts: 10458
Location: Leeds, UK
This is a generic comment and not on the specific extensions or category above.

I do not believe that the frequency of updates or the time since the last release should be a factor. Yes it could be that the extension has been abandoned but it could also be that the extension was released perfect and requires no updates. (Or it could be so simple that no updates are required)

_________________
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/


Top
 Profile  
 
PostPosted: Tue Jan 31, 2012 9:36 am 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
brian wrote:
Yes it could be that the extension has been abandoned but it could also be that the extension was released perfect and requires no updates. (Or it could be so simple that no updates are required)


I understand where you are coming from, and I don't necessarily disagree with you. That said, if you have a group of extensions like booking systems, which do require ongoing work then the update times are important. In a category like "Corner Banners" then not so much, and this is where grouping extensions into categories works for us. As extensions are compared against other extensions in the same category only it would become less of an issue, one would think.

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Tue Jan 31, 2012 9:53 am 
User avatar
Joomla! Master
Joomla! Master
Online

Joined: Fri Aug 12, 2005 7:19 am
Posts: 10458
Location: Leeds, UK
Even in your category your belief that the extension requires constant updates is subjective and not absolute. A developer might choose to make 6 monthly releases or daily releases it doesn't make the extension necessarily better or worse. In addition adding this criteria would be very easy for an unscrupulous developer to abuse

_________________
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/


Top
 Profile  
 
PostPosted: Tue Jan 31, 2012 10:08 am 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
Look at it from another perspective. The products I've highlighted are all obviously abandonware, to all intents and purposes, yet the systems "score" better than Jomres.

In the last couple of weeks Jomres has seen 4 new excellent reviews, as they've been posted Jomres' score went from 4.22 to 4.23. At that rate Jomres will need about another 250 reviews simply to get over the 4.5 mark, which is obviously ludicrous. It's taken 5 years to get just 88 reviews, because it's a low volume extension compared to something like a SEF plugin.

I'm not saying that my solution is best, but I am saying that the current situation is immensely frustrating. It isn't worth my asking users to review Jomres any more. As a result, the scores on the JED will become even more irrelevant. That can't be good for the future of the JED.

The point of the post was to help people to understand just how ineffectual the JED's current scoring system is, and to invite people to suggest alternative scoring mechanisms. We all know that the JED's maintainers are a law unto themselves and we can't force them to change anything, but we can highlight to them the silliness of things as they stand now and ask them to consider possible alternatives.

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Tue Jan 31, 2012 10:51 am 
User avatar
Joomla! Master
Joomla! Master
Online

Joined: Fri Aug 12, 2005 7:19 am
Posts: 10458
Location: Leeds, UK
I definitely see the value in a scoring system that gives greater weight to more recent reviews I just dont see the additional need to give greater weight to more recent extension updates

_________________
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/


Top
 Profile  
 
PostPosted: Tue Jan 31, 2012 11:10 am 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Sat Jun 03, 2006 4:33 pm
Posts: 250
Brian, Vince I think we all agree that the JED needs better scoring system. The real problem is that we have no idea what a better scoring system should be :)

I'm currently reading this book http://www.amazon.com/Programming-Colle ... B0028N4WM4 and there are some great examples of scoring algorithms and everyone that wants to change the JED score system should read it!

If you ask me - we need to consider a lot of factors. Frequency of the updates, number of reviews, who made the review? There are users that are generally give higher scores and other that give lower scores (how do we weight those users). Also - the latest reviews should receive a higher weight, than the reviews from 2 years ago etc. It is a really complex problem and we should try to find a formula that fits best.

And yes - the average score is a very poor way to solve this problem.

_________________
http://compojoom.com


Top
 Profile  
 
PostPosted: Tue Jan 31, 2012 11:25 am 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
THE_AI wrote:
I'm currently reading this book http://www.amazon.com/Programming-Colle ... B0028N4WM4 and there are some great examples of scoring algorithms and everyone that wants to change the JED score system should read it!


Thanks for that, I've added it to the kindle as I'm sure it'll be useful for my own software.

Meanwhile, I believe that it might be useful to the JED maintainers if we were to provide them with some sample algorithms for them to consider. Anybody got any suggestions they'd like to throw out there?

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Wed Feb 01, 2012 2:30 pm 
Joomla! Enthusiast
Joomla! Enthusiast

Joined: Thu Nov 17, 2005 9:39 pm
Posts: 202
Hi,

I believe including update frequency, amount of views and such will lead to heavy misuse.

What about something simple to keep it fair?

For example:

- Random display by default, combined with a two-column layout. Additionally a simple filter that will display how it it currently does.

- two to four random extensions from the category at top of the listings


Top
 Profile  
 
PostPosted: Wed Feb 01, 2012 3:03 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
Seems pretty pointless as you've already got that in the category above.

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Wed Feb 01, 2012 3:46 pm 
Joomla! Enthusiast
Joomla! Enthusiast

Joined: Thu Nov 17, 2005 9:39 pm
Posts: 202
Why pointless? The category overview pages seem not to be that well visited, plus showing a few extensions from all subcategories, completely different things.

However, more important was the suggestion about general random display of listings and having 2 columns, instead of endless lists.

With the proposed scoring algorithm, it would take epically long for a new but great extension to get a rank that reflects its quality. That would be no option as well.


Top
 Profile  
 
PostPosted: Thu Feb 02, 2012 3:37 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
Ah, lovely, another review from a happy customer bumps our rating up to 4.24. Jolly good.

*sigh*

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Thu Feb 09, 2012 1:52 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
Yay 4.25 {does a little dance}

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Thu Feb 09, 2012 3:06 pm 
Joomla! Apprentice
Joomla! Apprentice

Joined: Sun Jul 12, 2009 4:59 pm
Posts: 48
I think, the update frequency has nothing to do with the ranking and scoring. Some developers may update their listings only for major new versions, other may do it every day.

I honestly thing that this discussion could be avoided if JED simply allowed premium listings. I know - it is a whole new story, but that's what should happen.

_________________
Birnik


Top
 Profile  
 
PostPosted: Thu Feb 09, 2012 3:08 pm 
User avatar
Joomla! Enthusiast
Joomla! Enthusiast

Joined: Mon Sep 18, 2006 1:17 pm
Posts: 219
The problem is, these discussions have been going on for years. Even inside the JED team there has been talk (and almost steps!) to change things like rating. Even talk about whether to have a rating system like now at all. Or just have thumbs up/down. Or have a more complex voting system (having to vote on multiple aspects, like support and such).

Anyway, more to the point, there have been new rating algorithms on the table... BUT:

The way the JED is run, changes won't be implemented. And if they ever do, it will take ages.
The JED is too busy to worry about usability or to even listen to what 'outsiders' have to say. (It won't even listen to what insiders say).

As long as the way (and people) the JED is run changes, it isn't much use trying to discuss the changes.
Nothing will happen, going by the last 3-4 years.

The JED is too busy and doesn't take on new people. People who offer their help are neglected. And people that do get in, run away after some time because... well, you can fill in the dots...

_________________
NoNumber.nl - Extensions for Joomla!
Advanced Module Manager, Articles Anywhere, Better Preview, CDN for Joomla!, Cache Cleaner, Components Anywhere, DB Replacer, Modals, Modules Anywhere, ReReplacer, Sliders, Sourcerer, Tabs, Tooltips, What? Nothing!


Top
 Profile  
 
PostPosted: Fri Feb 17, 2012 11:39 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
Another 5 star review, still sitting on 4.25. Jolly good.

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Fri Feb 17, 2012 11:42 pm 
User avatar
Joomla! Enthusiast
Joomla! Enthusiast

Joined: Mon Sep 18, 2006 1:17 pm
Posts: 219
Has this thread turned into a Jomres rating updates" thread?
Will have to unfollow if it is.

_________________
NoNumber.nl - Extensions for Joomla!
Advanced Module Manager, Articles Anywhere, Better Preview, CDN for Joomla!, Cache Cleaner, Components Anywhere, DB Replacer, Modals, Modules Anywhere, ReReplacer, Sliders, Sourcerer, Tabs, Tooltips, What? Nothing!


Top
 Profile  
 
PostPosted: Sat Feb 18, 2012 2:31 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
nonumber wrote:
Has this thread turned into a Jomres rating updates" thread?
Will have to unfollow if it is.


Heheh.

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Mon Apr 09, 2012 11:04 am 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Sat Aug 13, 2005 2:13 am
Posts: 263
Location: Sweden
I would go with incremental changes. Simply give some advantage to extensions with larger number of votes, even if the rating is little lower.
That way we could avoid that an extension with rating 5 average out of 3 votes is listed above an extension with 4.75 average out of 50 votes.

See how that works before taking other factors into account.

_________________
Emir Sakic
http://www.sakic.net


Top
 Profile  
 
PostPosted: Mon Apr 09, 2012 12:19 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
Using that approach, and the numbers I collected in January* then a calculation could look like this :

(Cumulative scores * reviews) / sum of scores (?) (I don't really do maths, so couldn't say what the correct terms would be).

Code:
Jomres    ( (315+36+1) *1.88)   /73   =   (352*1.88)/73    = 9.06
Icebooking ( (110+3) *1.23)/23      =   (113*1.23)/23   = 6.04
Table boss ( (35) ) * 1.07)/7      =   (35*1.07)/7   = 5.35
Room booker ( (30)+(8) *1.1 ) /8   =   (38*1.1)/8      = 5.22
Jongman    ( (20)+(24) *1.1)/10      =   (44*1.1)10   = 4.84


This certainly feels more representative of the current state of play, I'd encourage other devs to try making the same calculations for the categories that pertain to them, perhaps they can feed back if it works for them too.


* Please remember, I'm trying to figure out an algorithm that doesn't require the collection of additional data that might not have been available when the original scores were collected. Certainly there are better ways of calculating scores (I liked the idea of adding "weight" to a reviewer's reputation).

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Mon Apr 09, 2012 1:13 pm 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Sat Aug 13, 2005 2:13 am
Posts: 263
Location: Sweden
Vince, can you explain what is "Cumulative scores", "reviews" and "sum of scores" in your equation?

_________________
Emir Sakic
http://www.sakic.net


Top
 Profile  
 
PostPosted: Mon Apr 09, 2012 3:15 pm 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Thu May 04, 2006 9:11 am
Posts: 479
Location: Germany
I'm afraid it's pointless to discuss this unless there is either an openly stated desire by the JED leadership to change this, or a broad support for this in the developer community. The first has yet to materialise and the second is unlikely because half of the developers are just fine because the old system gives them preferential treatment.

The only thing we can do is offer quick and good solutions, like I did with my last topic on removing non-review votes. Simple, uncontroversial, easy to implement. As for the stuff in this topic - maybe it would help, but I think we can achieve it a lot simpler:

Reduce the value of a vote by half for each year of age, like interest.

or in math:

Code:
$actual_rating = ( ( 0.5 / $vote->age_years ) * $vote->value ) * $vote->age_years;


The tricky part is making this work for an accumulative value. Right now, the computation is simple, it is keeping a running total:

Code:
$new_rating = ((($link->link_rating * $link->link_votes) + $rating) / ++$link->link_votes);


Instead of that, we would need to recompute the entire stack each time, but the trick is to reduce not the rating value, but the impact it has. As you can see above, it is a simple addition of all votes divided by the number of votes. So reducing the value of a vote would mean that you just fall shorter and shorter of your goal. Instead, you have to reduce both:

Code:
$vote_add = 0;
$vote_max = 0;
foreach ( $votes as $vote ) {
   if ( vote->age_years ) {
      $vote_max += ( ( 0.5 / $vote->age_years ) * 5 ) * $vote->age_years;
      $vote_add += ( ( 0.5 / $vote->age_years ) * $vote->rating ) * $vote->age_years;
   } else {
      $vote_max += 5;
      $vote_add += $vote->rating;
   }
}

$total = $vote_max / $vote_add;


This shows how differently extensions are evaluated (the rating being put in bold):

Image

I think it is obvious that it is still a very mild proposal. To further underline how crass the difference is, though, consider that in the current system, in case 1, the extension developer would have to get 17 straight 5 star reviews to get the same score as in the proposed system. Meaning he would have to almost double the number of votes.

In general, this keeps both sides on their toes - if there is an outdated extension that still does the job perfectly, people will continue to review it well. If a developer had a mixed start, she/he can push in more work, get people to use the software and not be destined to be rated based on years-old opinions.

(For instance, I computed that for AEC to get up 0.3 points, I would have to get 50 straight five star votes. And that would still place it below the next-in-line competitor. So I'm really not posting this to be self serving - the system just happens to be obviously unfair.)

_________________
Developer of the AEC Membership Management Component: http://valanx.org
Fellow of the Free Software Foundation Europe (and so can you: http://www.fsfe.org !)


Top
 Profile  
 
PostPosted: Mon Apr 09, 2012 4:31 pm 
User avatar
Joomla! Ace
Joomla! Ace

Joined: Fri Aug 19, 2005 12:14 am
Posts: 1524
Location: United Kingdom
Saka wrote:
Vince, can you explain what is "Cumulative scores", "reviews" and "sum of scores" in your equation?


Originally I posted

Code:
Jomres ( (63x5)+(9x4)+(1x1) -0)   / (63+9+1) = 352/73 = 4.82


( 63 x 5 stars + 9x4 stars + 1x1 stars, minus the months since the last update ) / sum 63+9+1

Jomres ( (315+36+1) *1.88) /73 = (352*1.88)/73 = 9.06

This time ( (315 = 63 x 5 ...etc...) / ((the number of reviews) / 100 +1) / sum 63+9+1

As I said, I'm not that hot on maths, but IMO this algorithm is simple yet seems to bear out my percieved value of the extensions referenced. YMMV, naturally.


skOre wrote:
I'm afraid it's pointless to discuss this unless there is either an openly stated desire by the JED leadership to change this, or a broad support for this in the developer community.


How many years have we been begging them to sort this out? I'd say a good 5 or 6 years. Nothing's happened up until now, I don't expect that to change in the near future.

skOre wrote:
the second is unlikely because half of the developers are just fine because the old system gives them preferential treatment.


It's benefitting the people who are least active in updating their plugins and requesting reviews, so in reality, the current system benefits nobody. The least active developers don't care enough, and they're the ones who're the most "well regarded" by the scoring system.

_________________
http://www.jomres.net THE online hotel booking and reservation system for Joomla


Top
 Profile  
 
PostPosted: Mon Apr 09, 2012 6:54 pm 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Sat Aug 13, 2005 2:13 am
Posts: 263
Location: Sweden
As pointless as this discussion is...
If I understand correctly, I don't think this algo is good enough as it would still give rate 5 to an extension that has two 5 stars out of 2 reviews total, while one that has 50 reviews and avarage 4.7 would be below. 48 reviews with 5 stars and 2 with 1 star must weight much more than total 2 5-star reviews.

_________________
Emir Sakic
http://www.sakic.net


Top
 Profile  
 
PostPosted: Mon Apr 09, 2012 7:44 pm 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Thu May 04, 2006 9:11 am
Posts: 479
Location: Germany
That's actually a tricky one and it shines a little more light into the deeper problem - that ratings don't solve a particular problem or provide any actual value besides constituting a rating. Once you start down that path, you end up where I ended up - seeing that what you actually need is to answer the question "does this software solve MY problems" - a situation that no popularity list in the world can solve because most users look for unique solutions (whether they actually have unique problems is a different discussion).

This is also why I'm a little more hesitant than Vimes on this - if an outdated extension does the job for a user, it does the job for a user. Period. I want users to be happy. It's a problem that you can never really solve, so the best you can do is somewhat approximate a solution that gets you 95% there.

For now, as much as it hurts, we have to do with pragmatism and the pragmatic approach to your problem is saying that if it really is one, it will, in time, solve itself. Because let's dissect this problem: In most cases, "few votes, but on top" means that the extension is very simple and solves the problems for a lot of the users. In my category there is, for instance, 'Nice Memberships' and it does what a good percentage of people want: Put PayPal into your registration process.

Now - do I really want those clients? I argue that no, I don't want that. Ergo, trying to compete with that vendor means that I put energy into trying to get after clients that don't even want what I offer. My place is offering more functionality should people need it. So once they figure out that they need invoices, tax computation etc. etc., I have to be there for them.

At the end of the day, users usually go through a list of 5 to 10 extensions until they have made a choice. So as long as the top 10 gives them the choice they deserve, the system works. Obsessing over having it objectively grade every software to perfection is pointless, but with simple tricks like the one I wrote about, you can at least keep it fresh.

The goal is for the list of choices to be relevant, not to stroke the ego of developers.

_________________
Developer of the AEC Membership Management Component: http://valanx.org
Fellow of the Free Software Foundation Europe (and so can you: http://www.fsfe.org !)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 26 posts ] 



Who is online

Users browsing this forum: No registered users and 11 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group