Proposal to improve the JED's scoring algorithms

Here you can contact the editors of our Extensions site, as well as access infomation relating to this site.

Moderator: JED Team

Forum rules
Forum Rules
READ ME <-- please read before posting, this means YOU.
Locked
User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Proposal to improve the JED's scoring algorithms

Post by Vimes » Mon Jan 30, 2012 6:50 pm

This issue gets mentioned time and again, but then it seems to slip down the back of the feature request's sofa and forgotten, so I'm going to raise it again.

Could you please resolve the review scoring system used in the JED? At the moment it is, to put not too fine a point on it, unfair, meaning that my core product scores "worse" than many of my less developed competitors despite the level of work we put into it being significantly higher.


Let's take a look at how things are now.
http://extensions.joomla.org/extensions ... ns/booking

Jomres is listed on the JED under Vertical Markets - Booking : Currently it is placed 8th in the scores but this seems odd, given the amount of work we put into it, and the reviews. Let's look at some of the competition :


Roombooker. 2nd.
Added 05/04/2010
Last updated 5/01/2011.
10 reviews.
http://extensions.joomla.org/extensions ... king/12082
Forum http://www.joomplace.com/forum/joomla-c ... ooker.html

Jongman. 5th.
Added 01/10/2010
Last updated 24/12/2010.
10 reviews.
http://extensions.joomla.org/extensions ... king/14221
Forum http://www.joomlant.org/forum/6-jongman ... ystem.html

Table boss. 7th.
Added 22/09/2007
Last updated 08/01/2011 7 reviews.
http://extensions.joomla.org/extensions ... oking/3110
Forum http://tableboss.com/component/option,c ... 1/lang,en/

Jomres 8th.
Added 05/03/2006
Last updated 30/01/2012
88 reviews.
http://extensions.joomla.org/extensions ... ooking/335
Forum http://www.jomres.net/forum


Jomres aside, none of them have been updated in the last year.

I've linked to these product's forum pages. From them you can see that there's almost no activity in them, at least two of them look dead. In comparison, our forum alone has had a dozen posts this morning, and that's not including the emails we've responded to via the ticket system.

Since these extensions were last updated, Jomres has seen 43 new versions. It's probably one of the most heavily developed of the Joomla components. It has two people supporting and developing for it, full time. Indeed, if you look at the reviews you'll see that we pride ourselves on our speedy and accurate support.


Yet, somehow, Jomres is still showing 8th in the list. Why is that? I believe that it's the number of reviews we've got that are actually holding us back from getting a better score. We have more users, whereas these stagnant projects aren't gaining new users therefore their scores aren't changing. It's their inactivity that's hurting Jomres', and more importantly potential user's impressions of Jomres. In a community that agrees that software that's under development is going to be of higher quality than code that's simply been ignored for the last few years, surely this is an anathema? It's actually to my competitor's benefit that they don't update their products and don't encourage users to review them on the JED. If that's not a bit mad, I don't know what is.

Naturally, this is very frustrating to me, and I think it's jolly unfair :)

How can we improve the scoring system on the JED?

Well, I've done some back of a fag packet calculations. My numbers might be slightly off, but if they are it's not by much.

First, let's look at the existing data.

Review score values (sum and count of reviews)

Code: Select all

Roombooker Score 48, reviews 10.
Jongman Score 44, reviews 10.
Table boss Score 35 reviews 7. 
Jomres Score 366 reviews 81* 
*Yes, I know JED says 88 reviews, but not all of them have ratings attached, as those reviews predate the rating functionality that was added later.

Months since last update

Code: Select all

Roombooker 0
Jongman 13
Table boss 12
Jomres 0

JED ratings (How JED currently scores extensions)

Code: Select all

Room booker 4.8
Jongman 4.57
Table boss 4.36
Jomres 4.23

Simple averages score

Code: Select all

Table boss 5
Room booker  4.8
Jomres 4.51
Jongman 4.4
Clearly, averages aren't going to help,

Weighted averages (n1 * N1) + (n2 * N2) / (n1 + n2) are a minor improvement, but they don't take an extension's inactivity into account, let's see the numbers :

Simple weighted averages

Code: Select all

Table boss (7x5)/5 = 35/7 = 5
Icebooking* (22x5)+(1x3)/(22+1) = 113/23 = 4.91
Jomres (63x5)+(9x4)+(1x1) / (63+9+1) = 352/73 = 4.82
Room booker (6x5)+(2x4)/(6+2) = 38/8 = 4.75
Jongman (4x5)+(6x4)/(4+6) = 44/10 = 4.4
* I've added Icebooking in too to ensure that these figures don't negatively impact currently well reviewed and favoured extensions. I'm not looking for an unfair advantage here. Instead, I'm hoping to generate some discussion around how the scoring system can be improved to fairly reflect an extension's ongoing value within the Joomla environment.


Now, Table boss, which has the fewest reviews but is second oldest after Jomres and is obviously ignored by it's developer, demonstrates that this formula isn't working as it's ranked the highest now. Is there anything we can do to improve this?

Yes, we can throw the number of months since the last update into the mix.

Weighted averages including last update

Code: Select all

Jomres ( (63x5)+(9x4)+(1x1) -0)   / (63+9+1) = 352/73 = 4.82
Icebooking* ( (22x5)+(1x3) -6) /(22+1) = 107/23 = 4.65
Table boss ( (7x5) -12 ) /5 = 23/7 = 3.28
Room booker ( (6x5)+(2x4) - 0 ) /(6+2) = 26/8 = 4.75
Jongman ( (4x5)+(6x4)/(4+6) - 13 )  = 31/10 = 3.1
Now, you may disagree with this calculation as it's favourable to Jomres. Icebooking has 23 reviews, of which 22 are 5 stars, so currently it appears to be more popular than Jomres, however bear in mind that it's been around a fraction of the amount of time that Jomres has and is not updated as often, so like everything you need to take things with a pinch of salt, however this calculation DOES appear to favour more recently updated extensions than abandoned ones. If Icebooking had been updated in the last couple of months then it would rank higher than Jomres.

Sure, there might be some people who will think that massaging their changelog will suffice to keep their extension rated better, but isn't that really the point, to get people updating their products?

Remember, I've based these calculations off the information that's currently available in the JED's db. There's no point in throwing other weighting factors into the mix, such as the frequency of updates, if we haven't captured that information. The JED might have done, but there's no evidence to say that it has therefore we'll assume it's not available therefore we can't use it.



NB. Before you mention it, yes I know that all of those mentioned extensions (except Icebooking) are J1.5 only and will be unpublished on the 1st of April, that's not my point. My point is that the scoring system used actually penalises me for having more reviews.
Last edited by Vimes on Thu Feb 02, 2012 3:41 pm, edited 1 time in total.
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Mon Jan 30, 2012 8:17 pm

Of course, in my hurry to not upset the missus and get grumbled at for letting dinner get cold, I forgot to wrap up the post. Now that I'm fed and watered...

I'd appreciate it if my developer colleagues who also have extensions listed on the JED would take a look at the figures posted here and let us know what they think about these proposals.
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
brian
Joomla! Master
Joomla! Master
Posts: 12781
Joined: Fri Aug 12, 2005 7:19 am
Location: Leeds, UK
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by brian » Tue Jan 31, 2012 9:27 am

This is a generic comment and not on the specific extensions or category above.

I do not believe that the frequency of updates or the time since the last release should be a factor. Yes it could be that the extension has been abandoned but it could also be that the extension was released perfect and requires no updates. (Or it could be so simple that no updates are required)
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Tue Jan 31, 2012 9:36 am

brian wrote: Yes it could be that the extension has been abandoned but it could also be that the extension was released perfect and requires no updates. (Or it could be so simple that no updates are required)
I understand where you are coming from, and I don't necessarily disagree with you. That said, if you have a group of extensions like booking systems, which do require ongoing work then the update times are important. In a category like "Corner Banners" then not so much, and this is where grouping extensions into categories works for us. As extensions are compared against other extensions in the same category only it would become less of an issue, one would think.
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
brian
Joomla! Master
Joomla! Master
Posts: 12781
Joined: Fri Aug 12, 2005 7:19 am
Location: Leeds, UK
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by brian » Tue Jan 31, 2012 9:53 am

Even in your category your belief that the extension requires constant updates is subjective and not absolute. A developer might choose to make 6 monthly releases or daily releases it doesn't make the extension necessarily better or worse. In addition adding this criteria would be very easy for an unscrupulous developer to abuse
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Tue Jan 31, 2012 10:08 am

Look at it from another perspective. The products I've highlighted are all obviously abandonware, to all intents and purposes, yet the systems "score" better than Jomres.

In the last couple of weeks Jomres has seen 4 new excellent reviews, as they've been posted Jomres' score went from 4.22 to 4.23. At that rate Jomres will need about another 250 reviews simply to get over the 4.5 mark, which is obviously ludicrous. It's taken 5 years to get just 88 reviews, because it's a low volume extension compared to something like a SEF plugin.

I'm not saying that my solution is best, but I am saying that the current situation is immensely frustrating. It isn't worth my asking users to review Jomres any more. As a result, the scores on the JED will become even more irrelevant. That can't be good for the future of the JED.

The point of the post was to help people to understand just how ineffectual the JED's current scoring system is, and to invite people to suggest alternative scoring mechanisms. We all know that the JED's maintainers are a law unto themselves and we can't force them to change anything, but we can highlight to them the silliness of things as they stand now and ask them to consider possible alternatives.
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
brian
Joomla! Master
Joomla! Master
Posts: 12781
Joined: Fri Aug 12, 2005 7:19 am
Location: Leeds, UK
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by brian » Tue Jan 31, 2012 10:51 am

I definitely see the value in a scoring system that gives greater weight to more recent reviews I just dont see the additional need to give greater weight to more recent extension updates
"Exploited yesterday... Hacked tomorrow"
Blog http://brian.teeman.net/
Joomla Hidden Secrets http://hiddenjoomlasecrets.com/

User avatar
THE_AI
Joomla! Explorer
Joomla! Explorer
Posts: 252
Joined: Sat Jun 03, 2006 4:33 pm
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by THE_AI » Tue Jan 31, 2012 11:10 am

Brian, Vince I think we all agree that the JED needs better scoring system. The real problem is that we have no idea what a better scoring system should be :)

I'm currently reading this book http://www.amazon.com/Programming-Colle ... B0028N4WM4 and there are some great examples of scoring algorithms and everyone that wants to change the JED score system should read it!

If you ask me - we need to consider a lot of factors. Frequency of the updates, number of reviews, who made the review? There are users that are generally give higher scores and other that give lower scores (how do we weight those users). Also - the latest reviews should receive a higher weight, than the reviews from 2 years ago etc. It is a really complex problem and we should try to find a formula that fits best.

And yes - the average score is a very poor way to solve this problem.

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Tue Jan 31, 2012 11:25 am

THE_AI wrote: I'm currently reading this book http://www.amazon.com/Programming-Colle ... B0028N4WM4 and there are some great examples of scoring algorithms and everyone that wants to change the JED score system should read it!
Thanks for that, I've added it to the kindle as I'm sure it'll be useful for my own software.

Meanwhile, I believe that it might be useful to the JED maintainers if we were to provide them with some sample algorithms for them to consider. Anybody got any suggestions they'd like to throw out there?
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

TheMuffinMan
Joomla! Explorer
Joomla! Explorer
Posts: 282
Joined: Thu Nov 17, 2005 9:39 pm

Re: Proposal to improve the JED's scoring algorithms

Post by TheMuffinMan » Wed Feb 01, 2012 2:30 pm

Hi,

I believe including update frequency, amount of views and such will lead to heavy misuse.

What about something simple to keep it fair?

For example:

- Random display by default, combined with a two-column layout. Additionally a simple filter that will display how it it currently does.

- two to four random extensions from the category at top of the listings

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Wed Feb 01, 2012 3:03 pm

Seems pretty pointless as you've already got that in the category above.
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

TheMuffinMan
Joomla! Explorer
Joomla! Explorer
Posts: 282
Joined: Thu Nov 17, 2005 9:39 pm

Re: Proposal to improve the JED's scoring algorithms

Post by TheMuffinMan » Wed Feb 01, 2012 3:46 pm

Why pointless? The category overview pages seem not to be that well visited, plus showing a few extensions from all subcategories, completely different things.

However, more important was the suggestion about general random display of listings and having 2 columns, instead of endless lists.

With the proposed scoring algorithm, it would take epically long for a new but great extension to get a rank that reflects its quality. That would be no option as well.

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Thu Feb 02, 2012 3:37 pm

Ah, lovely, another review from a happy customer bumps our rating up to 4.24. Jolly good.

*sigh*
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Thu Feb 09, 2012 1:52 pm

Yay 4.25 {does a little dance}
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

birnik
Joomla! Intern
Joomla! Intern
Posts: 50
Joined: Sun Jul 12, 2009 4:59 pm

Re: Proposal to improve the JED's scoring algorithms

Post by birnik » Thu Feb 09, 2012 3:06 pm

I think, the update frequency has nothing to do with the ranking and scoring. Some developers may update their listings only for major new versions, other may do it every day.

I honestly thing that this discussion could be avoided if JED simply allowed premium listings. I know - it is a whole new story, but that's what should happen.
Birnik

User avatar
nonumber
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 223
Joined: Mon Sep 18, 2006 1:17 pm
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by nonumber » Thu Feb 09, 2012 3:08 pm

The problem is, these discussions have been going on for years. Even inside the JED team there has been talk (and almost steps!) to change things like rating. Even talk about whether to have a rating system like now at all. Or just have thumbs up/down. Or have a more complex voting system (having to vote on multiple aspects, like support and such).

Anyway, more to the point, there have been new rating algorithms on the table... BUT:

The way the JED is run, changes won't be implemented. And if they ever do, it will take ages.
The JED is too busy to worry about usability or to even listen to what 'outsiders' have to say. (It won't even listen to what insiders say).

As long as the way (and people) the JED is run changes, it isn't much use trying to discuss the changes.
Nothing will happen, going by the last 3-4 years.

The JED is too busy and doesn't take on new people. People who offer their help are neglected. And people that do get in, run away after some time because... well, you can fill in the dots...
NoNumber.nl - Extensions for Joomla!
Advanced Module Manager, Articles Anywhere, Better Preview, CDN for Joomla!, Cache Cleaner, Components Anywhere, DB Replacer, Modals, Modules Anywhere, ReReplacer, Sliders, Sourcerer, Tabs, Tooltips, What? Nothing!

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Fri Feb 17, 2012 11:39 pm

Another 5 star review, still sitting on 4.25. Jolly good.
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
nonumber
Joomla! Enthusiast
Joomla! Enthusiast
Posts: 223
Joined: Mon Sep 18, 2006 1:17 pm
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by nonumber » Fri Feb 17, 2012 11:42 pm

Has this thread turned into a Jomres rating updates" thread?
Will have to unfollow if it is.
NoNumber.nl - Extensions for Joomla!
Advanced Module Manager, Articles Anywhere, Better Preview, CDN for Joomla!, Cache Cleaner, Components Anywhere, DB Replacer, Modals, Modules Anywhere, ReReplacer, Sliders, Sourcerer, Tabs, Tooltips, What? Nothing!

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Sat Feb 18, 2012 2:31 pm

nonumber wrote:Has this thread turned into a Jomres rating updates" thread?
Will have to unfollow if it is.
Heheh.
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
Saka
Joomla! Explorer
Joomla! Explorer
Posts: 263
Joined: Sat Aug 13, 2005 2:13 am
Location: Sweden
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Saka » Mon Apr 09, 2012 11:04 am

I would go with incremental changes. Simply give some advantage to extensions with larger number of votes, even if the rating is little lower.
That way we could avoid that an extension with rating 5 average out of 3 votes is listed above an extension with 4.75 average out of 50 votes.

See how that works before taking other factors into account.

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Mon Apr 09, 2012 12:19 pm

Using that approach, and the numbers I collected in January* then a calculation could look like this :

(Cumulative scores * reviews) / sum of scores (?) (I don't really do maths, so couldn't say what the correct terms would be).

Code: Select all

Jomres 	( (315+36+1) *1.88)	/73	=	(352*1.88)/73 	= 9.06
Icebooking ( (110+3) *1.23)/23		=	(113*1.23)/23	= 6.04
Table boss ( (35) ) * 1.07)/7		=	(35*1.07)/7	= 5.35
Room booker ( (30)+(8) *1.1 ) /8	=	(38*1.1)/8		= 5.22
Jongman 	( (20)+(24) *1.1)/10		=	(44*1.1)10	= 4.84
This certainly feels more representative of the current state of play, I'd encourage other devs to try making the same calculations for the categories that pertain to them, perhaps they can feed back if it works for them too.


* Please remember, I'm trying to figure out an algorithm that doesn't require the collection of additional data that might not have been available when the original scores were collected. Certainly there are better ways of calculating scores (I liked the idea of adding "weight" to a reviewer's reputation).
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
Saka
Joomla! Explorer
Joomla! Explorer
Posts: 263
Joined: Sat Aug 13, 2005 2:13 am
Location: Sweden
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Saka » Mon Apr 09, 2012 1:13 pm

Vince, can you explain what is "Cumulative scores", "reviews" and "sum of scores" in your equation?

User avatar
skOre
Joomla! Explorer
Joomla! Explorer
Posts: 474
Joined: Thu May 04, 2006 9:11 am
Location: Germany
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by skOre » Mon Apr 09, 2012 3:15 pm

I'm afraid it's pointless to discuss this unless there is either an openly stated desire by the JED leadership to change this, or a broad support for this in the developer community. The first has yet to materialise and the second is unlikely because half of the developers are just fine because the old system gives them preferential treatment.

The only thing we can do is offer quick and good solutions, like I did with my last topic on removing non-review votes. Simple, uncontroversial, easy to implement. As for the stuff in this topic - maybe it would help, but I think we can achieve it a lot simpler:

Reduce the value of a vote by half for each year of age, like interest.

or in math:

Code: Select all

$actual_rating = ( ( 0.5 / $vote->age_years ) * $vote->value ) * $vote->age_years;
The tricky part is making this work for an accumulative value. Right now, the computation is simple, it is keeping a running total:

Code: Select all

$new_rating = ((($link->link_rating * $link->link_votes) + $rating) / ++$link->link_votes);
Instead of that, we would need to recompute the entire stack each time, but the trick is to reduce not the rating value, but the impact it has. As you can see above, it is a simple addition of all votes divided by the number of votes. So reducing the value of a vote would mean that you just fall shorter and shorter of your goal. Instead, you have to reduce both:

Code: Select all

$vote_add = 0;
$vote_max = 0;
foreach ( $votes as $vote ) {
   if ( vote->age_years ) {
      $vote_max += ( ( 0.5 / $vote->age_years ) * 5 ) * $vote->age_years;
      $vote_add += ( ( 0.5 / $vote->age_years ) * $vote->rating ) * $vote->age_years;
   } else {
      $vote_max += 5;
      $vote_add += $vote->rating;
   }
}

$total = $vote_max / $vote_add;
This shows how differently extensions are evaluated (the rating being put in bold):

Image

I think it is obvious that it is still a very mild proposal. To further underline how crass the difference is, though, consider that in the current system, in case 1, the extension developer would have to get 17 straight 5 star reviews to get the same score as in the proposed system. Meaning he would have to almost double the number of votes.

In general, this keeps both sides on their toes - if there is an outdated extension that still does the job perfectly, people will continue to review it well. If a developer had a mixed start, she/he can push in more work, get people to use the software and not be destined to be rated based on years-old opinions.

(For instance, I computed that for AEC to get up 0.3 points, I would have to get 50 straight five star votes. And that would still place it below the next-in-line competitor. So I'm really not posting this to be self serving - the system just happens to be obviously unfair.)
Developer of the AEC Membership Management Component: http://valanx.org
Fellow of the Free Software Foundation Europe (and so can you: http://www.fsfe.org !)

User avatar
Vimes
Joomla! Ace
Joomla! Ace
Posts: 1675
Joined: Fri Aug 19, 2005 12:14 am
Location: United Kingdom
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Vimes » Mon Apr 09, 2012 4:31 pm

Saka wrote:Vince, can you explain what is "Cumulative scores", "reviews" and "sum of scores" in your equation?
Originally I posted

Code: Select all

Jomres ( (63x5)+(9x4)+(1x1) -0)   / (63+9+1) = 352/73 = 4.82
( 63 x 5 stars + 9x4 stars + 1x1 stars, minus the months since the last update ) / sum 63+9+1

Jomres ( (315+36+1) *1.88) /73 = (352*1.88)/73 = 9.06

This time ( (315 = 63 x 5 ...etc...) / ((the number of reviews) / 100 +1) / sum 63+9+1

As I said, I'm not that hot on maths, but IMO this algorithm is simple yet seems to bear out my percieved value of the extensions referenced. YMMV, naturally.

skOre wrote:I'm afraid it's pointless to discuss this unless there is either an openly stated desire by the JED leadership to change this, or a broad support for this in the developer community.
How many years have we been begging them to sort this out? I'd say a good 5 or 6 years. Nothing's happened up until now, I don't expect that to change in the near future.
skOre wrote:the second is unlikely because half of the developers are just fine because the old system gives them preferential treatment.
It's benefitting the people who are least active in updating their plugins and requesting reviews, so in reality, the current system benefits nobody. The least active developers don't care enough, and they're the ones who're the most "well regarded" by the scoring system.
http://www.jomres.net THE online hotel booking and reservation system for Joomla and Wordpress.

User avatar
Saka
Joomla! Explorer
Joomla! Explorer
Posts: 263
Joined: Sat Aug 13, 2005 2:13 am
Location: Sweden
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by Saka » Mon Apr 09, 2012 6:54 pm

As pointless as this discussion is...
If I understand correctly, I don't think this algo is good enough as it would still give rate 5 to an extension that has two 5 stars out of 2 reviews total, while one that has 50 reviews and avarage 4.7 would be below. 48 reviews with 5 stars and 2 with 1 star must weight much more than total 2 5-star reviews.

User avatar
skOre
Joomla! Explorer
Joomla! Explorer
Posts: 474
Joined: Thu May 04, 2006 9:11 am
Location: Germany
Contact:

Re: Proposal to improve the JED's scoring algorithms

Post by skOre » Mon Apr 09, 2012 7:44 pm

That's actually a tricky one and it shines a little more light into the deeper problem - that ratings don't solve a particular problem or provide any actual value besides constituting a rating. Once you start down that path, you end up where I ended up - seeing that what you actually need is to answer the question "does this software solve MY problems" - a situation that no popularity list in the world can solve because most users look for unique solutions (whether they actually have unique problems is a different discussion).

This is also why I'm a little more hesitant than Vimes on this - if an outdated extension does the job for a user, it does the job for a user. Period. I want users to be happy. It's a problem that you can never really solve, so the best you can do is somewhat approximate a solution that gets you 95% there.

For now, as much as it hurts, we have to do with pragmatism and the pragmatic approach to your problem is saying that if it really is one, it will, in time, solve itself. Because let's dissect this problem: In most cases, "few votes, but on top" means that the extension is very simple and solves the problems for a lot of the users. In my category there is, for instance, 'Nice Memberships' and it does what a good percentage of people want: Put PayPal into your registration process.

Now - do I really want those clients? I argue that no, I don't want that. Ergo, trying to compete with that vendor means that I put energy into trying to get after clients that don't even want what I offer. My place is offering more functionality should people need it. So once they figure out that they need invoices, tax computation etc. etc., I have to be there for them.

At the end of the day, users usually go through a list of 5 to 10 extensions until they have made a choice. So as long as the top 10 gives them the choice they deserve, the system works. Obsessing over having it objectively grade every software to perfection is pointless, but with simple tricks like the one I wrote about, you can at least keep it fresh.

The goal is for the list of choices to be relevant, not to stroke the ego of developers.
Developer of the AEC Membership Management Component: http://valanx.org
Fellow of the Free Software Foundation Europe (and so can you: http://www.fsfe.org !)


Locked

Return to “extensions.joomla.org - Feedback/Information”