JED Review/Vote Cleanup

[David-Andrew]

Wow Matt and the JED team, just wanted to say: thanks for listening! It's refreshing!

[Webdongle]

I agree that old votes should not count in the score. Software evolves over time. That said, losing old reviews is unfair. ....

Why not leave the old reviews alone without counting them towards the rating? ...

That sounds good, keeping the old reviews. Perhaps only if the developer is shown not to be active and gives no support ... remove old reviews for those ?

Removing old reviews of extensions of a developer who does not provide updates or some form of support ... documentation etc ... will remove the 'dead wood'.

Also an open policy when a review is disputed would be good. That would make it a fairer system and would give anyone (accused of false voting) a chance to prove their innocence. Also if JED published a review that it openly accepts that review is valid ... and does not retract its decision on a whim.

Perhaps a system where various criteria were given a value rather than just a rating for overall usage ? (With a space to comment on each criterion). That would make it clear what could be commented on ... as well as why the reviewer scored each criterion for that extension.

[nikosdion]

It's easy to figure out if an extension is no longer actively developed: the last released version will be two lifetimes ago. It's also easy to spot lousy service: the very month the service level drops to subpar the number of negative reviews goes skyhigh. No need for JED to try to create a thick layer of convoluted rules to substitute common sense.

[Webdongle]

My point about reviews for old unsupported extensions was a compromise between deleting all the old reviews and not deleting any.

My point about having criteria (with a comment for each) was to make things less ambiguous as to what could be included and what could not. There have been many posts in the past where there has been inconstancy as to what can and cant be included in a review. Having specific criteria would simplify things not complicate them. And it would provide the same 'measuring stick' for the extensions.

[nikosdion]

This is a common misconception. The criteria have to take into account the kind and complexity for an extension.

Something as fast evolving and complicated as Akeeba Subscriptions tends to have several dozens reviews every year. Due to its complexity it begs rapid development and that brings more reviews. Definitely it should be consider active and by your standards it is.

Something like my K2 - JCE advlink plugin only has a couple of reviews despite accruing several hundred downloads per month. For a year it was more downloaded than Akeeba Subscriptions. It is very simple. There is nothing to further develop, save a quantum change in K2 or JCE. For all intents and purposes it is a fully supported extension. But it has little to no reviews. I think it has no review in 2012. Should it be considered an inactive extension? Would it be fair if I had to actually pay someone to submit a couple of fake reviews every year and release dummy updates (which only change a comma here and a fullstop there) just to keep it from being expunged?

Don't suppose that only extensions with a gazillion updates every month are useful and relevant. Sometimes there are those too small to be further developed, yet highly useful when you need their functionality, extensions. As I said, don't add convoluted rules to replace common sense. In other words, if it ain't broken don't try to fix it

[Webdongle]

...
Something like my K2 - JCE advlink plugin only has a couple of reviews despite accruing several hundred downloads per month. ... I think it has no review in 2012. Should it be considered an inactive extension? Would it be fair if I had to actually pay someone to submit a couple of fake reviews every year and release dummy updates ...

I see what you mean ... the defining of active is difficult and could be arbitrary. And extensions with broken links are already removed from JED. OK scratch that idea.

[Vimes]

Somebody suggested a "keep alive" email, where an email is set once in a while where if a link in it isn't clicked then after a couple of retries the extension's marked as dead. Might be worth pursuing.

[KonstantinDK]

An option to view only those extensions, compatible with specific version of joomla will definitely be nice.
Also, an option to display ratings and most popular extensions /In the last/3 months/1 year/all-time/ may work good.
Also, since we are gonna have new Joomla version every 6 months reviews have to state which version of joomla they are using.

[Webdongle]

....
Also, since we are gonna have new Joomla version every 6 months reviews have to state which version of joomla they are using.

Perhaps by having a field in the form(used for reviews) would be good for enforcing that ?

[TheMuffinMan]

...another thing that would be nice to have is, either randomly ordered extensions in the listing categories by default and other sorting options on demand or randomly showcased extensions directly in the listing pages -- on top, instead on the category overview pages. Most people visit the listing pages rather than the category overview ones.

However, any solution to help newer extensions not go get "out of radar" would help.

But I guess there will be another announcement after 30th asking for feedback.

[mattbaylor]

Part 1 complete. All votes with out reviews have been removed.

[nonumber]

The votes that have no review attached have been removed yesterday/today.
So you will probably notice a drop in amount of votes for your extensions.
Also that could have an impact on your position in the category and top rated/voted list views.

The mention of number of votes has now been removed from the list views, as that is now the same as the number of reviews. So instead of saying: "123 votes 123 reviews", it now only says "123 reviews".

And the "Most Voted" link has been removed from the Directory Menu on the bottom left. As this now gives the same result as the "Most Reviewed" list.

[Vimes]

Damn, it's good to see some movement on this, finally. Keep up the good work guys/girls.

[Alex Dobrin]

There is a difference in the number of reviews displayed in the list with extensions ( Akeeba Backup has 770 reviews here : http://extensions.joomla.org/extensions/most-reviewed ) and the number of votes displayed in the details of the extension ( Akeeba Backup has 4.86 out of 5.00 from 752 users ).
I've seen this on multiple extensions including one of mines. I used Akeeba as an example just because it is very easy to find.

[nikosdion]

Akeeba Backup was around when it was possible to leave a review without a rating. If you click on the very last page (reviews from January 2007) you'll see what I mean. Maybe it's the same on your extensions, too?

[nonumber]

The difference in votes/reviews is now down to there being reviews without votes.
Sometimes votes get removed without removing the review for whatever reason.
See: http://extensions.joomla.org/extensions ... /reviews39

That will be even more when we remove all the 2+ year old votes.
So maybe good to place back the amount of votes and the Most Voted link then

[Vimes]

As an aside, has any decision been made about J1.5 extensions yet?

[Alex Dobrin]

Didn't thought about that.
That explanations with the reviews without vote are explaining why there are extensions that have more review then votes but one of my extensions seams to have more votes then reviews ( http://extensions.joomla.org/extensions ... forms/7901 ).
I'm not complaining. I just wanted to let you know that some things might have been missed. Or probably I am missing something.

[nonumber]

And that - my friend - is because there are currently 2 reviews pending approval.
Not-approved reviews do not count in the review count yet, however the vote does count (I know, it is stupid, have been complaining about that for years )

[Alex Dobrin]

So I was the one missing something.
Thanks for taking the time to clarify those questions.

[mattbaylor]

2+ year old votes have been removed. Reviews remained in place :-)

[germi]

it's very good thing that you guys are working on improving the review system.
great work Matt, thanks for your effort!

well, I have a bit of mix feelings about this.
of course removing non-review votes is a good thing. about the 2+ year old votes... well, I understand your intention is good, but its clear it will benefit some of us and will damage some others.
I think old review are still meaningful. they may be old but they are valid. what was said is still true, I think! of course the extensions evolve... and the newer review probably reflect this evolution.

in my particular case it's a bit bad. in my case it suposed the removal of 24reviews... and all of them were 5stars. that's a lot of average lost towards my competitors (in the same category). it suddently seems like my extension is poorer, just because the (few) bad reviews (actually only 1) have more impact in the average.

so, in someway, now I'm in disatvange towards the competitos.. simply because my extension is older! and actually an older extension probably means it has improved more because it has had more years to evolve.

I think old votes should count. but I'm not unhappy with what you're doing. I understand this is for a good reason and you aim is to do good!

keep up the good work.

[nikosdion]

I'd counter that old reviews reflect an older state of your extension's development, older Joomla! version, older batch of users and so on. There is a cure to the ailment your extension listing suffers: subtly, but actively, ask your users to submit a review and rating to your JED listing. Putting that info on your extension's main page and the page on your site are a good start. Putting the standard "if you are using X please post a review and rating etc." text with every release announcement your do on your site / newsletter / Facebook page / Twitter stream / Google+ profile / plane-drawn banner / megaphone announcement / whatever also helps I believe that the goal of JED with this change is to favour new, up-to-date reviews for extensions, a.k.a. an indication of a healthy community which is actively using and reviewing extensions. So, let's help our JEDIs, each one of us developers on his own and as the whole Joomla! developer community. Plus, having more reviews is also good for your reputation. Win-win!

Full disclosure: I lost over 10 pagefuls of 5-star Akeeba Backup reviews with that change.

[nonumber]

On the whole, the developers that lost the most (going by the top 100 extensions) are Akeeba, Ossolution Team and NoNumber (that's me). But it is all pretty relative. Some extensions go up, some go down. ANd nearly all have lost some votes.

Some of my extensions went from around 30 votes to below 10 (meaning they drop off the Top 100 listing)

But that's all good. An extension that only has 6 votes in the last 2 years is clearly not something that deserve a mention.

[Vimes]

I lost out as well, but that was to be expected and I'm confident that Matt will come up with a more meaningful algorithm that'll make the (hopefully) brief pain worthwhile.

[nonumber]

I already have an algorithm ready for use. But we will implement it when we rebuild the JED.
Essentially it takes into account the number of votes and age of votes.
Votes will decrease in value over a 2 year period. So a 1 year old vote is only counted for 50%.
And some other stuff to make it all fairer and less vulnerable for abuse (like not counting the top and bottom 2% of the votes).

[Vimes]

Looking forward to it, cheers.

[H13]

... But that's all good ...

I think, it is not. If you run a business you will never do an action which will disadvantage existing certified valuable clients (compare to JED: extensions with history/repute/goodwill which was lost) ... so for some developers it is not good, but of course, if the changes were needed, there is no other solution. Important is, JED is comunicating and developers can make a feedback.

And some other stuff to make it all fairer and less vulnerable for abuse (like not counting the top and bottom 5% of the votes).

Some kind of such rule will be great, because on JED some reviews look like fake reviews (top and bottom).

If the extension has e.g. 50 top ratings (4 - 5 stars) and one 1 star rating with confusing review, such rule can then make the extension rating more fairer.

Jan

[nonumber]

I think, it is not. If you run a business you will never do an action which will disadvantage existing certified valuable clients (compare to JED: extensions with history/repute/goodwill which was lost)

If an extension has a lot of great votes before 2010, he should have a lot of great votes after 2010 too.
If not, then that 'goodwill' and history is all just misleading.

So quality extensions don't have a lot to loose. Seeing this happened to ALL extensions, there is no real disadvantage. The game rules have just changed a little.

To take a business you have some experience with (going by your extensions names), a 2 year old 5-star vote for a restaurant means nothing. Especially not if it has received only 2-star votes in the last year. I am not interested in the steak they served 2 years ago, and how the service was back then.

[H13]

Hi, yes, we can quibble here. But I still think, history is important, this is why firm names include "since", etc., so e.g. for me old ratings/reviews are important too, but it is just my point of view. But I fully understand, changes were necessary and I accept them (like I accept your/JED team arguments about old ratings).

BTW restaurants :-) - yes you right but this is very specific area - when the cook changes, restaurant reviews change too - so in fact not the restaurants but chefs should be rated :-)

So looking forward, I hope "not counting the top and bottom" feature will be implemented.

Thank you, Jan