JED Review/Vote Cleanup

[mad4joomla]

While I think removing all silent votes was a desired action I am not sure about the 2+ year old reviews removal.
I also think the 2% of newest and oldest "no counting" would be not fair.

The JED team has a look behind the scene but only few of them know how JED works in real life.
I lost many votes, anyway, I don't want to complain. It is ok for me.
But getting votes is really hard work.
Getting votes if you have already over 100 or 200 reviews is much harder.
Quote: "Oh, I thought you have enough reviews and don't need no further ..."

Almost all reviews are done if you did a lot of support for your customers / users and ask them afterwards.
Even then 4 of 5 users do not vote. ( I assume because meanwhile many users skip the "If you use ......" text)

The only reason why a user writes an review without being asked is because (s)he is not satisfied by somthing and this can be just a wrong color in the CSS.
But only in very few cases if (s)he is "flashed" by the extension you will get a positive review without asking.
This means the nature of the review system is not fair by default.

You can imagine that it was hard to swallow if hundrets of votes has been removed at one day.
What I want to say is that a 2% rule would just disadvantage established developers.
2% of 5 votes = 0 vote x 2 = 0 votes
2% of 100 votes = 2 votes x 2 = 4 votes.

I think that many "dirt bags" got a second chance to cheat them to top again.
Before, the system had a "internal rule".
If someone had about 500 reviews the bad guys couldn't down rate "whales" that effective.
Now that many whales became small fishes they can easily be down rated by all the evil guys.
Exactly this situation will cause more work for JED than before.

[TheMuffinMan]

Hi,

Thank you for already starting to look forward in the JED updates!

About introducing the top and bottom 2% of votes, can somebody please explain how it prevents fraud voting? Maybe I missed something.

As far as I understand it would mean that bottom votes constantly need to be removed due to their age, otherwise the bottom 2% would always be the same ones. Then for the top ones, if a fraud voter wants to downrate an extension and knows about the 2% he simply needs to accelerate the amount of fraud down votes to get an immediate impact. Nothing would be more secure here but the impact will be just shifted to a later point (after new votes arrived).

The idea of counting 1 year old votes for only 50% is a really great idea, but I am not sure yet if the top-and-bottom-2%-thing will help on frauds. Maybe instead of 2% it should be a higher number or broken down to more levels (0%, 25%, 50%, 100%) so a fraud voter would need to do a large series of downratings, which would be in general unusual and then can be considered for closer investigation.

For example, the top 12 votes will be broken down to: 1st three votes count 0%, 2nd 3 votes count 25%, and so on.

This, coupled with the planned age of voting, fraud voters will then have a hard time to even do slow-pace fraud voting, plus we are having all recent and relavant votes / reviews that we wanted to have.

Though personally I would prefer to have a simpler solution for this, this appears to be working pretty well, too (if it hopefully doesn't harm the site speed so much).

Regards,
Markus

[H13]

If someone had about 500 reviews the bad guys couldn't down rate "whales" that effective.
Now that many whales became small fishes they can easily be down rated by all the evil guys.
Exactly this situation will cause more work for JED than before.

:-) This exactly happened in my case. This is why I am not happy about this change.

If your extension gets normally 4 or 5 star rating and then you get an email with such text: "You will modify a Phoca Gallery for my customer for free or I will make bad rating for your extension on JED" and after cca one month you see 1 star rating with fake review and such review has then more power (because old rating values were removed) then it is not easy to be happy about such changes :-(

Some purge of extreme values (not about time but about extreme values) will be great.

[Webdongle]

If the review form had set sections like 'Ease of installation', 'Configurability', 'How would you describe your experience level ... Expert ... average ... beginner' with radio buttons or boxes for input(as the case may be). Would that make it easier for the JED team to spot fake reviews than just one input area where the reviewer decides on the criteria ?

Also a more open and fair investigation process would help [* spam *] out the fake reviews. And give people (who are falsely accused of fake reviews) the opportunity to prove openly that their review was fair.

As a victim of being falsely accused I see the need for a fairer system of weeding out false reviews. Looking at the problems some are having with fake reviews I can also see the other side of the coin. The answer surely must be to create a review form which allows the criteria of the reviews to be standardised. And and an investigation process that is open.

This would allow the JED team to better asses a review, the developers and reviewers the chance to openly prove/disprove allegations. As well as deterring developers making false allegations and reviewers from making false reviews.

[TheMuffinMan]

Hi H13,

almost the same happened to me, too. Wasn't blackmailing but frequent slow pace down voting (1 down vote per review page) that started from out of nowhere 1.5 years ago, until today. Usually you can ask the jed team to get in contact with the reviewer. But only a tiny fraction ever replied and also weren't determinable in my customer database. This made me raise my eyebrows quite high.

However, what happened happened but nonumber's idea to count old votes for 50% will certainly help on things like this. If the 5-star rating system will be kept, it should be made as hard as possible to place fraud ratings -- wether downs or ups - and this can be achieved by a threshold that is large enough, but not too large, such that a fraud voter will loose his motivation.

@webdongle: I don't know how you see that but a simple thumbs up/like/+1 system with reviews would be fair. Many understand the JED as a directory, not as a sales platform like Amazon and thus shouldn't go too far with different rating sections.

Regards,
Markus

[Webdongle]

... I don't know how you see that but a simple thumbs up/like/+1 system with reviews would be fair. Many understand the JED as a directory, not as a sales platform like Amazon and thus shouldn't go too far with different rating sections.
...

With a thumbs up/down it is too easy for fake voting ... no real way to tell the motivation. With one input area harder to wade through for the JED team ... but agree that too many rating sections in the review form would be a problem. Was thinking of somewhere in-between no structure and strict(complicated) structure.

Another suggestion to 'through into the mix'. How many of the JED team are responsible for accepting a review ? i.e. any one person looks at a review and passes/rejects it or several people look at a review and a consensus is reached before passing/rejecting. Perhaps the latter would be good (if not done now) ?

[TheMuffinMan]

Hi webdongle,
Probably not if you force reviews to come with a thumbs up.

Markus

[Webdongle]

... not if you force reviews to come with a thumbs up.
...

and force everyone to write a positive review as well ?

[TheMuffinMan]

no, like it is now. if you give a thumbs up you must write a review. while reviews without thumbs up would be allowed (to write a negative one). it is basically what there is now, except nothing can be downvoted.

[mad4joomla]

Well I think the old rating system should be left as it is.
I plead for a new approach.
What end users want to know is: is this a good software, are the developers reliable, etc.
Therefore it would be great idea if the software is tested by experts.

My suggestion is establishing a paid service by JED where experts evaluate an extension.
Volunteers could get a little fee for their efforts.
The evaluation service should cost a fixed amount.
The evaluation should only apply for the software and not the service.
If end users like to learn more about the service they still can read the reviews.

The evaluation should be done by 3 experts which are rating several characteristics like: usability, stability, flexiblity etc.
If JED visitors are viewing an extension listing the extensions which have been tested by experts should be higlighted such as editor picks.

The JED extension site should not only show the ratings of the experts, but also a little summary.

The expert rating should not influence the normal rating.
Thereby nobody can complain about that only "paid" JED extensions are getting high positions.

The expert opinion would be just an extra add-on.
I am sure that almost all commercial extension developers would appreciate the expert opinion.



.

[Krishna16]

On or after October 30, 2012 all votes that are 2+ years old will be removed, reviews will remain but votes will not be factored into the averages.

That's fine Matt, 2 years = about 7 years in 'internet' time, so that is a good compromise.

[H13]

Recently, everytime I get an email from JED (new review), I am angry, because most of the reviews are fake, lies, etc. Some are made because user is somehow not able to read the documentation, some based on reasons which are not related to the using of extensions itself. (e.g. as I wrote in some previous post - if the developer will not accept threats by the user, etc. It is easy to recognize it - the extension has most of great reviews and one is the poor reivew with text which is a non sense).

I'm embarrassed to allways report such reviews and contact moderators (sometimes it is really hard to explain, that what user described, is a non sense) Moderator of course is not able to learn all the features of all the extensions. It is really hard to answer something to users who are in fact anonymous. And of course developer can see the thing more subjective that objective.

I really don't know if some "smart" feature can help (like existing: report the review, or answer to the user). I think, adding a contact of the person who made a review to the developer will not help, because if someone makes fake review, than it does not make any sense to contact such user :-( (the same when the user does not read the documentation - writing RTFM is not a solution)

For now I cannot imagine any such feature which will prevent from fake reviews or reviews based on some other reason which is not related to the using of extensions itself.

But will be great and I (like many developers) will apreciate it a lot if JED team will find some kind of such feature (to somehow eliminate the importance of reviews done with intent to harm).

Thank you, Jan

[zanderp]

Hi Jan,

The JED team is working on improving the JED review system, please see the slides of Matt Baylor's presentation during the Joomla World Conference about this http://www.slideshare.net/natselection/ ... l-throttle

I think this will reduce a lot of the current frustrations, for both developers as for the JED team.

You will find a timeline for this as well in the slides. You can expect more communication about this in the next weeks.

Thanks, Sander

[Webdongle]

Nope that set of slides meant nothing to me. No real information there ... just pretty pictures for good public relations. Nothing there with real specific details.

[brian]

Such a shame that you werent able to attend and participate. Hopefully when the videos from the Joomla World Conference are online it might shed some further light for you although I doubt very much it will be the same as being there. Events like JWC and JAB are about so much more than the sessions you attend they are about the people you meet and the conversations that you have face to face

[Webdongle]

Unfortunately not everyone can afford to attend such events. But If I did attend then my preference would be to attend something that would help me use Joomla better. To watch a PR presentation is not my bag. But I am eagerly awaiting to see the actual changes ... to see how much of an improvement they are.

[brian]

Then I guess you are forming an opinion, which I believe to be false, about the entire event based on a single session out of almost 80

[Webdongle]

Not about the entire event ... I really would love to attend one. Maybe sometime my personal circumstances will allow me to attend an event. But until(or unless) it does then I only have the forum posts and videos to go by.

[dig814]

Hi,

(before reading following sentences, you should know, my view on this issue is not objective, it is strong subjective - I think I don't need to explain why? :-) )

One of the most valuable SEO indicator for e.g. Google is a history of the site and the stability of the links on the site.

I worked hard for five years to produce Joomla! extensions and they are somehow rated on JED. Now the history (we can call the rating as some kind of history reputation for an extension) will be removed - in my case all 3 years of the history will be lost (some developer's history is even longer).

In case one of the rating criteria is count of ratings, this will be discriminative for established developers :-( (of course, you can say new developers are discriminated for now. So the question is: who should be in advantage - new or established? Not sure, but as a new member of JED I didn't never think about making new start (restart) for everyone - because you will never find the time (2 or 3 years back), you will never find the period (delete history all 2 or 5 years).

Deleting means, you will discriminate and dishonor established developers. I think this can be a problem for some of them.

Backward compatibility in Joomla!, removing history of established developers on JED (history is very valuable mark of quality for users who search an extension), etc. - maybe this is too much and established and recognized developers will skip their work or move to other project, or etc. :-( :-(

As I wrote, a subjective view, but I think, there are more developers like me with a history. And such action (restart) can be very problematic :-(

Jan

I have to say, I agree with Jan. I worked har don improvng an extension for over 3 years, then come to see that now after building up 3 years worth of rating and reviews, I now only have 3 rating. Yes, the reviews of the past are still present, but the ratings are gone. i understans the need for space, but like Jan, I feel that removing the history of established developers is a bit unfair.

Dan