JED reviews trusthworty?

[the apprentice]

I wrote an honest and respectful review about an extension (which I paid for) but it's not there anymore, the review count for that particular extension is 4 yet it only shows 3, also the scores don't add up. So I wonder if only very positive reviews are published.

[the apprentice]

Ok. I have now read this locked topic viewtopic.php?f=262&t=964002 and I see it's not an uncommon problem.
By publishing only brilliant reviews you're tempting us to part with our money, to buy apps that do not meet the expectations, it's not a little thing, we work hard for our money... JED will be dying soon I guess, trustworthiness has no place here as far as I'm concerned.

[romagromov]

Yes. Unfortunately it's true.
This is especially true for extensions that are expensive. And I still believe that moderators get money for deleting negative reviews. No one will remove a negative review for an unknown free extension. But expensive extensions do not have any negative reviews at all. It's unreal.

I had a negative experience described in the topic viewtopic.php?f=262&t=964002
The moderator deleted it, but could not explain the reason for the deletion.

But the saddest thing is that we can no longer add new reviews to this extension. We are blocked, like criminals(!).

As I said before - this is a dictatorship.

[sozzled]

Yes. Unfortunately it's true. This is especially true for extensions that are expensive. And I still believe that moderators get money for deleting negative reviews. No one will remove a negative review for an unknown free extension. But expensive extensions do not have any negative reviews at all.

As we've discussed before, these kinds of unproven conspiracy theories don't help anyone. The only facts (that we're aware of) are that there are positive reviews, negative reviews and no reviews. Some extensions have lots of reviews; it's possible that the quantity of reviews for some extensions may be artificially engineered. It's entirely possible that some reviews are removed by the JED review team members for reasons that are not clearly explained. As I've written elsewhere, there's a lack of transparency in the process.

However, it's a giant leap to go from accepting that reviews exist or do not exist because that's the nature of things to alleging that these things are manipulated by corrupt JED review staff! The really "sad thing" is that people take for granted the hard work that extension developers put into their products; they siphon products from extension developers' websites and never write so much as a "thank you" by way of posting a review of their experiences on the JED. That's the sad thing: not that the review system is corrupted but the lack of gratitude from the thousands of people who use the JED who never post reviews.

[the apprentice]

I didn't mention corruption or conspiracies, I'm reporting my personal case where I forked out good money for an extension with great scores. I'm presuming all the unflawless reviews where removed, and users banned, this attitude is just ridiculous, you have more free speech in North Korea. I paid the extension with PayPal so hopefully they will see sense and refund me.

[toivo]

As a general comment, I tend to stay away from extensions that do not have a free version. Checking the features of a 'lite' version and getting an idea of the quality of the coding by putting the extension through the JED Checker gives a good idea of what to expect from the paid version.

Even positive reviews get easily rejected by the automated JED review application in my own personal experience. The standard response to a JED ticket questioning the rejection does not show what was wrong with the review, but, without exception, defends the automated system and refers to the list of reasons at https://extensions.joomla.org/support/k ... published/.

The broken review application and the failure of the follow-up process does not encourage anyone to write more reviews, which is a real pity, because Joomla requires a healthy supply chain of extensions.

Could a certification system for paid extensions work, if a team of developers within OSM could at least qualify that the latest version of an extension can be installed without errors and works after a version update like 3.9 and 4.0? Of course the developer of the paid extension would need to pay a small fee for a couple of hours of developers' time, but the benefits would be significant, legal wrangles aside, if users can trust a seal of approval from JED. Currently the JED has many obsolescent 3.x extensions that do not stand up to scrutiny after the recent changes in the framework. Something for the Ideas forum perhaps?

[sozzled]

The JED is, essentially, a marketplace where people can offer for sale (or at no cost) their wares and other people can choose to obtain those products (whether or not they pay for them). Above all else, there are no guarantees, expressly stated or implied, that any of the products will work in all cases. Unlike other markets, where there may be consumer protection law governing the rights of merchants and consumers, the JED is merely a place where "merchants" (i.e. Joomla developers) can advertise their products at no cost.

This has always been a hallmark of the JED.

Over time, additional rules have been imposed on Joomla extension developers to ensure compliance with the whole J! "ecosystem" (that is to say, authors need to explicitly state that their products will generally work with the supported stable release of Joomla and that the products have been tested in regards to "compliance" against the JED Checker tool). It takes a long time for a software developer to get their product to the market and there are no certainties that a product will be listed (or remain listed) on the JED.

On the other side of the coin, it's not the role of JED team members to become involved in commercial disputes. The principle tenet in buying-and-selling via the JED is caveat emptor. Even so, extensions are often reported to the JED team as the result of a customer's disappointment with a product or because of a competitor's jealousy that a product is eating away at their own work. As a Joomla developer myself, I have (from time to time) had to justify myself, defend myself based on one anonymously-made assertion, to JED team members. It's not an easy task for a Joomla developer to establish a good reputation and to keep it.

I have been critical of the so-called "automated" review system. I've read many tales from people who claim that their extensions are not receiving any reviews; some of these people claim that the reason for not getting reviews is because the review system prevents people from posting reviews. This may be the reason; I really don't know. What I do know is that the reason the review system is "automated" is because there are not enough people within the JED team who are able to tell the difference between a genuine review and a fabricated one.

Reviews are totally subjective, anyway. Reviews that confine someone's ability to describe their experience by limiting it to 120 characters are not always worthwhile. But, just as a few hundred people may review a movie and describe it as being awesome or magnificent there are dozens of movies I've watched (after reading the glowing reviews) and thought, "what were they thinking?"

Are JED reviews "trustworthy"? Are any consumer reviews worthy of trust? And, lastly, the reputation of Joomla extension developers (as with anyone) is only as good as the trust that they establish with their customers.

[Webdongle]

Perhaps devs of paid extensions could offer free use of the paid version to volunteers who tested their extensions ?

[sozzled]

@Webdongle: yes, some developers already do that (but that's not really what this discussion is about). This discussion is about some people who lack confidence within the JED system to accurately reflect the "trustworthiness" of reviews. This topic is about assertions made (by two people, at least) that the review system is rigged in favour of (a) "expensive" extensions, (b) extensions that involved in some kind of payola scheme involving JED review staff, and (c) arbitrarily removing negative reviews in order to make it seem that certain products are worth the money involved in buying them. None of these allegations can be proven or disproven but, by the same token, there's a lack of openness about how the JED team members adjudicate the merits of reviews that may be posted.

[chrisguk]

As developers, we have a really hard time with a small number of customers who have unreasonable expectations. When a review is really unrepresentative of the extension, we can ask for it to be removed and the JED team will make a decision based mostly on the JED Terms of Service. In ambiguous cases they might, if we are lucky, give a long-established developer the benefit of the doubt. But I am absolutely certain that no developer would ever offer any kind of bribe or incentive for removing a review. I suspect that would be a sure-fire way for the developer to get banned from the JED forever. No-one is going to risk that. The idea that there is any kind of corruption is ridiculous and offensive.

Unfortunately the JED does still suffer from fake reviews posted by developers. It's a risky strategy for developers because if they get caught they get banned, but it does happen. It must be tricky for the JED to police because they can't risk wrongly banning an honest developer. It's possible for a new developer to skew the rating of a new extension for a while, but it's unsustainable as a long-term strategy. I could point to examples of extensions that have a high volume of great reviews shortly after the extension was first released, followed by a lower volume of more mixed reviews. In the end, the truth emerges, so I don't believe any serious developer would bother with fake reviews. I can spot them easily and I'm sure other people can.

So for extensions that have been on the JED a number of years, I think you can be pretty confident that the reviews paint an accurate picture. And if a well-established developer releases a new extension, I think you can trust that the quality will be at least as good as his older extensions.

For new extensions from new developers where there are very few reviews, you do need to be more careful. I would say look carefully at the developers' website, the documentation, and any demos that are available. New developers really need to establish themselves with some good free extensions before launching any paid extensions. Even as a very experience developer, I needed a lot of feedback from several high volume free extensions to learn how to build really robust extensions that work well out in the wild.

Sometimes potential customers ask us if they can have paid versions of our software for free to evaluate, and we always refuse. But if a potential customer gives good reasons why he is unsure whether or not one of our paid extensions will meet his needs, we are happy to reassure him that if there is a problem, or if the software does not satisfy his needs, we will happily refund him.

[Webdongle]

As developers, we have a really hard time with a small number of customers who have unreasonable expectations. ...

Those dorks are usually easily spotted by those of us who are reasonable. They stick out like a sore thumb ... 'I couldn't install it as I had to unzip it first', 'after I unzipped it the 3.5 version installed OK but the 2.5 version didn't ', 'The free version is useless because all the features are not enabled' etc.

It is always refreshing to see those types of reviews because it validates the good reviews.

[sozzled]

It's one of those zombie topics that's never going to die. Having infused myself into the discussion at viewtopic.php?f=262&t=973003, I have changed my opinions about the JED review feature. It's not just that reviews are untrustworthy as a general rule; the problem is that the review system is corrupted beyond redemption.

Isn't it remarkable that the majority of the thousands of discussions in this "feedback" forum category describe the pain, perceived prejudice and discrimination felt by extension developers? The single most common cause of that pain stems from JED reviews: the arbitrariness and lack of remediation to cure the problem.

Well, I think that I can speak for a large number of developers who are tired of the endless discussion—a discussion that's painful, a discussion that's really hurts, a discussion that's really on a hiding to nothing.

I vote for the immediate removal of reviews from the JED; all reviews; all extensions.

[chrisguk]

I'm with sozzled 100%. I have been a Joomla extension developer since the Mambo days and have several popular and highly-ranked extensions, so I hope my opinion also carries some weight. I'm sick of the fake reviews, sick of the stupid reviews, and sick of being held to ransom by moronic users.

Just nuke the whole snake-infested swamp and get rid of the problems once and for all.