JED Reviews - We Need Your Suggestions

[Webdongle]

... We need a few more people to get involved in this to widen the perspective....

Possibly difficult for the same reason #3 is a no go?

With #3 you might get some volunteers if the 'Official testers' were allowed to use the the paid extensions for free . The only way to know for certain would be to 'suck it and see' ... you might be surprised at the number of users that would volunteer if they could benefit as well.

I do like #2

We now have #5 by @Physicist. Much like how google verifies a url.

[sozzled]

Hmmm ... I'm not sure if using a similar method, such as what's used by Google and Bing, would prove that an extension has been installed (and thereby confer the "right to review"). One could overcome the "right to review" check by simply adding that I've-registered-as-a-reviewer file to a website, irrespective of whatever extensions the reviewer may have been installed.

I discarded the other idea, for the JED review system to check if an extension's manifest file existed on a server, because there are a few security concerns there. The first concern is that the check needs either (a) CORS to be enabled for the site for a script to read the XML files, and (b) the .htaccess needs to allow external folder browsing merely to determine if an XML file exists. I deliberately disallow folder browsing on my websites.

Lastly, the official testers thing is never going to work (for reasons I have explained in some detail earlier in this discussion). The concept is contrary to the JED terms and conditions; the principle of what is listed on the JED is based on "the developer takes responsibility for the accuracy of the information". It should not be a function of some "official" (or otherwise) testing group to question whether such-and-such an extension purports to live up to the developer's (or users') expectations.

As a developer (and I think other developer's would join with me in this), if we permit people who are part of some "independent" group that "reviewed" extensions to get extensions for free as a kick-back for reviewing them, then I would be pulling all my extensions from the JED or I would explicitly not give my consent to that. That's not what I signed up for.

[Webdongle]

... if we permit people who are part of some "independent" group that "reviewed" extensions to get extensions for free as a kick-back for reviewing them, then I would be pulling all my extensions from the JED or I would explicitly not give my consent to that. That's not what I signed up for.

It would be optional. In other words the developer would have the option to get free testing in exchange for allowing the tester to use it for free. That would give the developer (if they agreed) bug reports that would help them prevent loosing customers because they had put untested/buggy releases.

[sozzled]

Thanks, @Webdongle, for qualifying your suggestion with the word "optional". Not an idea that can be ruled out but it's not something that I or many other developers would give their consent to.

As I've tirelessly written on countless occasions before (see, for example, viewtopic.php?f=262&t=952408), it's a tough gig being a developer. Not only do we take an idea, design an implementation of that idea, write the code, test it, package it, market it, etc., we also have to follow a check list of requirements in order to use the JED as a means of advertising it. We aren't thanked for what we do; we're at the bottom of the food chain.

It costs us real money to do what we do. One small way that we achieve recognition is that people take the five or ten minutes to write a review. Sometimes the reviews we receive are congratulatory and sometimes they're not; sometimes reviews are honest and sometimes they're not. The number one issue here are the current obstacles placed in the way of people in being able to express their opinions about the value—tangible or intangible, value-for-money, ease of use, usefulness, etc.—of products listed on the JED. I feel that the review system, itself, is the main impediment and that's what I hope will be improved; for the benefit of developers, in particular, in the first instance and for the overall benefit of the community.

[Webdongle]

I only developed one extension. Most of my time was spent testing. Luckily I had two 'friends' on here that helped test it ... could have done with many more. But, hey ... each to their own way of doing things.

[sozzled]

Here's another way to generate a token (see previous example where I used Akeeba Admin Tools v 5.6.2). I wrote a script as a proof of concept that generates a MD5 hash, which changes every 24 hours. If you understand PHP, it shouldn't be too difficult to follow the logic:

Code: Select all

<?php

function token_generate ($extension, $version) {

         $first5 = substr ($extension, 0, 5);           // get the first five characters from the named extension
         $date = gmdate ('d M Y');                      // get the time as UTC
         $unencoded_extn_string = $first5 . $version . $date;
         return md5($unencoded_extn_string);            // return the MD5 hashed token
         
}

/* ===========================================================================================================
   Obviously, the token has to be generated on the website where an extension is installed and there needs to
   be a way to select which extension is the one being reviewed.  This snippet doesn't go into details about
   how the extension name and version is extracted.  The following is only an example using the extension name
   "Admin Tools" and version "5.3.2".
   
   The hashed token needs to be matched on the JED review site using the same technique.  The hash value
   changes every 24 hours.
   =========================================================================================================== */

echo token_generate ('Admin Tools', '5.3.2');

?>

Therefore, someone needs to build an extension that reviewers will download and install to generate the token and the JED review process has to use a similar technique to ensure that submitted-token = JED-received-hash. Assuming that the reviewer's server time is reasonably accurate—there may be a problems when tokens are generated around midnight UTC—to within a few minutes, this should work most times.

The token generated in the above example is

Code: Select all

c9b95b626fea4ae95e0c80d104c327b8

[Webdongle]

Is that similar to the way the Joomla statistics plugin works?

[sozzled]

Nope. The J! Statistics plugin pushes data automatically (as long as the plugin is enabled, but it seems that a lot of people disable the plugin because they're suspicious about what data is collected) out to some cloud-based storage. AFAIK, the J! Statistics plugin doesn't generate any token information.

Remember, too, that MD5 is one-way data encyption. The hash cannot be decrypted; it's the same technique used in passwords. In other words, data is encrypted before it's transferred [somewhere] and then it has to be matched against something stored [somewhere]. My idea would encrypt the data [by the reviewer] and, assuming that the transactional latency is short (i.e. within an hour or so), the same encryption method would have to be replicated by the JED review system and the two hashed values compared.

Obvious points of failure are

1) The server time on the reviewer's site is not properly synchronised with the world clock and, likewise, the JED review server time has to be similarly close; and
2) The developer needs to ensure that their listing on the JED is up-to-date. If the listing shows that the latest extension version is, say x.y.z and the version installed on the reviewer's site is something else, then the hashes will be different.

In other words, you can't post a review for an extension + version that isn't listed on the JED and you have to post a review before the token expires and a new tokens must be generated at both ends; a maximum of 24 hours. Not perfect, no; may result in failed attempts to review, yes.

[hdouglas]

Don't lose sight of the time it might take, or the hoops to jump through to be able to post a review. Too many requirements and all that will be submitted is those who want to post negative reviews in revenge of what they may (and at times unreasonably) view as bad service. Ask yourself how much time you would spend posting a positive review?

[deleted user]

Remember, too, that MD5 is one-way data encyption. The hash cannot be decrypted; it's the same technique used in passwords.

For the record, MD5 is quite easily brute forced and decoded back to its original data these days and should not be used in a system where you're dealing with security sensitive operations (it's part of the reason Joomla moved from MD5 to Bcrypt password hashes).

AFAIK, the J! Statistics plugin doesn't generate any token information.

The stats plugin uses a SHA1 hash which allows us to uniquely identify websites so that records are properly updated instead of every ping to the API inserting a new row (and before anyone freaks out here, the hash data is a random 28 character string and a timestamp of when it was generated, see https://github.com/joomla/joomla-cms/bl ... s.php#L356 to audit for yourself). SHA1 is a bit more secure than MD5, but still not a great recommendation for security sensitive operations.

Now, while I'm here...

Personally, I don't think I would put too much time into this type of system for JED reviews. It is a hugely complicated extra step for someone to be able to submit a review, and it puts up a needless barrier (what if you're using Joomla in an intranet where your site can't communicate with .org, as somewhere along the line the site is going to have to communicate out to be able to correctly build a token that can be validated). I'd just turn off reviews before I took this idea seriously personally. Yes, reviews are a very easy spam source, and yes they are going to require human interaction at all steps in the administrative process. No, I don't think this idea, or how JED is handling them now, helps matters.

[sozzled]

Thanks, @mbabker. I didn't know that the J! Statistics plugin sent a token.

I don't think it is significant to compare what is used by the J! Statistics plugin and the token idea I've suggested. I also don't think it's too important to over-egg [security] mechanics that might be used in "verifying" the authenticity of reviews. My code snippet example was merely an example, for illustrative purposes. If someone has a better idea then I'm keen to see it.

If the new mechanism adopts a token generation feature, and it uses MD5 or SHA1 or some other encryption, it's probably a case of "So what?". To use SHA1 instead of MD5, the generator function in my code would merely return a different value, in other words, change the line (in my example) like this:

Code: Select all

         return sha1($unencoded_extn_string);            // return the SHA-1 hashed token

But, again, so what?

I agree that no "automated" verification approach will catch everything; it's not bulletproof (and neither should it be). There's always a requirement for human intervention (or arbitration) where people report fake/dishonest/spam reviews.

Don't lose sight of the time it might take, or the hoops to jump through to be able to post a review ... Ask yourself how much time you would spend posting a ... review?

It generally takes me about 15 minutes to write a review. Generally my reviews are fairly "positive"—I wouldn't say that I'm fawning in my praise or hyper-critical in my assessment—but I would like to think that my reviews are honest and justifiable. I am lucky; I'm not among the cohort of people who currently spend time only to be disappointed that the process fails because of the current [IP/keyword-based] filter.

I don't know how many extensions I've reviewed over the years—the number is around twenty, I think—and it's hard for me to condense my opinions to 140 characters for each of the five metrics (and 280 characters for an overall assessment/final comment). However, the review submission process has become easier with the introduction of the "star" scores. Swings and roundabouts.

I don't know whether it's a good idea to, at this time, disable the existing review system. I think it is a good idea, however, for the JED team to engage with the community, acknowledge that the current system is imperfect and keep us appraised of what their plans are to remediate the current problems.

Personally, I think that the current "anti-spam" mechanism is a wasted effort. Personally, I think that the level of "spam" is insufficient to justify the JED team running up the white flag and, effectively, say that it's all too hard to juggle the value of reviews against the volume of fake ones that make their way into the system. That's just my opinion.

But let's be completely clear about what the JED is all about. The JED is loosely based around helping developers, giving developers some recognition, and [of secondary importance] allowing the developers' work to be shared with the community. If the community as a whole benefits from it then so much the better.

The process is based on a simple premise: someone thinks of an idea about how to improve a website's functionality. That "someone" is a "developer". It starts with someone saying to themselves, "I need something to do something." Now the genesis of that thought may be because they want something for themselves or they may need something to satisfy a client. It doesn't matter. After they've built that "something" they may want to announce to the world, "Hey, world, I've built something. Maybe this 'something' would be something you might like to use?"

Now, maybe other people would find these "somethings" useful to them, and maybe they won't. Everyone is different and everyone seems to want something different to what's on offer.

The whole point of the JED is to showcase the work of developers. Some developers are happy enough to obtain a pat-on-the-back and other developers actually want to earn some money. The motivation is different.

It's nice to get feedback. It's nicer to earn some pocket money. It's even better if we could live off the earnings. (Apologies for the rambling. I've been feeling out-of-sorts lately because, it seems to me, much of the "feedback" I've received has been fairly negative. So, maybe, I should just leave it at that. )

• Retain the ability to post reviews; reviews are an important part of the JED ecosystem.
• Ensure that the people can only submit a review based on the principle that the reviewer has actually installed the extension they're reviewing; leave it to the JED4 development team to implement a mechanism to ensure that reviewers can establish they've installed the extension they're reviewing. Some of the ideas in this discussion may contribute to that implementation.
• Get rid of the current IP/keyword-based checking algorithm.

[chrisguk]

One problem that I want to see solved in JED4 is developers living in fear of bad reviews. We are forced to entertain the whims and demands of users who don't read our documentation, who make unreasonable demands on our time, and who have all manner of unreasonable expectations. In the current system we don't have the option of politely saying to our users "sorry, we cannot provide the amount of support that you need at the price you paid".

If we can't find a simple solution for this problem, then I am in favour of scrapping reviews altogether. Sorry, but I do not agree that a user having installed an extension in any way qualifies them to post a review.

In my experience the vast majority of satisfied users do not leave a review, and those that do rarely convey any useful information. But a "bad" user can in moments destroy years of a developer's hard work. Subjective opinions are of very limited use anyway, because what works well for one person may or may not work well for another.

Fake reviews are also a huge problem, and I am quite sure that some developers have been faking reviews on an industrial scale. As many have observed, attempts to detect abuse have failed. We have received several reports from users saying that they have been unable to post their reviews, but I imagine a determined faker can still get through.

One possible solution is to remove the incentive for posting fake reviews, i.e. by scrapping the ordering by rating and/or number of reviews. Just randomise the ordering within categories. That would also be fairer to new developers, and it would make the occasional bad review much less of a problem.

[sozzled]

Agree with @chrisguk (and, to a certain extent, with @mbabker before that). The review system is the number one bugbear with the JED; it's the number one source of complaints; it's the number one fertilising agent for cultivating discrimination, uneven-handedness, unending debate about corruption.

To take a leaf from @Webdongle's book, we've tried; we've failed. Expecting a different outcome is insane.

To be totally honest, I don't give a damn anymore about JED reviews. I'm sick to death in reading about the problems, the unfairness, the lack of respect, etc. I'm sick to death of the JED team trying to prop up a rotting carcass of a system. I'm sick to death of living at the bottom of the food chain.

In a complete turnaround from my previously-stated position, I vote to end the suffering; end the discrimination; level the playing field for everyone. Disable the JED reviews. They don't benefit anyone; if they did then we wouldn't be having this discussion and the JED team would not be overburdened with our non-stop complaints.

[Webdongle]

From an end user point of view I like to see stupid reviews. They make the good reviews more believable. It is easy to tell when a bad review was left by a [redacted].

[sozzled]

This is as good a place as anywhere to write this. As the result of one person—a person I have defended—I have decided that this forum is not the place where one can offer one's opinion or challenge the status quo. As I feel that my contributions to this forum are next-to-worthless, I will take a leave of absence.

It's probably better for all concerned that it rests that way.

[Webdongle]

From an end user point of view I like to see stupid reviews. They make the good reviews more believable. It is easy to tell when a bad review was left by a [redacted].

Just for clarification. I am not suggesting that fake reviews are not a problem. What I am trying to say that it is the fake reviews (not the stupid ones) that do need to be prevented.

[hdouglas]

One problem that I want to see solved in JED4 is developers living in fear of bad reviews. We are forced to entertain the whims and demands of users who don't read our documentation, who make unreasonable demands on our time, and who have all manner of unreasonable expectations. In the current system we don't have the option of politely saying to our users "sorry, we cannot provide the amount of support that you need at the price you paid".

I see one of the key areas where problems arrise is the overall expectation of a number of users that because they can download Joomla for free, then all extensions and full professional support ought also to be free. I am very minded that developers need to make a living in order to continue to develop and support their extensions.

There is however a way to handle this with a owners' response to a bad review. I would suggest that a 'customer' who is not willing to pay for support to ensure the continued development of an extenion is perhaps not the customer you need to have.

On the other side of the coin, if you are going to charge for support make sure you are very clear about the level of support and response time you are offering. In many instances, the root source of complains is because the customer was led to believe something which was never delivered.

I can see two sides to this, but clear transparency goes a long way to prevent issues.

[sozzled]

There is however a way to handle this with a owners' response to a bad review.

I agree but, with the current system, reviews ("bad" as well as good/honest ones) are not getting there. The current system blocks most reviews except, it would seem, in those (rare?) cases where people are able to work their way through the IP/keyword-blocking filtering process.

In many instances, the root source of complaints is because the customer was led to believe something which was never delivered.

I don't know if the evidence supports that conclusion. If there wasn't any "anti-spam" blocking then we could analyse the incidence of negative reviews arising from a failure to meet user expectations.

However, as I wrote earlier:

The [JED system is based] on a simple premise: someone thinks of an idea about how to improve a website's functionality. That "someone" is a "developer". It starts with someone saying to themselves, "I need something to do something." Now the genesis of that thought may be because they want something for themselves or they may need something to satisfy a client. It doesn't matter. After they've built that "something" they may want to announce to the world, "Hey, world, I've built something. Maybe this 'something' would be something you might like to use?"

Now, maybe [prospective customers will] find these "somethings" useful to them, and maybe they won't. Everyone is different and everyone seems to want something different to what's on offer.

The whole point of the JED is to showcase the work of developers.

That's at the heart of what the JED is about.

However, because the reviews themselves taint the JED because honest opinions are not getting there and because dishonest opinions are getting there, and because whatever reviews exist don't actually say much, I feel that end-user testimonials are basically counter-productive to the ecosystem and all of them should be deleted. Not having reviews levels the playing field and, in the final analysis, end-users/customers should be able to decide for themselves whether they choose to install third-party extensions or not (and not because someone else says these things meet someone's expectations or not).

[Webdongle]

I would suggest that the JED is more than that ... we (the users) rely on the extensions in JED to be safe. That (imho) is the main purpose of JED .. along with that trust comes the ability to trust the reviews are genuine. That trust is (again imho) is far more important than the quality of someone else's opinion.

[sozzled]

All extensions currently listed on the JED are assessed for safety. Safety checks are conducted by the JED staff whenever an extension is listed or updated. All extension developers are subjected to those safety checks. Unsafe extensions are reported to the VEL team. Extensions can also be reported to the JED staff if people believe that they are non-compliant with the terms of use.

These safety checks are manually performed as per Terms of Service (clauses 1-5 inclusive). This is why it usually takes up to three weeks before new extensions are listed on the JED because of the background checks that are undertaken by JED team members.

There are no manual safety checks carried out by JED staff in relation to reviews unless specifically reported by end users or developers.

The penalties that apply to developers are contained in clause 7 of the TOS. There is no specific mention of penalties that may apply to reviewers, apart from

[6.5] Acceptance and/or removal of reviews and votes is at the sole discretion of the JED team and OSM.

[Webdongle]

All extensions currently listed on the JED are assessed for safety. Safety checks are conducted by the JED staff whenever an extension is listed or updated. All extension developers are subjected to those safety checks. Unsafe extensions are reported to the VEL team. Extensions can also be reported to the JED staff if people believe that they are non-compliant with the terms of use...

Precisely my point ... it is the trust in that safety (and the trust in the authenticity of the reviews) that is the main reason for JED. The quality of the reviews is of little importance compared to the trust in the safety and authenticity. We should be concentrating on the authenticity of the reviews not we agree with the review

[brian]

> All extensions currently listed on the JED are assessed for safety. Safety checks are conducted by the JED staff whenever an extension is listed or updated.

What makes you think that

[sozzled]

@brian: would you like to see my mail inbox? I have many examples where my extensions have initially been rejected or where I have had listings suspended (but subsequently re-instated) based on what has been brought to the attention of JED team members.

There are 20+ checks undertaken (semi-automatically) as part of the listing submission process. In addition, there are manual checks to insure copyright/trademark compliance (e.g. the name of the extension must not violate someone else's product "trademark"). There are other checks to ensure that URLs meet appropriate standards.

The developer must bear all of the costs (the costs of building, packaging, hosting, distributing, updating, marketing, etc.) for the product. All of this work can be ruined by one unsavoury (and subjective) review.

@brian: would you like to see my mail inbox? I have many examples where my extensions have initially been rejected or where I have had listings suspended (but subsequently re-instated) based on what has been brought to the attention of JED team members.

There are 20+ checks undertaken (semi-automatically) as part of the listing submission process. In addition, there are manual checks to insure copyright/trademark compliance (e.g. the name of the extension must not violate someone else's product "trademark"). There are other checks to ensure that URLs meet appropriate standards.

The developer must wear all of the costs (the costs of building, packaging, hosting, distributing, updating, marketing, etc.) for the product. All of this work can be ruined by one unsavoury (and subjective) review.

One final point about safety: just because I've written that "safety" checks are performed by the JED team, this doesn't guarantee that the listed extensions are necessarily safe (or fit for any—or every—situation). These checks only ensure that JED listings conform to the TOS and, in that regard, that's about the best that customers can expect. Nothing is safe in this world (and how true that is for software); things are only as safe as we can make them. A subjective review doesn't mean that a product is any more or less "safe" than any other; it's just one person's experience.

[bruno28]

Perhaps make it a requirement to submit real name, postal address and phone number when submitting a review? Or make it an option then only allow the ability to 'star' rate if they have supplied their details.

@Webdongle +1

It seems a good way to make sure of the identity of users and to prevent fake submissions

[deleted user]

Unless something has changed, extension uploads in the JED are not audited. At most, they are run through an automated checker to make sure you have a license in a file doc block and the defined or die check, but that's really the extent of it.

Likewise, JED is not a canonical source of extensions in the Joomla ecosystem, unlike the WordPress plugin directory or Drupal's contributed modules listing (both of which have resources that Joomla cannot sanely offer, which includes source code hosting, hosting of the resources needed for the update systems, and backing by their "core" security teams to help with identifying and resolving security issues). Realistically, the only thing JED offers over a Google search is a centralized location for reviews (when that subsystem works) and the false premise that the extension has passed some kind of Joomla assessment to be deemed worthy of listing.

[Webdongle]

And all this time we have been using JED because we thought the extensions were safer than other sorces. What a let down!!!

[sozzled]

It's been a month since this discussion started. We were asked to contribute to a survey (and I spent the few minutes required to participate). Has there been any discussion within the JED development team in relation to the survey responses?

I read from the JED4 sprint report,

Hugh explained the additions to the reviews functionality and discussed the ‘Right to Review’ idea. A number of options were debated, including ideas introduced by the community in the recent forum post. As this area is still a work in progress, updates will be posted as work continues.

So, while there's discussion or "debate" about the way forward w.r.t. extension reviews—and the existing problems still continue to be felt—have we narrowed the field of options being considered?

Thanks for any information that we may receive.

[sozzled]

I am a bit disappointed that, a month after the JED team asked for us to contribute to a survey about redesigning the JED—in particular, asking for our views about the current (broken) JED review sub-system that is one of the major sources of complaints—there has been no feedback about what came from that survey.

If people are asked to contribute to a survey (and there are many instances where we're asked to spend our time contributing to surveys) it seems only reasonable that those who request these surveys ought to be able to publish the findings and recommendations (if any) that emerge from those surveys. If this doesn't happen then it's a bit like "the boy who cried wolf", isn't it? I hope that we see some response soon because, if we don't, it tests our generosity to contribute to future surveys if they should happen.

[darb]

Dont miss to make a search filter menu link to "Last updated" pls. Thks.

[Tuscany]

Hi!

Cannot write any reviews on JED becaus of this error message

jed_error.jpg

I am using the IP released from my provider, no mask, no vpn

Don't like to publish IP here.

Just to let You know