Weird word filtering in search (bug?) Topic is solved

[froehli]

I just noticed that Joomla seems to apply a strange word filtering in its search component.
I would understand a filtering of simple stop words (such as "and", "of", etc.) - which indeed seem to be filtered. But in addition, several other words are filtered - and not really consistently; sometimes the submitted text doesn't even seem to make it to the search providers (see below).

In english, the filtering seems less harsh - at least on my quick tests now; my installation is in german though, and there, the words for digits for example are filtered - such as "sieben" (seven) - that is, if I enter "sieben Schwerter" (seven swords), the search will only be for "Schwerter".

I'm actually writing my own search plugin; and with that I noticed that search requests for text which are apparently fully filtered, such as for "Dinge" (things), or "sieben dinge" (seven things), result in my search plugin receiving an empty text field. One difference in the result of such empty queries is that my search result form in that case doesn't even show the "found xx results" text. In contrast, it shows "found 0 results" when it did search for some "proper" words, but there just weren't any actual results.

Is that a known bug? Or some weird strange stop word filtering (which to me would feel extremely bad usability - I would at least expect some feedback, something like a message to the user telling him that there is filtering going on). Or is nobody using the on-page search anymore anyways since google is much better at searching pages?

[imanickam]

Please look into the file com_finder.commonwords.txt that is located in the directory language\de-DE.

This file contains all the common words that would be filtered out from searching. Being a text file, you could edit this file. However, be informed that this file may be overwritten by future releases of the corresponding language pack. So, make sure to take a copy of the edited file so that you could incorporate the changes you have made in the newly overwritten file.

Note:
Assuming that you are using the German version of the language. If not, use the appropriate language code in the place of de-DE.

[froehli]

Thanks for the input! I thought I had searched for the stop words before, but apparently my search wasn't thorough enough.

I couldn't find the com_finder.commonwords.txt file, but searched around a bit, and found the ...localise.php scripts (probably the way the list works has changed in some recent Joomla version?)

The german file (language/de-DE/de-DE.localise.php) returns an extensive list of search words via `getIgnoredSearchWords`. The list returned by the english version (language/en-GB/en-GB.localise.php) contains only three words: and, in, on.

To me this stop word list is overly excessive and is actively working towards confusing users - if they search for some word and no result turns up, and there's not even a hint about what's going on, that's the worst user experience there can be - they will say the search does not work.

Is there maybe some way to report this to the german translation providers?

Edit: Just found an FAQ entry on the jgerman site, where they say, they filter 996 words which have "no relevance to the actual content of entries... that such a high number of words should be "not relevant" appears highly questionable to me.

Edit2: The language source on github has the com_finder.commonwords.txt (from which the localise.php is probably built during deploying...)?
I have created Issue on the excessive stop word filtering for the german translation.

[imanickam]

Please accept my apologies for my mistake. Somehow, I thought that I am answering the question in the Joomla! 4 forum and hence my answer.

Glad to know that you have found what you wanted in the file localise.php and trying to rectify the issue with the German translation team.