Search result giving no result for "the" keyword

Get help with installation and running phpBB 3.2.x here. Please do not post bug reports, feature requests, or extension related questions here.
Post Reply
Lsha
Registered User
Posts: 77
Joined: Mon Feb 25, 2019 10:38 am

Search result giving no result for "the" keyword

Post by Lsha »

this is the keyword (subject, message both has this keyword):

Code: Select all

Bless the Harts
if i search "Bless the Harts" search result gives zero result. if i search "Bless Harts" i get the search result

this is my search settings
https://i.imgur.com/0qTxaR5.png

User avatar
david63
Registered User
Posts: 17246
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Name: David Wood
Contact:

Re: Search result giving no result for "the" keyword

Post by david63 »

"the" is probably being classed as a too frequent word as it will appear in the majority of posts.
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email as they will be ignored

User avatar
Mick
Support Team Member
Support Team Member
Posts: 22061
Joined: Fri Aug 29, 2008 9:49 am
Location: Watching cricket.

Re: Search result giving no result for "the" keyword

Post by Mick »

What’s the result if you surround the words in quotes like “Bless the Harts”?
"The more connected we get the more alone we become" - Kyle Broflovski

Please read: “Am I In The Right Place?” before posting.

User avatar
ctrstudio
Registered User
Posts: 10
Joined: Wed Jan 01, 2020 4:32 am

Re: Search result giving no result for "the" keyword

Post by ctrstudio »

david63 wrote:
Mon Feb 03, 2020 11:54 am
"the" is probably being classed as a too frequent word as it will appear in the majority of posts.
That doesn't explain why the search excludes other keywords in his search phrase only when "the" is included. Somehow the "common word" exclusion is kicking out the entire phrase when "the" is included within the phrase at all.

A "common word" exclusion should only exclude the common word; it would be reasonable to expect the phrase "Bless the Hart" would give the same results as "Bless Hart" in such a case.

But, that's the problem being described by lsha; all keywords in the phrase are being excluded, not just the single "common word". That isn't a behavior that an average user would expect when entering a search phrase.
Last edited by ctrstudio on Mon Feb 03, 2020 1:03 pm, edited 3 times in total.

KYPREO
Registered User
Posts: 307
Joined: Fri Feb 02, 2018 9:56 am
Contact:

Re: Search result giving no result for "the" keyword

Post by KYPREO »

This also depends on the search backend you are using.

Phrase searching suggested by Mike is currently only possible with the mySQL fulltext backend. It is also possible with Sphinx (as used on this board) but that is currently broken (I have written a patch which is pending approval in Github). It is NOT possible with phpBB native fulltext.

The word "the" will either be in the stopword list or triggered by the common word threshold.

You need to post your search configuration settings from the ACP to better tell what is going on.
phpBB user since 2002
www.AusRotary.com

User avatar
AmigoJack
Registered User
Posts: 5697
Joined: Tue Jun 15, 2010 11:33 am
Location: グリーン ヒル ゾーン
Contact:

Re: Search result giving no result for "the" keyword

Post by AmigoJack »

As written by others: it differs in configuration/backend: on this board ...but most likely this board has other settings than you.
The worst thing about censorship is ███████████
Affin wrote:
Tue Nov 20, 2018 9:51 am
The problem is probably not my English but you do not want to understand correctly.
...
We will not come anybody anyway, nevertheless, it's best to shit this.

Lsha
Registered User
Posts: 77
Joined: Mon Feb 25, 2019 10:38 am

Re: Search result giving no result for "the" keyword

Post by Lsha »

sorry my bad. search keyword is without any quote. please check the image link for the search settings which i using. is any other settings for search?

david63 wrote:
Mon Feb 03, 2020 11:54 am
"the" is probably being classed as a too frequent word as it will appear in the majority of posts.
if this is the possibility, is there any settings that phpbb will not ignore most frequent word for search result?

User avatar
Lumpy Burgertushie
Registered User
Posts: 67231
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: Search result giving no result for "the" keyword

Post by Lumpy Burgertushie »

the point is that if you have the search set so that it returns every instance of the word "the" then every search someone does with that word will return every post on the board because chances are most every post would have that word in it.
the is the point of the common words system.

robert
I'm baaaaaccckkkk. still doing work on donation basis. PM your needs.

Premium phpBB 3.3 Styles by PlanetStyles.net

If nobody is in the forest, does a tree really fall?

Lsha
Registered User
Posts: 77
Joined: Mon Feb 25, 2019 10:38 am

Re: Search result giving no result for "the" keyword

Post by Lsha »

if i use quote like "Bless the Harts" search result giving output. without quote does not giving any result.

if i search without quote search result give output for this keyword (without common word).

Code: Select all

Bless Harts
bit off topic for this section: is it possible to put default

Code: Select all

""
sign in search box by editing template files? so if any user search keyword will have auto quote.
Lumpy Burgertushie wrote:
Tue Feb 04, 2020 1:45 pm
the point is that if you have the search set so that it returns every instance of the word "the" then every search someone does with that word will return every post on the board because chances are most every post would have that word in it.
the is the point of the common words system.

robert
i got the point. phpbb should ignore The (other common words) but should not ignore rest of word and give the output. it looks like if there is any common word, phpbb ignoring other keyword (not common) as well.

i am not sure if there is any other search setting, because phpbb official site giving search output fine even if there is common keyword.

ACP=>General=>Search Settings
this is the settings i set.
https://i.imgur.com/0qTxaR5.png

KYPREO
Registered User
Posts: 307
Joined: Fri Feb 02, 2018 9:56 am
Contact:

Re: Search result giving no result for "the" keyword

Post by KYPREO »

You are using mySQL fulltext as your search backend. Stopwords (such as "the") for mySQL fulltext searching is actually defined within the mySQL configuration file itself, ie it's not a phpBB issue. See here: https://wiki.phpbb.com/MySQL_Fulltext_Search

You can redefine stop words, which is explained in mySQL's documentation.
Lsha wrote:
Wed Feb 05, 2020 1:41 am


i am not sure if there is any other search setting, because phpbb official site giving search output fine even if there is common keyword.
This board uses the Sphinx search engine as the backend which uses completely different code. Accordingly, the results you get here are irrelevant to your situation. As noted above, there is a bug for Sphinx search which means phrase searching doesn't work (you'll get the same results for bless the harts (no quotation marks) and "bless the harts" (phrase search). That bug has a pending fix which might form part of a future release (I wrote the fix and have it working on my board).
phpBB user since 2002
www.AusRotary.com

Lsha
Registered User
Posts: 77
Joined: Mon Feb 25, 2019 10:38 am

Re: Search result giving no result for "the" keyword

Post by Lsha »

thank you for the explanation.

KYPREO
Registered User
Posts: 307
Joined: Fri Feb 02, 2018 9:56 am
Contact:

Re: Search result giving no result for "the" keyword

Post by KYPREO »

Lsha wrote:
Wed Feb 05, 2020 5:51 am
thank you for the explanation.
No problem.

More info on the mySQL fulltext search is here: https://dev.mysql.com/doc/refman/8.0/en ... words.html

As described in the mySQL documentation, a Boolean search that includes a stopword will return a negative result as that word does not appear in the search index:
A boolean search interprets the search string using the rules of a special query language. The string contains the words to search for. It can also contain operators that specify requirements such that a word must be present or absent in matching rows, or that it should be weighted higher or lower than usual. Certain common words (stopwords) are omitted from the search index and do not match if present in the search string. The IN BOOLEAN MODE modifier specifies a boolean search. For more information, see Section 12.9.2, “Boolean Full-Text Searches”.
(see here: https://dev.mysql.com/doc/refman/8.0/en ... earch.html)

I haven't inspected the phpBB-side implementation of mySQL fulltext, but I suspect what is happening here is that your search for "bless the harts" (no quotation marks) creates a query against the mySQL fulltext index for "bless" AND "the" AND "harts". The word "the" returns a negative result through the mySQL fulltext search as that word does not appear in the mySQL fulltext index, so the whole search query returns a zero result. There might be a way around this, but it would require a rewrite of the phpBB mySQL fulltext implementation to pull the stopword list from mySQL and drop any search terms for stopwords before passing on the query to mySQL.

The other solutions are to use a different search backend (phpBB native or Sphinx). Sphinx doesn't have any stopwords and it is powerful enough that you can create a search index right down to 1-character words with no drop in speed. On my board, I can search for a single character across 2 million posts and the result returns in under 100ms. :shock: Most searches take a few thousandths of a second only.
phpBB user since 2002
www.AusRotary.com

Post Reply

Return to “[3.2.x] Support Forum”