A Call for Positive Discussion on Advanced Search

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Ideas Centre
Post Reply
ChrisGreaves
Registered User
Posts: 17
Joined: Mon Jan 03, 2022 2:54 pm
Location: painful.inspiration.inhabited
Name: Chris Greaves
Contact:

A Call for Positive Discussion on Advanced Search

Post by ChrisGreaves »

The Advanced Search is suitable only for crude searches, and perhaps 95% or more of users of phpBB sites around the world rarely get past the first text search box and the author search box. In theory the Advanced Search pane offers and/or logic, but it does not offer full and/or logic as a logical expression with nested expressions shows.

Code: Select all

{remove | reduce } & { Click | Hiss | {Noise & - Notch } | POP}
A macro from the QuickLinks menu generates a URL. If “Search these results” is then used, a search URL is produced. This URL can be modified by the user. In that sense, the URL is owned by the user.
The Advanced Search generates a URL which can be modified by the user; in that sense, the URL is owned by the user.

Try this:-
(a) QuickLinks New Posts and then
(b) Search These Results for “posting” (as an example keyword)
You will see a URL with the &=parameter convention. That URL could be generated from within php code, VBA, C++ or whatever language you choose; the URL is just a string of characters that begins with http://www.

In many instances the generated search URL is not correct. Something corrupts the translation as documentation within and without forums shows.
The search URL is fed into the php search engine which is also corrupt, at least in the sense that it cannot cope with an isolated negative term. For example, in this forum, ask for “wmgr” and get results, but you cannot ask for “anyone except wmgr”, because that generates an error which is put in the error log - which is not visible to the user! The user is told “zero results”, but is not told why there are no results. Possibly two chunks of code do not communicate with each other.

Of course in the case of members names we humans figure that zero results MUST be wrong, but what if we searched for a term “anything but county”. Or “anything but shire”. Or “anything but province”?

The conclusion reached is that everything from the Advanced Search pane, through the URLs, to the search engine, is riddled with design faults, which means that sometimes, the user has little chance of finding what they want and worse: they may well believe that the data they require is not there, when in truth the data is there, stored in a base of data, but the phpBB system conspires to hide it from the user’s eyes.

A Better Search Interface
(a) The Advanced Search Pane can still exist, but with a different layout.
(b) The QuickLinks Menu can still hold macros.
(c) The Macros can still generate search URLs
(d) The Search URLs can still drive the phpBB search engine.
(e) The Search URLs can be modified manually by the user.
(f) The phpBB Search Engine can still be driven by the search URL.

So what changes?

Change could be rooted in logical expressions. Enhance the ability to communicate with phpBB in a logical manner. Communicating in an illogical manner is by definition, nonsense (“You can’t search just for “not something”; you must always include a positive search term ...”. Searching in phpBB is, in short, illogical.

Consider a logical expression such as {remove | reduce } & { Click | Hiss | {Noise & - Notch } | POP}, and how to allow the user to convey that request to the search engine. The expression here is just an example; it is not proposed as a communication standard! It could have been written (key=remove OR key=reduce ) AND ( key=Click OR key= Hiss OR (key=Noise AND NOT key= Notch ) OR key=POP)
There will be change in the phpBB code, but the overall design - Quick Links and Advanced Search through URL to search engine - need not change at all.

The page topicrow Block lists about one hundred fields from the database. The point has been made that “The users of the forums create a cloud of information among all of us,” and that brings us to the thought: “When a bunch of people add data to a public database, the data ought to belong to them, and in particular, they ought to be able to mine that data”. Anything that gets in the way of obtaining our data is a bad thing, and so a poor interface that will not allow access to data is a bad thing.

Users have a need to submit a logical expression that can do exactly what can be done with an SQL SELECT statement; users should be able to cause a search to be driven by any logical expression that can be dreamed up in SQL, based on those 100 data fields and the logical devices And, Or, Not, and Parentheses.
For this, Users need an Advanced Search pane that lets them drive the generation of a logical expression that generates a search URL that can access the data in the database within topics, posts, and whatever else is public.

A member ought to be able to search Private Messages Sent to or Sent by that user. Why not?

There must be something along the lines of SQL that grabs data from the message database, so why not SQL or whatever a database uses?
A recent post suggested change making SQL available to users. The forum’s immediate reaction was swift and, to my mind, poorly thought out. Indeed the reaction focused quickly on The End Of The World As We Know It when security is violated.

I fail to see how a user with the GUI form can violate security when the GUI form is the device that is generating SQL and translating that into a URL. phpBB right now allows me to manually generate a URL that interrogates the database and yields a page or more of results. Why would SQL make any difference?
In particular, the GUI form is generating only the equivalent of a string argument to a SELECT clause in MSAccess; it is generating a read-only screen of data extracted from the database.

The User-View

Here is the user’s view of augmenting the QuickLinks menu with search URLs that are suitable for a specific site.
phpBB_050.png
(1) QuickLinks Menu
The user chooses an item from the menu. Typically the menu has about five preset-search URLs (New Posts, Your Posts, ...) which I call “macros”
(2) Advanced Search Pane
One way or another the user can access the Advanced Search pane and enter a motley collection of search terms, search filters, and display filters.
(3) Generated Search URL
The Advanced Search pane generates a URL which can be immediately parsed(7) and passed to the search engine(8)
(4) Search These Results Box
The user can add more terms and thus have produced a more detailed search URL. There appears to be a limit to the mechanism after two or three shots at this. Search URLs with a series of consecutive “+” and “-“ signs in the URL parameters; sometimes the negation appears to be reversed.
(5) phpBB-Modified Search URL
“Active Topics” can return 31 matches. Enter “vba*” in the Search These Results box and the number of matches is reduced to 5. The term “vba*” does not appear in the URL, so this can be classed as a bit of behind-the-scenes work by phpBB.
(6) User-Modified Search URL
The user issues an Advanced Search for posts by the user containing “vba” and gets 371 matches.
The user can modify this URL (Ctrl+L to take me to the browser URL box) and reduce the number of matches to 26.
(7) phpBB URL Parser
When the user taps the <Enter> key, the browser emits the search URL to the phpBB machine which translates the parameters into syntax suitable for the search engine.
The string can be imagined as “A Language for Queries which (language) is Structured”, but I don’t mind using the abbreviation “SQL” for now. It matters only that the URL can be translated in a consistent manner that will correctly extract and filter data from the database.
(8) phpBB Search Engine
The search engine reads data from the database and filters it, sorts it and then presents the raw data to be formatted into a human-friendly form.
Important: “The Search Engine Reads Data from the Database”. The search engine ought never be writing data TO the database excepting for internal mechanisms for record management such as updating a date-last-accessed field or any other statistical value useful to designers of the search engine and the database.
(9) Search Results to User’s Screen
phpBB formats the extracted raw data into a form agreeable to the end user.
phpBB_051.png
We are familiar with screens that look like this, depending on the skin in use.

The Site Administrator’s View

Suppose now a member of a phpBB site comes along and has a great idea: Locate topics with your posts where the most recent poster is not yourself.

How can they implement this in Advanced Search?

In LibriVox (audio book recording) our concern is with Audacity’s ability to remove or reduce one or more of Click, Notch, Hiss, Noise or POP.

How to do this in phpBB Advanced Search?

A custom search URL that returns all topics that have received at least one new post (message/reply) during a specified period would be preferred to the current New Posts”. If I am steady about checking new posts first thing each morning, then I want to see all new posts since yesterday morning. In some instances users have complained that by visiting a forum to create a post, the New Posts results on their next visit is updated, and the user fails to be shown some posts.

Why can’t we search only for posts that do not mention “VBA”?

Why can’t we search for “layout” in NRMF? I understand that this is a common term on the NRMF site, but doesn’t a search engine have to examine all records of a database table? I certainly don’t need 90,000 matches, but I ought to be able to search for all posts that do NOT mention “layout” and thereby receive a subset of about 1,000 matches, from which I can refine my search.

If a user or a super-user comes up with a generalized search URL, then the site administrator(s) could add that to the QuickLinks menu as an extra macro, a superb convenience to all their users for all time.

My experience on various phpBB sites shows me that each site uses different conventions for Topics and Posts (LibriVox has excellent conventions) and so macro-capabilities will probably be unique to each phpBB site.
Which is why each site’s members are probably the best people to devise QuickLinks macros (custom search URLs) for that site.

With respect
Chris Greaves
Last edited by Kailey on Mon Jan 24, 2022 3:39 am, edited 1 time in total.
Reason: Removed unnecessary links
Post Reply

Return to “phpBB Discussion”