Slow searches on new Linode machine

Get help with installation and running phpBB 3.2.x here. Please do not post bug reports, feature requests, or extension related questions here.
Albert Wiersch
Registered User
Posts: 145
Joined: Sat Dec 11, 2004 6:00 pm
Location: Dallas, TX
Name: Albert Wiersch
Contact:

Re: Very slow searches for simple words

Post by Albert Wiersch » Thu Mar 29, 2018 11:18 pm

JoshyPHP wrote: ↑
Thu Mar 29, 2018 11:05 pm
Based on a cursory examination of the file, I don't believe there are any side-effects to disabling highlighting apart from the obvious.
Unfortunately I think I found one... I updated my previous post with it.

User avatar
JoshyPHP
Code Contributor
Posts: 1046
Joined: Mon Jul 11, 2011 12:28 am

Re: Very slow searches for simple words

Post by JoshyPHP » Thu Mar 29, 2018 11:26 pm

Alternatively, you can comment the 3 individual lines that contain the <span class="posthilit"> part. That's lines 1089, 1172 and 1173 on my local copy.
I wrote the thing that does BBCodes in 3.2.

User avatar
3Di
Former Team Member
Posts: 14348
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milan (IT) Frankfurt (DE)
Name: Marco
Contact:

Re: Very slow searches for simple words

Post by 3Di » Thu Mar 29, 2018 11:28 pm

Find this line (line 669 in 3.2.2)

$hilit = str_replace(' ', '|', $hilit);

replace it with

$hilit = false;

Not truly tested but I am sure there will be no more highlights visible.

edit: tested after I did recreate the index, it doesn't show the results in other pages.
Please PM me only to request paid works. Thx.
Want to compensate me for my interest? Donate
My development's activity ΒΊ PhpStorm's proud user
Extensions, Scripts, MOD porting, Update/Upgrades
πŸ‘¨β€πŸ« | Take a tour to | The Studio | πŸ‘¨β€πŸ«

Albert Wiersch
Registered User
Posts: 145
Joined: Sat Dec 11, 2004 6:00 pm
Location: Dallas, TX
Name: Albert Wiersch
Contact:

Re: Very slow searches for simple words

Post by Albert Wiersch » Fri Mar 30, 2018 12:32 am

JoshyPHP wrote: ↑
Thu Mar 29, 2018 11:26 pm
Alternatively, you can comment the 3 individual lines that contain the <span class="posthilit"> part. That's lines 1089, 1172 and 1173 on my local copy.
That got rid of the highlights but it didn't fix the search time problem... so you get the bad stuff but not the good stuff.

Maybe the problem is here (just a guess):

Code: Select all

		if ($hilit)
		{
			// Remove bad highlights
			$hilit_array = array_filter(explode('|', $hilit), 'strlen');
			foreach ($hilit_array as $key => $value)
			{
				$hilit_array[$key] = phpbb_clean_search_string($value);
				$hilit_array[$key] = str_replace('\*', '\w*?', preg_quote($hilit_array[$key], '#'));
				$hilit_array[$key] = preg_replace('#(^|\s)\\\\w\*\?(\s|$)#', '$1\w+?$2', $hilit_array[$key]);
			}
			$hilit = implode('|', $hilit_array);
		}
Nevermind... I commented that out and search was still way too slow.

User avatar
JoshyPHP
Code Contributor
Posts: 1046
Joined: Mon Jul 11, 2011 12:28 am

Re: Very slow searches for simple words

Post by JoshyPHP » Fri Mar 30, 2018 1:00 am

Interesting. It's impossible to know for sure what's making it slow on your installation without profiling it. If it's not the regexp used in preg_replace() then the second likeliest culprit would be the algorithm in get_context(), although I can't reproduce it locally.

You can test this hypothesis by replacing line 1152 from this:

Code: Select all

$row['post_text'] = get_context($row['post_text'], array_filter(explode('|', $hilit), 'strlen'), $return_chars);
To this:

Code: Select all

$row['post_text'] = get_context($row['post_text'], [], $return_chars);
Also, try enabling mbstring on your server.
I wrote the thing that does BBCodes in 3.2.

Albert Wiersch
Registered User
Posts: 145
Joined: Sat Dec 11, 2004 6:00 pm
Location: Dallas, TX
Name: Albert Wiersch
Contact:

Re: Very slow searches for simple words

Post by Albert Wiersch » Fri Mar 30, 2018 4:33 am

JoshyPHP wrote: ↑
Fri Mar 30, 2018 1:00 am
Interesting. It's impossible to know for sure what's making it slow on your installation without profiling it. If it's not the regexp used in preg_replace() then the second likeliest culprit would be the algorithm in get_context(), although I can't reproduce it locally.

You can test this hypothesis by replacing line 1152 from this:

Code: Select all

$row['post_text'] = get_context($row['post_text'], array_filter(explode('|', $hilit), 'strlen'), $return_chars);
To this:

Code: Select all

$row['post_text'] = get_context($row['post_text'], [], $return_chars);
Yes, that really speeds it up.
JoshyPHP wrote: ↑
Fri Mar 30, 2018 1:00 am
Also, try enabling mbstring on your server.
How important is this for phpBB? Is there a link with more information about enabling mbstring for phpBB? Thanks.

User avatar
3Di
Former Team Member
Posts: 14348
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milan (IT) Frankfurt (DE)
Name: Marco
Contact:

Re: Very slow searches for simple words

Post by 3Di » Fri Mar 30, 2018 4:46 am

Albert Wiersch wrote: ↑
Fri Mar 30, 2018 4:33 am
How important is this for phpBB? Is there a link with more information about enabling mbstring for phpBB? Thanks.
https://area51.phpbb.com/phpBB/search.p ... s=mbstring

https://www.phpbb.com/search/?q=mbstring
Please PM me only to request paid works. Thx.
Want to compensate me for my interest? Donate
My development's activity ΒΊ PhpStorm's proud user
Extensions, Scripts, MOD porting, Update/Upgrades
πŸ‘¨β€πŸ« | Take a tour to | The Studio | πŸ‘¨β€πŸ«

Albert Wiersch
Registered User
Posts: 145
Joined: Sat Dec 11, 2004 6:00 pm
Location: Dallas, TX
Name: Albert Wiersch
Contact:

Re: Very slow searches for simple words

Post by Albert Wiersch » Fri Mar 30, 2018 4:52 am

3Di wrote: ↑
Thu Mar 29, 2018 11:28 pm
Find this line (line 669 in 3.2.2)

$hilit = str_replace(' ', '|', $hilit);

replace it with

$hilit = false;

Not truly tested but I am sure there will be no more highlights visible.

edit: tested after I did recreate the index, it doesn't show the results in other pages.
Thanks. Too bad it also doesn't show the results in other pages. :(

Albert Wiersch
Registered User
Posts: 145
Joined: Sat Dec 11, 2004 6:00 pm
Location: Dallas, TX
Name: Albert Wiersch
Contact:

Re: Very slow searches for simple words

Post by Albert Wiersch » Fri Mar 30, 2018 1:30 pm

JoshyPHP wrote: ↑
Fri Mar 30, 2018 1:00 am
Also, try enabling mbstring on your server.
I asked my buddy to go ahead and enable that and HOLY CRAP!! That must have been it because searches are VERY FAST now.

There should be a warning about this somewhere in the phpBB Administration Control Panel if mbstring is not enabled!

Now I can turn back on searching for guests.

Thank you all for the help. I'm very happy to have finally figured this one out.

User avatar
rubencm
Development Team Member
Development Team Member
Posts: 82
Joined: Fri Oct 05, 2007 2:24 pm
Location: Spain

Re: Slow searches on new Linode machine

Post by rubencm » Wed Sep 26, 2018 11:28 am

Mbstring is not necessary, this was caused by a very inefficient function

User avatar
JoshyPHP
Code Contributor
Posts: 1046
Joined: Mon Jul 11, 2011 12:28 am

Re: Slow searches on new Linode machine

Post by JoshyPHP » Wed Sep 26, 2018 11:36 am

Sounds like you're talking about the mbstring polyfill or some kind of userland function that is not used if mbstring is enabled.
I wrote the thing that does BBCodes in 3.2.

User avatar
rubencm
Development Team Member
Development Team Member
Posts: 82
Joined: Fri Oct 05, 2007 2:24 pm
Location: Spain

Re: Slow searches on new Linode machine

Post by rubencm » Wed Sep 26, 2018 12:20 pm

JoshyPHP wrote: ↑
Wed Sep 26, 2018 11:36 am
Sounds like you're talking about the mbstring polyfill or some kind of userland function that is not used if mbstring is enabled.
I have rewritten get_context and it works fine for me now.
And yes, the php extension is faster than the polyfill library.

Post Reply

Return to β€œ[3.2.x] Support Forum”