MOD Search_partial_display

This forum is now closed as part of retiring phpBB2.
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

This forum is now closed due to phpBB2.0 being retired.
Merlin Sythove
Registered User
Posts: 2339
Joined: Tue Mar 16, 2004 7:42 am

MOD Search_partial_display

Post by Merlin Sythove »

In the topic on removing bbCode also the option came up to get a nicer display for truncated messages in the search results window. Rather than remove all bbCode so it is no longer clear what is a quote and so on.

I have chosen a different approach. I create the full message first. Then I count as many characters between html code as i want to display, and start cleaning text (again between html code only) from that point onwards. This will give a truncated message that is formatted exactly like the real message will be in viewtopic.

Documentation is given below. I'm not the one for dollying up mod documentation but if someone else is interested in trying it out and having a go at creating a proper mod ready for everyone else, be my guest :D

Code: Select all

##############################################################
## MOD Title: MOD post_partial_display
## MOD Author: Merlin Sythove 
## MOD Description: This mod improves the way search displays partial posts.
##   This mod ensures that bbcode displays properly when a user conducs a search
##   and displays less than the full text of the resulting posts.  
##
## MOD Version: 2.0.0
##
## Installation Level: (Easy)
## Installation Time:   5 Minutes
## Files To Edit:
##    search.php
##    includes/functions_search.php
##    language/lang_english/lang_main.php
##############################################################
## For Security Purposes, Please Check: http://www.phpbb.com/mods/ for the
## latest version of this MOD. Downloading this MOD from other sites could cause malicious code
## to enter into your phpBB Forum. As such, phpBB will not offer support for MOD's not offered
## in our MOD-Database, located at: http://www.phpbb.com/mods/
##############################################################
## Author Notes:   
##############################################################
## MOD History:
##  2008-20-05 - Version 2.0.0
##   2005-03-18 - Version 1.0.1
##           initial version
##
##############################################################
## Before Adding This MOD To Your Forum, You Should Back Up All Files Related To This MOD
##############################################################

#
#-----[ OPEN ]------------------------------------------
#
search.php

#
#-----[ FIND ]------------------------------------------
#
          if ( $return_chars != -1 )
					{
						$message = strip_tags($message);
						$message = preg_replace("/\[.*?:$bbcode_uid:?.*?\]/si", '', $message);
						$message = preg_replace('/\[url\]|\[\/url\]/si', '', $message);
						$message = ( strlen($message) > $return_chars ) ? substr($message, 0, $return_chars) . ' ...' : $message;
					}
					else

#
#-----[ REPLACE WITH ]------------------------------------------
#
               //START MOD post_partial_display
               //We'll do the trimming later
               //if ( $return_chars != -1 )
               //{
               //   $message = strip_tags($message);
               //   $message = preg_replace("/\[.*?:$bbcode_uid:?.*?\]/si", '', $message);
               //   $message = preg_replace('/\[url\]|\[\/url\]/si', '', $message);
               //   $message = ( strlen($message) > $return_chars ) ? substr($message, 0, $return_chars) . ' ...' : $message;
               //}
               //else
               //END MOD post_partial_display

#
#-----[ FIND ]------------------------------------------
#
      $message = make_clickable($message);

#
#-----[ AFTER, ADD ]------------------------------------------
#
    //MOD post_partial_display
    if ($return_chars)  $message = post_partial_display($message, $return_chars, $post_url);


#
#-----[ OPEN ]------------------------------------------
#
includes/functions_search.php

#
#-----[ FIND ]------------------------------------------
#
?>

#
#-----[ BEFORE, ADD ]------------------------------------
#
//START MOD post_partial_display
//Trim a post to $length characters but leave all html code intact.
//Note: do this AFTER bbcode has been processed into html!!
function post_partial_display($message, $length, $post_url)
{
  global $lang;

  $count = $in_html = $pos = 0;
  for ($j = 0; $j < strlen($message); $j++)
  {
    switch ($message[$j]) //Current character
    {
      case '<': //entering html code
      $in_html = 1;
      break;

      case '>': //exit html code
      $in_html = 0;
      break;

      default:
      if (!$in_html) //Not in html code, so count and/or clean
      {
        $count++;
        //Not yet cleaning, have as much as we want, plus up to next space: start cleaning
        if(!$pos && $count > $length && $message[$j] == ' ') $pos = $j;
        if ($pos)  $message[$j] = "~"; //Clean with "~"
      }
      break;
    }
  }

  //Have cleaned anything? Then tidy up tail and construct return message
  if ($pos)
  {
     $head = substr($message, 0, $pos);
     $tail = str_replace('~', '', substr($message, $pos));

     //Remove selected CSS
     $tail = str_replace(' class="quote"', '', $tail);
     $tail = str_replace(' class="code"', '', $tail);

     //Remove selected complete HTML
     $tail = str_replace('<li>', '', $tail);
     $tail = preg_replace('/<a.*?<\/a>/si', '', $tail);
     $tail = preg_replace("/\<img.*?\>/si", '', $tail);

     $message = $head . ' (...)' . $tail; //Inside quote for example
     //After all html (after all quotes etc) add a 'Read more' link
    if ($post_url && $lang['Read_more']) $message .= '<br /><span class="postdetails"><i><a class="postdetails" href="' . $post_url . '">' . $lang['Read_more'] . '</a></i></span>';
  }
  return $message;
}
//END MOD post_partial_display

#
#-----[ OPEN ]------------------------------------------
#
language/lang_english/lang_main.php

#
#-----[ FIND ]------------------------------------------
#
?>

#
#-----[ BEFORE, ADD ]------------------------------------------
#
//MOD post_partial_display
$lang['Read_more'] = 'Read more ...';


#-----[ SAVE/CLOSE ALL FILES ]------------------------------------------
#
# EoM

				
          
Please note that the very last line where some html is added to show "Read more..." as a link to the message, the classes and so forth may not be standard, it may need tweaking. Also, the language entry may not exist as standard, have not checked that yet.

EDIT 24 May 2008: Updated MOD file - I had reason to revisit the mod and update it.
Last edited by Merlin Sythove on Sat May 24, 2008 11:39 am, edited 5 times in total.
asinshesq
Registered User
Posts: 6266
Joined: Sun Feb 22, 2004 9:34 pm
Location: NYC
Name: Alan

Post by asinshesq »

Merlin, that's intensely cool. I haven't tried it yet but I understand the approach...very clever. My one hesitation you've already mentioned: speed. That's a lot of character by character work there. Any feel for how much this might weigh down a board? Or is it pretty much lost in the rounding for a medium size (couple of hundred users) board?
Merlin Sythove
Registered User
Posts: 2339
Joined: Tue Mar 16, 2004 7:42 am

Post by Merlin Sythove »

The code is done for as many posts as you have on 1 page only, like 15 or 25 posts, depending on the average size of posts and the settings you have chosen. I think page speed generation has dropped from 0.0025 seconds to 0.002612 seconds or something silly like that. You can check it with a page speed mod if you don't have something like that installed yet.

Anyway, the page generation time is only a minimal fraction of the download time for the user to see the page. My guess is that the actual SQL execution time is far longer than the post cleanup time.

So unless your host has a toy-windup server this is absolutely no problem (and you should move hosts in that case :D)

What WILL speed up the routine further (if you feel so inclined) is to check from your point where you have as much text as you want, if there is a further opening html bracket in the remainder, and if there is not, to just ditch the remainder and not loop through all of it to hunt for html code. If you have a board with lots of long posts without bbcode, you could do it. But I doubt if you would notice any speed improvement for all your work :D

Also: I have changed my own selection box in the main search window where you select how many characters you want returned, to "all", 250 and 500. If you test drive the routine, you will notice that even with the 250 setting many posts are already shorter than that and are not processed at all by this routine. So ideally find a search setting that will give you a page somewhere with 25 very long posts with lots of bbCode in it (maybe you have an author like that... :lol:), and then check the time difference between showing all text (less time) and showing only part of the text (more time, extra processing done).

My guess is that this part, trimming posts, is legacy-last-century when each byte you cut saved you noticeable time on servers and download. I always have my preference set to "all text" and see the cutting down as a visual aid, giving you a quicker overview of your results, not as a time saver.
Merlin Sythove
Registered User
Posts: 2339
Joined: Tue Mar 16, 2004 7:42 am

Post by Merlin Sythove »

Code: Select all

                    //Remember first cleaned character position. Clean with "~" 
                    if ($pos == 0) $pos = $j; 
                    $message[$j] = "~"; 
Change to

Code: Select all

                    //Clean with "~"
                    $message[$j] = "~"; 
                    //Remember first cleaned character position. 
                    if ($pos == 0) 
                    {
                      $pos = $j;
                      //See if there is any point in continuing
                      if (!  strpos($message, '<', $j ) )  $message = substr($message, 0, $j);
                    }                    
But I doubt if you will see any speed increase.
asinshesq
Registered User
Posts: 6266
Joined: Sun Feb 22, 2004 9:34 pm
Location: NYC
Name: Alan

Post by asinshesq »

Merlin, I haven't tested this yet, but here's your mod prettied up a bit to make it look phpbbish (it should be installable by easymod in this form):

Code: Select all

##############################################################
## MOD Title: search for partial posts display improvement
## MOD Author: Merlin Sythove < [Merlin's email address] > (Merlin Sythove) N/A
## MOD Description: This mod improves the way search displays partial posts.
##	This mod ensures that bbcode displays properly when a user conducs a search and displays less
##	than the full text of the resulting posts.  Without this mod, for example, the quote tag is simply stripped
##	when displaying partial posts...with this mod, the quote tag gets retained so that the partial post
##	shows which text is being quoted versus which text is the poster's new text; and the mod retains all
##	other bbcode as well.
##
## MOD Version: 1.0.1
##
## Installation Level: (Easy)
## Installation Time: 2 Minutes
## Files To Edit: 	search.php
##############################################################
## For Security Purposes, Please Check: http://www.phpbb.com/mods/ for the
## latest version of this MOD. Downloading this MOD from other sites could cause malicious code
## to enter into your phpBB Forum. As such, phpBB will not offer support for MOD's not offered
## in our MOD-Database, located at: http://www.phpbb.com/mods/
##############################################################
## Author Notes:	This mod changes the way that search.php processes partial post results
##				so that it:
##				-	first, gets the complete message with all highlighting, html and so forth;
##
##				-	next, goes through message 1 character at a time until it reaches the last
##					character that will be displayed;
##
##				-	finally, deletes the rest of the message (the characters that will not be
##					displayed) but leaves the html in place (the stuff between the '<' and the '>')
##					so that there will be no hanging open html tag without a corresponding
##					close html tag (in order to prevent an isolated open html tag from messing
##					up the rest of the page layout).
##############################################################
## MOD History:
##
##   2005-03-18 - Version 1.0.1
##			  initial version
##
##############################################################
## Before Adding This MOD To Your Forum, You Should Back Up All Files Related To This MOD
##############################################################

#
#-----[ OPEN ]------------------------------------------
#
search.php

#
#-----[ FIND ]------------------------------------------
#
					if ( $return_chars != -1 )
					{
						$message = strip_tags($message);
						$message = preg_replace("/\[.*?:$bbcode_uid:?.*?\]/si", '', $message);
						$message = preg_replace('/\[url\]|\[\/url\]/si', '', $message);
						$message = ( strlen($message) > $return_chars ) ? substr($message, 0, $return_chars) . ' ...' : $message;
					}
					else
					{

#
#-----[ REPLACE WITH ]------------------------------------------
#
					// start mod search for partial posts display improvement
					// commented out the following lines since cleaning the post and limiting it to the number of
					// requested characters will now happen after the post is ready to display the regular way...
					//if ( $return_chars != -1 )
					//{
					//	$message = strip_tags($message);
					//	$message = preg_replace("/\[.*?:$bbcode_uid:?.*?\]/si", '', $message);
					//	$message = preg_replace('/\[url\]|\[\/url\]/si', '', $message);
					//	$message = ( strlen($message) > $return_chars ) ? substr($message, 0, $return_chars) . ' ...' : $message;
					//}
					//else
					//{
					// end mod search for partial posts display improvement

#
#-----[ FIND ]------------------------------------------
#
					}

					if ( count($orig_word) )

#
#-----[ REPLACE WITH ]------------------------------------------
#
					// start mod search for partial posts display improvement
					// only need to do special processing if user asks to display less than all text from resulting posts and
					// the post is longer than the number of characters per post the user has asked to display
					if ( ($return_chars != -1) && (strlen($message) > $return_chars))
					{
						// initialize all variables...we start in 'counting' mode and have not yet found
						// the cutoff position where the display should stop
						$p_count = 0;
						$p_counting_or_cleaning = 1;
						$p_cleaning = 0;
						$pos = 0;

						// now, loop through every single character in the post...
						// if we are still in the part of the message that will be displayed (i.e. we are not yet in 'cleaning' mode)
						// and we are not inside of an html tag, then count the character;
						// but once we are beyond the part that will be displayed, change every character to a ~ (i.e. 'clean  it)
						// unless it's inside an html tag
						for ($j = 0; $j < strlen($message); $j++)
						{
							switch ($message[$j])
							{
								case '<': //entering html code: stop counting or cleaning
								$p_counting_or_cleaning = 0;
								break;

								case '>': //exit html code: start counting or cleaning
								$p_counting_or_cleaning = 1;
								break;

								default:
								if ($p_counting_or_cleaning) // checks to see if we are counting or cleaning versus being in an html tag
								{
									$p_count++;
									//Have as much as we want: start cleaning at first space
									//this will leave smilies code complete for parsing later.
									if ($p_count > $return_chars)
									{
										 // when we finally reach the number of chars to display, the next if statement
										 // assures we do not start cleaning till we reach the next space; this leaves smilies
										 // code complete for later parsing
										if ($message[$j] == ' ') $p_cleaning = 1;
									}
									if ($p_cleaning)
									{
										//Remember first cleaned character position; then clean with "~"
										if ($pos == 0) $pos = $j;
										$message[$j] = "~";
									}
								}
								break;
							}
						}

						// now finish this up...if only part of the post will display (because it is longer than the number of characters per post being displayed), then:
						// separate the part that will display (the 'head') from the part that will not (the 'tail'),
						// delete all '~' in the tail (so the only thing left in the tail are html tags with nothing in between), merge the head and tail, and
						// finally add "Read more..." link at the end of the post
						if ($pos > 0)
						{
							$head = substr($message, 0, $pos);
							$tail = str_replace('~', '', substr($message, $pos));
							
							// get rid of any list item in there
							$tail = str_replace('<li>', '', $tail);

							// now remove any class="quote" or class="code" designations in $tail
							$tail = str_replace(' class="quote"', '', $tail);
							$tail = str_replace(' class="code"', '', $tail);

							// Remove all tags that begin with 'a' or 'img'
							$tail = preg_replace('/<[a]([^<>]*?)>/', '', $tail);
							$tail = preg_replace('/<[i][m][g]([^<>]*?)>/', '', $tail);
							
							$message = $head . ' (...)' . $tail; //Inside quote for example
							//After all html (after all quotes etc) add a 'REad more' link
							$message .= '<br /><span class="postdetails"><i><a class="postdetails" href="' . $post_url . '">' . $lang['Read_more'] . '</a></i></span>';
						}

					}
					// end mod search for partial posts display improvement

					if ( count($orig_word) )

#
#-----[ OPEN ]------------------------------------------
#
language/lang_english/lang_main.php

#
#-----[ FIND ]------------------------------------------
#
?>

#
#-----[ BEFORE, ADD ]------------------------------------------
#
// start mod search for partial posts display improvement
$lang['Read_more'] = 'read rest of post';
// end mod search for partial posts display improvement

#-----[ SAVE/CLOSE ALL FILES ]------------------------------------------
#
# EoM
[edit: on March 21, I cleaned this up a bit from the original code to better deal with img tags, url tagss, email tags and code and quote block tags appearing in the tail of the message]

[edit: on March 23, I abandoned the idea of stripping out tds and trs from the tail; instead I now have it strip out ' class="quote'' and ' class="code"' so that there are no funny white spaces that appear in the tail. This is not a perfect solution since it means that if a post has multiple quote or code blocks in the tail (after the number of chars that the user has asked to display), the post will have extra skipped lines in it (the result of extra trs), but it doesn't look bad, especially if the user asks for the results to return a reasonable number of chars)]
Last edited by asinshesq on Wed Mar 23, 2005 11:50 pm, edited 3 times in total.
asinshesq
Registered User
Posts: 6266
Joined: Sun Feb 22, 2004 9:34 pm
Location: NYC
Name: Alan

Post by asinshesq »

By the way, are you still looking for a way to short cut some of the one-by-one character counting through clever use of regular expressions or do you think this will end up being just as fast so that there's no reason to struggle with that?
Merlin Sythove
Registered User
Posts: 2339
Joined: Tue Mar 16, 2004 7:42 am

Post by Merlin Sythove »

Hi Alan,

Looks great, thanks! Have to wait for user feedback now... :D

As for the preg_replace, you're welcome to have a go, I found that my chosen solution is simple enough to follow, and it works without problems.

With preg_replace you would still have to count the beginning bit to get your 200 or 400 "real" characters, then continue looping until you are out of html and have found a space to break the post, so only the last part of the string could be done with a preg_replace. It may very well be faster, but as I said before, so far I have not seen any speed problems.
Merlin Sythove
Registered User
Posts: 2339
Joined: Tue Mar 16, 2004 7:42 am

Post by Merlin Sythove »

Sorry- double post
asinshesq
Registered User
Posts: 6266
Joined: Sun Feb 22, 2004 9:34 pm
Location: NYC
Name: Alan

Post by asinshesq »

I just tried it...works fabulously. Nice to have a partial search that look good!

I took out the parenthetical around the '...' at the end since I think it looks better without it. And I'm still trying to decide whether the 'read more' link is useful, given that it simply repeats the link that is already there at the top of the post in question.
Merlin Sythove
Registered User
Posts: 2339
Joined: Tue Mar 16, 2004 7:42 am

Post by Merlin Sythove »

They're supposed to be visual clues that not the whole message is being displayed. But hey - your board, you decide... :lol:
asinshesq
Registered User
Posts: 6266
Joined: Sun Feb 22, 2004 9:34 pm
Location: NYC
Name: Alan

Post by asinshesq »

Merlin Sythove wrote: They're supposed to be visual clues that not the whole message is being displayed. But hey - your board, you decide... :lol:


I like the visual clue of '...' but I didn't like the look of the parentheiss around it. So I just took out the ( and ) but left in the '....' I guess I'll leave in the link too (as another 'visual clue').
Merlin Sythove
Registered User
Posts: 2339
Joined: Tue Mar 16, 2004 7:42 am

Post by Merlin Sythove »

Yes, matter of taste of course. (...) is a well established indication that a section has been cut out or cut off (in email-land for example).
asinshesq
Registered User
Posts: 6266
Joined: Sun Feb 22, 2004 9:34 pm
Location: NYC
Name: Alan

Post by asinshesq »

Here's a little problem. Even though this properly cleans out all characters past the number of characters the user wants displayed, it will show a small white strip accross the screen for each [ quote ] or [ code ] block that appears in the tail (since it leaves all the tail html entact). So I guess things should be tweaked so that when it is all done, it should do something clever with the residual html relating to quote and code blocks?
Merlin Sythove
Registered User
Posts: 2339
Joined: Tue Mar 16, 2004 7:42 am

Post by Merlin Sythove »

Ah, that is part of your own quote html code in your own bbcode.tpl file I think. If the quote area is in some way visible if there is a quote without any text at all, then you will have that problem.

You could try to replace a part of it that is causing this. A quick inspection shows that if you remove the text

Code: Select all

class="quote"  
from the tail part of the message, you have probably solved the problem.

So:

Code: Select all

### Find
$tail = str_replace('~', '', substr($message, $pos));
### Add after
$tail = str_replace('class="quote"', '', $tail);
and of course make sure that class="quote" is exactly the same as the corresponding text in the quote bbcode.tpl entry, same quotes, same (or absent) spaces etc.
asinshesq
Registered User
Posts: 6266
Joined: Sun Feb 22, 2004 9:34 pm
Location: NYC
Name: Alan

Post by asinshesq »

Merlin Sythove wrote: Ah, that is part of your own quote html code in your own bbcode.tpl file I think....


Nope, my bbcode.tpl file had all that stuff stripped out so that it could be used for plain text email. But you're right that it appears in the normal bbcode.tpl, and for that reason the quote class gets put in when you do a bbencode second pass run.
Merlin Sythove wrote: ...If the quote area is in some way visible if there is a quote without any text at all, then you will have that problem...


It acutally doesn't need to be visible to be visible ;) Your clever code strips out all the text for quote blocks that are in the tail but leaves in all html code, and for that reason you have that qutoe class stuff appear even if it is only in the tail.
Merlin Sythove wrote: ...You could try to replace a part of it that is causing this. A quick inspection shows that if you remove the text

Code: Select all

class="quote"  
from the tail part of the message, you have probably solved the problem.

So:

Code: Select all

### Find
$tail = str_replace('~', '', substr($message, $pos));
### Add after
$tail = str_replace('class="quote"', '', $tail);
and of course make sure that class="quote" is exactly the same as the corresponding text in the quote bbcode.tpl entry, same quotes, same (or absent) spaces etc.


Sounds good, I'll give that a try. I assume code blocks use quote class too so that this new line would do double duty? I'll check that tonight as well.
Post Reply

Return to “[2.0.x] MOD Writers Discussion”