[RC] Precise Similar Topics II

A place for MOD Authors to post and receive feedback on MODs still in development. No MODs within this forum should be used within a live environment!
Ideas Centre
User avatar
MattF
Extensions Development Coordinator
Extensions Development Coordinator
Posts: 5263
Joined: Sat Jan 17, 2009 9:37 am
Location: Los Angeles, CA
Name: Matt Friedman
Contact:

Re: [RC1] Precise Similar Topics II

Post by MattF »

DoYouSpeakWak wrote:That does not solve the problem. Topics with non stopwords in them listed in both forums will still be listed together. Like "I need help" can be in both forums, but the topics content can be totally diffrent.

We must have 4-5000 topics with the words "i" "need" "help" in the topic tittle, additional to that there is many many other topics with the same problem. Only way to solve it would be a switch not to seach other forums than the one your in.

I hope you can see the problems this missing feature can give in 8-10 yrs old forums with millions of topics. Like here.
But "I need help" are still all stop words, so none of those words would factor in to the Similar Topics results ;) Like, "I need help with my Camaro" and "I need help building my Dell", for example, have only Camaro, building and Dell as keywords... So the former would match up with any other "Camaro" topics and the latter with any "building, Dell" topics.

My main concern with your idea is that somebody would then want the options to make Forum X exclusively search Forum X and Forum Y, but not Forum Z... I tried creating that system - to assign multiple specific forums to search in - and it made things WAY too complicated. It made the MOD way more "heavy" than I wanted it to be and also meant the Admin would have to go through a cumbersome forum-by-forum set up process for each forum's specific similar topics settings. :cry:

I have been testing this on a forum with over 16,000 topics and for the most part it is working just fine! Of course, you do find the occassional topic whose title leads to un-similar similar topics :lol: But for the most part it seems to be finding things even from the "Off Topic" forum, that are relevant :)
Last edited by MattF on Wed Jun 16, 2010 5:33 am, edited 1 time in total.
Formerly known as VSEMy ExtensionsPlease do not PM me for support.
User avatar
MattF
Extensions Development Coordinator
Extensions Development Coordinator
Posts: 5263
Joined: Sat Jan 17, 2009 9:37 am
Location: Los Angeles, CA
Name: Matt Friedman
Contact:

Re: [RC1] Precise Similar Topics II

Post by MattF »

VSE wrote:My main concern with your idea is that somebody would then want the options to make Forum X exclusively search Forum X and Forum Y, but not Forum Z... I tried creating that system - to assign multiple specific forums to search in - and it made things WAY too complicated. It made the MOD way more "heavy" than I wanted it to be and also meant the Admin would have to go through a cumbersome forum-by-forum set up process for each forum's specific similar topics settings. :cry:
Forget this ^^^

I have come up with a good solution to allow Admins to limit a forum to pull similar topics only from itself, or from a select group of multiple forums... It's gonna be nice and flexible and best of all, won't require any new core file edits or SQL queries, etc. ;) Will post a new updated package later when it is ready

EDIT:
First Post Updated to v1.0.1
Upgrading: No new edits, just copy all the included files over the old ones, and run the new install_similar_topics.php file

New Feature: Added optional 'Advanced' Forum settings to allow Admins to customize where each forum can get its Similar Topics from, if they so desire.
Image
Formerly known as VSEMy ExtensionsPlease do not PM me for support.
User avatar
DoYouSpeakWak
Registered User
Posts: 2310
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Contact:

Re: [RC1] Precise Similar Topics II

Post by DoYouSpeakWak »

VSE wrote:
VSE wrote:My main concern with your idea is that somebody would then want the options to make Forum X exclusively search Forum X and Forum Y, but not Forum Z... I tried creating that system - to assign multiple specific forums to search in - and it made things WAY too complicated. It made the MOD way more "heavy" than I wanted it to be and also meant the Admin would have to go through a cumbersome forum-by-forum set up process for each forum's specific similar topics settings. :cry:
Forget this ^^^

I have come up with a good solution to allow Admins to limit a forum to pull similar topics only from itself, or from a select group of multiple forums... It's gonna be nice and flexible and best of all, won't require any new core file edits or SQL queries, etc. ;) Will post a new updated package later when it is ready

EDIT:
First Post Updated to v1.0.1
Upgrading: No new edits, just copy all the included files over the old ones, and run the new install_similar_topics.php file

New Feature: Added optional 'Advanced' Forum settings to allow Admins to customize where each forum can get its Similar Topics from, if they so desire.
Image
That sounds great. I cant wait to test it out. Thanks for putting so much effort into this feature. Im glad you feel its as important as i do, this makes its even more precise.
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid services. 10+ years of experience with phpBB3 and as System Administrator.
User avatar
PlanetStyles.net
Former Team Member
Posts: 4665
Joined: Wed Nov 04, 2009 11:16 pm
Location: UK
Name: Christian
Contact:

Re: [RC1] Precise Similar Topics II

Post by PlanetStyles.net »

Sweet MOD, Bookmarked it as I plan to use it in the near future :).

With regard to the "Number of years worth to search", are there any plans to make this more flexible? One thing I hate on a forum is when an ancient topic gets bumped. Whilst I'd love to install this MOD, I feel that displaying topics that are "too" old will provoke old-topic bumping.

Is something like this within the realms of possibility? :)
Image
User avatar
MattF
Extensions Development Coordinator
Extensions Development Coordinator
Posts: 5263
Joined: Sat Jan 17, 2009 9:37 am
Location: Los Angeles, CA
Name: Matt Friedman
Contact:

Re: [RC1] Precise Similar Topics II

Post by MattF »

Christian 2.0 wrote:Sweet MOD, Bookmarked it as I plan to use it in the near future :).

With regard to the "Number of years worth to search", are there any plans to make this more flexible? One thing I hate on a forum is when an ancient topic gets bumped. Whilst I'd love to install this MOD, I feel that displaying topics that are "too" old will provoke old-topic bumping.

Is something like this within the realms of possibility? :)
Image
that's a good idea... i'll implement that... good thing you posted this as I was just getting ready to submit it to the MODDB :lol:
Formerly known as VSEMy ExtensionsPlease do not PM me for support.
User avatar
PlanetStyles.net
Former Team Member
Posts: 4665
Joined: Wed Nov 04, 2009 11:16 pm
Location: UK
Name: Christian
Contact:

Re: [RC1] Precise Similar Topics II

Post by PlanetStyles.net »

Nice one, thanks. :D
User avatar
MattF
Extensions Development Coordinator
Extensions Development Coordinator
Posts: 5263
Joined: Sat Jan 17, 2009 9:37 am
Location: Los Angeles, CA
Name: Matt Friedman
Contact:

Re: [RC1] Precise Similar Topics II

Post by MattF »

Updated to v1.0.2 - See first post

Upgrading: No new edits, just copy all the included files over the old ones, and run the new install_similar_topics.php file and then purge your cache!

New feature: Per Christian 2.0's idea/request, you can now define Similar Topics search Period in terms of days, weeks, months or years. :mrgreen:
Formerly known as VSEMy ExtensionsPlease do not PM me for support.
User avatar
PlanetStyles.net
Former Team Member
Posts: 4665
Joined: Wed Nov 04, 2009 11:16 pm
Location: UK
Name: Christian
Contact:

Re: [RC1] Precise Similar Topics II

Post by PlanetStyles.net »

Thanks very much, wasted no time with that then did you? :) - you even used my demo text, I'm flattered :P

Seeing as it's 1am at the moment I'll install it tomorrow and let you know how it turns out. :)
User avatar
MattF
Extensions Development Coordinator
Extensions Development Coordinator
Posts: 5263
Joined: Sat Jan 17, 2009 9:37 am
Location: Los Angeles, CA
Name: Matt Friedman
Contact:

Re: [RC1] Precise Similar Topics II

Post by MattF »

Christian 2.0 wrote:Thanks very much, wasted no time with that then did you? :) - you even used my demo text, I'm flattered :P
Well, it made perfect sense to me me. :lol: I also gave you credit for the idea in the MODX file's version history.
Formerly known as VSEMy ExtensionsPlease do not PM me for support.
User avatar
DoYouSpeakWak
Registered User
Posts: 2310
Joined: Fri Jul 25, 2008 1:32 pm
Location: Island of Wak-Wak
Contact:

Re: [RC] Precise Similar Topics II

Post by DoYouSpeakWak »

After a bit of testing, i can conclude that this is not a mod for our forum. The functions are just perfect. But with millions of posts over the past 10 yrs some topics just take 3-6 seconds to load compared to normally 0.090 sec. There is not much that can be done about it i think. besides lowering the search back time from 10yrs to less.
Whatever you share comes back. Support the phpBB Communities
My Validated and Released Modifications
Offering paid services. 10+ years of experience with phpBB3 and as System Administrator.
emsik
Registered User
Posts: 102
Joined: Wed Jul 01, 2009 12:55 pm

Re: [RC] Precise Similar Topics II

Post by emsik »

User avatar
MattF
Extensions Development Coordinator
Extensions Development Coordinator
Posts: 5263
Joined: Sat Jan 17, 2009 9:37 am
Location: Los Angeles, CA
Name: Matt Friedman
Contact:

Re: [RC] Precise Similar Topics II

Post by MattF »

DoYouSpeakWak wrote:After a bit of testing, i can conclude that this is not a mod for our forum. The functions are just perfect. But with millions of posts over the past 10 yrs some topics just take 3-6 seconds to load compared to normally 0.090 sec. There is not much that can be done about it i think. besides lowering the search back time from 10yrs to less.
How many topics does your forum have? Because this MOD only looks for Topic titles. I just tested it on a forum with over 17,000 topics (188,500 posts) and on average the load times were the same :)
That MOD is using essentially the exact same SQL query as this one, plus it's doing a bunch of extra stuff on top, so it might actually be a bit heavier on ram usage and load times.
Last edited by MattF on Wed Jun 16, 2010 7:56 am, edited 1 time in total.
Formerly known as VSEMy ExtensionsPlease do not PM me for support.
User avatar
MattF
Extensions Development Coordinator
Extensions Development Coordinator
Posts: 5263
Joined: Sat Jan 17, 2009 9:37 am
Location: Los Angeles, CA
Name: Matt Friedman
Contact:

Re: [DEV] Precise Similar Topics II

Post by MattF »

And it continues to develop...

Updated to v1.1.0 - See first post

IMPORTANT NOTES ABOUT UPDATING:
1) If you are going to update from an older version (1.0.0 - 1.0.2) then you must first UNINSTALL your old version by running the install_similar_topics.php file that came with your older version and choose the UNINSTALL option. After that you can install this new version. (You can download older versions from my repository linked to in the first post if you lost your original install_similar_topics.php file.)

2) There is also a change to the edit instructions for the viewtopic.php page, so you will also need to make that change if upgrading from the older version. Sorry, no update instructions were provided, but it's only one edit ;)

3) All files included in the root folder with this MOD should be uploaded to your forum.

There are no new features in this version. But with all the other upgrades/features added, I really needed to reorganize the code and database variables. Because of this, version 1.1.0 is now considered the new ground-zero for this MOD.

Thanks to all who have tested so far - I welcome any more feedback, issues and comments :mrgreen:

Planned addition: Group-based permissions. So you can allow this feature for certain or all groups.
Formerly known as VSEMy ExtensionsPlease do not PM me for support.
User avatar
MattF
Extensions Development Coordinator
Extensions Development Coordinator
Posts: 5263
Joined: Sat Jan 17, 2009 9:37 am
Location: Los Angeles, CA
Name: Matt Friedman
Contact:

Re: [DEV] Precise Similar Topics II

Post by MattF »

knmt wrote:Can we have any caching? That would be great :D
You got any ideas on how best to do that? :?

EDIT: I got some things working...

I guess the question now, is how long should the cache data live?

A longer cache life means:
Pros: Much less MySQL searching
Cons: New topics won't show up yet in other similar topics that have already been cached, until re-cached
Cons: Deleted topics will still be listed in cached results, until re-cached

A shorter life has the opposite pros and cons ;)

-------------------------------------------------------

HERE IT IS: Enabled caching of similar topics results
The curious may download this new copy of similar_topics.php from my repository, with caching ability enabled. Please test it out and let me know how it works. It is currently set to cache results for topics for 30 minutes.
Formerly known as VSEMy ExtensionsPlease do not PM me for support.
Locked

Return to “[3.0.x] MODs in Development”