[MODDB] Smartfeed for phpBB 3

A place for MOD Authors to post and receive feedback on MODs still in development. No MODs within this forum should be used within a live environment!
Ideas Centre
Locked
User avatar
MarkDHamill
Registered User
Posts: 4364
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by MarkDHamill »

Not currently. Newsfeeds are designed to show current topical information. If there is enough demand it would not be hard to increase the value, or even let the user select how many days they want. The board owner can set a defined limit on how far back to go. This is not enabled by default but may be necessary for busy boards with lots of people getting feeds.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
User avatar
MarkDHamill
Registered User
Posts: 4364
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Smartfeed 2.0.0 Alpha, Version 1 Available

Post by MarkDHamill »

Okay, scratch that itch and put it through its paces. I didn't have time to work through the ModX installation problems I was having, so you have to deal with a .mod file instead. For the purpose of an Alpha release, it doesn't matter.

PLEASE DO NOT USE ON A LIVE BOARD -- THERE MAY BE SIGNIFICANT BUGS AND SECURITY ISSUES.

Of course, please report your experiences here. Read the installation notes carefully!

Download Here
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
User avatar
Puggs
Registered User
Posts: 80
Joined: Mon Oct 06, 2003 3:53 am
Location: Melbourne, Australia
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by Puggs »

Hi

Looks like i'm 1st... short answer, Installed and working on my test board at http://home.accesspointlive.com/phpBB3/

OK the longer answer, It needed mcrypt and i didn't have this installed, i fixed it by installing it, but other may not be able to do this, anyway to have it work without mcrypt?

Puggs
User avatar
MarkDHamill
Registered User
Posts: 4364
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by MarkDHamill »

In my phpBB 2 version, the Mod Team said it was a critical problem that I was not truly encrypting the MD5 hash of the password. I looked at various lightweight solutions and the simplest was to use mcrypt, which is typically but not always compiled with PHP.

Smartfeed will still run without mcrypt, but you can only select from public forums. If you have no forums that are not available to the public, then it's not an issue. Maybe in that case I should have a switch that turns off the annoying error message.

The mod team did point me to the PHP Secure Communications Library:

http://phpseclib.sourceforge.net/

I remember spending some time with this and having issues and finally running out of patience. I could give it another try. Of course I would have to include it in the package, and I suspect is would run slower than mcrypt, which is compiled.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
joe_postscript
Registered User
Posts: 109
Joined: Sun May 25, 2003 3:32 am
Location: Columbus Ohio USA
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by joe_postscript »

I went to the test interface, made a feed, hit the URL and there comes email addresses of the poster included in the feed. This really needs an ability to kill that info because you are feeding spammers the addresses of the members through rss.

So, another feature request would be to limit/kill the poster's info that goes out in a news feed. I'm adding it to the feature requests already made. If RSS is to be meaningful, it has to be topical, searchable, and pretty clean or it can't get the optimum traffic.

(1) minimum post requirement - number of words or characters - kills meaningless post like " I agree" or "ROFL, me too" from going out like it was a meaningful post..

(2) textual formatting and character replace - replace or kill useless formatting like html gibberish that makes newsfeeds hard to read and not as good for aggregation.

(3) Banword List - viagra, cel phones, etc. - can get eliminated from newsfeed based on words, thus your spammers aren't using your site for newsfeed spam, thus getting your site banned from aggregation.

(3) Kill/Limit Posters info - having the email of the poster go into a newsfeed is spammer food. This has to be parsed out and in general the name/ID of poster is of no use making newsfeeds full of non-topical noise.

(5) Strip html and pictures/images - useless logos, images, icons, link spam, and noisy text formatting causes rss feeds to be less than useful. lots of stuff in posts is not appropriate and useful in a feed.

HTH, would like to help with these features by compensation or can submit code to you for review and inclusion.
See My Home Wind Turbine Videos!
Prepress Forums - Graphic Design & Prepress Forum for Printing Industry
Corvette Forums - Corvette Owners, Mechanics, & Enthusiasts Community
User avatar
Puggs
Registered User
Posts: 80
Joined: Mon Oct 06, 2003 3:53 am
Location: Melbourne, Australia
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by Puggs »

joe_postscript wrote:I went to the test interface, made a feed, hit the URL and there comes email addresses of the poster included in the feed. This really needs an ability to kill that info because you are feeding spammers the addresses of the members through rss.

(3) Kill/Limit Posters info - having the email of the poster go into a newsfeed is spammer food. This has to be parsed out and in general the name/ID of poster is of no use making newsfeeds full of non-topical noise.

I would say this is number 1 priority, i may even fix it now my self...


Puggs
User avatar
MarkDHamill
Registered User
Posts: 4364
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by MarkDHamill »

joe_postscript,

There is a constant called SMARTFEED_PRIVACY_MODE that is set by default to true. However, this line regulates whether the email address shows:

Code: Select all

				$item->authorEmail = (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false) || $registered_user) ? $row['user_email'] : '';
As you can see if a user in his profile elects to allow other people to view his email address AND either the SMARTFEED_PRIVACY_MODE is true OR the user requesting the feed is registered (and authenticated), the email goes in the feed. Real email addresses will NOT appear in the feed if the user is a guest unless you specifically set SMARTFEED_PRIVACY_MODE to false. My thinking is that if a user specifically allows other users to see his email address on the forum, then the privilege should extend to the feed also. However, with enough convincing I could add another constant like SMARTFEED_NEVER_SHOW_REAL_EMAIL_ADDRESS_IN_FEED.

So you don't want the author's name to appear in the feed at all? I think most people would not want this feature, although of course logic could be added. Or did you want to not show signature lines? User signatures follow the same logic as email addresses: if you can see them on the forum you see them in the feed UNLESS SMARTFEED_PRIVACY_MODE is true and you are a guest.

In general I try to emulate phpBB's security features for consistency.

Minimum post words is definitely something to consider.

Most newsreaders handle HTML perfectly fine. Some ignore certain features, like colors and styles. But I could add a constant SMARTFEED_FANCY_HTML similar to my digest modification or allow the user to determine if they want HTML applied or not.

On the Banword List, text does go through the phpBB 3 censor_text function automatically. However, that removes the offending words, but does not kill the post. It might be possible for me to write a function to determine if any censored words were removed by the censor_text function and if so not show that item. The problem is the Admin would have to enter the censored words in the ACP manually, and that gets rather old after a while.

It would be better to call a function which would make a judgment "this is spam". This could perhaps be done with an Askimet integration, but that's not easy to do and sending each post through Akismet in real time adds latency and will probably make the feed rather slow to generate. User registration is pretty tight with phpBB 3. The visual confirmation is much better. Unless you allow wide open guest posting, I am thinking the registration system should make any spam problem largely moot.

Logos etc. in the feed could be stripped out depending on the implementation of SMARTFEED_FANCY_HTML.

As for including your code or getting compensation to add features, I prefer to implement and control the code myself so it is consistent. We can talk offline about adding a feature for your need for money, however understand if I do it, the feature will be included for general release.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
User avatar
Puggs
Registered User
Posts: 80
Joined: Mon Oct 06, 2003 3:53 am
Location: Melbourne, Australia
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by Puggs »

Puggs wrote:
joe_postscript wrote:I went to the test interface, made a feed, hit the URL and there comes email addresses of the poster included in the feed. This really needs an ability to kill that info because you are feeding spammers the addresses of the members through rss.

(3) Kill/Limit Posters info - having the email of the poster go into a newsfeed is spammer food. This has to be parsed out and in general the name/ID of poster is of no use making newsfeeds full of non-topical noise.

I would say this is number 1 priority, i may even fix it now my self...


Puggs
It seems to show emails if the user is registered, so i made the following changes in smartfeed.php

Find

Code: Select all

$item->authorEmail = (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false) || $registered_user) ? $row['user_email'] : '';
Replace with

Code: Select all

$item->authorEmail = ($registered_user) ? $row['username'] : '';

Puggs
User avatar
Puggs
Registered User
Posts: 80
Joined: Mon Oct 06, 2003 3:53 am
Location: Melbourne, Australia
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by Puggs »

MarkDHamill wrote:joe_postscript,

There is a constant called SMARTFEED_PRIVACY_MODE that is set by default to true. However, this line regulates whether the email address shows:

Code: Select all

				$item->authorEmail = (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false) || $registered_user) ? $row['user_email'] : '';
As you can see if a user in his profile elects to allow other people to view his email address AND either the SMARTFEED_PRIVACY_MODE is true OR the user requesting the feed is registered (and authenticated), the email goes in the feed. Real email addresses will NOT appear in the feed if the user is a guest unless you specifically set SMARTFEED_PRIVACY_MODE to false. My thinking is that if a user specifically allows other users to see his email address on the forum, then the privilege should extend to the feed also. However, with enough convincing I could add another constant like SMARTFEED_NEVER_SHOW_REAL_EMAIL_ADDRESS_IN_FEED.
The problem seems to be that when you upgrade the user setting for allow email seems to be set to true.

Puggs
User avatar
Puggs
Registered User
Posts: 80
Joined: Mon Oct 06, 2003 3:53 am
Location: Melbourne, Australia
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by Puggs »

MarkDHamill wrote:joe_postscript,

There is a constant called SMARTFEED_PRIVACY_MODE that is set by default to true. However, this line regulates whether the email address shows:

Code: Select all

				$item->authorEmail = (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false) || $registered_user) ? $row['user_email'] : '';
Actualy that code says, if the rss feed getter is a registered user, then show email address regardless || is or maybe it should be

Code: Select all

$item->authorEmail = (!$registered_user) ? $row['username'] : (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false)) ? $row['user_email'] : $row['username'];
This new code is if the view is not a registered user then just show the board username, else if the viewer is registered and the poster allows emails and (&&) the policy says show then show the email, else just show the board user name
joe_postscript
Registered User
Posts: 109
Joined: Sun May 25, 2003 3:32 am
Location: Columbus Ohio USA
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by joe_postscript »

[quote="Most newsreaders handle HTML perfectly fine. Some ignore certain features, like colors and styles.[/quote]

It's not that the readers won't handle it, it's more that if one uses the RSS to attrract non members to the forum, to create/attract new members, the amount of clownSTYLE formatting that many people put into their posts just makes the feed hard to read and ugly in formatting. If you saw how nice my RSS feeds look from my forum, it's more convincing than talk. Look here, scroll to the bottom of my forum and see how nice these RSS feeds look, done well like this, they will bring you much desired network traffic from blogs and aggregators. The feeds at bottom are my RSS example from 2.0X http://software-robotics.com/graphic-design-prepress/

All posts in RSS are of a decent length and read well, format well, search well. Like returns, if you don't kill all the returns the feeds look crappy too. Many forum members use too many returns and really make ugly posts that flows right into an ugly RSS feed if allowed to.

[quote="allow the user to determine if they want HTML applied or not.[/quote]

Most of my use of RSS from the forum is not for users/members at all. matter of fact, I don't want actual members even using my RSS. That would allow them to read and follow without any benefit of ad clicks etc. Other webmasters may like that, I don't at all. I use RSS to syndicate inbound links and traffic from blogs and such. So, having a members settings affect use and format of a feed is not really helpful to my use of RSS.

[quote="The problem is the Admin would have to enter the censored words in the ACP manually, and that gets rather old after a while. [/quote]

Yes, good point. But, I have this feature in my phpBB 2.0x feeds and haven't changed the censored words in months. Really, spam is simply porn, viagra, prescription, and cel phones. If you have the keywords that catch that spam, that's 99% killed with a list of about 25 or 30 words.

[quote="I am thinking the registration system should make any spam problem largely moot.[/quote]

This too a good point. I have really tight control of spam on my boards now in phpBB 2.0X and don't evepect 3.0 to improve anything for me there because of my MODs on the 2.0X that counter it really well. But, once in awhile spammers pounce on a Sunday night or something. The RSS is all over the internet before you can kill their account and block their IP etc. It does happen and when it does, it can be very damaging to have hundreds of inbound links with nasty spam words pointing to your site, very damaging.

[quote="As for including your code or getting compensation to add features, I prefer to implement and control the code myself so it is consistent. We can talk offline about adding a feature for your need for money, however understand if I do it, the feature will be included for general release.[/quote]

Yes, that's great. As long as it's reasonable amount I am happy to contribute. My RSS from my boards has been really successful and the features I'm requesting will help me continue same success with 3.0x. It would be great to see others benefit from the same great features that I am using with 2.0X. I really like the form style control panel you have and the interface part of this MOD is all very nice work.
See My Home Wind Turbine Videos!
Prepress Forums - Graphic Design & Prepress Forum for Printing Industry
Corvette Forums - Corvette Owners, Mechanics, & Enthusiasts Community
User avatar
MarkDHamill
Registered User
Posts: 4364
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by MarkDHamill »

Puggs wrote:
MarkDHamill wrote:joe_postscript,

There is a constant called SMARTFEED_PRIVACY_MODE that is set by default to true. However, this line regulates whether the email address shows:

Code: Select all

				$item->authorEmail = (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false) || $registered_user) ? $row['user_email'] : '';
Actualy that code says, if the rss feed getter is a registered user, then show email address regardless || is or maybe it should be

Code: Select all

$item->authorEmail = (!$registered_user) ? $row['username'] : (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false)) ? $row['user_email'] : $row['username'];
This new code is if the view is not a registered user then just show the board username, else if the viewer is registered and the poster allows emails and (&&) the policy says show then show the email, else just show the board user name
Excellent! Good catch! This is definitely a bug!!

Amendment. Actually this should be an email address. If left blank the feedcreator class will substitute no_email@example.com. In some cases an email address is needed to validate a feed. So the line of code should read:

Code: Select all

				$item->authorEmail = (!$registered_user) ? '' : (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false)) ? $row['user_email'] : '';
Last edited by MarkDHamill on Sun Jan 13, 2008 3:16 am, edited 1 time in total.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
User avatar
MarkDHamill
Registered User
Posts: 4364
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by MarkDHamill »

joe_postscript,

The post_text column contains bbcode delimiters like [quote:efef8]. Presumably you would like these taken out but if taken out then the context may get lost. In this case it is hard to tell where the quoted information starts or begins. It looks like your examples are not in multiple paragraphs. So you would like all post text to be consolidated into one paragraph?

Smartfeed was designed to solve the general question of "how can I see the forum as an RSS feed and get into my protected forums?" It was not designed as a general purpose feed solution. There were others before me that provided that. Now it may be possible to make Smartfeed work like you want it, but it kind of goes against the general design principles.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. eBook and paper versions available.
User avatar
Puggs
Registered User
Posts: 80
Joined: Mon Oct 06, 2003 3:53 am
Location: Melbourne, Australia
Contact:

Re: [DEV] Smartfeed for phpBB 3

Post by Puggs »

MarkDHamill wrote:Excellent! Good catch! This is definitely a bug!!

Amendment. Actually this should be an email address. If left blank the feedcreator class will substitute no_email@example.com. In some cases an email address is needed to validate a feed. So the line of code should read:

Code: Select all

				$item->authorEmail = (!$registered_user) ? '' : (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false)) ? $row['user_email'] : '';
Can we modify this a bit more to

Code: Select all

				$item->authorEmail = (!$registered_user) ? '' : (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false)) ? $row['user_email'] : (defined(SMARTFEED_EMAIL_SITE))? ($row['username'] . '@' . SMARTFEED_EMAIL_SITE) : '';
This will give the no_email@example.com to non-registered readers, but username@mysite.com for registered users, if i recieve any email on these address i can pass onto the user or put them in the bit bucket under my desk...

Puggs

edit: added a check to see if SMARTFEED_EMAIL_SITE was defined or not
Mickey14
Registered User
Posts: 88
Joined: Sat Mar 05, 2005 10:45 am

Re: [DEV] Smartfeed for phpBB 3

Post by Mickey14 »

MarkDHamill wrote:
Puggs wrote:
MarkDHamill wrote:joe_postscript,

There is a constant called SMARTFEED_PRIVACY_MODE that is set by default to true. However, this line regulates whether the email address shows:

Code: Select all

				$item->authorEmail = (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false) || $registered_user) ? $row['user_email'] : '';
Actualy that code says, if the rss feed getter is a registered user, then show email address regardless || is or maybe it should be

Code: Select all

$item->authorEmail = (!$registered_user) ? $row['username'] : (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false)) ? $row['user_email'] : $row['username'];
This new code is if the view is not a registered user then just show the board username, else if the viewer is registered and the poster allows emails and (&&) the policy says show then show the email, else just show the board user name
Excellent! Good catch! This is definitely a bug!!

Amendment. Actually this should be an email address. If left blank the feedcreator class will substitute no_email@example.com. In some cases an email address is needed to validate a feed. So the line of code should read:

Code: Select all

				$item->authorEmail = (!$registered_user) ? '' : (($row['user_allow_viewemail'] == '1') && (SMARTFEED_PRIVACY_MODE == false)) ? $row['user_email'] : '';
Hi,

is this bug also affecting the phpBB2 version of Smartfeed ?
Locked

Return to “[3.0.x] MODs in Development”