[CDB] Feed post bot

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Anti-Spam Guide
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
Locked
User avatar
BadBlock
Registered User
Posts: 80
Joined: Sat Oct 06, 2001 8:33 pm

Re: [3.2][RC] Feed post bot

Post by BadBlock » Tue Feb 13, 2018 1:02 am

Installed the mod on a test board, got a couple of notes:

1. YouTube embedded videos currently seem to just get stripped out.
May make it easy as follows:

Replace the embed code with a link to the video, i.e. replace:

Code: Select all

<iframe src="https://www.youtube.com/embed/wbSwFU6tY1c?ecver=2" width="640" height="360" frameborder="0" allow="autoplay; encrypted-media" style="position:absolute;width:100%;height:100%;left:0" allowfullscreen></iframe>
with:

Code: Select all

https://www.youtube.com/watch?v=wbSwFU6tY1c
and let phpbb media embed do the rest. Or at least there will be a link to the video.

2. Topic prefix doesn't seem to be processed in Unicode, i.e., enter topic prefix in Cyrillic and save, the prefix displays as a series of question marks ("????? ").

User avatar
Ger
Recognised Extension Developer
Posts: 1401
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Tue Feb 13, 2018 7:47 am

BadBlock wrote:
Tue Feb 13, 2018 12:53 am
Nice mod!
Are there any plans to add a feature to post feed(s) to a specific topic, rather than forum?
Just a simple topic_id text field against each feed and a dropdown selector to choose between "post to forum" and "reply to topic".
That would be really great.
I've been flooded with all kind of requests :)
I intend to get the current setup validated (in queue now) and than think of how to improve stuff without baffling people with thousands of options. Another user mentioned this one before so I think this will make it to the next version.

BadBlock wrote:
Tue Feb 13, 2018 1:02 am
Installed the mod on a test board, got a couple of notes:

1. YouTube embedded videos currently seem to just get stripped out.
May make it easy as follows:

Replace the embed code with a link to the video, i.e. replace:

Code: Select all

<iframe src="https://www.youtube.com/embed/wbSwFU6tY1c?ecver=2" width="640" height="360" frameborder="0" allow="autoplay; encrypted-media" style="position:absolute;width:100%;height:100%;left:0" allowfullscreen></iframe>
with:

Code: Select all

https://www.youtube.com/watch?v=wbSwFU6tY1c
and let phpbb media embed do the rest. Or at least there will be a link to the video.
Yeah, all iframes are stripped. Even more: everything a default phpBB board doesn't parse will be stripped. Leaving a source in place might be a good option though.
2. Topic prefix doesn't seem to be processed in Unicode, i.e., enter topic prefix in Cyrillic and save, the prefix displays as a series of question marks ("????? ").
I think you're just the first to try. :P I'll look into it.
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote

Like my work? Buy me a coffee to keep it coming. :ugeek:

traal
Registered User
Posts: 7
Joined: Thu Feb 15, 2018 5:00 pm

Re: [3.2][RC] Feed post bot

Post by traal » Thu Feb 15, 2018 5:05 pm

Hi, I try to add a feed and I get the following error:
Invalid feed URL. This may be the result of a duplicate in your feed list or simply an URL that does not meet the specifications
The feed validates in the W3C Feed Validator.

I had this problem with another feed from the same source on a different phpbb forum and finally got it to accept (somehow). But even that feed isn't accepted on the new forum.

User avatar
Ger
Recognised Extension Developer
Posts: 1401
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Fri Feb 16, 2018 7:38 am

If this is something you just want to inform me about or do you want me to help you with it? If you mean the latter, than by all means: share the URL that's bugging you. Otherwise I cannot help you ;)
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote

Like my work? Buy me a coffee to keep it coming. :ugeek:

traal
Registered User
Posts: 7
Joined: Thu Feb 15, 2018 5:00 pm

Re: [3.2][RC] Feed post bot

Post by traal » Fri Feb 16, 2018 4:49 pm

Sorry, here's a feed that validates in the W3C validator but doesn't work with the Feed Post Bot: http://www.redfin.com/stingray/do/rss-s ... h/23324687

Dunnocks
Registered User
Posts: 188
Joined: Tue Jul 05, 2011 9:16 pm
Location: Ireland

Re: [3.2][RC] Feed post bot

Post by Dunnocks » Sat Feb 17, 2018 12:21 am

thank you for this,working well on my forum

i presume if i want to disable automatic posting i set "Interval for automatic processing feeds" to 0 seconds?

Dunnocks
Registered User
Posts: 188
Joined: Tue Jul 05, 2011 9:16 pm
Location: Ireland

Re: [3.2][RC] Feed post bot

Post by Dunnocks » Sat Feb 17, 2018 12:28 am

actually no,that don't work :D :D

Dunnocks
Registered User
Posts: 188
Joined: Tue Jul 05, 2011 9:16 pm
Location: Ireland

Re: [3.2][RC] Feed post bot

Post by Dunnocks » Sat Feb 17, 2018 2:30 am

hi im creating my rss file from a mysql database and not sure how to get the published date of the entry

at the moment im using the row id instead of the published date

example : <pubDate>579</pubDate>

it seems to be working ok after a few tests,

im just wondering if its ok to be doing this or could it cause problems?

User avatar
Ger
Recognised Extension Developer
Posts: 1401
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Mon Feb 19, 2018 3:10 pm

traal wrote:
Fri Feb 16, 2018 4:49 pm
Sorry, here's a feed that validates in the W3C validator but doesn't work with the Feed Post Bot: http://www.redfin.com/stingray/do/rss-s ... h/23324687
I've looked at it, but it turns out that that when I use PHP (any version) to parse the content of that URL, it gets no data back. Therefore the feed can't be inspected properly and that's why it fails while adding since it will also fail upon fetching the data.

I think the server of redfin.com somehow blocks PHP fetching through file_get_contents(). I've now built in the cURL option for those who have it available on their host. That works for me, could you report back to me if it also works for you?
Dunnocks wrote:
Sat Feb 17, 2018 12:21 am
i presume if i want to disable automatic posting i set "Interval for automatic processing feeds" to 0 seconds?
I really didn't think of someone wanting this :)
Just pushed a small change to add this.
BadBlock wrote:
Tue Feb 13, 2018 1:02 am
2. Topic prefix doesn't seem to be processed in Unicode, i.e., enter topic prefix in Cyrillic and save, the prefix displays as a series of question marks ("????? ").
This should now be fixed as well.


Now, time to get some coffee :mrgreen:
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote

Like my work? Buy me a coffee to keep it coming. :ugeek:

User avatar
Ger
Recognised Extension Developer
Posts: 1401
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Mon Feb 19, 2018 3:14 pm

Dunnocks wrote:
Sat Feb 17, 2018 2:30 am
hi im creating my rss file from a mysql database and not sure how to get the published date of the entry

at the moment im using the row id instead of the published date

example : <pubDate>579</pubDate>

it seems to be working ok after a few tests,

im just wondering if its ok to be doing this or could it cause problems?
Pubdate is mostly used as a fallback to determine wether or not a feed item has been processed earlier (when a unique ID is missing). When your id's roughly follow the time (newer id = later time) it's OK.

Further it's used as post time when the option "Local date/time" is unchecked, this might lead to weird times. When you leave that box checked it *should* not be a problem, but no guarantees.
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote

Like my work? Buy me a coffee to keep it coming. :ugeek:

User avatar
Ger
Recognised Extension Developer
Posts: 1401
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Mon Feb 19, 2018 3:22 pm

BadBlock wrote:
Tue Feb 13, 2018 1:02 am
1. YouTube embedded videos currently seem to just get stripped out.
May make it easy as follows:

Replace the embed code with a link to the video, i.e. replace:

Code: Select all

<iframe src="https://www.youtube.com/embed/wbSwFU6tY1c?ecver=2" width="640" height="360" frameborder="0" allow="autoplay; encrypted-media" style="position:absolute;width:100%;height:100%;left:0" allowfullscreen></iframe>
with:

Code: Select all

https://www.youtube.com/watch?v=wbSwFU6tY1c
and let phpbb media embed do the rest. Or at least there will be a link to the video.
I've pushed a small fix to the html2bbcode function that should cover this.
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote

Like my work? Buy me a coffee to keep it coming. :ugeek:

traal
Registered User
Posts: 7
Joined: Thu Feb 15, 2018 5:00 pm

Re: [3.2][RC] Feed post bot

Post by traal » Tue Feb 20, 2018 2:03 am

Ger wrote:
Mon Feb 19, 2018 3:10 pm
I think the server of redfin.com somehow blocks PHP fetching through file_get_contents(). I've now built in the cURL option for those who have it available on their host. That works for me, could you report back to me if it also works for you?
It was able to add the feed this time, thanks! But when I try to "Fetch all feeds manually" it says "No (new) items to fetch" and the selected feed forum has no new topics.

Here's what my server says about cURL:

Code: Select all

cURL support	enabled
cURL Information	7.38.0

User avatar
Ger
Recognised Extension Developer
Posts: 1401
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Tue Feb 20, 2018 9:17 am

traal wrote:
Tue Feb 20, 2018 2:03 am
It was able to add the feed this time, thanks! But when I try to "Fetch all feeds manually" it says "No (new) items to fetch" and the selected feed forum has no new topics.

Here's what my server says about cURL:

Code: Select all

cURL support	enabled
cURL Information	7.38.0
Well, indeed. I see it blocked me again. Your cURL is fine, redfin is just trying to lock us out.

I just pushed a change so that when the first try fails, the feed post bot tries again with his Google T-shirt on (e.g.: overriding the user agent). This works for me, but it's a long shot I must say. It might as well work now but die within a couple of months. They simply don't seem to like automated fetching like this.
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote

Like my work? Buy me a coffee to keep it coming. :ugeek:

traal
Registered User
Posts: 7
Joined: Thu Feb 15, 2018 5:00 pm

Re: [3.2][RC] Feed post bot

Post by traal » Tue Feb 20, 2018 11:15 pm

Ger wrote:
Tue Feb 20, 2018 9:17 am
I just pushed a change so that when the first try fails, the feed post bot tries again with his Google T-shirt on (e.g.: overriding the user agent). This works for me, but it's a long shot I must say. It might as well work now but die within a couple of months. They simply don't seem to like automated fetching like this.
Thanks, it works now.

Dunnocks
Registered User
Posts: 188
Joined: Tue Jul 05, 2011 9:16 pm
Location: Ireland

Re: [3.2][RC] Feed post bot

Post by Dunnocks » Wed Feb 21, 2018 1:55 am

Ger wrote:
Mon Feb 19, 2018 3:14 pm
Dunnocks wrote:
Sat Feb 17, 2018 2:30 am
hi im creating my rss file from a mysql database and not sure how to get the published date of the entry

at the moment im using the row id instead of the published date

example : <pubDate>579</pubDate>

it seems to be working ok after a few tests,

im just wondering if its ok to be doing this or could it cause problems?
Pubdate is mostly used as a fallback to determine wether or not a feed item has been processed earlier (when a unique ID is missing). When your id's roughly follow the time (newer id = later time) it's OK.

Further it's used as post time when the option "Local date/time" is unchecked, this might lead to weird times. When you leave that box checked it *should* not be a problem, but no guarantees.

cool,thanks man,think ill be ok using id's

so if i want to upgrade to the latest version of this mod do i need to uninstall it fully? and then re install? or just upload the new files?

Locked

Return to “Extensions in Development”

Who is online

Users browsing this forum: jtb33 and 19 guests