[CDB] Feed post bot

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Anti-Spam Guide
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
User avatar
Ger
Recognised Extension Developer
Posts: 1593
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Fri Mar 02, 2018 12:59 pm

Got your PM.

It's a strange feed I must say, holding a LOT of contents. Some go well, like the one titled Zwischen Krise und Aufbruch <snip>
Others hold this kind of stuff:

Code: Select all

<![CDATA[<figure id="attachment_877" style="max-width: 1200px" class="wp-caption alignnone">
<img class="wp-image-877 size-full" src="***" alt width="1200" height="500" srcset="***"><figcaption class="wp-caption-text">© flickr: spoilt.exile (CC BY-SA 2.0)</figcaption></figure>
<div id="vc_row-5a993d940f925" class="vc_row wpb_row vc_row-fluid"><div class="wpb_column vc_column_container vc_col-sm-12"><div class="vc_column-inner "><div class="wpb_wrapper">
	<div class="wpb_text_column wpb_content_element ">
		<div class="wpb_wrapper">
			
		</div>
	</div>
</div></div></div></div><div id="vc_row-5a993d940fbc2" class="vc_row wpb_row vc_row-fluid vc_custom_1508251598805"><div class="wpb_column vc_column_container vc_col-sm-8"><div class="vc_column-inner vc_custom_1508252250311"><div class="wpb_wrapper">
	<div class="wpb_text_column wpb_content_element ">
		<div class="wpb_wrapper">
		
<snip>

div class="vc_row wpb_row vc_inner vc_row-fluid vc_custom_1509937984313 vc_row-has-fill"><div class="wpb_column vc_column_container vc_col-sm-12"><div class="vc_column-inner "><div class="wpb_wrapper">
	<div class="wpb_text_column wpb_content_element  vc_custom_1508167210959">
		<div class="wpb_wrapper">
			<h3><span style="color: #ffffff;">Newsletter bestellen</span></h3>

		</div>
	</div>

	<div class="wpb_text_column wpb_content_element  vc_custom_1510657799241">
		<div class="wpb_wrapper">
			<p>Tragen Sie sich in unseren Newsletter ein und bleiben Sie auf dem Laufenden.</p>
<script type="text/javascript">(function() {
	if (!window.mc4wp) {
		window.mc4wp = {
			listeners: [],
			forms    : {
				on: function (event, callback) {
					window.mc4wp.listeners.push({
						event   : event,
						callback: callback
					});
				}
			}
		}
	}
})();
</script><!-- MailChimp for WordPress v4.1.15 - https://wordpress.org/plugins/mailchimp-for-wp/ --><form id="mc4wp-form-13" class="mc4wp-form mc4wp-form-434" method="post" data-id="434" data-name="*** Newsletter"><div class="mc4wp-form-fields"><p>
	<input type="email" name="EMAIL" placeholder="Ihre E-Mail-Adresse" required>
	<input type="submit" value="Absenden">
</p><label style="display: none !important;">Leave this field empty if you’re human: <input type="text" name="_mc4wp_honeypot" value tabindex="-1" autocomplete="off"></label><input type="hidden" name="_mc4wp_timestamp" value="1519992212"><input type="hidden" name="_mc4wp_form_id" value="434"><input type="hidden" name="_mc4wp_form_element_id" value="mc4wp-form-13"></div><div class="mc4wp-response"></div></form><!-- / MailChimp for WordPress Plugin -->

		</div>
	</div>
</div></div></div></div></div></div></div></div>
That's a LOT.OF.CRAP that shouldn't be in a feed. Never seen javascript in a feed before, or subscriptions to newsletters.

You can take a look at the W3 feed validation service and enter your feed URL: https://validator.w3.org/feed/
It doesn't validate.

Now what you *could* do, is remove these lines from the feed post bot: https://github.com/GerB/feedpostbot/blo ... #L366-L369
It than ignores everything in the <content:encoded> tag and simply return what's in the description.
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote
Newest: Quoted Where + anonymize

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

whocarez
Registered User
Posts: 137
Joined: Sat Apr 18, 2009 9:19 pm

Re: [3.2][RC] Feed post bot

Post by whocarez » Fri Mar 02, 2018 1:04 pm

yes, I see, but the bot should ignore the crap and only use the stuff inside the description.

User avatar
Ger
Recognised Extension Developer
Posts: 1593
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Fri Mar 02, 2018 1:08 pm

whocarez wrote:
Fri Mar 02, 2018 1:04 pm
yes, I see, but the bot should ignore the crap and only use the stuff inside the description.
Content:encoded is usually favoured above description. You can change the lines I pointed at above at your will, by removing them or move them below the description ones. I won't change that by default though since many other feeds benefit from a proper content:encoded tag.
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote
Newest: Quoted Where + anonymize

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

User avatar
KYPREO
Registered User
Posts: 68
Joined: Fri Feb 02, 2018 9:56 am
Contact:

Re: [3.2][RC] Feed post bot

Post by KYPREO » Mon Mar 05, 2018 11:24 pm

BadBlock wrote:
Tue Feb 13, 2018 12:53 am
Nice mod!
Are there any plans to add a feature to post feed(s) to a specific topic, rather than forum?
Just a simple topic_id text field against each feed and a dropdown selector to choose between "post to forum" and "reply to topic".
That would be really great.
Another vote for this feature in the next version!

I can think of lots of useful and powerful applications for this, particularly when combined with the Social Media Embed extension.

For example, you can create an RSS feed from an Instagram account or hashtag using numerous online tools. You could then have the feed post bot generate a new post for every new Instagram photo from that account or featuring that hashtag. This might go into a phpBB topic called "Daily photos from Instagram" or similar. If everything but the URL to the Instagram post is stripped from the RSS data by the feed post bot, then combined with the Social Media Embed extension, the instagram post will appear as rich content in the feed bot's phpBB post.

Voila - full Instagram feed integration into a phpBB topic.

You could do this for Twitter or Facebook or countless other sites as well.

I will experiment with it for myself in due course and report back, but someone let me know if there is an easier way to do this (as a forum topic, rather than for example using a sidebar and Instagram/Twitter/Facebook's APIs).
phpBB user since 2002
www.AusRotary.com

traal
Registered User
Posts: 9
Joined: Thu Feb 15, 2018 5:00 pm

Re: [3.2][RC] Feed post bot

Post by traal » Wed Mar 07, 2018 2:37 am

Feature request: if the title of a feed item changes (such as to fix a misspelling), change the title of the topic that was previously created by the same feed item. Or post a message to the topic indicating the title change.

User avatar
Ger
Recognised Extension Developer
Posts: 1593
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Wed Mar 07, 2018 8:19 am

traal wrote:
Wed Mar 07, 2018 2:37 am
Feature request: if the title of a feed item changes (such as to fix a misspelling), change the title of the topic that was previously created by the same feed item. Or post a message to the topic indicating the title change.
That seems very hard to me on first glance. Usually I simply ditch everything that's older than the latest processed item for performance reasons. Checking each item and looking up their topic counterparts would also mean I'd have to keep track of every posted topic for each item in a feed. This will have a major impact on the performance.
And then again: what would have to be done if a moderator of your board had changed the topic title for whatever good reason he or she may have? Once a difference is detected, should the Feed Post Bot overwrite that change again? Maybe not when the feed source title hasn't changed, but if both feed source and topic title have changed there is no simple rule to cover all that.

IMO, it would require a lot of effort to program this and have a massive negative impact on performance (slowing down your board in the process) with very little benefit.
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote
Newest: Quoted Where + anonymize

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

traal
Registered User
Posts: 9
Joined: Thu Feb 15, 2018 5:00 pm

Re: [3.2][RC] Feed post bot

Post by traal » Thu Mar 08, 2018 9:11 am

Ger wrote:
Wed Mar 07, 2018 8:19 am
traal wrote:
Wed Mar 07, 2018 2:37 am
Feature request: if the title of a feed item changes (such as to fix a misspelling), change the title of the topic that was previously created by the same feed item. Or post a message to the topic indicating the title change.
Checking each item and looking up their topic counterparts would also mean I'd have to keep track of every posted topic for each item in a feed. This will have a major impact on the performance.
I think all it needs is a lookup table (one per feed) with three fields: {item_uri}, {item_title}, and phpBB topic ID#

The {item_uri} is required and unique for each item according to the RDF spec, and it persists when the {item_title} changes. Whenever the Feed Post Bot pulls the feed, it would check to see if the {item_title} has changed for a given {item_uri} and if so, it would change the board topic for the given topic ID#.

As items specified by their {item_uri} disappear from the feed, the Feed Post Bot should prune the matching records from the lookup table.
Ger wrote:
Wed Mar 07, 2018 8:19 am
And then again: what would have to be done if a moderator of your board had changed the topic title for whatever good reason he or she may have?
The simplest and most performant thing is for the Feed Post Bot to ignore any changes a moderator has made to the topic title. The moderator can fix the title again if necessary.
Ger wrote:
Wed Mar 07, 2018 8:19 am
IMO, it would require a lot of effort to program this and have a massive negative impact on performance (slowing down your board in the process) with very little benefit.
Maybe I'm the only one who has ever encountered the issue, so unless someone else speaks up you can ignore this entire post. :D

Anyway, thanks for your hard work making this extension!

User avatar
Ger
Recognised Extension Developer
Posts: 1593
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Thu Mar 08, 2018 9:20 am

traal wrote:
Thu Mar 08, 2018 9:11 am
The {item_uri} is required and unique for each item according to the RDF spec, and it persists when the {item_title} changes.
If I have learned one thing during development of this extension, it's that hardly any feed respects the entire specifications. That's one of the reasons I needed to build-in a lot of fallbacks to get the stuff working in the first place. ;)
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote
Newest: Quoted Where + anonymize

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

ottischwenk
Registered User
Posts: 13
Joined: Tue Mar 06, 2018 2:14 pm

Re: [3.2][RC] Feed post bot

Post by ottischwenk » Mon Mar 12, 2018 3:04 pm

I got an error message when trying https://www.presseportal.de/rss/medien-kultur.rss2
Information
Invalid feed URL. This may be the result of a duplicate in your feed list or simply an URL that does not meet the specifications
What happens?

User avatar
Ger
Recognised Extension Developer
Posts: 1593
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Mon Mar 12, 2018 3:37 pm

ottischwenk wrote:
Mon Mar 12, 2018 3:04 pm
I got an error message when trying https://www.presseportal.de/rss/medien-kultur.rss2
Information
Invalid feed URL. This may be the result of a duplicate in your feed list or simply an URL that does not meet the specifications
What happens?
Works for me. Added it to my feed list, fetched and got 15 new topics.
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote
Newest: Quoted Where + anonymize

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

ottischwenk
Registered User
Posts: 13
Joined: Tue Mar 06, 2018 2:14 pm

Re: [3.2][RC] Feed post bot

Post by ottischwenk » Mon Mar 12, 2018 3:56 pm

Yes, it works for others too, but not for me and i donot know, where could be my problem
My activated extensions:
Collapsible Forum Categories 1.0.2
External Links Open in New Window 1.0.6
Feed post bot 1.0.10
mChat 2.0.3
Show Guests in viewonline 0.1.0
Tapatalk Plugin for phpBB 3.1/3.2 2.0.6
Thanks for posts 3.2.1
Last edited by ottischwenk on Mon Mar 12, 2018 4:00 pm, edited 1 time in total.

User avatar
Ger
Recognised Extension Developer
Posts: 1593
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Mon Mar 12, 2018 3:59 pm

You might want to start with giving me some information...

Do you have the latest version of Feed Post Bot?
Is cURL enabled on your server?
Have you tried other feeds?
What PHP version do you have?
What phpBB version do you have?
Any other extensions?
Any other useful information?
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote
Newest: Quoted Where + anonymize

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

ottischwenk
Registered User
Posts: 13
Joined: Tue Mar 06, 2018 2:14 pm

Re: [3.2][RC] Feed post bot

Post by ottischwenk » Mon Mar 12, 2018 4:17 pm

Ger wrote:
Mon Mar 12, 2018 3:59 pm
You might want to start with giving me some information...

Do you have the latest version of Feed Post Bot?
Yes - 1.0.10
Is cURL enabled on your server?
Yes
Have you tried other feeds?
Yes, 12
What PHP version do you have?
PHP Version 7.0.15 - but other versions are possible
What phpBB version do you have?
3.2.2
Any other extensions?
My activated extensions:
Collapsible Forum Categories 1.0.2
External Links Open in New Window 1.0.6
Feed post bot 1.0.10
mChat 2.0.3
Show Guests in viewonline 0.1.0
Tapatalk Plugin for phpBB 3.1/3.2 2.0.6
Thanks for posts 3.2.1
Any other useful information?
Im a beginner
BTW: it doesnt work on a complete fresh installation of phpBB 3.2.2 - the same error message

User avatar
Ger
Recognised Extension Developer
Posts: 1593
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.2][RC] Feed post bot

Post by Ger » Tue Mar 13, 2018 8:52 am

Can you post a screenshot from your Feed post bot settings page?
My extensions: Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update and Modern Quote
Newest: Quoted Where + anonymize

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-

ottischwenk
Registered User
Posts: 13
Joined: Tue Mar 06, 2018 2:14 pm

Re: [3.2][RC] Feed post bot

Post by ottischwenk » Tue Mar 13, 2018 9:07 am

Ger wrote:
Tue Mar 13, 2018 8:52 am
Can you post a screenshot from your Feed post bot settings page?
Which do you mean? - from extension (Erweiterungen) tab? Or from common (Allgemein) tab?
Names are in German
Image

All these feeds are working correct

Locked

Return to “Extensions in Development”

Who is online

Users browsing this forum: andares, obagley, pierredu and 17 guests