Page 11 of 26

Re: [3.1][3.2][RC] Feed post bot

Posted: Wed Aug 30, 2017 11:38 pm
by scriptman
Is there a way to set the limit of feeds fetched each time to maybe 1 or 2? i.e The Depth.
It seem like 5 is the default.

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 5:28 am
by sysz
Got this error on a freshly installed phpbb 3.2.1
[phpBB Debug] PHP Warning: in file [ROOT]/ext/ger/feedpostbot/classes/driver.php on line 73: Invalid argument supplied for foreach()
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions_acp.php on line 137: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions_acp.php on line 137: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions_acp.php on line 137: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 7:31 am
by Ger
sysz wrote:
Thu Aug 31, 2017 5:28 am
Got this error on a freshly installed phpbb 3.2.1
[phpBB Debug] PHP Warning: in file [ROOT]/ext/ger/feedpostbot/classes/driver.php on line 73: Invalid argument supplied for foreach()
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions_acp.php on line 137: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions_acp.php on line 137: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)
[phpBB Debug] PHP Warning: in file [ROOT]/includes/functions_acp.php on line 137: Cannot modify header information - headers already sent by (output started at [ROOT]/includes/functions.php:3262)
Thanks for reporting. I have fixed this in the latest download version (check Github).
scriptman wrote:
Wed Aug 30, 2017 11:38 pm
Is there a way to set the limit of feeds fetched each time to maybe 1 or 2? i.e The Depth.
It seem like 5 is the default.
The Feed post bot just fetches everything that's offered in the feeds you added to the list and is newer than the latest entry previously fetched. Most feeds don't offer their entire history though, but just the latest n items.

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 10:23 am
by scriptman
Ger wrote:
Thu Aug 31, 2017 7:31 am
The Feed post bot just fetches everything that's offered in the feeds you added to the list and is newer than the latest entry previously fetched. Most feeds don't offer their entire history though, but just the latest n items.
Could you please help add this feature?

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 11:27 am
by Ger
scriptman wrote:
Thu Aug 31, 2017 10:23 am
Ger wrote:
Thu Aug 31, 2017 7:31 am
The Feed post bot just fetches everything that's offered in the feeds you added to the list and is newer than the latest entry previously fetched. Most feeds don't offer their entire history though, but just the latest n items.
Could you please help add this feature?
I think I don't really understand. When it comes to fetch more items, it's simply impossible if the feed doesn't offer more; e.g. what doesn't exist cannot be fetched. When it comes to fetching less items, it seems pretty illogical; usually when you track a feed, you'd want to fetch all items. Just fetching a couple of them seems a bit random to me.

If this isn't what you mean, than by all means elaborate. :)

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 12:42 pm
by scriptman
Ger wrote:
Thu Aug 31, 2017 11:27 am
I think I don't really understand. When it comes to fetch more items, it's simply impossible if the feed doesn't offer more; e.g. what doesn't exist cannot be fetched. When it comes to fetching less items, it seems pretty illogical; usually when you track a feed, you'd want to fetch all items. Just fetching a couple of them seems a bit random to me.

If this isn't what you mean, than by all means elaborate. :)
Am not talking about fetching more. Am talking about fetching less.

E.G
Feed item limit:
Set the maximum number of feed items that should be processed. The smaller the number, the faster the new feed is produced.
Minimum is "1" through "10" and maximum could be "Fetch All Items".

AND YES, some item will be left out if limit is applied as it will only fetch from latest items.

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 2:15 pm
by Ger
Hmmm... My first reaction would be that such a limit is against the whole idea of feeds, which is to keep track of every item. Most feeds offer only a couple of new items each hour, many just a couple of items each week.

If you are tracking a feed that puts it's entire history in, than it would be very slow on first fetch (possibly leading to a time-out) but that's not really what feeds are intended to do. But even then; after the first fetch the extension only posts new items. And the slowness will stay since the extension needs to check every item in given feed for identifiers or the timestamp to decide whether or not to post it to your board.

If you have too much items per batch, you can decide to lower the cron interval so the extension will process the feeds more frequently. Each time it runs, you'll have less new items probaly. It's the option Interval for processing feeds (seconds) (default 1800 seconds).

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 4:21 pm
by scriptman
Ger wrote:
Thu Aug 31, 2017 2:15 pm
Hmmm... My first reaction would be that such a limit is against the whole idea of feeds, which is to keep track of every item. Most feeds offer only a couple of new items each hour, many just a couple of items each week.

If you are tracking a feed that puts it's entire history in, than it would be very slow on first fetch (possibly leading to a time-out) but that's not really what feeds are intended to do. But even then; after the first fetch the extension only posts new items. And the slowness will stay since the extension needs to check every item in given feed for identifiers or the timestamp to decide whether or not to post it to your board.

If you have too much items per batch, you can decide to lower the cron interval so the extension will process the feeds more frequently. Each time it runs, you'll have less new items probaly. It's the option Interval for processing feeds (seconds) (default 1800 seconds).
Why not look at it from another perspective. In a scenario where you are fetching feeds from about 10 source. How do you cope if those 10 sources release about 2 post each time?

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 5:50 pm
by Ger
Why would that be a problem?

Re: [3.1][3.2][RC] Feed post bot

Posted: Thu Aug 31, 2017 7:22 pm
by scriptman
Ger wrote:
Thu Aug 31, 2017 5:50 pm
Why would that be a problem?
It's a forum not a blog. Other users post should also be relevant.
That's where the problem comes in.

Although i have a solution i use by limiting the rss feed posts before using the output link on Feed Post Bot. But it would be nice to do it directly on ACP.

Re: [3.1][3.2][RC] Feed post bot

Posted: Sat Sep 02, 2017 4:31 pm
by scriptman
Wanted to ask, is there a way to include some pre-text for the source url?
E.G Source: http://example.com

Re: [3.1][3.2][RC] Feed post bot

Posted: Sat Sep 02, 2017 7:13 pm
by Thunder_one
Hi

Sorry for my english. I do not understand so well. :(

Question

Can this be adjusted at what intervals he should download the feeds and post into the forum?

Many Thanks

Re: [3.1][3.2][RC] Feed post bot

Posted: Sun Sep 03, 2017 10:59 am
by scriptman
scriptman wrote:
Thu Aug 31, 2017 7:22 pm
Ger wrote:
Thu Aug 31, 2017 5:50 pm
Why would that be a problem?
It's a forum not a blog. Other users post should also be relevant.
That's where the problem comes in.

Although i have a solution i use by limiting the rss feed posts before using the output link on Feed Post Bot. But it would be nice to do it directly on ACP.
Despite the feed having over 10 feeds, the bot pulls just 5 like i said earlier.
I still suggest you add that feature.

Re: [3.1][3.2][RC] Feed post bot

Posted: Mon Sep 04, 2017 6:35 am
by wolfofone
Thank you for this! You are awesome!

I have been looking for a replacement to SMIX.MODS RSS importer for a long, long time now and I recently had given up and updated to phpBB 3.2 and just went without an rss posting bot... then I happened to come across this!

It was super simple to set up and get working. https://verrytechnical.com/forums/viewt ... f=16&t=786

The latest post was by your tool, all the older ones were done by the SMIX.MODs feed that has been abandoned for forever (I did write two articles on supporting getting it working with 3.0.x forums but didn't have the know-how to update the code to work with 3.1+). I will need to up the text limit count though as I chose 50 just to test, I think 150 will be better :-).

I will certainly point people towards your project and include a link to your donation link in an upcoming blog post :-).

Re: [3.1][3.2][RC] Feed post bot

Posted: Mon Sep 04, 2017 6:38 am
by wolfofone
Thunder_one wrote:
Sat Sep 02, 2017 7:13 pm
Hi

Sorry for my english. I do not understand so well. :(

Question

Can this be adjusted at what intervals he should download the feeds and post into the forum?

Many Thanks
You can set it to post manually when you go into ACP and manually tell it to go grab the feed and post or by adjusting the auto-posting interval in the "Interval for processing feeds (seconds):" box. 1800 seconds (default is every 30 minutes, you can change this so that it checks and post more or less often).