Feed post bot

Only processes the first feed; then stops - Feed post bot

Only processes the first feed; then stops

by jtb33 » Fri Jun 22, 2018 1:03 am

This is a GREAT mod! Good job - I love it.

The one issue I am having is that I've got about 5 RSS feeds in my config. I'm running the latest version from your GitHub site (just downloaded a few minutes ago) on phpBB v3.2.2.

I verified one-by-one that all my RSS feeds work. ie, I had all disabled but one and then ran it manually. Did that with each and it worked. However, if I have them all enabled, it only works with the very first RSS feed and then stops without processing any others in the list.

What am I missing?
jtb33
Registered User
Posts: 49
Joined: Thu Dec 02, 2004 11:36 pm

Re: Only processes the first feed; then stops

by Ger » Fri Jun 22, 2018 6:52 am

Not sure. Can you post a screenshot from your ACP module and provide (copy-pastable) URLS for the feeds you have?
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-
User avatar
Ger
Recognised Extension Developer
Posts: 1659
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:


Re: Only processes the first feed; then stops

by Ger » Fri Jun 22, 2018 2:20 pm

Your settings page states that fetching was started but not finished for some reason. Might be a server fault of somekind. You might want to release the lock with the second button from the top. If this persists you may need to check the error logs of both phpBB as well as your servers' PHP error log.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-
User avatar
Ger
Recognised Extension Developer
Posts: 1659
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: Only processes the first feed; then stops

by jtb33 » Fri Jun 22, 2018 5:00 pm

Ger wrote:Your settings page states that fetching was started but not finished for some reason. Might be a server fault of somekind. You might want to release the lock with the second button from the top. If this persists you may need to check the error logs of both phpBB as well as your servers' PHP error log.
I really appreciate the help. It's great having a content creator providing assistance.

I did manually release the lock using the button many times during the testing I did. If it helps track this down, I have two of the feeds that work without issue. I can put them both in and process them and it works - and the fetching process completes and releases like it should. Feeds show up properly.

Then there are a several that get the feeds, but do not ever finish the processing and I have to stop it - and then it will not move down the list to the next one.

Here are the two that work perfectly:

Here are two that work, but don't complete:
It also doesn't seem to be related to the amount of content, since one of the two that works brings back ~15 items, and one that doesn't work properly only brings back one feed item. Further, when it fails, it doesn't seem to be waiting any longer than when it succeeds. Should I be looking at anything specific in my php.ini file? I even set the max_execution_time to 500 from the default of 30, but that didn't help.
jtb33
Registered User
Posts: 49
Joined: Thu Dec 02, 2004 11:36 pm

Re: Only processes the first feed; then stops

by Ger » Fri Jun 22, 2018 5:51 pm

I have to investigate those two further but for now I recommend that you disable them and try if things work out for the others. Check if that goes well for a couple of days and then try the two you mentioned one by one. Maybe it's only one of them that's bugging you.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-
User avatar
Ger
Recognised Extension Developer
Posts: 1659
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: Only processes the first feed; then stops

by jtb33 » Fri Jun 22, 2018 6:03 pm

Yep - that's exactly what I have done. I tried them all individually (I would disable all but one and then test). Verified functionality of one at a time that way, then disabled that one, even if it worked and moved on to the next one so I'd know if they worked even by themselves. Luckily, there was no inconsistency. They either worked fully (like the first two above - from Foxnews and youtube), or didn't work, meaning that they processed, but never 'finished'. That would also prevent it from moving on down the list to process the next one if I tested with multiples.

The Foxnews and youtube ones above work just fine - both individually, and together. Once I add one of the other two that don't complete, it runs into the problem. Same problem if it's just one of those by itself.
jtb33
Registered User
Posts: 49
Joined: Thu Dec 02, 2004 11:36 pm

Re: Only processes the first feed; then stops

by Ger » Sun Jun 24, 2018 7:46 am

I need to dive into those two feeds deeper. I'm on my phone right now so I need to remember to do this later on. Please bump this topic of I haven't replied within a couple of days, that would mean I forgot ;)
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-
User avatar
Ger
Recognised Extension Developer
Posts: 1659
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: Only processes the first feed; then stops

by Ger » Mon Jun 25, 2018 10:00 am

I did some testing. I've added all feeds you have to my test board in the same order, cleared all other feeds and assigned each feed to a dedicated forum. Set the timeout to 30 seconds just like you did.

They've all (but 1) fetched fine actually. The process took some time but it finished without getting the lock. There was 1 error though; https://www.range365.com/rss.xml?loc=footer&lnk=rss
It didn't fetch any items but there was an entry in the phpBB error log:
I took a peek and it seems logical to me: visiting the site results in a cookie wall. Once cookies are accepted everything is fine but that won't work for automated processes like Feed Post Bot. Now I don't see any benefit in setting cookies in a feed and there also aren't any analytics or advertisements loaded. So my advice would be for them to remove that cookie wall from their RSS feed or -if they are unwilling or unable- to disable that feed for the moment.

However, this doesn't really apply to the error you describe. The two that don't complete for you work perfectly for me. Guns and Ammo delivers 25 topics and Personal Defence world delivers 10. So unless Range 365 is blocking something on your server for other feeds, I don't see anything that would help you so far.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-
User avatar
Ger
Recognised Extension Developer
Posts: 1659
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: Only processes the first feed; then stops

by jtb33 » Mon Jun 25, 2018 2:19 pm

Thanks for taking the time to look into this. I will look deeper on my site - I can't find any limit being hit on the server side, but will investigate further. It helps knowing that they do work for you, so it is likely something in my configuration, php.ini or overall server settings.
jtb33
Registered User
Posts: 49
Joined: Thu Dec 02, 2004 11:36 pm

Re: Only processes the first feed; then stops

by Ger » Mon Jun 25, 2018 2:31 pm

Good luck :)
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:
-Available for custom work-
User avatar
Ger
Recognised Extension Developer
Posts: 1659
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: Only processes the first feed; then stops

by jtb33 » Tue Jun 26, 2018 12:36 am

dupe
jtb33
Registered User
Posts: 49
Joined: Thu Dec 02, 2004 11:36 pm