Page 19 of 26

Re: [3.2][RC] Feed post bot

Posted: Fri Dec 22, 2017 11:21 am
by david63
I have just started using this and I am having some mixed results. In principle I have it working apart from one feed. Having tried to do some debugging I think the problem is that the content is within <content:encoded> tags and this is being dropped when this extension is process the data.

Is this something beyond the scope of this extension or do I need to do somethong else?

Re: [3.2][RC] Feed post bot

Posted: Fri Dec 22, 2017 12:15 pm
by Ger
It could be that the content you're looking for in the feed isn't within designated tags, but it's hard to say without the source. Can you provide the feed URL and tell me what you'd expect to be processed vs. the result?

Re: [3.2][RC] Feed post bot

Posted: Fri Dec 22, 2017 2:13 pm
by Ger
Got your feed trough PM.
david63 wrote:
Fri Dec 22, 2017 11:21 am
I think the problem is that the content is within <content:encoded> tags and this is being dropped when this extension is process the data.
You are right.
As you may or may not have seen, I use the SimpleXML extension to parse the feed contents. For RSS, I iterate through the <item> list, check for existence of required or possible fallback properties and push it to the list.

In your case, you use namespacing. I haven't implemented that, so when I dump the contents of an item from your feed, I get these properties:

Code: Select all

object(SimpleXMLElement)[10]
  public 'title' => string '...' (length=20)
  public 'link' => string '...' (length=78)
  public 'category' => string '...' (length=4)
  public 'comments' => string '...' (length=87)
  public 'author' => string '...' (length=26)
  public 'pubDate' => string '...' (length=31)
  public 'guid' => string '...' (length=62)
So your content is missing, and the feed post bot falls back to the title.

In this case I would have to check the namespaces and if the content namespace is available, check if it has the encoded attribute and if it exists, use it for the post content.

I might implement it, but I have to think about that (and what's the optimal approach). It's essentially another fallback in an ever-growing list of fallbacks. When I started this extension I had no idea about the dozens of formats of feeds. Can't you just keep things simple on your end? :P

On a side note: I got some recommendations when validating your feed

Re: [3.2][RC] Feed post bot

Posted: Fri Dec 22, 2017 2:32 pm
by david63
Thanks Ger
Ger wrote:
Fri Dec 22, 2017 2:13 pm
On a side note: I got some recommendations when validating your feed
I had seen those but as this is not my code and to be honest I do not really understand much about feeds :? - although I now know more than I did 24 hours ago :)
Ger wrote:
Fri Dec 22, 2017 2:13 pm
Can't you just keep things simple on your end?
I think that I may have found a way around the problem by replacing the <content:encoded> tags in the template from my blog with <description> tags.

On another side note - when I was searching for a solution I noticed that <content:encoded> is used in WordPress so it might be worth considering adding this before all the WP users start complaining :shock: :shock: :o

Re: [3.2][RC] Feed post bot

Posted: Fri Dec 22, 2017 2:52 pm
by Ger
Yeah, most of them use both the description as the content:encoded part.

However, I think you're right, I have to try and tackle this one.

Re: [3.2][RC] Feed post bot

Posted: Fri Dec 22, 2017 3:48 pm
by Ger
david63 wrote:
Fri Dec 22, 2017 2:32 pm
poke
I think I've managed to handle the content:encoded style. It favours the description. Would you mind testing it for me?

I've also added a locking mechanism that sets an extra config value, so make sure you disable and enable the extension to run the migration.

Re: [3.2][RC] Feed post bot

Posted: Fri Dec 22, 2017 3:58 pm
by david63
Ger wrote:
Fri Dec 22, 2017 3:48 pm
I think I've managed to handle the content:encoded style. It favours the description. Would you mind testing it for me?
No problem - is it on GitHub? Might not be until tomorrow though.

Re: [3.2][RC] Feed post bot

Posted: Fri Dec 22, 2017 4:14 pm
by david63
Just downloaded 1.0.5 from Git and it works fine :D :D

Thanks.

Re: [3.2][RC] Feed post bot

Posted: Sat Dec 23, 2017 7:04 am
by Ger
Great, thanks for testing!

Re: [3.2][RC] Feed post bot

Posted: Sat Dec 23, 2017 8:07 am
by david63
I think I have found a couple of minor buglets.

I have three feeds set up and when I look at the Admin log after the feeds have been fetched the username of the first feed is that of the user who invoked the cron job (which is what I would expect), but the subsequent feed entries use the username of the user who is set in the ACP as the Feed user id.

The table of feeds and their options in the ACP is not responsive and can result in the table extending beyond the width of the page.

I notice that you have added a lock to this version. I tried this once and there is a possibility that it can cause problems if you have a situation where the job does not complete for some reason then you can end up with the system locked - I would suggest that you either have an "unlock" button or at the start of the cron job (or elsewhere) auto unlock before you start.

Now for some queries (or feature requests ;) )

1. Is it possible to shorten the title?
2. Is it possible when setting up a feed to only import them from a "start date"?
3. Would it be possible to post a feed to a topic instead of a forum?

Re: [3.2][RC] Feed post bot

Posted: Sat Dec 23, 2017 2:44 pm
by david63
Another little bug.

When you are setting the feedpostbot_locked variable in driver.php you need to make it dynamic each time.

Re: [3.2][RC] Feed post bot

Posted: Sun Dec 24, 2017 1:10 pm
by Ger
Thanks. I'll look into it after the holidays

Re: [3.2][RC] Feed post bot

Posted: Wed Dec 27, 2017 7:56 am
by Ger
david63 wrote:
Sat Dec 23, 2017 2:44 pm
Another little bug.

When you are setting the feedpostbot_locked variable in driver.php you need to make it dynamic each time.
When I look at https://github.com/phpbb/phpbb/blob/mas ... g.php#L124
it seems the third parameter isn't used at all?

Re: [3.2][RC] Feed post bot

Posted: Wed Dec 27, 2017 8:12 am
by david63
Ger wrote:
Wed Dec 27, 2017 7:56 am
david63 wrote:
Sat Dec 23, 2017 2:44 pm
Another little bug.

When you are setting the feedpostbot_locked variable in driver.php you need to make it dynamic each time.
When I look at https://github.com/phpbb/phpbb/blob/mas ... g.php#L124
it seems the third parameter isn't used at all?
But when you are changing it like this you do not want it to use the cached setting you need it to pick up the revised setting, which is why you need to define it as being dynamic

Re: [3.2][RC] Feed post bot

Posted: Wed Dec 27, 2017 9:23 am
by Ger
david63 wrote:
Sat Dec 23, 2017 8:07 am
I have three feeds set up and when I look at the Admin log after the feeds have been fetched the username of the first feed is that of the user who invoked the cron job (which is what I would expect), but the subsequent feed entries use the username of the user who is set in the ACP as the Feed user id.
User is only switched when there are items to post, so the logging is a bit off due to that. However: in this case I don't think it really matters, does it?
The table of feeds and their options in the ACP is not responsive and can result in the table extending beyond the width of the page.
Yeah, that's a thingy indeed. The table was very simple in the beginning of this extension, but got wider along the months. ;)
From a simple list of urls, forumlist and a delete button it got to this. Not ideal, but I'm keeping it this way for the current version which I'm trying to get past validation. Redesigning the ACP part will probably lead to larger issues.
I notice that you have added a lock to this version. I tried this once and there is a possibility that it can cause problems if you have a situation where the job does not complete for some reason then you can end up with the system locked - I would suggest that you either have an "unlock" button or at the start of the cron job (or elsewhere) auto unlock before you start.
Yeah the lock is there to prevent a double process when running from cron and while that process runs, starting it manually from ACP again. It can lead to double postings. I can imagine it can lead to a permalock when the process interrupts. I think I'll add a button for unlocking when needed, thanks for mentioning.
Now for some queries (or feature requests ;) )

1. Is it possible to shorten the title?
2. Is it possible when setting up a feed to only import them from a "start date"?
3. Would it be possible to post a feed to a topic instead of a forum?
All good ideas, but I'll pass them to a second version. If you want to you can create your own addon extension to enforce 1 and 3 when hooking into ger.feedpostbot.submit_post_before.