[CDB] Feed post bot

[david63]

I have just started using this and I am having some mixed results. In principle I have it working apart from one feed. Having tried to do some debugging I think the problem is that the content is within <content:encoded> tags and this is being dropped when this extension is process the data.

Is this something beyond the scope of this extension or do I need to do somethong else?

[Ger]

It could be that the content you're looking for in the feed isn't within designated tags, but it's hard to say without the source. Can you provide the feed URL and tell me what you'd expect to be processed vs. the result?

[Ger]

Got your feed trough PM.

I think the problem is that the content is within <content:encoded> tags and this is being dropped when this extension is process the data.

You are right.
As you may or may not have seen, I use the SimpleXML extension to parse the feed contents. For RSS, I iterate through the <item> list, check for existence of required or possible fallback properties and push it to the list.

In your case, you use namespacing. I haven't implemented that, so when I dump the contents of an item from your feed, I get these properties:

Code: Select all

object(SimpleXMLElement)[10]
  public 'title' => string '...' (length=20)
  public 'link' => string '...' (length=78)
  public 'category' => string '...' (length=4)
  public 'comments' => string '...' (length=87)
  public 'author' => string '...' (length=26)
  public 'pubDate' => string '...' (length=31)
  public 'guid' => string '...' (length=62)

So your content is missing, and the feed post bot falls back to the title.

In this case I would have to check the namespaces and if the content namespace is available, check if it has the encoded attribute and if it exists, use it for the post content.

I might implement it, but I have to think about that (and what's the optimal approach). It's essentially another fallback in an ever-growing list of fallbacks. When I started this extension I had no idea about the dozens of formats of feeds. Can't you just keep things simple on your end?

On a side note: I got some recommendations when validating your feed

[david63]

Thanks Ger

On a side note: I got some recommendations when validating your feed

I had seen those but as this is not my code and to be honest I do not really understand much about feeds - although I now know more than I did 24 hours ago

Can't you just keep things simple on your end?

I think that I may have found a way around the problem by replacing the <content:encoded> tags in the template from my blog with <description> tags.

On another side note - when I was searching for a solution I noticed that <content:encoded> is used in WordPress so it might be worth considering adding this before all the WP users start complaining

[Ger]

Yeah, most of them use both the description as the content:encoded part.

However, I think you're right, I have to try and tackle this one.

[Ger]

poke

I think I've managed to handle the content:encoded style. It favours the description. Would you mind testing it for me?

I've also added a locking mechanism that sets an extra config value, so make sure you disable and enable the extension to run the migration.

[david63]

I think I've managed to handle the content:encoded style. It favours the description. Would you mind testing it for me?

No problem - is it on GitHub? Might not be until tomorrow though.

[david63]

Just downloaded 1.0.5 from Git and it works fine

Thanks.

[Ger]

Great, thanks for testing!

[david63]

I think I have found a couple of minor buglets.

I have three feeds set up and when I look at the Admin log after the feeds have been fetched the username of the first feed is that of the user who invoked the cron job (which is what I would expect), but the subsequent feed entries use the username of the user who is set in the ACP as the Feed user id.

The table of feeds and their options in the ACP is not responsive and can result in the table extending beyond the width of the page.

I notice that you have added a lock to this version. I tried this once and there is a possibility that it can cause problems if you have a situation where the job does not complete for some reason then you can end up with the system locked - I would suggest that you either have an "unlock" button or at the start of the cron job (or elsewhere) auto unlock before you start.

Now for some queries (or feature requests )

1. Is it possible to shorten the title?
2. Is it possible when setting up a feed to only import them from a "start date"?
3. Would it be possible to post a feed to a topic instead of a forum?

[david63]

Another little bug.

When you are setting the feedpostbot_locked variable in driver.php you need to make it dynamic each time.

[Ger]

Thanks. I'll look into it after the holidays

[Ger]

Another little bug.

When you are setting the feedpostbot_locked variable in driver.php you need to make it dynamic each time.

When I look at https://github.com/phpbb/phpbb/blob/mas ... g.php#L124
it seems the third parameter isn't used at all?

[david63]

Another little bug.

When you are setting the feedpostbot_locked variable in driver.php you need to make it dynamic each time.

When I look at https://github.com/phpbb/phpbb/blob/mas ... g.php#L124
it seems the third parameter isn't used at all?

But when you are changing it like this you do not want it to use the cached setting you need it to pick up the revised setting, which is why you need to define it as being dynamic

[Ger]

I have three feeds set up and when I look at the Admin log after the feeds have been fetched the username of the first feed is that of the user who invoked the cron job (which is what I would expect), but the subsequent feed entries use the username of the user who is set in the ACP as the Feed user id.

User is only switched when there are items to post, so the logging is a bit off due to that. However: in this case I don't think it really matters, does it?

The table of feeds and their options in the ACP is not responsive and can result in the table extending beyond the width of the page.

Yeah, that's a thingy indeed. The table was very simple in the beginning of this extension, but got wider along the months.
From a simple list of urls, forumlist and a delete button it got to this. Not ideal, but I'm keeping it this way for the current version which I'm trying to get past validation. Redesigning the ACP part will probably lead to larger issues.

I notice that you have added a lock to this version. I tried this once and there is a possibility that it can cause problems if you have a situation where the job does not complete for some reason then you can end up with the system locked - I would suggest that you either have an "unlock" button or at the start of the cron job (or elsewhere) auto unlock before you start.

Yeah the lock is there to prevent a double process when running from cron and while that process runs, starting it manually from ACP again. It can lead to double postings. I can imagine it can lead to a permalock when the process interrupts. I think I'll add a button for unlocking when needed, thanks for mentioning.

Now for some queries (or feature requests )

1. Is it possible to shorten the title?
2. Is it possible when setting up a feed to only import them from a "start date"?
3. Would it be possible to post a feed to a topic instead of a forum?

All good ideas, but I'll pass them to a second version. If you want to you can create your own addon extension to enforce 1 and 3 when hooking into ger.feedpostbot.submit_post_before.