[3.2][RC] Feed post bot

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Suggested Hosts
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
User avatar
3Di
Registered User
Posts: 11826
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milano - Frankfurt
Name: Marco
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by 3Di » Sun Oct 01, 2017 6:52 am

I doubt there is any of those with Emoji, those aren't allowed in topic's title ATM hence the error.
Simply that post has been not posted IMO.

Down the error page there are info about where to locate the error in the php file though, which is important to fix it. I know Ger knows how to do that. ;)
Want to compensate me for my interest? Donate
Please PM me only to request paid works. Thx.
Extensions, Scripts, MOD porting, Update/Upgrades

User avatar
Ger
Registered User
Posts: 1104
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger » Mon Oct 02, 2017 8:45 am

sysz wrote:
Sat Sep 30, 2017 9:17 pm
Got this error when i tried to manually fetch the feeds again.

Code: Select all

SQL ERROR [ mysqli ]

Incorrect string value: '\xF0\x9F\x98\x82 y...' for column 'topic_title' at row 1 [1366]
It's indeed the unicode for emoji 😂
I didn't expect that this would be a problem though, I thought phpBB would strip disallowed characters in the topic title. I'll look into it, thanks for bringing it to my attention.
Checkout my extensions: Simple CMS, Feed post bot, Modbreak, Magic OGP links and Live topic update

Like my work? Buy me a coffee to keep it coming. :ugeek:

User avatar
3Di
Registered User
Posts: 11826
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milano - Frankfurt
Name: Marco
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by 3Di » Mon Oct 02, 2017 9:00 am

Ger, you may substitute it with a line of code I posted months ago here AFAIR.

preg_replace('/[\x{10000}-\x{10FFFF}]/u', "\xef\xbf\xbd"

Catchs all within the Emoji'ss range and relace those with a standard appropriated 3-bytes glyph (?)
Want to compensate me for my interest? Donate
Please PM me only to request paid works. Thx.
Extensions, Scripts, MOD porting, Update/Upgrades

User avatar
Ger
Registered User
Posts: 1104
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger » Mon Oct 02, 2017 9:53 am

Thanks, but I've found how phpBB handles this. I'll just use the utf8_clean_string() function from utf_tools.php. That handles some other stuff as well and it's good not to invent the wheel twice. :)

A new version is being pushed to Github ATM.
Checkout my extensions: Simple CMS, Feed post bot, Modbreak, Magic OGP links and Live topic update

Like my work? Buy me a coffee to keep it coming. :ugeek:

dynserver
Registered User
Posts: 70
Joined: Thu Apr 06, 2017 11:06 pm

Re: [3.1][3.2][RC] Feed post bot

Post by dynserver » Tue Oct 17, 2017 4:19 am

So I have feeds being grabbed from steampowered.com which has like daily game deals and stuff.. and sometimes it will grab things accordingly.. like every hour 1 or 2 things.. but then sometimes it just grabs like a thousand at once? And creates thousands of topics.. What's going on here O_O

User avatar
Ger
Registered User
Posts: 1104
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger » Tue Oct 17, 2017 6:26 am

Please provide the exact Feed URLs
Checkout my extensions: Simple CMS, Feed post bot, Modbreak, Magic OGP links and Live topic update

Like my work? Buy me a coffee to keep it coming. :ugeek:

dynserver
Registered User
Posts: 70
Joined: Thu Apr 06, 2017 11:06 pm

Re: [3.1][3.2][RC] Feed post bot

Post by dynserver » Tue Oct 17, 2017 5:22 pm

http://store.steampowered.com/feeds/newreleases.xml <-- feed type rdf

http://store.steampowered.com/feeds/daily_deals.xml <-- feed type rss


not exactly sure if my feed types are picked correctly.. but it kinda autopicked it once I put the links in.

User avatar
Ger
Registered User
Posts: 1104
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger » Wed Oct 18, 2017 6:41 am

Well, when I look at the source of the new releases feed, they seem to sometimes have many new items in their feed on one day, while other days there is only 1 new item.

The other one goes back to Tue, 17 May 2011 14:16:00 -0700 so yeah, seemingly having an item for each day. So you can easily count how many items that would be :mrgreen:
Should happen only once though, unless they choose to rebuild their feed with new ids and timestamps.
Checkout my extensions: Simple CMS, Feed post bot, Modbreak, Magic OGP links and Live topic update

Like my work? Buy me a coffee to keep it coming. :ugeek:

zorni
Registered User
Posts: 82
Joined: Mon Mar 23, 2009 10:29 pm
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by zorni » Sun Oct 22, 2017 7:46 am

Hey Ger,

since one of your last updates, headlines are only posted in lowercase. Is that intentional?

User avatar
Ger
Registered User
Posts: 1104
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger » Mon Oct 23, 2017 7:53 am

zorni wrote:
Sun Oct 22, 2017 7:46 am
Hey Ger,

since one of your last updates, headlines are only posted in lowercase. Is that intentional?
Hello,

Thanks for mentioning it. Seems to happen in utf8_clean_string() mentioned earlier. I have now used @3DI's method to ditch emojis from the subject, that does the job as well and keeps the casing original.

I've also incorporated some suggestions from the validation team to harden this extension. It should get validated pretty soon now. Latest version -as always- on github.
Checkout my extensions: Simple CMS, Feed post bot, Modbreak, Magic OGP links and Live topic update

Like my work? Buy me a coffee to keep it coming. :ugeek:

User avatar
Ger
Registered User
Posts: 1104
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][BETA] Feed post bot

Post by Ger » Mon Oct 23, 2017 8:00 am

Mesto wrote:
Mon Mar 20, 2017 10:23 am
German Translation
Hello,

You might want to know that the translation file has been changed due to a suggestion by the validation team. All language keys have been prefixed with "FPB_" to avoid collisions.
Checkout my extensions: Simple CMS, Feed post bot, Modbreak, Magic OGP links and Live topic update

Like my work? Buy me a coffee to keep it coming. :ugeek:

User avatar
nhseacoast
Registered User
Posts: 471
Joined: Sun Sep 22, 2002 10:31 pm
Location: NH, USA
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by nhseacoast » Fri Oct 27, 2017 7:37 pm

I installed and seems to work great...BUT...I have the Text limit set to "0", yet I'm getting only a portion of a sentence then ...
Any help would be appreciated. Thank you.
Last edited by nhseacoast on Fri Oct 27, 2017 10:30 pm, edited 1 time in total.

User avatar
PaulusB
Registered User
Posts: 207
Joined: Sun May 28, 2006 2:08 pm
Location: Holland
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by PaulusB » Fri Oct 27, 2017 9:24 pm

Hello i updated to the last version. but one of my feeds gets briked because of a img tag

The feed is oke http://www.autoblog.com/category/volkswagen/rss.xml

Example https://www.volkswagen-forum.nl/forum/v ... 47&t=12546 there is no text anymore
and the link of the img is broken.

Can u fix this.

User avatar
Ger
Registered User
Posts: 1104
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger » Mon Oct 30, 2017 10:23 am

PaulusB wrote:
Fri Oct 27, 2017 9:24 pm
Hello i updated to the last version. but one of my feeds gets briked because of a img tag

The feed is oke http://www.autoblog.com/category/volkswagen/rss.xml

Example https://www.volkswagen-forum.nl/forum/v ... 47&t=12546 there is no text anymore
and the link of the img is broken.

Can u fix this.
Autoblog appears to have their image tag changed, which led the HTML2BBcode function fail. It seems to be valid HTML though, so I've changed the regex to match. If you update, it should work again as expected.
nhseacoast wrote:
Fri Oct 27, 2017 7:37 pm
I installed and seems to work great...BUT...I have the Text limit set to "0", yet I'm getting only a portion of a sentence then ...
Any help would be appreciated. Thank you.
I keep saying this over and over again: I cannot provide support when you don't provide the feed URL that's bothering you. I have no crystal ball...
And have you even checked the feed content? Does it even provide full texts or only short intros?
Checkout my extensions: Simple CMS, Feed post bot, Modbreak, Magic OGP links and Live topic update

Like my work? Buy me a coffee to keep it coming. :ugeek:

User avatar
nhseacoast
Registered User
Posts: 471
Joined: Sun Sep 22, 2002 10:31 pm
Location: NH, USA
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by nhseacoast » Mon Oct 30, 2017 11:54 pm

You were correct. My apologies. The RSS feed must have been limited.
Thank you.

Post Reply

Return to “Extensions in Development”

Who is online

Users browsing this forum: Tbot [Bot] and 12 guests

cron