[CDB] Feed post bot

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Get Involved
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
Locked
User avatar
3Di
I've Been Banned!
Posts: 17538
Joined: Mon Apr 04, 2005 11:09 pm
Location: I'm with Ukraine 🇺🇦
Name: Marco
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by 3Di »

I doubt there is any of those with Emoji, those aren't allowed in topic's title ATM hence the error.
Simply that post has been not posted IMO.

Down the error page there are info about where to locate the error in the php file though, which is important to fix it. I know Ger knows how to do that. ;)
🆓 Free support for our extensions also provided here: phpBB Studio
🚀 Looking for a specific feature or alternative option? We will rock you!
Please PM me only to request paid works. Thx. Buy me a coffee -> Image
My development's activity º PhpStorm's proud user º Extensions, Scripts, MOD porting, Update/Upgrades
User avatar
Ger
Registered User
Posts: 2108
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger »

sysz wrote: Sat Sep 30, 2017 9:17 pm Got this error when i tried to manually fetch the feeds again.

Code: Select all

SQL ERROR [ mysqli ]

Incorrect string value: '\xF0\x9F\x98\x82 y...' for column 'topic_title' at row 1 [1366]
It's indeed the unicode for emoji 😂
I didn't expect that this would be a problem though, I thought phpBB would strip disallowed characters in the topic title. I'll look into it, thanks for bringing it to my attention.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:

-Don't PM me for support-
User avatar
3Di
I've Been Banned!
Posts: 17538
Joined: Mon Apr 04, 2005 11:09 pm
Location: I'm with Ukraine 🇺🇦
Name: Marco
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by 3Di »

Ger, you may substitute it with a line of code I posted months ago here AFAIR.

preg_replace('/[\x{10000}-\x{10FFFF}]/u', "\xef\xbf\xbd"

Catchs all within the Emoji'ss range and relace those with a standard appropriated 3-bytes glyph (?)
🆓 Free support for our extensions also provided here: phpBB Studio
🚀 Looking for a specific feature or alternative option? We will rock you!
Please PM me only to request paid works. Thx. Buy me a coffee -> Image
My development's activity º PhpStorm's proud user º Extensions, Scripts, MOD porting, Update/Upgrades
User avatar
Ger
Registered User
Posts: 2108
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger »

Thanks, but I've found how phpBB handles this. I'll just use the utf8_clean_string() function from utf_tools.php. That handles some other stuff as well and it's good not to invent the wheel twice. :)

A new version is being pushed to Github ATM.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:

-Don't PM me for support-
dynserver
Registered User
Posts: 92
Joined: Thu Apr 06, 2017 11:06 pm

Re: [3.1][3.2][RC] Feed post bot

Post by dynserver »

So I have feeds being grabbed from steampowered.com which has like daily game deals and stuff.. and sometimes it will grab things accordingly.. like every hour 1 or 2 things.. but then sometimes it just grabs like a thousand at once? And creates thousands of topics.. What's going on here O_O
User avatar
Ger
Registered User
Posts: 2108
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger »

Please provide the exact Feed URLs
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:

-Don't PM me for support-
dynserver
Registered User
Posts: 92
Joined: Thu Apr 06, 2017 11:06 pm

Re: [3.1][3.2][RC] Feed post bot

Post by dynserver »

http://store.steampowered.com/feeds/newreleases.xml <-- feed type rdf

http://store.steampowered.com/feeds/daily_deals.xml <-- feed type rss


not exactly sure if my feed types are picked correctly.. but it kinda autopicked it once I put the links in.
User avatar
Ger
Registered User
Posts: 2108
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger »

Well, when I look at the source of the new releases feed, they seem to sometimes have many new items in their feed on one day, while other days there is only 1 new item.

The other one goes back to Tue, 17 May 2011 14:16:00 -0700 so yeah, seemingly having an item for each day. So you can easily count how many items that would be :mrgreen:
Should happen only once though, unless they choose to rebuild their feed with new ids and timestamps.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:

-Don't PM me for support-
zorni
Registered User
Posts: 119
Joined: Mon Mar 23, 2009 10:29 pm
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by zorni »

Hey Ger,

since one of your last updates, headlines are only posted in lowercase. Is that intentional?
User avatar
Ger
Registered User
Posts: 2108
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger »

zorni wrote: Sun Oct 22, 2017 7:46 am Hey Ger,

since one of your last updates, headlines are only posted in lowercase. Is that intentional?
Hello,

Thanks for mentioning it. Seems to happen in utf8_clean_string() mentioned earlier. I have now used @3DI's method to ditch emojis from the subject, that does the job as well and keeps the casing original.

I've also incorporated some suggestions from the validation team to harden this extension. It should get validated pretty soon now. Latest version -as always- on github.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:

-Don't PM me for support-
User avatar
Ger
Registered User
Posts: 2108
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][BETA] Feed post bot

Post by Ger »

Mesto wrote: Mon Mar 20, 2017 10:23 am German Translation
Hello,

You might want to know that the translation file has been changed due to a suggestion by the validation team. All language keys have been prefixed with "FPB_" to avoid collisions.
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:

-Don't PM me for support-
User avatar
nhseacoast
Registered User
Posts: 625
Joined: Sun Sep 22, 2002 10:31 pm
Location: NH, USA
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by nhseacoast »

I installed and seems to work great...BUT...I have the Text limit set to "0", yet I'm getting only a portion of a sentence then ...
Any help would be appreciated. Thank you.
Last edited by nhseacoast on Fri Oct 27, 2017 10:30 pm, edited 1 time in total.
http://nhgunforum.com phpBB v3.2.1
http://wtprise.org phpBB v3.3.2
User avatar
PaulusB
Registered User
Posts: 232
Joined: Sun May 28, 2006 2:08 pm
Location: Holland
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by PaulusB »

Hello i updated to the last version. but one of my feeds gets briked because of a img tag

The feed is oke http://www.autoblog.com/category/volkswagen/rss.xml

Example https://www.volkswagen-forum.nl/forum/v ... 47&t=12546 there is no text anymore
and the link of the img is broken.

Can u fix this.
User avatar
Ger
Registered User
Posts: 2108
Joined: Wed Jan 02, 2008 7:35 pm
Location: 192.168.1.100
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by Ger »

PaulusB wrote: Fri Oct 27, 2017 9:24 pm Hello i updated to the last version. but one of my feeds gets briked because of a img tag

The feed is oke http://www.autoblog.com/category/volkswagen/rss.xml

Example https://www.volkswagen-forum.nl/forum/v ... 47&t=12546 there is no text anymore
and the link of the img is broken.

Can u fix this.
Autoblog appears to have their image tag changed, which led the HTML2BBcode function fail. It seems to be valid HTML though, so I've changed the regex to match. If you update, it should work again as expected.
nhseacoast wrote: Fri Oct 27, 2017 7:37 pm I installed and seems to work great...BUT...I have the Text limit set to "0", yet I'm getting only a portion of a sentence then ...
Any help would be appreciated. Thank you.
I keep saying this over and over again: I cannot provide support when you don't provide the feed URL that's bothering you. I have no crystal ball...
And have you even checked the feed content? Does it even provide full texts or only short intros?
My extensions:
Simple CMS, Feed post bot, Avatar Resize, Modbreak, Magic OGP, Live topic update, Modern Quote, Quoted Where (GDPR) and Autoresponder.
Newest: FAQ manager for 3.2

Like my work? Buy me a coffee to keep it coming. :ugeek:

-Don't PM me for support-
User avatar
nhseacoast
Registered User
Posts: 625
Joined: Sun Sep 22, 2002 10:31 pm
Location: NH, USA
Contact:

Re: [3.1][3.2][RC] Feed post bot

Post by nhseacoast »

You were correct. My apologies. The RSS feed must have been limited.
Thank you.
http://nhgunforum.com phpBB v3.2.1
http://wtprise.org phpBB v3.3.2
Locked

Return to “Extensions in Development”