[RC][1.1.0-r2] FIND

A place for MOD Authors to post and receive feedback on MODs still in development. No MODs within this forum should be used within a live environment!
Ideas Centre
User avatar
PoPoutdoor
Registered User
Posts: 217
Joined: Fri Oct 31, 2008 2:58 pm
Contact:

Re: [1.1.0] FIND

Post by PoPoutdoor » Mon Sep 09, 2013 5:31 am

For those downloaded before this post, please checkout security update and applied the changes here.
Last edited by PoPoutdoor on Mon Sep 09, 2013 5:38 am, edited 1 time in total.

User avatar
Oyabun1
Former Team Member
Posts: 23162
Joined: Sun May 17, 2009 1:05 pm
Location: Australia
Name: Bill

Re: [1.1.0] FIND

Post by Oyabun1 » Mon Sep 09, 2013 5:34 am

Hippie459MN wrote:Question, how do I download this from github? I am not seeing any download links for the file. Sorry, never used github before.
In the right-hand column the bottom button you should see is Download ZIP.
                      Support Request Template
3.0.x: Knowledge Base Styles Support MOD Requests
3.1.x: Knowledge BaseStyles SupportExtension Requests

User avatar
PoPoutdoor
Registered User
Posts: 217
Joined: Fri Oct 31, 2008 2:58 pm
Contact:

Re: [1.1.0] FIND

Post by PoPoutdoor » Mon Sep 09, 2013 5:36 am

@Hippie459MN, update with direct download link on first post.

User avatar
PoPoutdoor
Registered User
Posts: 217
Joined: Fri Oct 31, 2008 2:58 pm
Contact:

Re: [1.1.0] FIND

Post by PoPoutdoor » Wed Sep 11, 2013 8:11 am

1.1.0-r1 released, please download the latest version or checkout the last commit on GitHb.

Release note: fix typo on feed_import.php

User avatar
prototech
Former Team Member
Posts: 5406
Joined: Mon Mar 19, 2007 2:04 pm
Location: Southern California

Re: [1.1.0] FIND

Post by prototech » Wed Sep 11, 2013 10:42 pm

Hi,

As per the MODs in Development forum rules, topic titles must contain a prefix designating the stage of development that the MOD is in. Your topic is missing this piece of information. Please adjust the title accordingly as soon as possible.

Thanks,
The phpBB Modifications Team
Need help with MOD/style installations or other phpBB problems? Contact me for a quote.

User avatar
PoPoutdoor
Registered User
Posts: 217
Joined: Fri Oct 31, 2008 2:58 pm
Contact:

Re: [RC][1.1.0] FIND

Post by PoPoutdoor » Thu Sep 12, 2013 4:21 am

Topic title updated.

User avatar
PoPoutdoor
Registered User
Posts: 217
Joined: Fri Oct 31, 2008 2:58 pm
Contact:

Re: [RC][1.1.0-r2] FIND

Post by PoPoutdoor » Fri Sep 13, 2013 6:18 am

1.1.0-r2 released and submitted for validation, please get updates from GitHub or get the download link from first post.

InTempest
Registered User
Posts: 26
Joined: Tue Feb 01, 2011 8:58 pm

Re: [RC][1.1.0-r2] FIND

Post by InTempest » Sat Oct 05, 2013 11:28 pm

As soon as I have time i'll check it out

BernhardSiwek
Registered User
Posts: 5
Joined: Wed Nov 13, 2013 7:44 pm

Re: [RC][1.1.0-r2] FIND

Post by BernhardSiwek » Wed Nov 13, 2013 8:04 pm

Hello I do have a urgent problem with this mod.
I tried to use this url:
http://feeds.feedburner.com/nandurion/news

But getting SQL errors. It seems to be a problem with encoding there.

I tested some englisch feeds i.e. from bbc and it worked fine.
As I could figure out you have change a bit of the code & encoding thing ?

Could you please lend me a helping hand out of there?

My setup:
phpbb3 version: 3.0.12
Mod: latest release from github
database collation: utf8_general_ci

I can add the feed without any error messages.
But if I start to import it from ACP it occurs:

Code: Select all

Allgemeiner Fehler
SQL ERROR [ mysqli ]

Incorrect string value: '\x80&brvb...' for column 'post_text' at row 1 [1366]

SQL

INSERT INTO phpbb_posts (forum_id, poster_id, icon_id, poster_ip, post_time, post_approved, enable_bbcode, enable_smilies, enable_magic_url, enable_sig, post_username, post_subject, post_text, post_checksum, post_attachment, bbcode_bitfield, bbcode_uid, post_postcount, post_edit_locked, topic_id) VALUES (16, 68, 0, '127.0.0.1', 1384372835, 1, 1, 0, 1, 0, '', 'Sex & Crime und die richtige Ausrüstung', '[url=http://nandurion.de][size=125]Nandurion[/size][/url]\n[size=85][color=indigo]DSA News[/color][/size]\nUpdated at: [color=green]Mi 13. Nov 2013, 10:29[/color]\n\n--------\nPost at: [color=green]Mo 11. Nov 2013, 18:08[/color]\n\nDas nächste DSA-Spiel von Daedalic, Blackguards, kommt zwar erst im Januar als Vollversion heraus, man kann es aber über Steam als Early Access-Version in einer Art offenen Beta kaufen und testen. Die Teilzeithelden durften dem Spiel einen Besuch abstatten und � [url=http://nandurion.de/blog/2013/11/11/sex-crime-und-die-richtige-ausrustung/]Weiterlesen �[/url]\n\n[url=http://feedproxy.google.com/~r/nandurion/news/~3/RmaDJ7LM3wU/][i]Read more...[/i][/url] - [url=http://nandurion.de/blog/2013/11/11/sex-crime-und-die-richtige-ausrustung/#comments][i]Post your comments[/i]\n[/url]', '46466ad28ede4cb665b2c89201d984f6', 0, '', '', 1, 1, 52)

BACKTRACE

FILE: (not given by php)
LINE: (not given by php)
CALL: msg_handler()

FILE: [ROOT]/includes/db/dbal.php
LINE: 754
CALL: trigger_error()

FILE: [ROOT]/includes/db/mysqli.php
LINE: 189
CALL: dbal->sql_error()

FILE: [ROOT]/includes/functions_posting.php
LINE: 2004
CALL: dbal_mysqli->sql_query()

FILE: [ROOT]/includes/functions_find.php
LINE: 797
CALL: submit_post()

FILE: [ROOT]/includes/functions_find.php
LINE: 622
CALL: autopost()

FILE: [ROOT]/includes/acp/acp_find.php
LINE: 136
CALL: post_feed()

FILE: [ROOT]/includes/functions_module.php
LINE: 507
CALL: acp_find->main()

FILE: [ROOT]/adm/index.php
LINE: 74
CALL: p_master->load_active()
I would be glad for a helping hand and say thanks in advantages :D

User avatar
PoPoutdoor
Registered User
Posts: 217
Joined: Fri Oct 31, 2008 2:58 pm
Contact:

Re: [RC][1.1.0-r2] FIND

Post by PoPoutdoor » Thu Nov 14, 2013 4:06 am

Most code rewritten from the ground up, indeed.

Just test import the url provided, import OK without error.

I do get the similar SQL error from hourly import sources, with old and current code. The error goes away after particular feed item/entry with specified encoded character which is not valid with declared encoding from the xml header (the spec. prefer UTF-8 but most non-Latin feed source use other encoding method).

The source encoding to unicode conversion is done by php native function before parsing, and data fetched pass to phpBB's utf functions via submit post function.

I'm not 100% sure this is a bug from phpBB's utf functions but I do think this is the source.

BernhardSiwek
Registered User
Posts: 5
Joined: Wed Nov 13, 2013 7:44 pm

Re: [RC][1.1.0-r2] FIND

Post by BernhardSiwek » Thu Nov 14, 2013 8:14 am

First of all: WOW! didn't expected such a fast reaction. Thank you very much for that :shock: :D

As i could say for now it has something to do with the encoding.
I tried several other german rss feeds and they are all messed up ending in the same results of SQL errors.

I looked for UTF-8 inside your files and changed it to ISO-8859-1.
So from now on there were posted most times 2,3 topics as a guest bevor I do get a SQL error.

I'm really sorry. I'm not deep into the world of coding so I can't be really a help.
Could you please tell me if your code is encoding the rss feed into propper UTF-8 before putting it into the db?
Don't get me wrong. i don't want to offend you. Just trying to understand a bit more. I do really like this mod and want to use it!

As i already told it worked with an older version of your code, a 3.0.8 bord and the same rss feed URL.

UPDATE: I checked my db and found out that the collation ist UTF-8_bin could this cause this problem?

BernhardSiwek
Registered User
Posts: 5
Joined: Wed Nov 13, 2013 7:44 pm

Re: [RC-5] RSS Input

Post by BernhardSiwek » Thu Nov 14, 2013 3:36 pm

I took the old one (rss_input 1.0.1) which is working fine...except one thing:

It is posting as "guest".
If you click on it you will be redirected to the details, where the name of the bot is like it should.

Any ideas?
So close to it :-) :D

User avatar
PoPoutdoor
Registered User
Posts: 217
Joined: Fri Oct 31, 2008 2:58 pm
Contact:

Re: [RC][1.1.0-r2] FIND

Post by PoPoutdoor » Thu Nov 14, 2013 4:37 pm

My reply is lost for session timeout, again!

So, just brief answers...

1. Encoding and mod version

Old version have code to do encoding conversion before passing data to parser code.
Current version drop conversion code and use simplexml functions from recent php versions, for these native function auto-magically do the conversion and do the parsing with-in a single call.

2. Database charset
For example, in German and some other languages “ß” is equal to “ss”. utf8_unicode_ci also supports contractions and ignorable characters. utf8_general_ci is a legacy collation that does not support expansions, contractions, or ignorable characters. It can make only one-to-one comparisons between characters.
Please check this out.

3. Old code, current phpBB version

phpBB submit_post function been updated since old version code freeze, I'm not going to fix the old code.


Try changing the database charset setting with current code, tell me if this fixed or not.

BernhardSiwek
Registered User
Posts: 5
Joined: Wed Nov 13, 2013 7:44 pm

Re: [RC][1.1.0-r2] FIND

Post by BernhardSiwek » Thu Nov 14, 2013 6:28 pm

I'm sorry but I can't follow you 100%. :roll:
I don't understand for what reasons I should alter my database table to what?
:?: :?:
Isn't it smarter to fetch the feed test its encoding convert if necessary and then place it into the database?
And didn't you mention SQL errors either testing the URL? I'm a little bit confused.

User avatar
PoPoutdoor
Registered User
Posts: 217
Joined: Fri Oct 31, 2008 2:58 pm
Contact:

Re: [RC][1.1.0-r2] FIND

Post by PoPoutdoor » Thu Nov 14, 2013 7:10 pm

BernhardSiwek wrote: Isn't it smarter to fetch the feed test its encoding convert if necessary and then place it into the database?
Seems you don't get what I'm trying to say from my first and last reply.

It is 0300 here now... you are free to do what you consider "smart", 'cos this mod is GPLed.

Locked

Return to “[3.0.x] MODs in Development”