Page 1 of 7

[CDB] Top Poster Of The Month (reloaded)

Posted: Wed Sep 13, 2017 1:46 am
by 3Di
Author: 3Di

Extension version: v2.0.9
Image (No support for github)

Extension download:
CDB v2.0.9

Want to say thank you?
Image

Update Extension: disable / delete data & files / upload new files / re-enable

Issues: https://github.com/3D-I/tpotm/issues

Special thanks to: canonknipser & Steve

Requirements
  • phpBB 3.1.1 or 3.2.1
  • PHP >= 5.4.0

  • User/admin permissions
  • Multiple template locations
  • Improved performance
  • Raise version min requirements to 3.1.11/3.2.1
  • Twig templating
  • Custom cache TTL (Enable disabling as well)
  • Allow admins/mods to be included
  • Allow Founders to be included
  • Allow banneds to be included
  • Miniavatar on output (default badge if user is avatarless)
  • TPOTM mini-badge in miniprofile next to posts
  • FontAwesome icons for Rhea
  • ACP logs
  • Hall of fame
  • version checker
  • Everything's managed in the new ACP module

Re: [3.1][3.2][DEV] Top Poster Of The Month (reloaded)

Posted: Wed Sep 13, 2017 1:46 am
by 3Di
Hall of fame - RC5
Image
.
ACP - 205
Image
Image
Image

Re: [3.1][3.2][DEV] Top Poster Of The Month (reloaded)

Posted: Wed Sep 13, 2017 1:47 am
by 3Di
Index ^TOP^
Image
.
Miniprofile (fa-icon for 3.2)
Image
.
UCP board prefs
Image

Re: [3.1][3.2][DEV] Top Poster Of The Month (reloaded)

Posted: Thu Sep 14, 2017 8:02 am
by 3Di
Ok, now remains only :lol: the Hall of fame to be coded.

Due to the fact it will be a long job (or maybe not) I will probably release a first fully functional version within the next days. As I said, without the last scheduled feature. See first post.

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Thu Sep 14, 2017 11:20 pm
by 3Di
v1.1.0-rc1 released. See first post.

Feedback is welcome.
Hall of fame not yet available but everything else is working. Added support for FontAwesome icons (Rhea) and some more things (descriptions in the first post)

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Mon Sep 18, 2017 2:49 am
by 3Di
v1.1.0-rc2 released. See first post.
v1.1.0-rc2 - changes log
- Task: Improved badge logic and multi-styles compatibility
- Task: Added check-point on the badge filename
- Issue #13 feature: Hall of fame (still base as a proof-of-concept)
- Feature: total posts count (board/current month) and percentage of TPOTM in statistics (thx Steve)
- All of the logic splitted in chunks to a service
Update Extension: disable / delete data & files / upload new files / re-enable

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Tue Sep 26, 2017 5:44 am
by 3Di
v1.1.0-rc3 released. See first posts.
v1.1.0-rc3 - changes log
- Working Hall of fame
- code rewritten for the hall of fame part
- a lot of improvements and fixes
Updating: disable / delete data & files / upload new files / re-enable

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Tue Sep 26, 2017 12:35 pm
by takufner
Hello 3Di,

I use to use this extension in phpBB 3.1, and it worked like a charm, very well.

After updating to 3.2.1, the old extension stopped working. And, with this new version is not different, same problem: whenever I enable the extension my server goes to extreme high loads, and the forum crashes.

I just tried with the v1.1.0-rc3, and the behavior is the same.

Is this ok? Is it doing the initial process, looking for all the top posters? I tried to leave it processing for some 7 minutes.. but it never ends, then I have to turn off the extension and restart the server.

Kind regards,

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Tue Sep 26, 2017 7:00 pm
by 3Di
As I said there in the validated ext support topic:
without any error log (see your server error logs) isn't possible to tell you what's going on there.
How many users/posts do you have there?
Server configuration?


Once you have the error logs we can discuss your issue. Thx.

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Wed Sep 27, 2017 3:59 am
by takufner
Hello 3Di, thanks for your answer.

My forum:
  • messages: 145184
  • topics: 54237
  • users: 13247
My server:
  • 8 GB RAM
  • 4 CPU Cores
  • Ubuntu 16.04
  • Apache 2.4
  • PHP Version 7.0.22-0ubuntu0.16.04.1
  • MySQL(i) 5.7.19-0ubuntu0.16.04.1-log
In order to see the log I issued the following command:

sudo tail -f /var/log/apache2/error.log

And leaved it in my terminal while enabling the extension. The server log didn't show nothing, but the load went skyrocket again.

Looking for the MySQL slow log with tail -f, I could see several entries just like this (same SELECT command):

Code: Select all

# Time: 2017-09-27T03:55:08.426670Z
# User@Host: phpbb_user[phpbb_user] @ localhost []  Id:    10
# Query_time: 110.682814  Lock_time: 0.000102 Rows_sent: 78  Rows_examined: 25381386
SET timestamp=1506484508;
SELECT ug.user_id, a.forum_id, r.auth_setting, r.auth_option_id, ao.auth_option
			FROM phpbb_acl_groups a, phpbb_user_group ug, phpbb_groups g, phpbb_acl_roles_data r, phpbb_acl_options ao
			WHERE a.auth_role_id = r.role_id AND r.auth_option_id = ao.auth_option_id 
				AND a.group_id = ug.group_id
				AND g.group_id = ug.group_id
				AND ug.user_pending = 0
				AND NOT (ug.group_leader = 1 AND g.group_skip_auth = 1)
				
				
				AND ao.auth_option = 'm_';
Looks like something from autogroups, don't you think?

Kind regards,

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Wed Sep 27, 2017 4:14 am
by 3Di
Is it the only error log you got? Feel free to send me via PM the whole log in a compressed archive.
I will examine it for the sake of this.

Btw, did you try to run the extension on a local environment (a copy of your production board).
Disabling all of the extensions but using this one? We could see this way where the issue starts.

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Wed Sep 27, 2017 4:44 am
by takufner
Hello 3Di,

Following your advice, I just disabled all extensions and enabled just the TPOTM.

The apache server log has nothing.

MySQL log is the same as above, I reproduce it here in full:

Code: Select all

/usr/sbin/mysqld, Version: 5.7.19-0ubuntu0.16.04.1-log ((Ubuntu)). started with:
Tcp port: 3306  Unix socket: /var/run/mysqld/mysqld.sock
Time                 Id Command    Argument




# Time: 2017-09-27T04:35:26.903903Z
# User@Host: phpbb_user[phpbb_user] @ localhost []  Id:     4
# Query_time: 16.396520  Lock_time: 0.000060 Rows_sent: 1  Rows_examined: 25381309
use phpbb;
SET timestamp=1506486926;
SELECT ug.user_id, a.forum_id, r.auth_setting, r.auth_option_id, ao.auth_option
			FROM phpbb_acl_groups a, phpbb_user_group ug, phpbb_groups g, phpbb_acl_roles_data r, phpbb_acl_options ao
			WHERE a.auth_role_id = r.role_id AND r.auth_option_id = ao.auth_option_id 
				AND a.group_id = ug.group_id
				AND g.group_id = ug.group_id
				AND ug.user_pending = 0
				AND NOT (ug.group_leader = 1 AND g.group_skip_auth = 1)
				
				
				AND ao.auth_option = 'a_';
# Time: 2017-09-27T04:35:43.007427Z
# User@Host: phpbb_user[phpbb_user] @ localhost []  Id:     4
# Query_time: 15.995557  Lock_time: 0.000097 Rows_sent: 78  Rows_examined: 25381386
SET timestamp=1506486943;
SELECT ug.user_id, a.forum_id, r.auth_setting, r.auth_option_id, ao.auth_option
			FROM phpbb_acl_groups a, phpbb_user_group ug, phpbb_groups g, phpbb_acl_roles_data r, phpbb_acl_options ao
			WHERE a.auth_role_id = r.role_id AND r.auth_option_id = ao.auth_option_id 
				AND a.group_id = ug.group_id
				AND g.group_id = ug.group_id
				AND ug.user_pending = 0
				AND NOT (ug.group_leader = 1 AND g.group_skip_auth = 1)
				
				
				AND ao.auth_option = 'm_';
# Time: 2017-09-27T04:36:20.736735Z
# User@Host: phpbb_user[phpbb_user] @ localhost []  Id:     6
# Query_time: 16.263490  Lock_time: 0.000061 Rows_sent: 1  Rows_examined: 25381309
SET timestamp=1506486980;
SELECT ug.user_id, a.forum_id, r.auth_setting, r.auth_option_id, ao.auth_option
			FROM phpbb_acl_groups a, phpbb_user_group ug, phpbb_groups g, phpbb_acl_roles_data r, phpbb_acl_options ao
			WHERE a.auth_role_id = r.role_id AND r.auth_option_id = ao.auth_option_id 
				AND a.group_id = ug.group_id
				AND g.group_id = ug.group_id
				AND ug.user_pending = 0
				AND NOT (ug.group_leader = 1 AND g.group_skip_auth = 1)
				
				
				AND ao.auth_option = 'a_';
# Time: 2017-09-27T04:36:36.810325Z
# User@Host: phpbb_user[phpbb_user] @ localhost []  Id:     6
# Query_time: 15.962662  Lock_time: 0.000136 Rows_sent: 78  Rows_examined: 25381386
SET timestamp=1506486996;
SELECT ug.user_id, a.forum_id, r.auth_setting, r.auth_option_id, ao.auth_option
			FROM phpbb_acl_groups a, phpbb_user_group ug, phpbb_groups g, phpbb_acl_roles_data r, phpbb_acl_options ao
			WHERE a.auth_role_id = r.role_id AND r.auth_option_id = ao.auth_option_id 
				AND a.group_id = ug.group_id
				AND g.group_id = ug.group_id
				AND ug.user_pending = 0
				AND NOT (ug.group_leader = 1 AND g.group_skip_auth = 1)
				
				
				AND ao.auth_option = 'm_';
# Time: 2017-09-27T04:37:11.569462Z
# User@Host: phpbb_user[phpbb_user] @ localhost []  Id:     9
# Query_time: 16.345419  Lock_time: 0.000061 Rows_sent: 1  Rows_examined: 25381309
SET timestamp=1506487031;
SELECT ug.user_id, a.forum_id, r.auth_setting, r.auth_option_id, ao.auth_option
			FROM phpbb_acl_groups a, phpbb_user_group ug, phpbb_groups g, phpbb_acl_roles_data r, phpbb_acl_options ao
			WHERE a.auth_role_id = r.role_id AND r.auth_option_id = ao.auth_option_id 
				AND a.group_id = ug.group_id
				AND g.group_id = ug.group_id
				AND ug.user_pending = 0
				AND NOT (ug.group_leader = 1 AND g.group_skip_auth = 1)
				
				
				AND ao.auth_option = 'a_';
# Time: 2017-09-27T04:37:27.761269Z
# User@Host: phpbb_user[phpbb_user] @ localhost []  Id:     9
# Query_time: 16.086757  Lock_time: 0.000100 Rows_sent: 78  Rows_examined: 25381386
SET timestamp=1506487047;
SELECT ug.user_id, a.forum_id, r.auth_setting, r.auth_option_id, ao.auth_option
			FROM phpbb_acl_groups a, phpbb_user_group ug, phpbb_groups g, phpbb_acl_roles_data r, phpbb_acl_options ao
			WHERE a.auth_role_id = r.role_id AND r.auth_option_id = ao.auth_option_id 
				AND a.group_id = ug.group_id
				AND g.group_id = ug.group_id
				AND ug.user_pending = 0
				AND NOT (ug.group_leader = 1 AND g.group_skip_auth = 1)
				
				
				AND ao.auth_option = 'm_';
# Time: 2017-09-27T04:38:20.648244Z
# User@Host: phpbb_user[phpbb_user] @ localhost []  Id:    11
# Query_time: 16.018374  Lock_time: 0.000049 Rows_sent: 1  Rows_examined: 25381309
SET timestamp=1506487100;
SELECT ug.user_id, a.forum_id, r.auth_setting, r.auth_option_id, ao.auth_option
			FROM phpbb_acl_groups a, phpbb_user_group ug, phpbb_groups g, phpbb_acl_roles_data r, phpbb_acl_options ao
			WHERE a.auth_role_id = r.role_id AND r.auth_option_id = ao.auth_option_id 
				AND a.group_id = ug.group_id
				AND g.group_id = ug.group_id
				AND ug.user_pending = 0
				AND NOT (ug.group_leader = 1 AND g.group_skip_auth = 1)
				
				
				AND ao.auth_option = 'a_';
I am working now in my localhost.

Do you recognize these SELECT statements? Looking around phpbb code I just could find something in the phpbb/auth/auth.php, but I don't know what is the purpose of the code.

Kind regards,

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Wed Sep 27, 2017 5:24 am
by Steve
In acp settings for the extension disable, Admin and Moderators. See if this helps.

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Wed Sep 27, 2017 6:44 am
by 3Di
The ext is shipped with all the option set to NO and the cache to 30min.

For "big" boards it is strongly suggested to use the cache (ACP option)
Try setting the cache to just 5 minute and all the option you want and you will see. (INDEX PAGE = 0 queries))
So at the end of all, if you don't click on the Hall's link and therefore let the related code run but you use just the index page... what happens?

About the HALL OF FAME i use a different cache (DB TTL not filesystem) and effectively also if cached is being retrieved from the cache of the DB, in my case the result is:
Before: 2.73001s | After: 2.73001s | Elapsed: 0.00000s
May be I will adpot the same filesystem method also for the hall of fame, I already have the code. Dunno yet.

But I am wondering also if your server MySql needs to be fine tuned, optimized?
The same topic for the same query for this extension has been discussed here:
viewtopic.php?p=14688571#p14688571
viewtopic.php?p=14696951#p14696951 (have a read)

Let me know pls. Thank you.

Re: [3.1][3.2][RC] Top Poster Of The Month (reloaded)

Posted: Fri Sep 29, 2017 12:48 pm
by 3Di
takufner wrote:
Wed Sep 27, 2017 4:44 am
@
Please give this a try, thx. :) Should at least (for your case) install and run till the admin/mods aren't excluded, see my above post for a solution in case, or report.
Steve wrote:
Wed Sep 27, 2017 5:24 am
In acp settings for the extension disable, Admin and Moderators. See if this helps.
Disabling Admin/Mods increases the queries, if you select YES (include) those queries aren't run.
See SQL in set. :)
v1.1.0-rc4 released. See first posts.
Changes Log v1.1.0-rc4 since rc3

Include admin and mods as default in ACP (selectable still)- saves queries
Tuning SQLs (optimize)
Prettify cells in viewport and styling here and there
DAE full compatibility (Default Avatar Extended)
Fix Hall's TPOTM avatar
Function change (Hall Top's avatar)
Housekeeping
Take care about the UCP prefs view avatars but the Hall
Hall's default avatars must be only TPOTM IMG (changeables per user's style)
Updating: disable / delete data & files / upload new files / re-enable