[2.0.x] Tweaks for large forums

The 2.0.x discussion forum has been locked; this will remain read-only. The 3.0.x discussion forum has been renamed phpBB Discussion.
Locked
bbmarty
Registered User
Posts: 2
Joined: Fri Mar 10, 2006 11:34 am

Post by bbmarty »

Would it not be an idea to post patch diff's?
BJv
Registered User
Posts: 18
Joined: Sun Mar 06, 2005 4:40 pm

Post by BJv »

Is there noone who can help me with the viewtopic mod form the first post in this topic? My forum is crashing constantly because of large topics :(
bbmarty
Registered User
Posts: 2
Joined: Fri Mar 10, 2006 11:34 am

Post by bbmarty »

Seems phpbb.com also suffers the same, it just timed out on me several times loading this thread.
Image
User avatar
Dog Cow
Registered User
Posts: 2499
Joined: Fri Jan 28, 2005 12:14 am
Contact:

For power-users only!

Post by Dog Cow »

Marnboorapa wrote: Some one please summarize this topic coz some code changes was posted for a long time which different phpBB version.

I think when anyone post their own code change ideas please specify that code can apply for which phpBB version.

BJv wrote: Is there noone who can help me with the viewtopic mod form the first post in this topic? My forum is crashing constantly because of large topics :(


Several others have been asking this similar thing and I would like to repeat what has been said before:

This topic has code changes for power-users who know about PHP and MySQL. There probably isn't going to be anyone posting here who is going to give you step-by-step instructions on what to do. So, if you are not able to apply the changes, understand what they mean or anything else, then they aren't for you.
Moof!
Mac GUI Vault: Retro Apple II & Macintosh computing archive.
Inside Allerton bookMac GUIMac 512K Blog
User avatar
Joe User
Registered User
Posts: 71
Joined: Mon Sep 13, 2004 9:56 am
Location: Germany
Name: Markus Kohlmeyer
Contact:

Post by Joe User »

Some months ago I implemented MySQL-Replication and now I would like to redirect the SELECT-Statements to the MySQL-Slave. My first idea was to replace all the related $db->foo with $db2->foo and adding an appropriate $db2 = new sql_db(bar); to include/db.php. Therefor i grepped my sources and found ~500 SELECT-Statements, which I'll have to change and maintain on updates. That is much work and I'm sure that there is a simpler solution than mine. So, how do you handle this?
PayPal.Me/JoeUserFreeBSD Remote Installation
Wings for LifeWings for Life World Run

„If there’s more than one possible outcome of a job or task, and one
of those outcomes will result in disaster or an undesirable consequence,
then somebody will do it that way.“ -- Edward Aloysius Murphy Jr.
NeoThermic
Security Consultant
Posts: 2141
Joined: Thu Dec 25, 2003 1:33 am
Location: United Kingdom
Contact:

Post by NeoThermic »

Joe User wrote: Some months ago I implemented MySQL-Replication and now I would like to redirect the SELECT-Statements to the MySQL-Slave. My first idea was to replace all the related $db->foo with $db2->foo and adding an appropriate $db2 = new sql_db(bar); to include/db.php. Therefor i grepped my sources and found ~500 SELECT-Statements, which I'll have to change and maintain on updates. That is much work and I'm sure that there is a simpler solution than mine. So, how do you handle this?


Just a henious guess, but since everything goes through sql_query() in the {dbal}.php file, why not do a quick strstr() for 'select' (don't forget strtolower() on the query before you search for 'select') and direct the query to the slave there? Then you've only got one file to edit.

NeoThermic
NeoThermic.com... a well of information. Ask me for the bit bucket so you can drink its goodness. ||新熱です
da_badtz_one
Registered User
Posts: 376
Joined: Thu Jan 29, 2004 8:25 pm

Post by da_badtz_one »

Hello there R45, could you elaborate on your new search system a bit more? I'm very interested as it could lead to a faster or better search algorithm.

I'm thinking something along your lines however split the tables up with a search_wordlist_a, b, c, d, and so on so that if you should for something like phpbb mod it would search for phpbb first from the p wordlist table and then mod from the m table.
lanzer
Registered User
Posts: 152
Joined: Wed Oct 10, 2001 10:00 am
Contact:

Life without Lanzer

Post by lanzer »

Hello Caspert Ghost:

1: I literally started learning PHP months before starting Gaia. I never wanted to be a professional programmer, but between me and 3 other artists, I was the only guy capable of programming for Gaia. :) Having C and Perl background, learning PHP was not difficult. Aside from Google search on PHP tutorials, I learned a whole lot from reading all the codes from opensource software like phpbb itself. What better ways to get started than to dive right in? lol

2: Gaia has about 200 moderators and helpers now. There are 6 admins and 2 of them are devoted to helping moderators full time. While others including myself continue to develop Gaia while keeping an eye on the community.

3: Popularity is a blessing and a curse. Many sites had shut down because too much traffic had caused servers to go down or bandwidth cost to skyrocket. As your site pick up momentum, be sure to work with ad solutions like Google AdSense to generate some money so that you can upgrade or pay for your hosting costs. It's a win-win situation. At the end, you gotta be the creative brain in making your site popular, and you also gotta think of how to handle everything outside of engineering.

Hope this helps
caspert_ghost wrote: 1: you mentioned earlier you are NOT a php or sql expert, mind sharing your idea on best practices for learning, or is it "dive in and see if it works LOL?

2: I saw your interview (both of them) and you mentioned you only have about 100 moderators, may I ask how many admins or is it just you. (do you run Jr admins as well?

3: What advice would you give to prepare someone if their board reaches the sky as yours has?
Last edited by lanzer on Fri Apr 14, 2006 8:26 am, edited 1 time in total.
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

Post by dcz »

What would be interesting to know is what is gaiaonline server setup ?

Single server, cluestered ?
Separated sql, with slaves ?
Which db type ?
What is your session trick (30 000 at a time, must be hard to deal with sometimes) ?

And since obviously you tweaked it a lot, did you implement other things than the one demonstarted here?

This may have olready been said somewhere, but I am wondering.

By the way, bravo for gaia, hudge, and still fast.

Thanks to you, nobody can really say phpbb is not a working script (even though you must have changed a lot of things).

++

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS
User avatar
drathbun
Former Team Member
Posts: 12204
Joined: Thu Jun 06, 2002 3:51 pm
Location: TOPICS_TABLE
Contact:

Post by drathbun »

dcz wrote: What would be interesting to know is what is gaiaonline server setup ?

I believe the answers to your questions are earlier in this topic.
I blog about phpBB: phpBBDoctor blog
Still using phpbb2? So am I! Click below for details
Image
mgutt
Registered User
Posts: 346
Joined: Tue Sep 21, 2004 2:54 pm
Location: Germany, Sankt Augustin
Contact:

Post by mgutt »

Forums: Honda || phpBB Categories Hierarchy
Mods: Spamfilter against bot registrations || Seo Urls
dcz
Registered User
Posts: 787
Joined: Sun Feb 13, 2005 5:37 am
Contact:

Post by dcz »

Thank you both ;)

phpBB SEO || phpBB3 SEO Premod || SEO phpBB3
GYM Sitemaps & RSS for phpBB3: GYM Sitemaps & RSS
ryan1918dotcom
I've Been Banned!
Posts: 478
Joined: Wed Dec 10, 2003 3:10 am
Location: michigan
Contact:

Post by ryan1918dotcom »

wow that's crazy.
Image
http://www.ryan1918.com More than security.
User avatar
Dog Cow
Registered User
Posts: 2499
Joined: Fri Jan 28, 2005 12:14 am
Contact:

Post by Dog Cow »

ryan1918dotcom wrote: wow that's crazy.


Not to mention expensive! :roll:
Moof!
Mac GUI Vault: Retro Apple II & Macintosh computing archive.
Inside Allerton bookMac GUIMac 512K Blog
lanzer
Registered User
Posts: 152
Joined: Wed Oct 10, 2001 10:00 am
Contact:

Post by lanzer »

Hello dcz:

Today Gaia has a about 150 web servers behind our load balancer, and about 20 databases handling different features. Many of our old servers had been either de-commissioned or converted to storage servers. The new servers we're installing are all Opteron servers that are 4 times faster than older machines with the same power requirement. Since the ISP charges a lot for power, it's much more economical for us to upgrade.

We don't really believe in clustering. Rather we changed our database structure and the database abstraction layer to divide up data into seperate physical databases. That way we only need to add more of the same servers when upgrading, and we don't have to spend money on expensive clustering solutions and risk supporting a complex system.

We've just gotten another 20 database servers to replicate the existing ones for redundency. No replication is used for scalability or handling bandwidth. We still use MyISAM for small logs and data that doesn't get updated more than once a second, otherwise everything else is InnoDB.

The session handler had been completely re-written to use the new database structure and to accomidate a whole bunch of Gaia related features. A caching system for gold earning for example. The user id determines which of the 4 database server to use. When we need more performance we will expand that up to 8 and change the session handler to accomidate 8 servers.

The database structure had also changed for the ability to handle threads of unlimited posts. Changes like these involved a few guys working together and a span of months to complete. Once I find the time I'll publish them for the phpbb team. Right now I'm still pulling my hair trying to get the latest features to release. :?

There are many other nifty things that we're doing, thanks to the new IT guy that joined our team, and many software developed by the guys at Livejournal such as memcache, PerlBal and MogleFS. More details will be posted once the dust settles. :)

-Lanzer
dcz wrote: What would be interesting to know is what is gaiaonline server setup ?

Single server, cluestered ?
Separated sql, with slaves ?
Which db type ?
What is your session trick (30 000 at a time, must be hard to deal with sometimes) ?

And since obviously you tweaked it a lot, did you implement other things than the one demonstarted here?

This may have olready been said somewhere, but I am wondering.

By the way, bravo for gaia, hudge, and still fast.

Thanks to you, nobody can really say phpbb is not a working script (even though you must have changed a lot of things).

++
Locked

Return to “2.0.x Discussion”