Save All My Posts

BigVolv · Post by **BigVolv** » Fri Nov 18, 2011 6:21 pm

One of my favorite phpBB forums is in danger of being shut down. I'd like to save all my posts (over 5000) to my computer, as there are a lot of good memories. I recall reading something here about a way to do this back in the v2 days, but I can't find the post today. IIRC someone wrote a tool that would search and copy all your posts to a text file. Can this still be done in v3?

marian0810 · Post by **marian0810** » Fri Nov 18, 2011 6:37 pm

Yes I remember that very well...

It was never released though.

http://www.phpbb.com/community/viewtopi ... 7&t=562619

There's no such function in phpBB3 so I've moved your topic to MOD requests.

BigVolv · Post by **BigVolv** » Fri Nov 18, 2011 6:42 pm

I'm not sure this fits as a mod request, becasue I am not an administrator of this board, only a member.

marian0810 · Post by **marian0810** » Fri Nov 18, 2011 6:45 pm

In that case I think your only option is to copy and paste them all by hand.

4_seven · Post by **4_seven** » Fri Nov 18, 2011 6:50 pm

it gives web-spider/grab programms. with this programms you can spider/grab complete sites.
eg. http://www.xaldon.de/products_webspider.html

on big sites it can take some days, but it works.

BigVolv · Post by **BigVolv** » Sat Nov 19, 2011 11:42 pm

Moderatadores, could you please move this back to the phpBB Discussion forum, since this is not a Mod request?

4_seven · Post by **4_seven** » Sat Nov 19, 2011 11:43 pm

Have you start the spider engine. no time to loose

BigVolv · Post by **BigVolv** » Mon Nov 21, 2011 7:38 am

I don't read German.

marian0810 · Post by **marian0810** » Mon Nov 21, 2011 11:25 am

Moved back to discussion

4_seven · Post by **4_seven** » Mon Nov 21, 2011 3:19 pm

BigVolv wrote:I don't read German.

Than translate it or search for a freeware with multilanguage..

igorw · Post by **igorw** » Wed Nov 23, 2011 10:48 am

If you know how to write a scraper, it should be relatively easy to implement. Let me describe how one would do it.

Make a request to http://www.phpbb.com/community/search.p ... igorw&ch=0 (substitute "igorw" for your username)
Adjust the "start" parameter to loop through all the pages. First page is 0, second is 15, third is 30, and so on.
Extract all the post IDs
For each post id, make a request to http://www.phpbb.com/community/posting. ... e=edit&p=x, replace x with the post id. Extract the content of the textarea tag, htmlspecialchars_decode it, store it.
Use something like curl_multi to parallelize the requests.
Use something like Symfony2's BrowserKit component to easily parse the contents of the page and read data out of it using CSS selectors
That's it!

Not too hard for a coder to implement.

3Di · Post by **3Di** » Wed Nov 23, 2011 12:05 pm

an example here: https://github.com/indigolemon/phpBB-Scraper

drathbun · Post by **drathbun** » Thu Nov 24, 2011 10:40 pm

For what it's worth, some board owners / hosts frown on such activity because it can use excessive bandwidth and system resources.

advertisements