securitynut wrote:The legal side is not an issue.
I don't see how you can say this. In my opinion, the legal side is very much an issue, especially if you don't have permission to take the content. If you have a legitimate right to the data, you should be able to use some other method to do this, like getting the information directly from a database link.
Can you provide a legitimate reason for doing such an activity? Do you have the permission from the target board(s) that you are trying to scrape?
These questions will probably appear to be a bit of a challenge, and they probably are.
I know that I would be extraordinarly upset to find out that someone was doing this to one of my boards, and I would take legal action against the person(s) involved.
There may be a legitimate reason; I'm open for responses. For example, google does a "scrape" of the content in order to index my board. But they don't attempt to store my content in their own database, potentially in a format (another board) that would compete with my own. So I don't have a quibble with them. But someone taking my content, and then making it appear as though my own users were posting the same content on another board would raise some concerns. And not just from me, but from my users as well I would imagine.