Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Anti-Spam Guide
Post Reply
WWu777
Registered User
Posts: 788
Joined: Tue Aug 14, 2007 12:40 pm
Contact:

Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by WWu777 »

Hi everyone,
I noticed that some of my posts from phpbb2 that were converted to phpbb3 have this weird thing where the quotation marks and apostrophes are turned into strange symbols. I noticed this from some of my web pages created in Microsoft Word too. Here is an example.

https://www.happierabroad.com/forum/vie ... p?f=1&t=45
In the Philippines, I observed that generally within most families, children don’t tend to argue, whine, complain or fight for control with their parents and siblings. There seems to be this natural symbiotic harmony and balance within the family that makes such ego/control battles unnecessary. It’s as if the family is one symbiotic unit that gets along without effort. I see this especially in my Filipina girlfriend’s family (and in fact they are what inspired me to write this chapter). There isn’t this “separate individual identity� that needs to assert its will and struggle for control.

But in the US of course, disagreements, arguing and fighting for control between family members are all too common. Children yell and whine to get their way. And parents inflict constant control, yelling daily orders such as, “I told you to put away your toys! It’s bedtime!� and “Alright I’m gonna count to three and if you don’t go, then…�. The parents often have to resort to disciplinary measures to counter disobedience such as threatening to take away privileges. It’s a constant struggle for control in the typical American family.
As you can see, quotation marks and apostrophes have turned into strange symbols. Originally they were like this:

quotation marks
"text"

apostrophes
'text'

This happened with many of my old webpages too. Maybe with those, I can use a search and replace text tool to fix them. But how to fix them on the phpbb forum? And does anyone know how and why this type of thing happens? Have any of you seen it before?
User avatar
Elias
Registered User
Posts: 5137
Joined: Sat Feb 25, 2006 4:31 pm
Location: In the Water!
Name: Elias

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by Elias »

I would guess that it has something to do with UTF-8 and encoding.
"Mystery creates wonder, and wonder is the basis of man's desire to understand." - Neil Armstrong
|Installing Extensions|Writing Extensions|Extension Validation Policy|
Offering private web hosting. Contact me for details.
User avatar
AmigoJack
Registered User
Posts: 5757
Joined: Tue Jun 15, 2010 11:33 am
Location: グリーン ヒル ゾーン
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by AmigoJack »

WWu777 wrote:
Thu Jun 04, 2020 8:02 pm
some of my posts
Look closer: it happened to all your posts when using anything outside ASCII. If you would have used " and ' always (or not let Word automatically replace/choose special characters) instead of only sometimes then those wouldn't come up now.

Now the damage is done (that's why converting/upgrading a board should always be tested multiple times) and infested multiple database tables (it's not just the post text, it could also be the post's subject, forum names, etc...). "Just" directly modifying the database tables to replace the rather unique sequences with what you actually want might work, but there's no guarantee these findings are the only ones that exist. Do you feel confident executing SQL?
The worst thing about censorship is ███████████
Affin wrote:
Tue Nov 20, 2018 9:51 am
The problem is probably not my English but you do not want to understand correctly.
...
We will not come anybody anyway, nevertheless, it's best to shit this.
WWu777
Registered User
Posts: 788
Joined: Tue Aug 14, 2007 12:40 pm
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by WWu777 »

AmigoJack wrote:
Fri Jun 05, 2020 9:08 pm
WWu777 wrote:
Thu Jun 04, 2020 8:02 pm
some of my posts
Look closer: it happened to all your posts when using anything outside ASCII. If you would have used " and ' always (or not let Word automatically replace/choose special characters) instead of only sometimes then those wouldn't come up now.

Now the damage is done (that's why converting/upgrading a board should always be tested multiple times) and infested multiple database tables (it's not just the post text, it could also be the post's subject, forum names, etc...). "Just" directly modifying the database tables to replace the rather unique sequences with what you actually want might work, but there's no guarantee these findings are the only ones that exist. Do you feel confident executing SQL?
I don't understand. The quotation marks and apostrophes are all the same right? I mean if you push the key for them on the keyboard, it all comes up the same right? I think some of these messed up posts were copied and pasted from MS Word files. Is that what happened? I also see the same symbols come up in articles that were copied and pasted from other sites. I have noticed though that some quotation marks look different on different boards. Some are slightly curved while others are straight. I've always wondered why. I didn't know they were different. Strange.

No I am not comfortable executing SQL. I can follow simple instructions. But these are all older posts in the forum archive. No big deal. But some of my articles written in MS Word were affected. Some have tons of question marks too. I would use a search and replace tool to replace them but the messed up symbols are not always consistent.
User avatar
AmigoJack
Registered User
Posts: 5757
Joined: Tue Jun 15, 2010 11:33 am
Location: グリーン ヒル ゾーン
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by AmigoJack »

Was full quoting my post necessary?

WWu777 wrote:
Sun Jun 07, 2020 8:14 pm
The quotation marks and apostrophes are all the same right?
No: https://en.wikipedia.org/wiki/Quotation ... oint_table will give you an overview of how many different ones exist.

WWu777 wrote:
Sun Jun 07, 2020 8:14 pm
if you push the key for them on the keyboard, it all comes up the same right?
No: type the following sentence in i.e. Word and you should notice both same quotation marks have been replaced while you typed (which has been the default auto correct configuration over decades): Do "you" see? becomes Do “you” see? - I didn't write "look closer" for fun: do that and you'll count 3 different quotation marks in this paragraph.
The worst thing about censorship is ███████████
Affin wrote:
Tue Nov 20, 2018 9:51 am
The problem is probably not my English but you do not want to understand correctly.
...
We will not come anybody anyway, nevertheless, it's best to shit this.
User avatar
Lumpy Burgertushie
Registered User
Posts: 67986
Joined: Mon May 02, 2005 3:11 am
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by Lumpy Burgertushie »

and yes, copying and pasting from word or any other word processor will cause all sorts of problems.

MSWord code for an apostrophe is a huge bunch of garbage that a web browser has no idea what to do with.


robert
I'm baaaaaccckkkk. still doing work on donation basis. PM your needs.

Premium phpBB 3.3 Styles by PlanetStyles.net

If nobody is in the forest, does a tree really fall?
WWu777
Registered User
Posts: 788
Joined: Tue Aug 14, 2007 12:40 pm
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by WWu777 »

That's so weird. Why are there different quotation marks? Why can't they be all the same? Why the extra layer of complexity? Isn't it unnecessary? Does Microsoft just like to create trouble?

Is there any easy way to convert all the quotation marks to normal or the weird characters back to normal? Or does it have to be manually done one by one, which takes forever?
User avatar
AmigoJack
Registered User
Posts: 5757
Joined: Tue Jun 15, 2010 11:33 am
Location: グリーン ヒル ゾーン
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by AmigoJack »

WWu777 wrote:
Mon Jul 06, 2020 3:24 am
That's so weird. Why are there different quotation marks? Why can't they be all the same?
No. Because different languages exist throughout the world. They can't be the same for huge historical reasons.
WWu777 wrote:
Mon Jul 06, 2020 3:24 am
Is there any easy way
I already answered that with a counter question which you declined. There is no cheat code available to overcome this.
The worst thing about censorship is ███████████
Affin wrote:
Tue Nov 20, 2018 9:51 am
The problem is probably not my English but you do not want to understand correctly.
...
We will not come anybody anyway, nevertheless, it's best to shit this.
User avatar
warmweer
Jr. Extension Validator
Posts: 4915
Joined: Fri Jul 04, 2003 6:34 am
Location: Van Allen Bel ... gium

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by warmweer »

WWu777 wrote:
Mon Jul 06, 2020 3:24 am
Is there any easy way to convert all the quotation marks to normal or the weird characters back to normal? Or does it have to be manually done one by one, which takes forever?
"easy way " is relative. What is easy for AmigoJack is probably not easy for you (nor for me).
However there is a safe way which doesn't involve rocket science.
Find a post with the wrong quotation marks.
Look up that post in the database (and copy the field value).
Edit the post on your board. (edit only 1 thing so that you don't get confused).
Look up that edited post in the database and note the changes.

Now there's a choice:

- edit all posts directly in the database using SQL, or
- edit an sql backup of that table using search and replace with a good text editor (not Windows Notepad)

Whatever you do, always make sure you can revert back to a "before" situation : i.o.w. Backup!
And when you decide you'll fix all the posts, make sure you put your board in Disabled mode so that when you import your (edited) backup you don't have to worry about new post.
Spelling is freeware, which means you can use it for free.
On the other hand, it is not open source, which means you cannot change it or publish it in a modified form.
WWu777
Registered User
Posts: 788
Joined: Tue Aug 14, 2007 12:40 pm
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by WWu777 »

Thanks. Can MySQL just auto replace all instances of the symbols with the right quotation marks? Rather than one at a time? Or what about SSH? I heard you can use SSH to reparse the forum too. Can that be used to automate a mass edit on every post?
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 3910
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by thecoalman »

You can use simple find and replace but fully read the bolded below before proceeding with anything.

In you hosting control panel open phpmyadmin >> open the database >> open phpbb_posts table >> click search tab on top >> on next page should be link on top "Find and Replace"

In find box paste what to find, in replace box put what to replace it with, in the drop down select the column post_text and click go. Next page should present you with preview of changes, clicking go again will commit changes.

Make sure to have backup of the database. Find and replace is sledge hammer. Finding cat and replacing it with dog turns catastrophe into dogastrophe. Try an be as psecific as possible with find text, e.g. search for cat and replace with dog ... note the spaces.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison
User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 51459
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: Quotation marks/apostrophes from phpbb2 posts turned into strange symbols. Why did this happen?

Post by stevemaury »

Those are MSWord's "smart quotes" where the open quote is and the close quote is . The smart apostrophe is . Completely different characters with different encoding.

See https://smartquotesforsmartpeople.com/
For REALLY good and VERY inexpensive hosting CLICK HERE

I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)
Post Reply

Return to “phpBB Discussion”