Ripefruit

You are here: Home / IT Talk / Publishing / How We Solved a Encoding Issue (Thanks to ChatGPT)

How We Solved a Encoding Issue (Thanks to ChatGPT)

For over two years, we wrestled with a bizarre issue in our custom-built CMS — corrupted symbols like:

â€, “, ”, ”, ’, ', ‘, ™, •, ®, —, –, ¯, ‰ (and many others)

These strange characters replaced quotation marks, apostrophes, trademark symbols, and more. It was not only ugly — it was affecting content quality and search engine results.

Even more confusing: two identical sites using the same CMS and the same database setup showed different results.

One displayed characters perfectly, the other replaced them with garbage.

We consulted:

  • Programmers

  • Database specialists

  • Web developers

  • Industry veterans

🧠 Nobody could fix it.

🔍 The Early Assumptions & Investigations

The obvious first step was checking encoding:

  • ✅ Meta tags in the HTML: UTF-8

  • ✅ Database collation: utf8_general_ci

  • ✅ File encoding: saved as UTF-8 (no BOM)

  • ✅ Symbol entities: encoded properly (’, ™, etc.)

We tried:

  • Converting special characters to entities

  • Using htmlspecialchars, htmlentities, etc.

  • Changing how files were saved (ANSI → UTF-8)

  • Forcing charset headers in the HTML and HTTP

Yet… the problem persisted.

🧩 The Turning Point — ChatGPT

While working on something unrelated (email HTML layout and simplification), we stumbled back onto the encoding issue. As we casually mentioned the symbol corruption again, ChatGPT zeroed in on something critical:

“Have you set the character set for your MySQL connection?”

💡 It suggested:

mysqli_set_charset($db, “utf8mb4”);

We added this line immediately after connecting to the database:

$db = mysqli_connect("host", "user", "pass", "dbname");mysqli_set_charset($db, "utf8mb4");

🎉 Instant fix. Symbols rendered perfectly — consistently — on both sites.

✅ Why This Works
By default, some MySQL setups use latin1 as the character set for client-server communication. Even if your database and tables are in utf8, if the connection uses latin1, characters get corrupted on the way in or out.

Setting the connection to utf8mb4 ensures:

Full Unicode support (including emoji and rare symbols)

Accurate character transmission between MySQL and PHP

🧠 Final Fix
Just add this after establishing the database connection:

$db = mysqli_connect("host", "user", "pass", "dbname");mysqli_set_charset($db, "utf8mb4");

And make sure your:

HTML is correct

Files are saved as UTF-8 (without BOM)

Database/table columns are utf8mb4_general_ci (or similar)

🙌 A Massive Thanks to ChatGPT
We’ve had talented people look at this issue for years — but it took a little AI magic to see the real problem.

💬 ChatGPT didn’t just give an answer — it:

  • Asked the right questions
  • Suggested multiple avenues to explore
  • Never gave up on finding the real cause
  • Finally nailed the solution with a single line of PHP

Thank you, ChatGPT — you saved us!

Encoding Issue Fix for â€, “, ”, ”, ’, ‘, ‘, ™, •, ®, —, –, ¯

mysqli_set_charset($db, "utf8mb4");

Author note: I included the characters in this post multiple times to help anyone searching for an answer. We tried everthing with the hardest part being finding posts/pages discussing the issue. Yes, I found lots of suggestions but only this one actually resolved it.

Oh, and in case it doesn’t fix your issue, run this query in phpmyadmin and it will remove/replace the bad characters in your content. You will have to change two things: table_name and field_name.

Post Views: 176

Also See..



Recent Posts