Pymysql Warning: (1366, "incorrect String Value: '\\xf0\\x9f\\x98\\x8d T...')

I'm attempting to import data (tweets and other twitter text information) into a database using Pandas and MySQL. I received the following error: 166: Warning: (1366, 'Incorrect s

Solution 1:

You need utf8mb4, not utf8, when connecting to MySQL and in the columns involved.

More python tips: http://mysql.rjweb.org/doc.php/charcoll#python (Except use utf8mb4 in place of utf8. UTF-8 should not be changed.)

A more detailed explanation to this can be found here.

Solution 2:

Change the character set and collation properties of the databases, tables, and columns to use utf8mb4 instead of utf8. docs

# Foreach database:
ALTER DATABASE database_name CHARACTERSET= utf8mb4 COLLATE= utf8mb4_unicode_ci;
# Foreachtable:
ALTERTABLE table_name CONVERTTOCHARACTERSET utf8mb4 COLLATE utf8mb4_unicode_ci;
# Foreachcolumn:
ALTERTABLE table_name CHANGE column_name column_name VARCHAR(191) CHARACTERSET utf8mb4 COLLATE utf8mb4_unicode_ci;
# (Don’t blindly copy-paste this! The exact statement depends on the column type, maximum length, and other properties. The above line is just an example for a `VARCHAR` column.)