[Python] UnicodeDecodeError codec can't decode error using pandas read_csv

I'm opening a csv file using pandas.

import pandas as pd
df = pd.read_csv('/file/planned.csv')


I'm opening a file that contains about 2,000 records collected from all over the places in the world. When I'm trying to open this file with pandas, I'm getting the following errors for

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xec in position 34: invalid continuation byte


After I searched through the web, I was able to put the following encoding options hoping that I could open the file. However, I'm still getting the following error messages for each encoding options I tried.

utf-8


df_planned = pd.read_csv('/content/sample_data/planned.csv', encoding='utf-8')
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xec in position 34: invalid continuation byte

utf-16


df_planned = pd.read_csv('/content/sample_data/planned.csv', encoding='utf-16')
> UnicodeDecodeError: 'utf-16-le' codec can't decode bytes in position 234-235: illegal encoding

euc-kr


df_planned = pd.read_csv('/content/sample_data/planned.csv', encoding='euc-kr')
UnicodeDecodeError: 'euc_kr' codec can't decode byte 0x84 in position 37: illegal multibyte sequence


I'm still not able to open the file into the dataframe using the pandas.

cp949


df_planned = pd.read_csv('/content/sample_data/planned.csv', encoding='cp949')
UnicodeDecodeError: 'cp949' codec can't decode byte 0xe8 in position 43: illegal multibyte sequence


Could anyone help? Thank you so much.

Continue reading...