[Python] Parse a CSV translation file that contains "None" as a standalone string [duplicate]

I am working on a large CSV file that contains number IDs for translations followed by entries for different languages. These entries represent localization strings in an application. I was tasked with parsing and making changes to these entries but have encountered an issue when the string "None" as is an entry in the CSV file.

For example I prepared the following test CSV file (test.csv):

2759,Keine,Keine,null,nan,,Nessuno,Žádné,Ingen,Nenhum,Nada,无,無し,Brak,無
2762,Keine,"None",Нет,Aucun,None,Nessuno,Žádné,Ingen,Nenhum,Nada,无,無し,Brak,無


I made some manual changes to the entries just so I can test the pandas parser with different null connotations and see how it reacts. One entry is empty because it's normal to have empty entries, this is sometimes used when translations are worked on and it is important that we parse the empty entries as empty strings.

I ran the following script:

import os, pandas

testDir = "{some_directory}\\Test"
testCSV = os.path.join(testDir, "test.csv")

csvContent = None


# read export CSV translations file
if os.path.exists(testCSV):
csvContent = pandas.read_csv(testCSV, sep=',', header = None, dtype=str)
else:
print("Error", f"Could not locate a suitable translations file at:\n{testCSV}")

print(f"Translation CSV file read with {len(csvContent)} rows and {len(csvContent.columns)} columns")

print(csvContent)


Output would look like:

Translation CSV file read with 2 rows and 15 columns
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
0 2759 Keine Keine NaN NaN NaN Nessuno Žádné Ingen Nenhum Nada 无 無し Brak 無
1 2762 NaN NaN Нет Aucun NaN Nessuno Žádné Ingen Nenhum Nada 无 無し Brak 無


as expected all instances of None, "None", nan, null, and empty entries are parsed as NaN. But I want to make sure that the strings are parsed literally and that I capture the empty entries as empty strings.

I tried first using fillna() like this:

csvContent[::] = csvContent[::].fillna('')


This would help me preserve the empty fields but also erases any instance of None. I tried to replace any None with a filler string and then convert back using:

csvContent[::] = csvContent[::].replace([None], "$None").fillna('').replace("$None", "None")


I did this assuming that pandas parser would distinguish between empty and ´[None]´ entries and fix my issue with the "None" localize strings. This however turns everything into "None" string and the output looks like this:

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
0 2759 Keine Keine None None None Nessuno Žádné Ingen Nenhum Nada 无 無し Brak 無
1 2762 None None Нет Aucun None Nessuno Žádné Ingen Nenhum Nada 无 無し Brak 無


At this point I am stumped, is there a way I can literally parse my output as is conserving the values? My fallback will be to manually parse the document myself but then I would lose all the nice Pandas functionalities.

Continue reading...