[Python] open foodfacts - issue to solve parsing due to memory error [closed]

I'm trying to use database from openfoodfacts but I encounter parser error. when trying to solve it, i encounter memory issue. I was looking to apply chunk approach in order to replace issues per batch and save it as a new file. After research, I found a better code than mine but not taking into acount the memory issue https://fleuryc.github.io/oc_ingeni...-un-organisme-de-sante-publique/notebook.html

Database from OpenFoodFacts: "https://world.openfoodfacts.org/data/en.openfoodfacts.org.products.csv"

Hereunder, my try

import os.path
import pandas as pd

# define csv path
local_path = os.getcwd() + '\\'
csv_filename = 'fr.openfoodfacts.org.products.csv'
csv_path = local_path + csv_filename

# define the name of the file where I will record dataset corrected
corrected_file = 'OFF_corrected.csv'
corrected_file_path = local_path + corrected_file

# Apply correction and save it
with open(csv_path, 'r') as csv_file, open(corrected_file_path, 'a') as clean_file:
chunk_size=1000
for chunk in csv_file (chunksize=chunk_size):
clean_file.write(chunk.replace('\n\t', '\t'))


my brain is liquid, thanks for help

Continue reading...