[Python] Optimize the python code to remove words from very large dataframe

Discussão em 'Python' iniciado por Stack, Janeiro 26, 2025.

  1. Stack

    Stack Membro Participativo

    My aim is to remove rare words from dataframe of size 3 million. Below code is taking very long time. Is there way i can optimize it?

    rare_word=[]
    for k,v in frequency_word.items():
    if v<=1:
    rare_word.append(k)

    df['description']=df['description'].apply(lambda x:[i for i in x if i not in rare_word] )

    Continue reading...

Compartilhe esta Página