[Python] Does python reads all lines of a file when numpy.genfromtxt() is executed?

Discussão em 'Python' iniciado por Stack, Fevereiro 15, 2025.

  1. Stack

    Stack Membro Participativo

    I have really large ASCII file (63 million lines or more) that I would like to read using numpy.genfromtxt(). But, it is taking up so much memory. I want to know what python actually does when numpy.genfromtxt() is executed. Does it read all the lines at once?

    Look at the below code, for example.

    import numpy as np
    data = np.genfromtxt("large.file.txt")


    When I execute the code above, would python read all the contents in large.file.txt and load it on to the memory? If yes, is there another way of reading a large file line-by-line so that python would not use large memory?

    Continue reading...

Compartilhe esta Página