I need to read xlsx file 300gb. Count of rows ~ 10^9. I need to get values from one column. File consists of 8 columns. I want to do it as fast as it possible. from openpyxl import load_workbook import datetime wb = load_workbook(filename="C:\Users\Predator\Downloads\logs_sample.xlsx", read_only=True) ws = wb.worksheets[0] count = 0 emails = [] p = datetime.datetime.today() for row in ws.rows: count += 1 val = row[8].value if count >= 200000: break emails.append(val) q = datetime.datetime.today() res = (q-p).total_seconds() print "time: {} seconds".format(res) emails = emails[1:] Now cycle needs ~ 16 seconds to read 200.000 rows. And time complexity is O(n). So, for 10^6 rows will be read for 1.5 minutes nearly. Bit we have 10^9. And for this we must wait 10^3 * 1.5 = 1500 minutes = 25 hours. It's too bad... Help me, please, to solve this problem. Continue reading...
