[Python] Slow performance with large dataset processing in Python

Code processing millions of records is slow. How can I optimize it to calculate the average word length more efficiently?

import time

def get_avg_word_length(records):
total_length = 0
word_count = 0
for record in records:
words = record.split()
for word in words:
total_length += len(word)
word_count += 1
return total_length / word_count


records = ["This is a sample record." * 10] * 10**6

start_time = time.time()
avg_length = get_avg_word_length(records)
end_time = time.time()

print(f"Average word length: {avg_length}")
print(f"Time taken: {end_time - start_time} seconds")

Continue reading...