[Python] List comprehension vs for loop over same data multiple times (speed-wise)

I know that list comprehension is supposed to be faster as it does less things more effectively but what I measure is contra-intuitive to me.

If I have one big chunk of words:

words = ("Hello. How are you? I am fine. What about you? Me too " * 10_000).split(" ")


and I want to perform some statistic over that list of words - I have two option for-loop and list comprehension:

%%timeit

#1
a = [x for x in words if x.islower()]
>> 3.13 ms ± 802 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)


and

a = []
%%timeit
#2
for x in words:
if x.islower():
a.append(x)
>> 2.54 ms ± 101 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)


Which means that comprehension is slower. What I do not understand at all is second example where I need to do multiple tests:

%%timeit

#1
a = [x for x in words if x.islower()]
b = [x for x in words if x.isupper()]
c = [x for x in words if x.istitle()]
d = [x for x in words if x.isalpha()]
e = [x for x in words if x.isnumeric()]
f = [x for x in words if x.isalpha() and x.isupper()]
>> 42.1 ms ± 375 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)


and

%%timeit
#2
for x in words:
if x.islower():
a.append(x)
if x.isupper():
b.append(x)
if x.istitle():
c.append(x)
if x.isalpha():
d.append(x)
if x.isnumeric():
e.append(x)
if x.isalpha() and x.isupper():
f.append(x)
>> 70 ms ± 19.7 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)


I thought that having only one for-loop will be more efficient than having multiple list comprehension going over that list over and over again. What is the key to understand this behavior?

Continue reading...