[Python] Regex-matching a dictionary efficiently in Python

I have a dictionary of word:count pairs; I would like to extract the entries matching any item in a list of strings.

I found a solution using regular expressions, but it takes ~10 hours. It must get faster

import re

dicti={'the':20, 'a':10, 'over':2}
regex_list=['the', 'an?']

extractddicti= {k:v for k,v in dicti.items() if any (re.match("^"+regex+"$",k) for regex in regex_list)}


The dictionary has about 60,000 entries; the regex_list is ~ 1,000. The items in the regex list might match several entries in the dictionary.

Update / Edit


I first compiled my regexes:

regex_list=['the', 'an?']
regex_list_compiled=[re.compile("^"+i+"$") for i in regex_list]


and then slightly adapted my search function:

extractddicti= {k:v for k,v in dicti.items() if any (re.match(regex,k) for regex in regex_list_compiled)}


A run with a dictionary of 14800 items and a list of 1,100 regexes took 34 minutes without compilation, < 1 minute with compilation!

Continue reading...